Depends on the goal. If staying at a certain number is the goal yes, climbing two weeks back up will possibly take more and more time. If staying energetic and healthy is the goal the number becomes only one of the moving benchmarks.

With that said I think doing some kind of workout even on vacation is important.