The Wrong Way:

Actually, if the refactor is doing too late, it will become “rewrite”.

The Right Way:

If the code quality is dropped a little bit, you should seize the opportunity to do the modification and make it better. Constantly do a little bit of improvement. The “refactor” task should be think as breaking down to small pieces and be doing day by day.

This is the right way to think about refactor.