I think you could drastically minimize any impact by doing the time travel in space and merely observing from high orbit, assuming your time machine has no form of exhaust, which if you have a time machine seems like a relatively small engineering challenge by comparison.
You might displace a few atoms in the void, but it's the safest way one could go about it.