Instead of registering the desired change of position and rely on physics simulation to apply it to the world, immediately change the position in the world without reset the simulation.