Play as audio - resume

When I play a video as audio, and then play the video directly, the video continues from where the audio finished. However, if I then play as audio again it will start from where the audio previously finished. It would be nice if you could go back and forth between the two and have it remember the position better.

Thanks!