Immediate rebirth or not?

In Buddhism the idea of rebirth is split into two main schools: The Theravada school with its Abhidhamma would insist that the continuum of consciousness from one body to another has no gaps between it. The Mahayana and Vajrayana (and the Pali source text) would suggest that there is an in-between stage between rebirths.

For the book Rebirth by Francis Story where he described some of the case interviews, he encountered a few describing in-between memories of rebirth. For Theravada scholars, they interpret it as being temporary in hungry ghost realm, it is a form of rebirth.

Bring in Physics, and we can see that there might be a sort of resolution to this disagreement. In Physics, specifically in general relativity or just special relativity, time is relative, it is personal, depending on the relative velocity of the observer. So, if the kamma and ignorance, or whatever that passes on from one life to another is transferred, if they are physical or information, then they can be subject to the laws of Physics. Then the transfer cannot be faster than the speed of light at least according to special relativity. And if it is massless, it should travel at the speed of light, where at that speed, to itself, it doesn't experience time passing from its starting location to its destination. (Same reason as photons not having experiencing time.)

Yet to an outside observer, there is some time lapse between the death and the subsequent rebirth if the distance is non-zero. Thus, to the person experiencing rebirth, there is no gap in between one body and the next. But to everyone else, there is a time gap. Satisfying both views from different schools. Yet, I suspect that this explanation would not work as in the case studies stories, the ghost in between lives seems to hang around some place, not moving, and since they can describe the experience, that means they experience time during the gap. Thus this is a nice idea, but refuted by evidence.