If the point of contention was not in the existence of a temple beneath the mosque, then whether it was destructed forcibly cannot be a point of contention too, in what other ways can there be a mosque over the ruins of a temple? Destroying temples and building mosques over them was not unusual in those days. And the birth place of Ram is a matter of faith as we have no sure way to prove with 100% certainty whether Ram was a historical character or a mythological character, it's like proving the existence of god with 100% certainty.
The only point of contention and debate can be about whether it is right or desirable to attempt to reset history by building temples (or three specific temples) now by destroying mosques that were built by destroying those temples in the past.