That so-called "kashmiri ruler" was a hindu invader from jammu area and people of jammu are NOT kashmiri they are called dogras and they have their own dogri language and culture. And there is a high mountain range that separates kashmir valley from these dogri areas in the east.
Look at the map of the districts of the state of indian controlled kashmir
View attachment 132944
Kashmiri speaking areas = Kupwara + Baramula + Srinagar + Badgam + Pulwama + Anantnag
Dogri speaking areas = Jammu + Kathua + Udhampur
Pothohari/pahari speaking areas (similar to AJK) = Poonch + Rajauri
Kashmiri and dogri speaking mixed area = Doda
Balti + Shina + Ladakhi speaking area = Kargil
Pure ladakhi speaking area = Leh