You have created your own definition of the caste system.
Every heterogeneous society has communities that are looked down upon. Europeans historically (and currently) looking down upon Gypsies, Omanis looking down upon Khaddams, Amhara Ethiopians looking down upon Southern Ethiopians, etc... does not mean that these peoples are a part of some caste system, even when intermarriage between these groups is considered taboo. The communities that are looked down upon in Pakistan are menial and itinerant (eg. Gypsies) groups that have their origins in modern-day India.
The caste/varna system is a Brahmanic concept of hereditary class hierarchy with strict duties and conventions assigned to each Varna and no class-mobility.
The Biradari/Qabaila is a system of tribes, divided into clans. There is no social hierarchy, except one that can be gained and lost through renown and land-ownership. Occupations are not inherited, rather they are fluid, a Jatt can be a zamindar, merchant, soldier, farmer, etc...
I wonder why Brahmins did not have a sizable population here? Scroll further below and you will find out.
Endogamy is not a trait unique to the caste system and is found among all tribal/clan based societies, it is how these tribes and clans maintain themselves. Also endogamy is not enforced, just preferred. Inter-Biradari marriages are not uncommon, however marriages are mostly arranged.
Exactly, the Biradari/Qabaila system of the Indus region has existed since ancient times and has it's origins in the tribal system of the early Indo-Aryans. The Caste System was developed within Brahmanism in modern-day North India as a means to maintain hierarchy over the native population, it had no hold over the Indus region.
Ancient Hindu texts stretching back to the Mahabharata forbid Brahmins from visiting the Indus region as their authority there is not respected and they are often even met with hostility.
Here are some of many excerpts by Brahmins describing the people and culture of the Indus region:
Madrakas are the people of North Punjab while the Sindhu-Sauviras are the people of Sindh and South Punjab.
"How, indeed, would the Madrakas and the Sindhu-Sauviras know anything of duty, being born, as they are, in a sinful country, being mlecchas in their practices, and being totally regardless of all (varna) duties?"
Aratta is a general term for the entire region. While Vahikas/Bahikas (meaning outsiders) are the inhabitants of the region.
''There where forests of Pilus stand, and those five rivers flow, viz., the Satadru, the Vipasa, the Iravati, the Candrabhaga, and the Vitasa and which have the Sindhu for their sixth, there in those regions removed from the Himavat, are the countries called by the name of the Arattas. Those regions are without virtue and religion. No one should go thither. The gods, the pitris, and the brahmanas, never accept gifts from those that are fallen, or those that are begotten by Shudras on the girls of other castes, or the Vahikas who never perform sacrifices and are exceedingly irreligious.''
"There where the five rivers flow just after issuing from the mountains, there among the Aratta-Vahikas, no Arya should dwell even for two days. There are two Pishacas named Vahi and Hika in the river Vipasa. The Vahikas are the offspring of those two Pishacas. They are not creatures created by the Creator. Being of such low origin, how can they be conversant with the duties ordained in the scriptures?"
"But on seeing the dharma practiced in the land of the five rivers, the grandfather cried, "Shame!". They are outcasts. They are born from servants. They are the performers of wicked deeds. That is the reason the grandfather condemned the dharma in the land of the five rivers. Though they followed their own dharma and that of their varna, he did not honour it."
"The Kauravas with the Pancalas, the Salwas, the Matsyas, the Naimishas, the Koshalas, the Kasapaundras, the Kalingas, the Magadhas, and the Cedis who are all highly blessed, know what the eternal religion is. The wicked even of these various countries know what religion is. The Vahikas, however, live without righteousness. Beginning with the Matsyas, the residents of the Kuru and the Pancala countries, the Naimishas as well and the other respectable peoples, the pious among all races are conversant with the eternal truths of religion. This cannot be said of the Madrakas and the crooked-hearted race that resides in the country of the five rivers."
"In days of yore, when the eternal religion was reverenced in all countries, the Grandsire, observing the practices of the country of the five rivers, cried fie on them. When even in the krita age, Brahman had censured the practices of those fallen people of evil deeds who were begotten by Shudras on others' wives, what would you now say to men in the world? Even thus did the Grandsire condemn the practices of the country of the five waters. When all people were observant of the duties of their respective (varna) orders, the Grandsire had to find fault with these men."
The tribes/peoples mentioned roughly comprised the entire Indus region.
"The regions are called by the name of Arattas. The people residing there are called the Vahikas. The lowest of brahmanas also are residing there from very remote times. They are without the Veda and without knowledge, without sacrifice and without the power to assist at other's sacrifices. They are all fallen and many amongst them have been begotten by Shudras upon other peoples' girls. The gods never accept any gifts from them. The Prasthalas, the Madras, the Gandharas, the Arattas, those called Khasas, the Vasatis, the Sindhus and the Sauviras are almost as blamable in their practices.'"
This excrept clearly highlights the lack of a caste system in the region and how the Brahmin considers it a despicable taboo:
'" I dwelt for a long time on a peak of the Himavat quite alone. Since then I have seen diverse countries following diverse religions. Never, however, have I seen all the people of a country act unrighteously. All the races I have met will admit that to be true religion which has been declared by persons conversant with the Vedas. Travelling through various countries following various religions, I at last, O king, came among the Vahikas. There I heard that one at first becomes a brahmana and then he becomes a kshatriya. Indeed, a Vahika would, after that, become a Vaishya, and then a Shudra, and then a barber. Having become a barber, he would then again become a brahmana. Returning to the status of a brahmana, he would again become a slave. One person in a family becomes a brahmana: all the others, falling off from virtue, act as they like. The Gandharas, the Madrakas, and the Vahikas of little understanding are even such. Having travelled through the whole world I heard of these practices, destructive of virtue, of these sinful irregularities amongst the Vahikas."
Sindh and Punjab never had a Hindu majority population. Mainstream Hinduism in either of these regions did not truly begin to spread until the late Bhakti movement.
Both Sindh and Punjab only had single short-lived Hindu dynasties each which emerged after Brahmin ministers usurped the Buddhist rulers.
The Brahmin Dynasty lasting from 632 to 712 CE was strongly resisted by the Buddhist-majority local population which was one of the reasons why it fell so easily to MBQ's invasion.
While the Hindu Shahis only managed to hold onto the Punjab proper for a mere 26 years.
I have stated why Punjabis and Pakistanis in general refer to their tribe or clan as "caste":
Caste is not even a local word while Zaat is of Arabic origin that denotes tribe or clan.
