Well, Mediterranean people already have "olive", so we got "yellow".
Plus, Chinese people originated from the Yellow River, and we are the descendants of the Yellow Emperor. So I guess it just came to be accepted that we are the yellow race, like you I have never actually seen anyone who was properly yellow before.
Huh that's interesting, I never knew about that. What a coincidence!
Our current flag is actually red and yellow as well.
According to hindu religion
Red indicates both
sensuality and purity. In Hindu religion, red is of utmost significance and
the color most frequently used for auspicious occasions like marriages, birth of a child,
festivals, etc. A red mark is put on the forehead during ceremonies and important occasions.
As a sign of marriage, women put red powder on the hair parting. They also wear a red sari
during marriage. Red powder is usually thrown on statues of deities and phallic symbols
during prayers. It is also the color of Shakti (prowess). A red colored dress is put on deities
who are charitable, brave, protective, and who have the capacity to destroy evil. On the death
of a woman, her body is wrapped in a red cloth for the cremation.
Yellow is the color of knowledge and learning. It
symbolizes happiness, peace, meditation, competence
and mental development. It is the color of spring and
activates the mind. Lord Vishnu’s dress is yellow
symbolizing his representation of knowledge. Lord Krishna and Ganesha also
wear yellow dresses. Yellow clothes are worn and yellow food is eaten at spring
festivals. Single girls wear yellow to attract a mate and
keep evil spirits away.