Duh. This is how the entire world economy works and it's been like this for a very long time now.
Some examples of leading Muslim nations.
Ofcourse, the developed world has a lot higher external debts owed to each.
So, I am not sure which fictitious world you're living in where you're imagining interest free loans, debt-free economies and budgets not dependent on Internal/external borrowing.
The phone/computer you're using to post here, is most likely made by a company that has a lot of debts to pay off.