truthfollower
BANNED
- Joined
- Mar 8, 2019
- Messages
- 1,841
- Reaction score
- -4
- Country
- Location
Need to host content locally
https://blog.apnic.net/2020/03/10/why-is-so-much-internet-traffic-leaving-pakistan/Aftab Siddiqui is a Technical Engagement Manager for the Internet Society.
At the recent SANOG meeting held in my homeland, Pakistan, I wanted to provide the local community with some insight into the importance of Internet Exchanges (IXs), specifically the need to host content locally.
Knowing that data is king among network operators, I set up a virtual machine as soon as I arrived to collect information on several key metrics, including latency and the hosting location of .pk domains. Needless to say, the results were surprising.
How long does it take to connect to public DNS services?
First, I tested for latency, specifically the time it takes to PING three of the most popular public DNS services: Cloudflare DNS (1.1.1.1), Google Public DNS (8.8.8.8), and Quad9 (9.9.9.9). (PING is not the best way to test DNS but this is for reachability purpose only.)
Before leaving my home in Sydney, Australia, I did the same to offer a comparison. As you can see from the results below (Figure 1), all were below 1ms.
Figure 1 — Latency measurements to connect to public DNS services in Sydney, Australia.
The results for Pakistan were less consistent.
Cloudflare was the best of the bunch with an average of 3ms, thanks largely to the three data centres they opened in Karachi, Lahore and Islamabad late-2018.
traceroute to 1.1.1.1 (1.1.1.1), 64 hops max, 52 byte packets
1 110.18.48.1 (10.18.48.1) 5.901 ms 2.933 ms 3.664 ms
2 2202.163.100.105 (202.163.100.105) 5.297 ms 5.205 ms 7.234 ms
3 * * *
4 202.163.100.66 (202.163.100.66) 6.572 ms 4.340 ms 4.051 ms
5 119.159.240.53 (119.159.240.53) 3.728 ms 3.403 ms 3.399 ms
6 khi77.pie.net.pk (221.120.251.205) 9.160 ms
khi275.p01.pie.net.pk (221.120.251.201) 5.765 ms 7.170 ms
7 rwp44.pie.net.pk (221.120.251.101) 5.268 ms 4.654 ms
rwp44.pie.net.pk (221.120.251.88) 4.956 ms
8 static.khi77.pie.net.pk (221.120.200.3) 5.270 ms 5.438 ms 5.009 ms
9 one.one.one.one(1.1.1.1) 3.722 ms 3.915 ms 3.467 ms
Google was second and even though it has several caches hosted in Pakistan, its latency hovered around 25ms.
traceroute to 8.8.8.8 (8.8.8.8), 64 hops max, 52 byte packets
1 10.18.48.1 (10.18.48.1) 7.683 ms3.913 ms 4.363 ms
2 202.163.100.105 (202.163.100.105) 4.654 ms 5.716 ms 6.581 ms
3 * * *
4 202.163.100.66 (202.163.100.66) 7.731 ms 5.862 ms 4.388 ms
5 119.159.240.53 (119.159.240.53) 5.593 ms 7.199 ms 7.226 ms
6 10.253.4.26 (10.253.4.26) 8.717 ms
10.253.4.8 (10.253.4.8) 6.228 ms
10.253.4.26 (10.253.4.26) 5.140 ms
7 74.125.118.170 (74.125.118.170) 25.506 ms 23.927 ms 22.310 ms
8 * * *
9 dns.google(8.8.8.8) 25.356 ms 24.182 ms 24.443 ms
Quad9, which has no local POPs, came in at a very slow 203ms.
traceroute to 9.9.9.9 (9.9.9.9), 64 hops max, 52 byte packets
1 10.18.48.1 (10.18.48.1) 2.921 ms 5.942 ms 3.498 ms
2 202.163.100.105 (202.163.100.105) 5.388 ms 5.041 ms 4.442 ms
3 202.163.100.25 (202.163.100.25) 5.492 ms
202.163.100.121 (202.163.100.121) 5.179 ms 5.183 ms
4 202.163.100.66 (202.163.100.66) 3.842 ms 4.455 ms 2.862 ms
5 119.159.240.53 (119.159.240.53) 4.525 ms 4.177 ms 4.179 ms
6 10.253.4.133 (10.253.4.133) 6.869 ms 4.925 ms 6.503 ms
7 et7-1-5.catania8.cat.seabone.net (93.186.129.38) 104.627 ms 104.593 ms 104.202 ms
8 ae24.franco31.fra.seabone.net (195.22.211.50) 199.860 ms 134.881 ms 137.868 ms
9 de-cix.woodynet.net (80.81.194.42) 249.100 ms 205.069 ms 204.172 ms
10 dns9.quad9.net (9.9.9.9) 203.927 ms !Z 201.411 ms !Z 204.876 ms!Z
What this experiment shows is the importance of hosting services locally. We could expect to see a further reduction in latency if such services were also hosted at a local IX, as it would reduce the number of hops for those that peer with it to one; the benefits of which would be reduced transit costs, with the savings being handed down to customers.
Currently, Pakistan has one IX, PKIX, which has two exchange points — one in Islamabad and one in Karachi — with 17 Members peering with it.
Where are ‘local’ websites hosted?
As we’ve seen so far, the best way to reduce latency for DNS services is make them local, so packets don’t have to travel extreme distances. The same can be said for web content too, whereby websites hosted locally should, in theory, be faster to connect to than those hosted internationally.
So, for my second experiment, I looked at where websites with .pk country code top-level domains (ccTLDs) were being hosted.
The time of my test was short so I could only get public data, which was around 15,000 .pk domains. However, like the latency test, it provided a very clear picture of the current situation.
Of the 14,927 .pk domains, only 1,541 are being hosted by ISPs or entities registered in Pakistan, and another 1,300 are being hosted by Cloudflare. That’s less than 20% of sites that, you would think, would be targeting local users. Instead, more than 80% of ‘local content’ is being hosted in servers outside of Pakistan, and, as such, is subject to international transit costs and increased latency.
Figure 2 — DNS records for .pk domains.
Digging deeper, I also looked it up for gov.pk, who you would think are targeting the people in Pakistan, right? Out of 431 gov.pk domains I found in the 15,000 domain addresses, only 72 were registered or advertised by Pakistani entities, 28 are with Cloudflare, and the remaining 331 are outside of Pakistan.
Figure 3 — DNS records for gov.pk domains.
Needless to say, I was quite surprised that so many local council websites are hosted outside of Pakistan, most likely because they were cheap.
Finally, I looked at edu.pk. For a university to host something outside of Pakistan and not within the university is blasphemy for me. Unfortunately, this is the case for the majority of edu.pk sites with fewer than 20% hosted locally.
Figure 4 — DNS records for edu.pk domains.
How many websites have Quad A records?
The IPv6 Task Force was established in Pakistan around 2006/07 during which time it built a test network between several ISPs. Unfortunately, IPv6 never caught the limelight beyond testing and enabling in core networks, and capability remains less than 1%.
I was hoping that because many websites are hosted outside Pakistan, that content might have been accessible via IPv6. Unfortunately, this wasn’t the case either, with the exception of Cloudflare.
Figure 5 — AAAA records for .pk domains.
One thing that’s not clear from Figure 5 is that of the 1,304 websites hosted by Cloudflare, why were only 1,087 responding to AAAA requests? Have some turned off IPv6? If so, why? I plan on digging deeper into this using the virtual machine generously provided by Rapid Compute (a cloud services provider in Pakistan).
Finally, it’s worth pointing out that all of the above would be a lot worse if it weren’t for Cloudflare setting up its Content Delivery Networks (CDNs) in Pakistan in 2018. However, if more CDNs are set up in Pakistan, there needs to be more local content available for them to host. This surely cannot be too difficult for an economy with more than 200 million people.
Aftab Siddiqui is a Technical Engagement Manager for the Internet Society.
Last edited:
