Hi Guys
This may be deemed off topic because it is not strictly directly Linux related, although all the machines involved are running Linux. However, there are some networking experts here and I would welcome a view on something that is driving me nuts.
The background is that I am experimenting with moving all my DNS lookups to DNS over TLS (to preserve privacy). I run local dnsmasq caching resolvers on two of my internal networks and for a while now I have forwarded upstream DNS requests to one of four unbound resolvers running on my OpenVPN end points. (So if I use the VPN on the machine called tap, I forward requests to unbound on tap).
Because any observer between my VPN endpoint and the root, TLD or other domain DNS servers can see my requests I want now to encrypt all requests leaving my networks.
I /could/ retain my unbound resolvers with suitable configuration changes so that they accept DoT requests from my local nets and then in turn forward upstream using DoT, but there are a limited number of servers accepting DoT requests in the wider world. So if I retain unbound, I would end up forwarding my upstream requests over DoT to third party resolvers such as Google, Quad9 or Cloudflare. I see no point in doing that when I can (and indeed have) install stubby on my local DNS resolvers and simply pass requests from dnsmasq through stubby to a public resolver (or actually a round robin list).
Because I really don't want to pass my DNS requests to a large resolver which then logs them (as does Google) or both logs and interferes with them (as do cleanbrowsing, OpenDNS or Quad9) I have been testing outside DNS resolvers using a shell script tool called dnsperftest I found here[1].
I modified that script to add some additional resolvers taken from the defaults provided in stubby's configuration plus some others taken from privacytools.io [2] and dnsprivacy [3]. I also added some other domains to the test set so that I had 20 domains to test across some 18 public resolvers. Some of those resolvers (such as google 8.8.8.8 and Quad9 at 9.9.9.9) I will not actually use in real life, but they are big anycast systems and I wanted a benchmark for my likely slower choices which are privacy conscious.
I then ran the test script from my desktop using an untampered connection through my ISP, followed by connections through each of my four VPN endpoints. As expected, I got faster lookups without the VPNs than with. I then ran the same tests from the machines I use as VPN endpoints. And again, as expected, and because those VMS are in large datacentres dotted around europe, I got much faster lookups from there.
Herein lies the interesting bit. The bit I really don't understand. Take a look at the results (averages only) in the attached text file.
You will see there that on all the tests run from my desktop either straight through my ISP or through my VPNs, cloudflare's anycast servers on 1.1.1.1 and 1.0.0.1 come out fastest, But, and this is the bonkers bit, if I run the same test directly on the VPN end points, 1.1.1.1 is slowest (and in fact I cannot ping 1.1.1.1 from any of my VPN enpoints but I can when I run the connection through a VPN to that same endpoint.)
This bothers the hell out of me because it should make no difference whether I try to reach 1.1.1.1 through my VPN or direct from the machine which constitutes the endpoint of that VPN /UNLESS/ my traffic is taking a route I do not understand. If that is the case, then my protective security stance has a major problem.
So: does anyone out there have any idea what is going on?
(Oh, and I know that cloudflare got the 1.1.1.1 address from APNIC, and I also know that 1.1.1.1 has been used as a "test" address in a lot of places in the past and that can cause some routing oddities, but if that were the problem then I would see the same difficulty from my desktop through the VPN as I see at the VPN endpoint. Wouldn't I?)
Thanks in advance
Mick
[1] https://github.com/cleanbrowsing/dnsperftest
[2] https://www.privacytools.io/providers/dns/#icanndns
[3] https://dnsprivacy.org/wiki/display/DP/DNS+Privacy+Public+Resolvers
--------------------------------------------------------------------- Mick Morgan gpg fingerprint: FC23 3338 F664 5E66 876B 72C0 0A1F E60B 5BAD D312 https://baldric.net/about-trivia ---------------------------------------------------------------------