Hi Guys
This may be deemed off topic because it is not strictly directly Linux related, although all the machines involved are running Linux. However, there are some networking experts here and I would welcome a view on something that is driving me nuts.
The background is that I am experimenting with moving all my DNS lookups to DNS over TLS (to preserve privacy). I run local dnsmasq caching resolvers on two of my internal networks and for a while now I have forwarded upstream DNS requests to one of four unbound resolvers running on my OpenVPN end points. (So if I use the VPN on the machine called tap, I forward requests to unbound on tap).
Because any observer between my VPN endpoint and the root, TLD or other domain DNS servers can see my requests I want now to encrypt all requests leaving my networks.
I /could/ retain my unbound resolvers with suitable configuration changes so that they accept DoT requests from my local nets and then in turn forward upstream using DoT, but there are a limited number of servers accepting DoT requests in the wider world. So if I retain unbound, I would end up forwarding my upstream requests over DoT to third party resolvers such as Google, Quad9 or Cloudflare. I see no point in doing that when I can (and indeed have) install stubby on my local DNS resolvers and simply pass requests from dnsmasq through stubby to a public resolver (or actually a round robin list).
Because I really don't want to pass my DNS requests to a large resolver which then logs them (as does Google) or both logs and interferes with them (as do cleanbrowsing, OpenDNS or Quad9) I have been testing outside DNS resolvers using a shell script tool called dnsperftest I found here[1].
I modified that script to add some additional resolvers taken from the defaults provided in stubby's configuration plus some others taken from privacytools.io [2] and dnsprivacy [3]. I also added some other domains to the test set so that I had 20 domains to test across some 18 public resolvers. Some of those resolvers (such as google 8.8.8.8 and Quad9 at 9.9.9.9) I will not actually use in real life, but they are big anycast systems and I wanted a benchmark for my likely slower choices which are privacy conscious.
I then ran the test script from my desktop using an untampered connection through my ISP, followed by connections through each of my four VPN endpoints. As expected, I got faster lookups without the VPNs than with. I then ran the same tests from the machines I use as VPN endpoints. And again, as expected, and because those VMS are in large datacentres dotted around europe, I got much faster lookups from there.
Herein lies the interesting bit. The bit I really don't understand. Take a look at the results (averages only) in the attached text file.
You will see there that on all the tests run from my desktop either straight through my ISP or through my VPNs, cloudflare's anycast servers on 1.1.1.1 and 1.0.0.1 come out fastest, But, and this is the bonkers bit, if I run the same test directly on the VPN end points, 1.1.1.1 is slowest (and in fact I cannot ping 1.1.1.1 from any of my VPN enpoints but I can when I run the connection through a VPN to that same endpoint.)
This bothers the hell out of me because it should make no difference whether I try to reach 1.1.1.1 through my VPN or direct from the machine which constitutes the endpoint of that VPN /UNLESS/ my traffic is taking a route I do not understand. If that is the case, then my protective security stance has a major problem.
So: does anyone out there have any idea what is going on?
(Oh, and I know that cloudflare got the 1.1.1.1 address from APNIC, and I also know that 1.1.1.1 has been used as a "test" address in a lot of places in the past and that can cause some routing oddities, but if that were the problem then I would see the same difficulty from my desktop through the VPN as I see at the VPN endpoint. Wouldn't I?)
Thanks in advance
Mick
[1] https://github.com/cleanbrowsing/dnsperftest
[2] https://www.privacytools.io/providers/dns/#icanndns
[3] https://dnsprivacy.org/wiki/display/DP/DNS+Privacy+Public+Resolvers
--------------------------------------------------------------------- Mick Morgan gpg fingerprint: FC23 3338 F664 5E66 876B 72C0 0A1F E60B 5BAD D312 https://baldric.net/about-trivia ---------------------------------------------------------------------
On Sun, 17 May 2020 16:57:44 +0100 mick mbm@rlogin.net allegedly wrote:
This may be deemed off topic because it is not strictly directly Linux related, although all the machines involved are running Linux. However, there are some networking experts here and I would welcome a view on something that is driving me nuts.
All
Forgive me. I am an idiot. I have just remembered that my iptables rules (on all my VMs, including the VPN endpoints) contain the two rules:
# drop all zeroes and all ones $IPTABLES -A INPUT -s 0.0.0.0/8 -j DROP $IPTABLES -A INPUT -d 0.0.0.0/8 -j DROP $IPTABLES -A INPUT -s 1.1.1.1 -j DROP $IPTABLES -A INPUT -d 1.1.1.1 -j DROP
(for historic reasons)
and since my VPN traffic goes out through a masquerade at:
# Now allow forwarding over the tun interface for openvpn
$IPTABLES -A FORWARD -i tun0 -o eth0 -j ACCEPT $IPTABLES -A FORWARD -i eth0 -o tun0 -m state --state ESTABLISHED,RELATED -j ACCEPT
of course the block above doesn't apply.
Apologies for the noise. But the act of documenting the issue helped my diagnosis.
Mick
--------------------------------------------------------------------- Mick Morgan gpg fingerprint: FC23 3338 F664 5E66 876B 72C0 0A1F E60B 5BAD D312 https://baldric.net/about-trivia ---------------------------------------------------------------------
On 17/05/2020 16:57, mick wrote:
Hi Guys
This may be deemed off topic because it is not strictly directly Linux related, although all the machines involved are running Linux. However, there are some networking experts here and I would welcome a view on something that is driving me nuts.
The background is that I am experimenting with moving all my DNS lookups to DNS over TLS (to preserve privacy). I run local dnsmasq caching resolvers on two of my internal networks and for a while now I have forwarded upstream DNS requests to one of four unbound resolvers running on my OpenVPN end points. (So if I use the VPN on the machine called tap, I forward requests to unbound on tap).
{SNIP}
OK, I know you've found a solution to your problem.
Random musings:
Opendns - It logs, that's a shame. It alters results? It can remove links to certain sites, e.g. adult or gambling etc, which is why some people use it. If memory serves, it sometimes also take you to a search page if the domain does note exist. If you use dnsmasq, you can enter the IP of this landing page under "bogus-nxdomain" which means that dnsmasq knows that the search didn't work. Dunno if that's of interest.
Which name servers don't log you? The ones at the end of your post?
Have you considered using pihole? It does site-wide dns lookups and drops "spammy/malwarey/bad/advertisey" domains. I think it also caches your lookups. Dnsmasq can also cache your lookups.
You could setup dnsmasq to lookup via a pihole server, and pihole to lookup via whichever non-logging server, or list of servers you choose. With caching, only your initial lookups would be slow.
Any use?
Steve
On Sat, 23 May 2020 15:03:13 +0100 steve-ALUG@hst.me.uk allegedly wrote:
Random musings:
Opendns - It logs, that's a shame. It alters results? It can remove links to certain sites, e.g. adult or gambling etc, which is why some people use it. If memory serves, it sometimes also take you to a search page if the domain does note exist. If you use dnsmasq, you can enter the IP of this landing page under "bogus-nxdomain" which means that dnsmasq knows that the search didn't work. Dunno if that's of interest.
Steve
Thanks - yes I knew that and do exactly that on my dnsmasq configuration. I also use dnsamsq's "addn-hosts=" option to point to hosts files listing domains I /don't/ want to resolve properly (Dan Pollock's list for one, plus my own list of no-nos - google analytics, fb stats, ad sites etc. ad nauseam). I do not like DNS resolvers which examine your request before deciding how they will reply (as does opendns and quad9 for example. Except of course when I manage those servers....)
Which name servers don't log you? The ones at the end of your post?
The ones which /claim/ not to log you are listed at the dnsprivacy references I gave and at the privacytools.io reference. I deliberately avoid all public resolvers known to log. And I have configured stubby to round robin on six different (ok five really because two are cloudflare) resolvers so no one resolver ever gets a full picture of my DNS lookups.
Have you considered using pihole? It does site-wide dns lookups and drops "spammy/malwarey/bad/advertisey" domains. I think it also caches your lookups. Dnsmasq can also cache your lookups.
You could setup dnsmasq to lookup via a pihole server, and pihole to lookup via whichever non-logging server, or list of servers you choose. With caching, only your initial lookups would be slow.
Yes, I have considered pihole (particularly since I run my local resolvers on pis) but I see no advantage over my (new) current set up using dnsmasq as my local caching resolver and forwarding through stubby for the upstream. Ideally all the root and other authoritative servers should accept DOT requests. I could then go back to using unbound and not rely on any public resolver. One day maybe.
Thanks
Mick
--------------------------------------------------------------------- Mick Morgan gpg fingerprint: FC23 3338 F664 5E66 876B 72C0 0A1F E60B 5BAD D312 https://baldric.net/about-trivia ---------------------------------------------------------------------
-
mick -
steve-ALUG＠hst.me.uk