I have an Ubuntu 18.04 server running VirtualMin.
I logged in today to do routine updates to discover it has lost its ability to resolve DNS queries. The server otherwise seems fine but I can't update it.
/etc/resolv.conf tells me that it's using 127.0.0.53 for DNS, which is systemd-resolvd.
systemd-resolve --status lists no nameservers, which will be where my problem is, but I work out where that information should come from.
I temporarily set Google DNS in resolv.conf and did the updates, but a reboot restored the 127.0.0.53 DNS configuration (as expected) and the problem is back.
I *think* VirtualMin is a red herring (it's not used for much and I've been planning to decommission it for years but not today!), and in any case it's just a glorified configuration tool with a GUI so any relevant settings should still be hiding under /etc somewhere, but where I have no idea. Suggestions?
On Fri, Feb 19, 2021 at 10:16:25AM +0000, Mark Rogers wrote:
I have an Ubuntu 18.04 server running VirtualMin.
I logged in today to do routine updates to discover it has lost its ability to resolve DNS queries. The server otherwise seems fine but I can't update it.
/etc/resolv.conf tells me that it's using 127.0.0.53 for DNS, which is systemd-resolvd.
systemd-resolve --status lists no nameservers, which will be where my problem is, but I work out where that information should come from.
I temporarily set Google DNS in resolv.conf and did the updates, but a reboot restored the 127.0.0.53 DNS configuration (as expected) and the problem is back.
I *think* VirtualMin is a red herring (it's not used for much and I've been planning to decommission it for years but not today!), and in any case it's just a glorified configuration tool with a GUI so any relevant settings should still be hiding under /etc somewhere, but where I have no idea. Suggestions?
The systemd implementation of resolv[conf] is, IMHO, a real can of worms. I've had various DNS issues on my little home LAN ever since systemd got involved with it. None of the issues I've had have been show stoppers but they have been annoying, e.g. failing to return address for short local names, a long delay for the first request, etc.
In the end I turned off the systemd caching DNS (i.e. the one on 127.0.0.53) and now rely solely on dnsmasq running on a Raspberry Pi on my LAN.
My /etc/resolv.conf on my LAN systems is now:-
nameserver 192.168.1.2 search zbmc.eu
Sadly I can't immediately remember what configuration changes I made to do this. If you want details I'm sure I can dig them out, it'll just take a bit of digging into the resolv.conf setup.
On Fri, Feb 19, 2021 at 11:25:43AM +0000, Chris Green wrote:
On Fri, Feb 19, 2021 at 10:16:25AM +0000, Mark Rogers wrote:
I have an Ubuntu 18.04 server running VirtualMin.
I logged in today to do routine updates to discover it has lost its ability to resolve DNS queries. The server otherwise seems fine but I can't update it.
/etc/resolv.conf tells me that it's using 127.0.0.53 for DNS, which is systemd-resolvd.
systemd-resolve --status lists no nameservers, which will be where my problem is, but I work out where that information should come from.
I temporarily set Google DNS in resolv.conf and did the updates, but a reboot restored the 127.0.0.53 DNS configuration (as expected) and the problem is back.
I *think* VirtualMin is a red herring (it's not used for much and I've been planning to decommission it for years but not today!), and in any case it's just a glorified configuration tool with a GUI so any relevant settings should still be hiding under /etc somewhere, but where I have no idea. Suggestions?
The systemd implementation of resolv[conf] is, IMHO, a real can of worms. I've had various DNS issues on my little home LAN ever since systemd got involved with it. None of the issues I've had have been show stoppers but they have been annoying, e.g. failing to return address for short local names, a long delay for the first request, etc.
In the end I turned off the systemd caching DNS (i.e. the one on 127.0.0.53) and now rely solely on dnsmasq running on a Raspberry Pi on my LAN.
My /etc/resolv.conf on my LAN systems is now:-
nameserver 192.168.1.2 search zbmc.euSadly I can't immediately remember what configuration changes I made to do this. If you want details I'm sure I can dig them out, it'll just take a bit of digging into the resolv.conf setup.
A short look and it turns out to be *fairly* simple. If you change the symbolic link for /etc/resolv.conf from the default:-
/run/systemd/resolve/stub-resolv.conf
to:
/run/systemd/resolve/resolv.conf
Then searches will bypass the local cached DNS.
The man page for systemd-resolved does explain it all but in a very confusing way!
On Fri, 19 Feb 2021 at 11:54, Chris Green cl@isbd.net wrote:
A short look and it turns out to be *fairly* simple. If you change the symbolic link for /etc/resolv.conf from the default:-
/run/systemd/resolve/stub-resolv.confto:
/run/systemd/resolve/resolv.confThen searches will bypass the local cached DNS.
Interesting...
If I look at the contents of /run/systemd/resolve/resolv.conf, it has no DNS settings in it, just a comment saying "No DNS servers known". I assume that this is the root of my issues but it gives me somewhere new to go digging.
On Fri, 19 Feb 2021 at 12:44, Mark Rogers mark@more-solutions.co.uk wrote:
If I look at the contents of /run/systemd/resolve/resolv.conf, it has no DNS settings in it, just a comment saying "No DNS servers known". I assume that this is the root of my issues but it gives me somewhere new to go digging.
Fixed it.
Adding my DNS to /etc/systemd/resolved.conf and restarting systemd-resolved fixed it.
I found the clues here: https://askubuntu.com/questions/1292650/resolv-conf-no-dns-servers-known
No idea what caused it to break but for now I'm happy just to have a working system!
On Fri, Feb 19, 2021 at 01:04:06PM +0000, Mark Rogers wrote:
On Fri, 19 Feb 2021 at 12:44, Mark Rogers mark@more-solutions.co.uk wrote:
If I look at the contents of /run/systemd/resolve/resolv.conf, it has no DNS settings in it, just a comment saying "No DNS servers known". I assume that this is the root of my issues but it gives me somewhere new to go digging.
Fixed it.
Adding my DNS to /etc/systemd/resolved.conf and restarting systemd-resolved fixed it.
I found the clues here: https://askubuntu.com/questions/1292650/resolv-conf-no-dns-servers-known
No idea what caused it to break but for now I'm happy just to have a working system!
The only place I have my DNS server's IP set on my systems is in the Network Manager configuration, and that only on ones with static IP.
E.g. on my desktop (no WiFi), it's in /etc/NetworkManager/system-connections/Wired connection 1.nmconnection
[ipv4] address1=192.168.1.3/24,192.168.1.1 dns=192.168.1.2; dns-search=zbmc.eu; method=manual
Otherwise of course the DNS server address is handed out by 192.168.1.2 when systems start up and send DHCP requests.
On Fri, 19 Feb 2021 at 13:38, Chris Green cl@isbd.net wrote:
No idea what caused it to break but for now I'm happy just to have a working system!
The only place I have my DNS server's IP set on my systems is in the Network Manager configuration, and that only on ones with static IP.
This server has been through a number of OS upgrades so it's a mixture of historical configurations and current ones.
However, I did an update on another server, and it pulled in an updated version of resolved.conf, offering the choice to replace my installed copy. Accepting it wiped my DNS settings (although I was ready for that and added them back). So although I don't recall having blindly accepted an updated config on the first server I assume that's what I must have done.
-
Chris Green -
Mark Rogers