Upgrading to 22.10 breaks resolv.conf

I just upgraded a VM from 22.04 to 22.10. Short version, the upgrade process broke resolv.conf by changing the symlink for /etc/resolv.conf to /run/systemd/resolve/stub-resolv.conf. To be honest, I have no idea where it pointed in 22.04, since that configuration is gone with the upgrade. But courtesy of this post:

I was able to determine that the correct location is /run/systemd/resolve/resolv.conf.

I searched Bugs : Ubuntu MATE for "22.10 resolv.conf" and got no relevant results. Should I report this as a bug? Seems like a significant one to me.

Thanks.

Hi, @guyr (Guy Roulliers).

You wrote:

I'm running Ubuntu MATE 22.04 in a laptop that, if I remember correctly, started with a fresh install of Ubuntu MATE 18.04 LTS ("Bionic Beaver") that I upgraded, a few years later, to Ubuntu MATE 20.04.1 LTS ("Focal Fossa") and that, 2 months ago, I upgraded once more to Ubuntu MATE 22.04.1 LTS ("Jammy Jellyfish"). At least in my case, I see that "symlink" (symbolic link) is exactly the same that you wrote ("/etc/resolv.conf" points to "/run/systemd/resolve/stub-resolv.conf"):

$ ls -l /etc/resolv.conf 
lrwxrwxrwx 1 root root 39 mar  4  2019 /etc/resolv.conf -> ../run/systemd/resolve/stub-resolv.conf

You also wrote / asked the following :

Well, I've searched and I eventually found the following directly related bug opened about 4 years ago (!) on 2018-06-01 (1st June 2018) - "Bug #1774632 “The symbolic link /etc/resolv.conf points to the wrong file by default” : Bugs : systemd package : Ubuntu" - originally reported by Glenn J. Schworak for a fresh install of "standard" / "vanilla" Ubuntu 17.10 ("Artful Aardvark") and so far currently 13 users marked the bug as affecting them:

In that bug, the original reporter wrote, among other things, the following (that is very similar to what you've found in that "DNS resolver stopped working after upgrade to 20.04" question and answers in AskUbuntu"):

" When using nslookup for local machine names, the local DNS was being ignored (not queried) and none of the local machines could be found.

After much research and digging, it was discovered that the cause was the incorrect symbolic link /etc/resolv.conf file.

The default install caused systemd-resolve to configure the link to point to the stub file:
/etc/resolv.conf -> ../run/systemd/resolve/stub-resolv.conf

Reomving that link and pointing it to the correct file solved the DNS lookup issue. The correct link looks like this:
/etc/resolv.conf -> /run/systemd/resolve/resolv.conf

(...)"

There are other users commenting in that bug that they have also found the same "issue" in "normal" Ubuntu 18.04 and Ubuntu 20.04.

The following question in the "UNIX StackExchange" seems also to be relevant about this matter:

The user duthils wrote the following answer to that question on 20th February 2021: "What's the difference between /run/systemd/resolve/stub-resolv.conf and /run/systemd/resolve/resolv.conf? - Unix & Linux Stack Exchange":

"Using resolv.conf instead of stub-resolv.conf will bypass a lot of systemd-resolved configuration, such as DNS answer caching, per-interface DNS configuration, DNSSec enforcement, etc.

Explanations:

When using stub-resolv.conf, applications will make DNS requests to the DNS stub resolver provided by systemd on address 127.0.0.53. This stub resolver will proxy the DNS requests to the upstream DNS resolvers configured in systemd-resolved, applying whatever logic it wants to those requests and answers, like caching them.

When using resolv.conf, applications will directly make DNS requests to the "real" (aka. upstream) DNS resolvers configured in systemd-resolved. In this case, systemd-resolved only acts as a "resolv.conf manager", not as a DNS resolver itself.

Source: systemd-resolved manpage "
answered Feb 20, 2021 at 18:06

I hope this reply of mine is useful :slight_smile:

2 Likes

Thank you very much for your exhaustive research. I added my comment to the bug report you found.

Well, I did not plan on dedicating my evening to this :frowning: , but I may have figured out the issue. See my comment on the bug report for details, but in short, I can stay with /run/systemd/resolve/stub-resolv.conf, but I had to first put my custom DNS servers in /etc/systemd/resolved.conf. Ugh, systemd has really made a mess of things.

[UPDATE] Please see my final comment in the bug report. I was encouraged by @ricmarques posts here, and by Lukas Märdian's response in the bug report, to put a little more effort into this to see if I could get it working as intended. At long last, I was successful. I'm documenting it in case others find this via search; perhaps I can save others some effort.

2 Likes

Your scenario is identical to mine, fresh 18.04 install on Dell laptop, upgrade to 20.04, then to 22.04. I can temporarily resolve issue by disconnecting from network, then reconnecting. But the lack of DNS returns after rebooting. To be honest, I get lost reading the various bug reports. I do not want a fixed DNS such as Google or Cloudfare, My understanding is that I need to have /etc/resolv.conf linked to /run/systemd/resolve/resolv.conf vice /run/systemd/resolve/stub-resolv.conf. Before I try this change, I was hoping for some confirmation I am following the topic correctly.

@jaybo, you say "I do not want a fixed DNS", but from your discussion, you actually do. Without it, you end up with DNS entries 172.98.193.42 and 198.206.14.241, which do not work for you.

In your other post, you mention when your system is working properly, your DNS setting is

DNS Servers: 192.168.1.1

You say you are using Network Manager. Per my final comment in the bug report, with Network Manager, I got the system to behave consistently by putting the proper DNS entries into Network Manager, separately for each configured interface. From the system menu, select System -> Preferences -> Internet and Network -> Advanced Network Configuration. Then, for you primary connection (mine is Ethernet -> Wired connection 1), highlight it and then click the edit icon on the bottom. On the IPv4 Settings tab (use IPv6 Settings if that is applicable to you), under DNS servers, add in the appropriate values for your DNS servers. For me, this is

192.168.2.1, 1.1.1.1, 1.0.0.1

The first is my router, and the next two are CloudFare, the public DNS servers I prefer. I don't know if this is strictly required, but my thinking was that if the router DNS entries get corrupted, I can fall back to CloudFare. I have these same CloudFare DNS servers configured in the router.

In your case, you would enter the 192.168.1.1 value. Let me know if you still have questions, and I will post pictures of my Network Manager configuration to show where I entered the DNS entries.

1 Like