Solved: do-release-upgrade hangs in Ubuntu on WSL

04 June 2021

skip the story, what do I do

I've been using Ubuntu either natively or via Windows Subsystem for Linux pretty consistently for a year or so, and it's been the base of our production baseimages at work for years - I was even involved in the migration from Trusty Tahr (14.04LTS) to Bionic Beaver (18.04 LTS). For audio-crackling related reasons, for the past few days I've been on Windows and using WSL to have better audio in zoom meetings before I figure out the issue.

I needed to reverse-i-search my fish shell history, and noticed that my reverse-i-search binding was broken because I was missing the fzf package. No problem, I thought, I'll just download it.

sudo apt install fzf

# No package found

Some investigation revealed that fzf does not publish a package for 18.04LTS, the distro I was using in WSL. Not ideal. But, I checked, and it is safe to do dist upgrades on WSL installations. Hooray!

do-release-upgrade

An upgrade was found, but somewhere in the call tree, something hung, and timed out after 2 minutes, aborting the upgrade process, and not giving much feedback as to what might be wrong. Once I had dug around on launchpad.net, the web system where the Ubuntu project tends to store packages, I found that do-release-upgrade exists in the ubuntu-release-upgrader package, which calls out to the update-manager packages. These packages handle the release upgrade service for Ubuntu, calling out to a web server to check for relevant updates, preserve LTS status where possible, and serve signed bundles of scripts to handle the update. Great! I was now able to find the relevant branch of the code that ran during the do-release-upgrade process.

Once the script has pulled the bundle, I found that it stored the package in a directory in /tmp. Browsing this bundle, I was able to confirm that it was from a branch of the ubuntu-release-upgrader package ubuntu/focal. I cloned the repo and had a look.

The log in /var/log/dist-upgrader/main.log showed me that there was a call to focalPostInitialUpdate, a suspiciously exact two minute pause in logging, and then a call to the abort function probably in a timeout. Inspecting this function, I saw that it did some intial validation, and then called out to a function on the snap package manager to check connectivity. That sounded suspicious. I could see that the command was snap debug connectivity and found that sure enough, snap was timing out, and failing to connect via a socket to snapd. Was snapd, the daemon that serves snap requests, running?

It turns out that it was not. In WSL, Ubuntu runs without systemd running in normal mode. This meant that the snapd service was not running correctly, causing the timeout!

I'm not sure whether snap came installed on the WSL ubuntu image, or if I installed it some time over the course of the last two years. But I did find, in a cursory Google, that it's safe to remove. So after a quick sudo apt remove snap, I was able to run do-release-upgrade, but this time it quickly made it past the spot where it hanged before, and the upgrade was a success! My WSL image is now running on Ubuntu 20.04LTS Focal Fossa. Big success!

But for me a bigger success was learning about the Ubuntu package ecosystem, the ability to search through the code that my OS was running, diagnose the problem, and fix it. Would it have been better if the do-release-update script had "just worked"? Not for me, I had fun. For lots of others, yeah absolutely. Computers, maaaaaaaan.

This is the real end of the article, the next section is for people who just wanted to skip to the solution without going on the journey.

What do I do to solve?

# Snap causes the hang in the upgrade script
sudo apt remove snap

# Do the upgrade, the kind of hang that was happening on my system shouldn't happen.
sudo do-release-upgrade

Later, you can reinstall snap if you need it. For generic instructions on how to upgrade your distribution on wsl, check this article out