A bug in the updated system package caused a DNS error in Ubuntu VMs running version 18.04. This also disrupted large parts of Azure’s Kubernetes services.
A recent system update creates an error when resolving DNS requests on virtual machines with Ubuntu Bionic Beaver (version 18.04). Systemd version 237-3ubuntu10.54 contains a bug that causes network connections to drop. The automatic installation of the update on VMs in Azure led to various failures. The Azure Kubernetes Service (AKS) in particular is severely affected globally. The Ubuntu and Azure teams are working to fix the bug and its impact. The problem is limited to Ubuntu 18.04.
Workaround available
The Canonical / Ubuntu team names two possible workarounds on Launchpad. Affected users either restart their instances or give up udevadm trigger -cadd -yeth0 && systemctl restart systemd-networkd as root. As a starting point, the developers identified open-vm-tools, from which the udevam trigger comes. By the way that driver-Match is set up for networking in Azure, the key-value pair is lost when running the udevam trigger. This leads to the loss of DNS information from the server.
Same problem two years ago
At Microsoft, the Azure team is working to restore the Kubernetes Service. Automatic recovery was able to repair most of the clusters, but individual nodes are still resisting, according to Azure. The systemd update on Azure is suspended until a solution is found. According to the Ubuntu team, the error appeared two years ago as Bug 1902960.
The current situation at Azure can be found on the status page, the Ubuntu report on Launchpad.
