A Day In The Life: Upgrade Woes

What a bunch of days.  On October 6th at around 3:20am I got an email from JetPack informing me that my site was down.  I was asleep at this time.

I woke up, saw it, and saw a follow-up from JetPack.  I immediately assumed it was the “Everything is OK now” email.  It was not.  As I started about my day I made my coffee I checked and saw that my site was still down.  Both of my sites were down, in fact.  They’re both on the same Digital Ocean droplet.  Greatttttttt.  I was already running on nil sleep, and it was going to be a day.  Thursday wanted to put up a fight.

So I log into Digital Ocean since I can’t ping or SSH into my host and find that it is, in fact, running.  Crap.  If it was off that’d have been a simple solution.

This was going to be anything but simple.

I used the Console connect utility and ran ifconfig.

Horror.

All I saw was lo, the loopback interface.

What.  The.  Hell.

I ran uptime.  6 hours.

Double What The Hell.

A little more digging and I saw some feedback from lshw -c net: I had 2 new network adapters: ens3 and ens4.

Weird.

I edited my network config file (/etc/network/interfaces) and renamed eth0 instances to ens4.  No dice.  Still no networking.

I then re-edited my network config file and renamed the ens4 to ens3.  This fixed it.  I don’t know what the hell happened, but I was confused as hell.  Total down-time: approximately 9.5 hours.  Thank goodness this isn’t a really super serious server.  Blah.

Now that I was sitting down with a working box, I decided to delve deeper.  Around 3:15am I saw a random “reboot” command issued to my droplet via the console.  I looked through the logins, thinking that SOMEHOW someone got access to my account.  No logins except for my IP address.

SOMEONE (or something) at Digital Ocean rebooted my droplet.

Why this reboot caused eth0 to become ens3 I do not know.

It has something to do with Ubuntu distro upgrading, but I upgraded to 16.04.1 a few weeks ago.

Super frustrating.

So: if you have a VM and find that you have lost network connection and your network adapters are missing: check the output of  lshw -c net and verify that the interface is named what you think it should be.

-M, out

Leave a Reply