July 14, 2017

VM inside a VM inside Azure

VM inside a VM inside Azure

The promise of modern technology. The amazingness of Azure. The wonder of Nested Virtualization in Server 2016. Am I full of hyperbole? Yep. Am I running a Windows 10 VM inside a Windows 2016 VM on top of Azure? Yep. Oh my, I shudder with glee. Now you can too!

On 7/13/17, Microsoft announced support for Nested Virtualization, effectively ending all previous IaaS platform concerns within any public cloud. Conceptually, this is what nested virtualization looks like:

At the time of this blog, the following SKUs support VM in VM virtualization:

  1. Dv3
  2. Ev3

Also, only in these specific regions:

  1. West US 2
  2. East US 2

These restrictions are there because these are the VM SKUs that are running on the new Azure Host Fleet which runs on Windows Server 2016, the glue that makes all of this possible. Also, the above server SKUs support a variety of features that make these things palatable like hyper threading. So, what are you waiting for? Let's spend some Azure money!

Log in to Azure and create a Windows Server 2016 VM in one of those regions with one of those SKUs. Make sure to pick a VM SKU with enough RAM and CPU to make the nested VM tolerable. For me, I chose this shiny bell:

One note: I had a real bear of a time getting the deployment to work as there is a new feature here that is not currently working in the Azure portal:

This causes the deployment to fail. Not sure why, probably just something hosed internally at the moment but just disabled it to move on as I won't need beefy networking (yet).

Once the VM is deployed, do as one does, log in, patch it, restart.

After you've restarted, here's where the fun begins! Install HyperV and DHCP (I'll explain why in a bit). Marvel that HyperV doesn't complain that you're already in a VM. Reboot and create a VM - Huzzah!!!

Now, here's the part you may need some assistance with (I did). I downloaded the latest Windows 10 Insider ISO and installed the OS. A few notes:

  1. The in VM console got stuck in the PBE during OS installation. Like it was hung stuck at the step where it prepares to unpack the files. I just let it sit there for 10 minutes or so and then ejected the DVD and restarted the nested VM. Not sure what was going on there but the VM came up into first run for Windows 10 just fine. Perhaps the console forgot to redraw the screen or something as it proceeded, dunno.

  2. Networking. Hmmm, now, you might be thinking, oh, this is Hyper V, I just use the default Hyper V switch network attached to External and this sucker will work. Not so much. Reason? Azure only issues IP addresses to its VMs, not your nested ones!

Inside your nested VM, you are still marveling though that you are running Windows 10 though, right? But I need me some internets. So, let's do that. Credit for original guidance goes here.

First, I'm gonna need some another HyperV switch, in this case, an Internal Only switch that we'll NAT up and then hop on board the Azure network train.

Next, we need to get his index:

Now, we need to create an IP address in there so we can NAT off of it:

Note: this command spits out a bunch of stuff and some of them look to be in an errored state - ignore.

Now, let's create a NAT:

Almost there! Now we just need to be able to dish out some IP addresses. Remember I had you install DHCP when you installed HyperV? Fire up DHCP console and create a DHCP scope that matches the IP4 properties of the NatNetwork you just created:

All that is left is to point the VM that we created at that network and poof:

And within about two seconds, my VM was on the Internet!

It's so pretty! I'll likely expand on this in decently short order to spawn off my own VDI GPU farm, but for now, my Azure credits are gone and, well, the G series VMs aren't supported yet. Server 2016 as the Azure Host OS is rolling out now to the entire Azure fleet so new SKUs and regions will come online soon. Happy nesting!