Nested Hyper-V Lab Environment

Oh man! Ad-blocking software has been detected! :'(

This website is run by the community, for the community... and it needs advertisements in order to keep running. Blocking our ads means your killing our stats!
Please disable your ad-block, or become a premium member to hide all advertisements and this notice.

Hi guys, I'm putting together a new lab environment and would prefer to avoid running multiple physical machines. I've always gone this way in the past and it's always more effort than it should have to be.

I was thinking of setting up a single high performance machine with VMWARE Workstation 8 and running a virtualised 2/3 host Hyper-V setup within VMWARE. Performance will obviously be a concern (this needs to perform smoothly) which is why I'm hesitating a little.

I'm looking at an i7 processor, 32GB RAM, RAID Array setup etc... I suspect the bottleneck may be the disks and I could reduce this by using solid state drives but I think these could push me well out of budget.

Is anyone else doing this, or tried it in the past?

All comments and thoughts welcome!

 

Oh man! Ad-blocking software has been detected! :'(

This website is run by the community, for the community... and it needs advertisements in order to keep running. Blocking our ads means your killing our stats!
Please disable your ad-block, or become a premium member to hide all advertisements and this notice.

Not that I can be too much help but can I ask:
- why are you having issues? What sort of network are you trying to simulate?
- can this be done by using ESXi or similar where you are not actually using a MS host OS but straight onto the bare metal?

 

Interesting post as I built a lab a while ago using AMD processor G5 quad core server and an Intel G6 quad server both pretty high specs. I did however run into problems when trying to use VMware vmotion as the hardware on both host were different and had to be of the same spec etc.

I would look at cheaper hard drives seeing as this is a lab and also going for VMware workstation 8 you can run 64bit systems. The 32GB Ram is impressive and would like to know how you get on once done with the lab.

I recently passed the VCP4 exam and waiting for my workstation 8 copy for Windows from VMware.

All the best and keep us posted on your progress.

 

Did I understand correctly that you'll try to run a Win2k8(R2?) server with Hyper-V enabled, as a VMware guest? Does that even work?

Anyway, the CPU won't be a bottleneck and 32GB ram lets you pay around a fair bit. Yes, the disks will be a bottleneck however...

If you use the dynamically expanding disks option of H-V, you could stuff 20 guests within 500GB's (a guesstimate on my part, depends a little on the stuff you'll install). A single SSD could do the job, but won't be cheap. Multiple cheaper mechanical disks works, but you'll burn through more juice and a potential disk failure will lose you a disk. Reason why I say you'll need more traditional disks, is due to the need to spread out the IO impact from 20 or so VM's running in tandem. An SSD will simply cope better.

SSD's have fallen in price making it a better option now, even a cheaper option. The IOPS on an SSD is simply a hundred times better (if not more).

 

Hi Derek, I didn't do a very good job of explaining myself. In the past I've setup labs with three physical ESX hosts and run a vast array of VMs on them to simulate real world corporate networks. However this was far from ideal in terms of energy consumption, space etc...

I'd be using this setup to assist me in updating my current certifications, looking specifically at:

MCITP: Enterprise Administrator on Windows Server 2008
MCITP: Enterprise Messaging Administrator on Exchange 2010
MCITP: Virtualization Administrator on Windows Server 2008 R2

So there would be some fairly extensive labs, in particular I will want to setup a full Hyper-V setup (possibly 3 Hyper-V hosts). Basically put, a virtual environment within a virtual environment.

 

I was previously running ESX3.5 on 3 HP Proliant ML115s, maxed out with 8GB RAM each and if memory serves me right 3TB of drives for shared storage. Turning it on was like being on a runway with planes whizzing past and my household energy spend tripled. Hence looking for other options....

 

That's right. At least two Hyper-V guest VMs with further guest VMs running inside those VM instances. It can be done but as I've said the performance isn't anything to write home about.

With normal disks I'd have to run a stripe set to ensure the fastest read/write speeds possible. SDD are probably too expensive for that at the moment at £130 for 120GB.

 

In all honesty you are probably better off running a single Hyper-V node on two boxes and just down grade the specs.

To give you an example I run VMware ESXi here at my home lab, the servers cost me £300 excluding the 16gb of ram per box, I use a shared storage solution (which is what you would need if you want to play with CSV's anyway) on a dedicated NAS\SAN solution (something like the HP Microserver with SSD's or maybe some Raptors), alternatively I would look at my PX6 replacement build (a small(ish) NAS server with a Mini ITX board capable of running 6 SATA drives).

For the cost of 32gb of ECC ram you could probably get 2 servers configured with 16gb of non-ecc ram for not too much more, the NAS environment would set you back anywhere from £4-500. The more disks you have (and you're better to stripe them then raid 5 them for performance) the better you will find it.

 

Pretty cool that you can run a H-V host as a VMware guest But if one already has VMware licenses, I don't see the value of running a nested hypervisor due to the performance loss.

Anyway, from my lab experience, the VM's get IO starved if running off a single disk. Especially if you're doing updates (I sometimes need to "pause" some VM'sbto give breathing room to those which are updating themselves due to the VM's sitting on the same spindles... Yeah, my home lab setup doesn't use SSD's or multiple traditional disks to spread the load).

Ever since I installed a RAID0 SSD with two 240GB disks for my gaming machine, the installation of updates and the like is blazingly fast. I suspect the improvement is due to the chaotic reads a task like this takes. As seek times are incredibly fast an update which took a minute to install on a normal disk, takes 1 second now. Thus I could run like 50 VM's off this setup (just an example). Of course, the disks are more expensive and only give me 480GB to work with...

But food for thought

 

I thought you were planning to virualise inside a virtual world - I haven't reached that far in my study so don't know the requirements for the courses and therefore what you need to do - but thanks for explaining it

As for reading Shinigami and SimonD's posts - when I was playing with WSUS for my 291, it was incredibly slow at times - I use a single laptop, 4GB RAM, dual-core at 2.1GHz: pretty slow! - but reading around what could help me would have been multiple discs in a RAID. I think in future, I'm looking at getting a HP little server, possibly 2, to fulfill my needs.

 

Thanks for all the advice guys. I've asked elsewhere too to get a wide variety of opinions and the general consensus seems to be - stick with multiple physical hosts for the Hyper-V nodes.

I'll have to think carefully about this before jumping in. I need to look into prices for some of the budget tower servers (that have room for 16GB RAM each, but preferably more).

@derkit - Multiple disks in a stripe set is a must really if you want decent performance. Although the 5400rpm drive in your laptop would have really limited you.

 

Could you not reuse the ML115 you had for your ESXi environment?

 

Couldn't you get away with just using normal pc's kitted out with more disks and ram? Much quieter and it's only a lab after all.

 

I don't have them any more unfortunately.

I paid less for my ML115s than most PCs cost and generally you can't get cheap PCs that handle the amount of RAM I need.

 

Glad to hear you've decided to go for the physical hosted route judging from your setup this would be the most feasible. Best wishes and keep us posted on your progress

 

In lab - PoC or test environments we often use cheaper, scaled-down systems to play with – which means we have to cut corners with certain luxuries, like using dynamic disks instead of fixed size disks, and being frugal with the number of VM's running simultaneously at any given time.

 

Unfortunately what you want to acomplish is not possible. To be able to install Hyper-V you need the VT-x extensions, and these are not found in the hypervisor.
There's no way around it, so you have to use physical machines.
If you're going after the Virtualisation Administrator certification, you have to go through Clusters, Live Migration, ISCSI shared storage, etc. As a bare minimum you will need four phisical machines - one for a DC and SystemCenter Virtual Machine Manager, two Hyper-V cluster nodes and one machine with the Microsoft iSCSI software target installed on it (or an ISCSI capable shared storage like a mid-range NAS). It is strongly advisable to equip your machines with a minimum of two physical NICs, although you may end up going to up to 5 NICs on the cluster nodes - management, iSCSI, cluster heartbeats, etc. Your power meter will probably hate you for this ;)

 

Yes you can with VMWARE you can pass-through the Intel VT-x/EPT support to a VM.

You can run Hyper-V in both VMWARE Workstation 8 and ESX(i), the same as you can run ESX within ESX. In fact I know people who have used virtual VMWARE ESX labs for VShpere certs etc. Obviously performance is pants.

For Hyper-V you also need to change the VMWARE Workstation configuration files i.e. hypervisor.cpuid.v0 to False.

 

Holy thread revival batman

I haven't been around for quite some time but having passed by I noticed this old thread is still hanging around. I wanted to backup what supernova said in his reply to darkblade. It is indeed possible to run Hyper-V as a nested VM instance as I've been doing it for quite some time.

Funnily enough I am just about to upgrade my lab system again and will be sticking with a single machine to run my virtual labs. Something in the region of an i7 (6 core), 64GB ram and several fast SSDs in a stripe set.