VMware vCloud Director – Q&A – Part 2

This is part 2 of the Q&A article published last week.  If anyone has any more questions regarding VMware vCloud Director please post a comment and I will try to answer your questions as soon as possible.

vCloud Director Network Isolation (vCD-NI)

Q. What limits exist when creating a vCD-NI-backed network pool?
A. You are limited to the maximum number of ephemeral portgroups you can have per vCenter Server, which is 1016. (This is the same as a VLAN-backed network pool.

Q. How does vCD-NI scale past the 802.1q limit of 4096?
A. The 802.1q protocol (in standard implementations) tops out at 4096 VLANs. The use of vCD-NI requires at most one VLAN per network pool for the transport network.

Q. Is vCD-NI provided as part of vShield Edge?
A. No, it is built into the VMkernel. This is why the product supports a minimum of vSphere 4.0 U2 or 4.1. (Previously it was implemented through the use of service VMs.)

Q. Is a VLAN ID required for the transport network?
A. No, by default if you leave it blank it will use VLAN ID 0. Entering in an actual VLAN ID will add a layer of protection on top of the MAC-in-MAC encapsulation. The requirements for the transport network are:
– Dedicated, not used in the providers environment for anything else
– Non-routed
– Available to all ESX hosts under vCD management and no one else.

The reason it has to be unused and non-routed is that we do not want anyone else to have access to the transport VLAN.  Once you are on the transport VLAN you can see all traffic.  It will be trivial to inject packets into the network on any of the vCD-NI networks, as well as read data flowing through any of the networks.  As such no physical machines should be able to attach to the transport VLAN. For even more security, you can use dedicated physical switches and not use the transport VLAN.

Q. if you have 2 organizations, can you create a vCD-NI-backed network pool for each organization but use the same VLAN ID?
A. Yes, you can have the 2 organizations use the same VLAN tag as long as they are using networks on 2 different vDS. The vDS gets really confused with the same VLAN on 2 different ports.

vCloud Director/vShield Edge: “NAT-Routed” Networks

Q. Is Edge really a routing appliance, i.e. does it support standard routing protocols and broadcasting of what networks it is routing for? Or is it really just a NAT device?

A. Edge is doing NAT translation. It doesn’t support normal routing protocols. vShield Edge is purely a L3 (NAT) device in a VM. At the same time, it can be called a router since it’s technically routing packets at the L3 level. And for this reason we say “NAT-Routed” within vCloud Director.

NAT = Network Address Translation. The translation part usually implies a look-up table.  When most (networking) people think of routing, it usually involves a routing protocol, from a collection of static routes (basic) to intelligent routing protocols such as BGP, OSPF, or even RIP. What vShield Edge does right now is just NAT. You might get confused by the fact that the inside and outside networks are sometimes different and at times they are the same network. It does not matter, as all it does is look at its table of inside and outside mappings to determine where the packet should go. If you have a firewall running then it considers if the packet is allowed through or not.

Q. Does vShield Edge ever show up as the default gateway to a VM?

A. Yes, the internal IP address of the Edge device is always the default gateway for the VMs connected to the internal interface of it. This shows up whenever DHCP is configured on the vShield Edge and the VM is getting the IP address from it.

VLANs and Security

Q. Is VLAN used for security?
A. No, it is intended for separating broadcast traffic and should not be relied upon as mechanism for security. Private networks using a VLAN-backed network pool will each have their own VLAN. For one tenant to try to attack another tenant, they will need to create their own spoofed frames and send them down to the switch. The virtual switch will typically stop such VLAN hopping since it inspects every frame going out and looks at the source MAC and frame header. Since we know where every VM lives on the virtual switches, we can prevent spoofed frames and VLAN hopping. We’ve had this feature since ESX 1.5. Note that you can turn this off in vSphere. Still, to create that spoofed frame, all a hacker has to know is another VLAN ID and he can guess that (there are only 4096 choices).

With vCD-NI-backed network pools, a hacker needs to spoof a frame with MAC-in-MAC encapsulation. Now they need to know the MAC of the destination VM, the MAC of the physical NIC in the source ESX host, and the MAC of the physical NIC in the destination host. For each of these there are over 16 million combinations. Taken all 3 in combination, and you get over 60 trillion combinations to guess at. So, with VLANs you get to guess at 4096 choices and once you get one right you see all of the traffic in that VLAN. With vCD-NI, you get to do 60 trillion guesses and once you get it right you just get to see the traffic going between those 2 VMs—you still don’t get to see all of the other vCD-NI traffic until you guess that match. If you throw in the fact that the provider admin can’t even mis-configure this in any way it’s a pretty good option.

vCloud Director: NFS Requirement

Q. Is an NFS share required for vCD installations?

A. No, it is only needed when you have multiple vCD Cells for transporting ISOs and vApps. For a single cell environment, you don’t need an external NFS share; in this case, you need to make sure that /opt/vmware/cloud-director/data/transfer has enough space to host your transient uploads. When you try to upload ISOs / vApps that exceed the free space you start getting “transfer failures”  errors.

The reason we require an NFS share in a multi-cell environment for vApp uploads is that if one of the cells fails, without NFS (or CIFS) none of the other cells would have access to the upload area of the failed cell. Importing an uploaded vApp or media into a datastore is not necessarily performed by the cell that handled the file upload. Also, the resume function might fail as the upload could be resumed by a different cell.

During installation of vCloud Director, the installer will not attempt to verify the existence of an NFS share. It’s only when the cell starts that verification of the share exists is done. If it cannot verify it, it will write a warning to the log file (not considered an error because other cells may not be up yet but may have previously checked in with the DB).

vCloud Director and vCenter Disabling

Q. If I disable a vCenter Server in vCloud Director, will this prevent me from creating new vApps/VMs?
A. No, vCD makes direct calls to ESX for specific tasks, and as such some tasks might still work when vCenter is unavailable. Disabling a vCenter does not prevent new vApps from being created/powered on. What it does do is disallow any new provider vDCs from being created on the vCenter. Disabling a vCenter does not disable any existing provider/org vDCs on the vCenter.

Scaling vCenters in a vCD Environment

Q. When should I add more vCenter Servers?

A. vCD still issues VIM API calls to vCenter to deploy VMs. A potential bottleneck is there with only up to 8 concurrent operations supported. In a multiple vCenter Server environment, the value from vCD comes in placement of VMs. You don’t have to try and figure out which host to place objects on or how to keep isolation between different tenants. Instead you simply tell the vCloud API to deploy a certain VM in a certain service level and then vCD figures out the placements and maintains the isolation between resources. This removes a lot of the logic that you’d have to program into your own scripts which people do find a lot of value in.

If you have multiple vCenters, then you can increase concurrent operations but it introduces a whole bunch of other problems such as catalog management. Since we’re talking deploying VMs in particular in this thread then you have to consider the nightmare of cross-vCenter templates.

Q. Are there particular network constraints for the networking between vCD and the vCenter(s) that are being managed? That is, could one of the vCenters be in a different datacenter, understanding of course other constraints, but specifically will vCD be able to work with the other vCenter?

A.You should keep everything local to a datacenter (site) in v1.0. While you could manage a vCenter anywhere ,you still have to worry about distribution of load and the fact that you’ll be coming back to the main datacenter for every request. For v1.0 you should set up a different cloud instance in each datacenter where you want to have resources in your cloud.

vSphere: Storage vMotion and vCD Workloads

Q. Is Storage vMotion supported for moving vCD-managed VMs?
A. Yes. Storage VMotion is supported today– you just need to be careful in choosing source and destination LUNs that belong to the same Provider vDC.