Network Virtualization with SR-IOV in simple terms

It's exciting time for Hyper-V. There are so many new features with Windows 8 Hyper-V that it is worth mentioning. One of interesting new feature is SR-IOV which stands for "Single-Root I/O Virtualization". This feature is to offload all the work done by CPU to network card. Previously, Host OS had to process all the incoming/outgoing network packets which means that it required lots of CPU time just to process which VM the packet belongs to. However, with SR-IOV we can avoid that overhead!

Again, this is another cool feature that hardware brings into the virtualization space. From high-level point of view what this does is that the network card itself has virtual functions that act like ports. For regular network card, we just see one physical port as in the picture below.

However, SR-IOV network card has implemented virtual ports associated with physical port. Therefore, when the VM starts up, we can assign these virtual ports to the VM and from then on the VM can directly talk to network card. This way we do not have to consume all the CPU to figure out which VM the network packet belongs to. Instead we can cut down all the overhead and directly connect to the network card. Following is a diagram that I borrowed from MSDN.

Before I conclude today's post, let me introduce one more terminology that is equally important to get this feature working properly. That is IOMMU / Intel VT-d. When these network card interacts with the rest of the system, it either uses interrupt or DMA to either read from or write to memory location. With these virtual functions talking directly to VMs, it is also required that we process interrupts and memory access properly. What does this mean? For instance, when the network card wants to read the data from memory location that belongs to the VM, it accesses the memory location as if that is real physical address. However, that address might not be the right address for the VM as we have to share the memory address among all the VMs. Therefore, we need to translate the memory address and this is done by IOMMU. This is very similar to page table walk to translate virtual address to physical address. Yet, this is for virtual machines. Likewise, we have to map interrupts from these network device to the appropriate CPU by finding out which virtual port the interrupt is associated with and forwards it to the appropriate CPU.

I copied the following diagram from wiki page. Hopefully the idea will make sense with the picture below.

In summary, SR-IOV enables to offload all the processing work from CPU with the help from both network card and CPU feature.

• Virtual Ports/VIrtual Functions from SR-IOV network card
• DMA/Interrupt remapping from CPU