More Processor Cores Did Not Automatically Mean More Network Processing
Virtual machines could be given several virtual processors and substantial computing resources, but network traffic presented a different scaling problem. Receiving large amounts of data required packets to be classified, queued, and processed efficiently before the guest operating system could use them.
As network adapters became faster and virtual machines handled heavier workloads, concentrating too much of that processing through a limited queue could prevent the networking path from taking full advantage of the available processors.
Windows Server 2016 introduced Virtual Machine Multi-Queue, or VMMQ, to allow the networking work associated with an individual virtual machine to spread more effectively across multiple queues.
A Powerful VM Could Still Be Constrained by How Packets Reached It
Adding virtual processors helped only when the networking architecture could distribute enough of the incoming work for those processing resources to participate.
Hardware Could Sort Traffic Before Windows Had to Do It in Software
Virtual Machine Queue, or VMQ, was designed to improve virtualization networking by allowing supported network hardware to classify incoming packets and place traffic into queues associated with virtual machines.
Without this type of hardware assistance, the host would have more packet-sorting work to perform before delivering network traffic to the appropriate virtual machine.
VMQ therefore helped move part of the receive-processing responsibility toward the physical network adapter and reduced some of the overhead created by virtualized networking.
The Network Adapter Could Help Decide Where Traffic Belonged
Hardware classification allowed packets intended for different virtual machines to be separated earlier in the receive path instead of requiring all of that work to occur later in software.
A Busy VM Needed More Than Its Own Dedicated Traffic Lane
Giving a virtual machine its own hardware queue was useful, but a very active VM could still produce more networking work than one queue could efficiently handle.
The server might contain many processor cores and the guest itself might have several virtual processors, yet a single receive queue could concentrate packet processing instead of distributing it broadly across those resources.
As network speeds increased, this imbalance became increasingly important for virtual machines expected to move large amounts of data.
Having a Queue Was Not the Same as Scaling the Queue
A dedicated path could separate one VM’s traffic from another’s while still becoming a bottleneck when that individual virtual machine required substantially more network throughput.
One Virtual Machine Could Receive Traffic Through Several Hardware Paths
Virtual Machine Multi-Queue expanded the queue model by allowing multiple hardware queues to be allocated for a single virtual machine.
Instead of treating the VM’s receive path as one queue handled through one processing path, VMMQ could distribute that traffic across several queues.
The virtual machine could therefore make better use of the parallel processing capacity available in modern server hardware.
The VM Was No Longer Limited to One Hardware Queue
VMMQ changed the default queue concept into a collection of queues that could work together for the same virtual machine.
Network Processing Could Spread Across Physical CPU Resources
The advantage of multiple queues was not merely having several waiting areas for packets. Those queues could be processed by different logical processors in the Hyper-V host.
This allowed receive processing for a high-traffic virtual machine to occur in parallel rather than concentrating the workload on a narrower CPU path.
For network-intensive guests, distributing packet processing could help the networking stack scale with the computational resources already available in the server.
Parallel Hardware Needed Parallel Network Processing
Multiple queues provided a way for the networking workload to take advantage of several processors instead of allowing one receive path to become disproportionately busy.
Traffic Could Continue Across Multiple Virtual Processors Inside the VM
Network processing does not end when packets reach the virtual machine. The guest operating system must still process the traffic using its own virtual processors.
VMMQ worked with the broader receive-side scaling architecture so that networking activity could be distributed beyond a single processing path as traffic moved into the VM.
This helped preserve parallelism from the physical networking hardware through the virtualization layer and into the guest.
Scaling Could Continue After the Packet Entered the VM
The benefit of several hardware queues was greater when the guest could also distribute network processing across the virtual processors assigned to it.
A Lightly Used VM Did Not Need an Army of Receive Queues
Not every virtual machine generates enough network traffic to expose a queue-processing limitation. A small utility server or lightly used application might operate comfortably without demanding substantial parallel networking resources.
The value of VMMQ became more apparent when a VM was expected to sustain high throughput and its networking workload was large enough to benefit from several processing paths.
This made the feature particularly relevant as faster physical adapters allowed individual virtual machines to consume increasingly significant portions of the host’s available network bandwidth.
The Bottleneck Appeared When One VM Became Busy Enough
Multi-queue processing was about scaling demanding networking workloads rather than multiplying queues merely because a virtual machine existed.
Windows Could Not Create Physical Queues the Adapter Did Not Provide
VMMQ depended on capabilities in both software and network hardware. The operating system could coordinate the feature, but the physical adapter and its driver had to support the queue functionality required underneath it.
This distinguished VMMQ from a purely software-defined feature that could behave identically regardless of the network interface installed in the server.
Server networking performance therefore depended not only on enabling an operating-system feature but also on selecting hardware capable of participating in that design.
The Feature Extended Into the Physical NIC
A configuration could not gain the intended multi-queue hardware acceleration simply by having many CPU cores if the network adapter and driver could not provide the necessary queue support.
Network Throughput Could Grow Faster Than One Processing Path
When network interfaces delivered relatively modest amounts of traffic, processing limitations elsewhere in the system could remain difficult to notice. Higher-speed adapters changed that balance.
A physical NIC capable of moving large quantities of data could deliver traffic faster than a narrowly concentrated receive-processing path could efficiently consume it.
VMMQ helped prevent the software and processor side of virtual networking from unnecessarily restricting throughput that the physical network hardware was capable of delivering.
A Faster Cable and NIC Were Only Part of the Performance Path
High throughput also required the server to classify and process packets quickly enough after they arrived at the physical network interface.
Separating Guests Did Not Automatically Scale Each Guest
VMQ could help distinguish traffic belonging to different virtual machines, allowing packet processing for separate guests to use separate queues and processors.
But a host with one especially network-intensive virtual machine presented another challenge. Giving every VM a queue did not necessarily solve the concentration of traffic inside that one demanding guest.
VMMQ addressed this second problem by allowing several queues to serve the same VM rather than requiring additional virtual machines before more receive queues became useful.
The Scaling Unit Could Become the Individual Virtual Machine
Multiple queues could be useful even when the network load came primarily from one guest instead of being evenly divided among many virtual machines.
The Network Path Had to Keep Pace With Multicore Servers
Server processors had long been moving toward increasing numbers of cores and logical processors. Virtualization could divide those resources among many guests or concentrate several virtual processors inside one demanding VM.
Networking needed similar opportunities for parallelism. Otherwise, a highly parallel computing environment could still encounter a comparatively narrow packet-processing path.
VMMQ represented one part of the effort to make virtual networking scale more naturally with modern multicore server architecture.
More Compute Resources Were Useful Only When Data Could Reach Them Efficiently
Distributing network processing helped prevent a busy receive path from undermining the benefit of the additional processor capacity available to a virtual machine.
Software Inside the VM Did Not Need to Understand Hardware Queue Management
An application running inside a virtual machine generally cares about receiving and transmitting network data, not about how the physical host assigns hardware queues underneath the virtualization layer.
VMMQ allowed the Hyper-V networking architecture and supported hardware to improve that underlying processing path without requiring ordinary applications to manage the physical queue arrangement themselves.
The optimization therefore occurred deep in the virtual networking stack while applications continued using familiar network interfaces inside the guest operating system.
One virtual network adapter could look simple inside the guest while several hardware queues worked underneath it to keep the traffic moving.
VMMQ Let One Virtual Machine Spread Its Networking Work
Virtual Machine Multi-Queue extended the earlier VMQ model by allowing the network traffic for an individual virtual machine to use multiple hardware queues instead of depending on a single queue.
Those queues could distribute receive processing across different physical processors, while the guest could continue spreading network work across its own virtual processors. For demanding virtual machines, this provided a more scalable path between high-speed network hardware and the applications consuming the traffic.
The important change was not simply that the server had more queues. Windows Server could dedicate several of them to the same virtual machine, allowing one heavily used guest to take better advantage of the parallel networking and processing resources available throughout the host.
