Why are Apple silicon VMs so different?

Running macOS virtual machines (VMs) on Apple silicon Macs may not seem popular, but it has long been one of Apple’s important goals. Yet, if you do use a virtualiser on an M-series Mac, you’ll know how different it is from those that virtualise macOS and other operating systems on Intel Macs. This article explains why virtualisation is so important, and how it has become so different.

Wind the clock back to WWDC in June 2020, when Craig Federighi announced details of what was coming in Apple silicon, and he identified three pillars to support a rich diversity of software on these new Macs: Universal apps that run on both architectures, Rosetta 2 to translate Intel code to Arm instructions, and virtualisation. At the time, before Apple had even released its Developer Transition Kit, Andreas Wendker went on to demonstrate a pre-release version of Parallels Desktop running Linux as a guest, but there was no mention at all of Windows.

Although the first version of macOS to run on Apple silicon, Big Sur, didn’t support the lightweight virtualisation that was to follow in Monterey, it brought what Apple termed its Virtualization Extensions, an implementation of Arm’s AArch64 virtualization.

In the early days of virtualisation, two distinct types were distinguished. Type 1 runs a hypervisor (the core of the virtualiser) direct on the computer’s hardware. Type 2, also known as hosted, runs a primary host operating system on the hardware, and hypervisors then run on top of, or in close conjunction with, that to deliver the same range of services to guest operating systems. A host operating system can run more than one hypervisor at a time, and each hypervisor in turn can run more than one guest, but ultimately all converge on the host operating system and its kernel. These are now much more common on desktop computers, and include popular products such as VMware Fusion, Parallels Desktop and Oracle VirtualBox.

Running unmodified guest operating systems is made simpler when there’s hardware support for a hypervisor. Those adopting that approach can use Intel’s VT-x feature set, typically with Extended Page Tables (EPT) and Unrestricted Mode, or, in the case of Arm CPUs, AArch64 virtualization. This still leaves the problem of device support, which can either be left to the virtualiser to address, or can be provided for in a higher-level virtualisation framework. Over the years, and thanks to the effort of many engineers, virtualisers running on Intel hardware including Macs have grown extensive device support, and that’s what we rely on when virtualising macOS on Intel Macs.

Every single hardware device in an Apple silicon chip is different from its equivalent (if there is one) in Intel Macs. Even if Apple had wanted to document them fully for external use, the engineering effort to match device support in Intel Macs would have been too costly for any third party. Thus starting with a hypervisor and expecting others to build a complete virtualiser wasn’t feasible, nor was it likely to result in the high performance that Apple and users expected. What Apple did instead was to build device support into macOS, in the form of Virtio drivers.

Virtio is a standard originally developed by Rusty Russell that provides an abstraction layer over I/O devices. When the guest operating system calls to open a file, for example, that’s passed to a front-end Virtio storage device para-driver, and from there into a Virtio back-end driver that interacts with the storage device. Although this might seem less efficient than traditional virtualisation, in practice it can prove far more efficient. Its most obvious advantage is that creating a virtualiser app becomes a matter of configuring and opening the required Virtio devices, and letting the guest, Virtio and the host get on with it. And that’s essentially what an app using Apple’s Virtualisation framework does.

Apple’s choice of Virtio was undoubtedly swayed by the fact that Linux already has good Virtio support, but at the time macOS had none. In the couple of years preceding the release of Monterey, Apple’s engineers thus set about building Virtio support into macOS, which explains why macOS lightweight virtualisation is only available on Monterey and later hosts, and when running Monterey and later guests. As implemented in macOS (both as guest and host), there are also extensions to support keyboard and pointing devices, a shared clipboard (‘Spice’), and high-performance graphics with Metal and GPU support.

In the Virtio model, providing such support is the task of the operating system, not the virtualiser. For vendors like VMware and Parallels this reduces not only the cost of development, but also the commercial value of their products; there’s no scope for either of them to engineer better or faster graphics support, as that’s determined by features provided in both guest and host operating systems, via Virtio or an equivalent. That puts Apple in charge of what hardware and features are supported by virtualisation on Apple silicon, and the difficulties that have arisen over Apple ID access for VMs.

On the other hand, it guarantees optimum performance in VMs. Not only is their CPU and GPU code run direct on the hardware, just as in the host, but Virtio devices deliver almost as good performance as on the host.

The reward for Apple is flexibility in the future of macOS. Running older versions of macOS in a VM enables users to run Intel-only apps long after Rosetta 2 support is dropped from the current macOS, and for newer Apple silicon Macs to run software that’s incompatible with their minimum version of macOS. Using either Linux or macOS, developers can distribute Docker-like lightweight VM packages, something already done by Cirrus Labs’ Tart.

Virtualisation may seem to be a minority pursuit just now. That’s likely to change over the coming few years, thanks to lightweight virtualisation and Virtio.

Further reading

Edouard Bugnion, Jason Nieh, Dan Tsafrir (2017) Hardware and Software Support for Virtualization, Morgan & Claypool. ISBN 978 1 62705 693 9. Bugnion was a co-founder of VMware.