Why apps need to Accelerate
Over the last few years, public attention has been drawn to ‘AI’ in the form of Large Language Models (LLM) that seem able to accomplish almost anything, so long as you can live with their ‘hallucinations’. Meanwhile we’ve become accustomed to Live Text recognising words in our pictures, object recognition telling us the breed of cats and dogs, and more powerful image processing apps, without wondering how features that used to take several seconds now work almost instantly.
One over-simple answer to how these have come about is faster processors, in particular the arrival of Apple silicon Macs. But that glosses over what happens in between apps and processors, and the role of Apple’s Accelerate library.
Much of what modern apps accomplish so quickly is mathematics. For a 3D figure to rotate in front of our eyes, all its coordinates have to be transformed using mathematical methods. Machine learning and neural networks used in AI also rely heavily on crunching numbers. Many of those come in the form of vectors, one-dimensional arrays, or matrices, in two dimensions. Manipulating those is the function of specialist processing units.
In Apple silicon chips, apps have an almost bewildering choice: there’s a NEON vector processing unit built into each CPU core, the GPU used in Compute mode, Apple’s neural engine or ANE, and of course the secret matrix co-processor AMX, that no one outside Apple is supposed to know about, let alone program. Working out which to use for any given task in each type of Mac isn’t easy, and most require specialist code.
Apple’s solution is a vast library of mathematical functions, Accelerate, that not only does the hard work for those coding apps, but picks the optimal code to run on each model. For example, when the coder needs to multiply two matrices together, instead of having to work out the fastest method for each class of Intel CPU, and separately for Apple silicon Macs, then implementing each in hardware-specific code, all they have to do is call an Accelerate function, and that will select which code to run where.
The Accelerate library is vast, and sprawls over several major sub-libraries like BLAS (Basic Linear Algebra Subprograms) and LAPACK (Linear Algebra Package). They include fundamental functions, like calculating a generalised vector addition of
y ← αx + y
or ‘a x plus y’, known for short as axpy. From that, there’s a single-precision saxpy, double-precision daxpy, complex caxpy, and double complex zaxpy.
If this still seems destined for the fine-print, then here’s a sample of some of the areas catered for by Accelerate: neural networks for machine learning and AI; image processing, conversion and manipulation; audio and other signal processing including filtering; and video processing. Further examples, code and documentation are available on its front page.
For the developer, these are a no-brainer. Coding your own subroutines is hard graft, and getting the best performance from the range of processors and chips in modern Macs can be a daunting task for a team of specialists. The alternative is discovering which functions are available in Accelerate, and using them in a single call that’s going to run code optimised for that Mac’s hardware. Those who switch from relying on their own code to using Accelerate often report enormous performance improvements, by a factor of ten or more.
For Apple silicon, there really aren’t alternatives to Accelerate: writing your own code to use the NEON vector processor requires the use of assembly language, using Compute on the GPU requires Metal code, and neither the neural engine nor AMX are directly accessible to third-parties. Unfortunately, the Accelerate library is so vast that it’s largely undocumented, or relies on what’s provided for its sub-libraries like BLAS and LAPACK, which are generic and descended from Fortran originals. And I certainly wouldn’t want to rely on an AI to explain them to me.
Accelerate is invisible to the user, empowering to the developer, and does its job superlatively. But it’s not sexy, controversial, or unpredictable, so seldom gets the credit it deserves.
