From quarantine to provenance: extended attributes
One of the innovative features in classic Mac OS was its use of resource forks, allowing structured metadata to be attached to any file. When Mac OS X merged that with the more traditional Unix approach adopted by NeXTSTEP, those were nearly lost. Classic Mac apps were restructured from storing most of their components, including their executable code, in their resource fork, when Mac OS X flattened those into an app bundle consisting of a hierarchy of separate files in folders, without any resources.
For the first four years of Mac OS X resource forks were reluctantly tolerated, until the solution came in 10.4 with the introduction of extended attributes, including one to contain what had previously been stored in the resource fork, which became an extended attribute or xattr with the name com.apple.ResourceFork.
All files in HFS+ and APFS (and other file systems) contain a fairly standard set of metadata known as attributes, information about a file such as its name, datestamps and permissions. Xattrs are extensions to those that contain almost any other type of metadata, the first notable xattr coming in Mac OS X 10.5, named com.apple.quarantine. That contains quarantine information for apps and other files downloaded from the internet, in a format so ancient that the quarantine flag is stored not in binary but as text.
The quarantine xattr provides a good demonstration of some of the valuable properties of xattrs: it can be attached to any file (or folder) without changing its data, and isn’t included when calculating CDHashes for code signatures. It can thus be added safely without any danger of altering the app or its code, although it does change the way in which macOS handles the code, by triggering security checks used to verify it isn’t malicious. Once those have been run, the flag inside the quarantine xattr can be changed to indicate it has been checked successfully.
Far from being a passing phase, or dying out as some had expected, xattrs have flourished since those early days. This has happened largely unseen by the user: few change anything revealed in the Finder’s Get Info dialog, although they’re used to store some forms of visible metadata such as Finder tags, and the URL used to download items from the internet. Editing xattrs is normally performed silently: you’re not made aware of changes in the quarantine xattr, and in most cases the only way to manage xattrs is to use the xattr command tool, or one of very few apps like xattred that can edit and manage them.
Examples
Among the well-known and important xattrs you can encounter are:
com.apple.quarantine the quarantine xattr, containing a quarantine flag
com.apple.rootless marks items individually protected by System Integrity Protection (SIP)
com.apple.provenance contains data about the origin of apps that have been quarantined
com.apple.metadata:kMDItemCopyright records copyright info
com.apple.metadata:kMDItemWhereFroms the origin of downloaded file as a URL
com.apple.metadata:_kMDItemUserTags Finder tags
com.apple.TextEncoding reveals text file encoding
com.apple.ResourceFork a classic Mac resource fork
Storage
In APFS and HFS+, xattrs aren’t stored with file data, nor with a file or folder’s normal attributes.
For smaller extended attributes up to 3,804 bytes, their data is stored with the xattr in the file system metadata. Larger extended attributes are stored as data streams, with separate records, but still separately from the file data. Apple doesn’t give a limit on the maximum size of xattrs, but they can certainly exceed 200 KB, and each file and folder can have an effectively unlimited number of them.
Persistence
Most file systems to which macOS can write either handle xattrs natively (HFS+, APFS), or macOS uses a scheme to preserve them, as in the hidden files written to FAT and ExFAT volumes. NFS is an important exception, and files copied to NFS will have all their xattrs stripped. Neither are extended attributes unique to Macs: most file systems used by Linux support them, and even Windows can at a push.
Because xattrs contain a wide range of metadata, some are treated as being ephemeral, others as persistent. Moving files with xattrs around within the same volume shouldn’t affect their xattrs, as that takes place within the same file system. Copying files to another volume, even if both use APFS, may leave some xattrs behind if they’re considered to be ephemeral.
The most complex situation is when a file with xattrs is moved to iCloud Drive. The Mac that originated that file is likely to retain most if not all of its xattrs, because the local copy remains within the same volume and file system. However, not all xattrs are copied up to iCloud storage, so other Macs accessing that file may only see a small selection of them. The rules for which xattrs are to be preserved during file copying, including in iCloud Drive, are baked into macOS, and the subject of the next article.
