Backup errors, iCloud Drive and the limits of APFS

One of the oddest things to happen to my Macs occurred after I had upgraded them to Sonoma. The first I noticed was when Carbon Copy Cloner (CCC) started reporting errors in its nightly backups of my iMac Pro’s Data volume. Assuming that this was a passing compatibility glitch, I delved into the folder that was causing the errors, and excluded it from CCC’s backups. As it was something buried deep in ~/Library/Application Support, I thought no more of it, and that backup problem was fixed.

The next strange occurrence was on my Mac Studio, when preparing the article on how Time Machine backups work, and I came across multiple error reports there, like
Illegal event path – /Users/hoakley/Library/Application Support/FileProvider
/4DD77633-87CD-44E2-9BCA-C1F640F91471/wharf/wharf/ingest/23230797/MainOriginalDeepCopy.rtfd
/MainOriginalDeepCopy.rtfd/MainOriginalDeepCopy.rtfd/MainOriginalDeepCopy.rtfd
/MainOriginalDeepCopy.rtfd/MainOriginalDeepCopy.rtfd/MainOriginalDeepCopy.rtfd
/MainOriginalDeepCopy.rtfd/MainOriginalDeepCopy.rtfd/MainOriginalDeepCopy.rtfd
/MainOriginalDeepCopy.rtfd/MainOriginalDeepCopy.rtfd/MainOriginalDeepCopy.rtfd
/MainOriginalDeepCopy.rtfd/MainOriginalDeepCopy.rtfd/MainOriginalDeepCopy.rtfd
/MainOriginalDeepCopy.rtfd/MainOriginalDeepCopy.rtfd/MainOriginalDeepCopy.rtfd
/MainOriginalDeepCopy.rtfd/MainOriginalDeepCopy.rtfd/MainOriginalDeepCopy.rtfd
/MainOriginalDeepCopy.rtfd/MainOriginalDeepCopy.rtfd/MainOriginalDeepCopy.rtfd
/MainOriginalDeepCopy.rtfd/MainOriginalDeepCopy.rtfd/MainOriginalDeepCopy.rtfd
/MainOriginalDeepCopy.rtfd/MainOriginalDeepCopy.rtfd/MainOriginalDeepCopy.rtfd
/MainOriginalDeepCopy.rtfd/MainOriginalDeepCopy.rtfd/MainOriginalDeepCopy.rtfd
/TXT.rtf was not on volume /System/Volumes/Data
(As this appears in one single continuous log message, I have broken it into lines to assist its reading.)

The recursive RTFD

This rang a bell: back in 2019, well before the first M1 Macs, I had somehow generated a recursive RTFD document using TextEdit in Mojave 10.14.6. Regular RTFD documents are bundles containing a single file TXT.rtf and a folder for their graphics files. For some reason, this document had within that another RTFD bundle, and inside that was another, and so on, like Russian dolls. Although not truly recursive, as they did come to an end, the path to the innermost TXT.rtf file was rather longer than some file systems might wish for.

When I had tried copying that document to iCloud Drive, it failed with the following error

I thought no more of it, and assumed that I had trashed the original recursive document. After these latest errors, I looked again, and found that problematic RTFD document sitting quietly in TextEdit’s folder in iCloud Drive, where it hadn’t been before upgrading to Sonoma.

While others had major problems with iCloud Drive following their upgrades to Sonoma, what had happened to me was that a long-forgotten failed attempt to copy a recursive RTFD document had suddenly re-materialised four years later, only to give me grief with local backups. But that didn’t explain why the offending file was sat happily in iCloud Drive, and its echo was buried in some wharf ingest folder in my Home folder library.

APFS?

The cause of the backup errors could have been the file path length, which is an astonishing 1,051 UTF-8 ‘characters’. However, I’ve been unable to discover what the limit for path length is in APFS; Internet rumour claims that could be as short as 1,024 UTF-8 characters, although other modern file systems have declared maxima of more than 32,000.

iCloud Drive as FileProvider

The biggest mystery was why this document, that I could see in iCloud Drive, was causing problems in ~/Library/Application Support/FileProvider
/[UUID]/wharf/wharf/ingest/[integer]. To understand that, I had to go back to Johannes Fortmann’s presentation on the FileProvider framework at WWDC in 2021, which few of us watched at that time because we don’t develop software to support third-party cloud services, neither did it mention iCloud.

Inside this FileProvider is a mine of Apple’s obfuscating codenames like tombstone, wharf, ingest and propagate, and some that reveal how iCloud Drive now seems to have adopted this new FileProvider architecture. There’s an AccessControl database, and five folders apparently concerned with iCloud Drive, Mobile Documents and Photos. Within those are property lists specifying Domains and Providers giving fascinating glimpses into this new world of iCloud.

Troubleshooting

If you encounter errors in backing up or anywhere similar, look carefully at the paths concerned. If they are anywhere inside ~/Library/Application Support/FileProvider, that indicates an iCloud Drive problem. In this case, all I needed to do was remove an old file from iCloud Drive, that hadn’t been there before Sonoma, and both iCloud Drive and backing up now work perfectly as they did before.

We need to do more work to discover how iCloud Drive now functions as a FileProvider in Sonoma, and whether this might finally resolve its longstanding problems with choked syncing.