Last Week on My Mac: iCloud of unknowing

The one thing we seem to know least about is iCloud, the brand name that Apple gives to its cloud and related services.

For a start, there isn’t one iCloud, but a whole bunch of different services, some available in plain iCloud, others restricted to those who subscribe to iCloud+. In broad terms, these fall into three groups:

iCloud Drive, a cloud file storage system not unlike Dropbox or Microsoft OneDrive, but more deeply integrated into macOS. This is the most visible of these services, in that you can see the files and folders of your iCloud Drive in the Finder, and within apps on your Macs and Apple’s devices.
Shared databases accessed by both Apple and third-party apps using the CloudKit service. These apps aren’t document-based, and don’t share files or documents that you can see in your iCloud Drive in the Finder. Instead, they write to databases that are shared in iCloud, from where they can be accessed by that app on your Macs and Apple’s devices.
Miscellaneous services, some of which don’t rely on any cloud service at all. These include shared preferences and other settings from App Store apps, and features like iCloud+ Private Relay. Some such as Apple’s notarization checking service are built into macOS, don’t require an Apple ID, and use iCloud as a convenient platform.

Confusion most commonly occurs between iCloud Drive and CloudKit databases. There’s a straightforward test you can apply to distinguish between these: if the app saves documents that you can see in iCloud Drive in the Finder, then it’s using iCloud Drive; if instead it saves data somewhere and that somewhere is shared in iCloud, then it’s using CloudKit databases. Pages and Numbers share documents using iCloud Drive; Notes and Contacts share their data using CloudKit. These seemingly arbitrary distinctions are important in practice, as iCloud Drive and CloudKit work completely differently.

iCloud Drive

Apple creates the illusion that what’s stored in iCloud Drive is just another folder within your Home folder, but it’s inevitably more complicated than that. iCloud Drive mimics the file system on your local storage, but has some important differences. These stem from its two modes of operation in Sonoma:

With Optimise Mac Storage turned off, iCloud Drive runs in replicated File Provider mode. Its task is then to mirror the files stored locally with those in iCloud. Those local copies are included in Spotlight’s indexes, backed up by Time Machine, and contained within local snapshots, and never become dataless.
With Optimise Mac Storage turned on, iCloud Drive runs in nonreplicated File Provider mode, and some of the files you store in iCloud can become dataless locally when you or macOS evict them. Although you can always download those files again to your Mac and devices, download is required before you can access them again. This means that those dataless files aren’t indexed by Spotlight so can’t be searched, they can’t be backed up by Time Machine, and their data isn’t included in local snapshots.

Previous versions of macOS don’t make the same distinction between these modes, although the practical consequences of files being evicted from local storage remain the same.

iCloud Drive’s two limits are your storage quota, determined by the subscription you pay Apple, and the speed of transfer of files between your Macs and devices, and the iCloud servers. Now that Apple’s top tier subscription provides a quota of up to 12 TB, there’s no evidence that Apple caps the amount of data that can be transferred, or the number of files. However, as with all services, there are physical limits to the speed of transfer that are likely to vary according to demand and service availability.

Apple currently imposes a maximum size on files stored in iCloud Drive of 50 GB.

CloudKit databases

While you determine which files are shared in iCloud Drive, databases accessed through CloudKit are managed by the apps that access them, together with CloudKit locally and iCloud’s remote database management. Some apps, like Notes, offer the facility to store some data locally and others in iCloud, but that’s determined by the app.

Shared databases aren’t generally accessible to Spotlight search, although they may be to in-app search using Core Spotlight, as illustrated by the content in Notes. The only way that other apps can gain access to search CloudKit data is through a plug-in to the app managing the CloudKit database. You also shouldn’t expect to be able to recover their contents from Time Machine backups or in local snapshots, as they are normally restored by syncing with iCloud.

Apple has recently confirmed that CloudKit databases can be throttled, which effectively blocks all access to them for requests for a set period of time. This isn’t a limitation in transfer rate in the way that iCloud Drive might experience, but an intentional denial of service until the retry interval has elapsed. This can occur as a result of local conditions on the device trying to access CloudKit, for example when a device has entered low-power mode because its battery is almost discharged. In those circumstances, the throttle only applies to that device, and will be in place until its battery charge has reached a level sufficient for the throttle to be released.

Apple currently imposes limits on the number of items that can be stored in shared databases and elsewhere in iCloud. These are given here for Contacts, Calendars, Reminders, Bookmarks and Maps, here for mailboxes and message size, and here for Shared Albums.

Throttling, as described by Apple, doesn’t make any sense in the context of iCloud Drive, as CloudKit doesn’t manage that, and no app is making requests of CloudKit in the process.

iCloud problems

With the architectural change in iCloud Drive, and the hidden complexities of CloudKit and its shared databases, resolving problems isn’t as simple as it might have seemed in the past. The general panacea of turning iCloud off and back on again often only makes problems worse, and if you don’t understand what you’re doing, killing arbitrary processes like bird lacks any rational basis.

The only actions that are likely to resolve problems are restarting the affected Mac or device, which shuts iCloud connections down in an orderly way, then starts them up again as they’re intended, and leaving your Mac running and awake for a period of many hours, perhaps overnight, with a reliable Internet connection. If those don’t lead to resolution, then your best course of action is to contact Apple Support, who have access to iCloud engineers and are likely to be your best hope.

Did someone mention Photos?

Photos libraries are the most complex of all, as they appear to sit between iCloud Drive, providing the storage for images and other files within the Photos library, and the database inside that library, which could be accessed via CloudKit. This is the point where I wish you a relaxing weekend, look forward to seeing you all next week, and run away to hide.