Please don’t delete your logs

Several of my most popular apps make use of the main system log in macOS, and I occasionally get questions from those who can’t get them to work properly because their Mac’s log files have mysteriously disappeared. This has puzzled me for the last few years, but I think I’ve discovered the cause: some apps delete those log files in a misguided attempt to clean up your Mac and save storage space. This article explains why you shouldn’t do that.

Which logs?

In theory, since macOS Sierra introduced the macOS Unified log, there should be only one log, and one folder containing its log files, hidden away in /var/db/diagnostics.

In practice, many apps and even parts of macOS ignore those and write their own log files. Those can appear almost anywhere, but popular locations include:

~/Library/Logs, the worst offender;
/Library/Logs, which should be better maintained;
/var/log, normally properly maintained;
/var/logs, small and properly maintained.

Those locations contain a mixture of ephemeral information of little value, and some potentially important records, such as the outcome of all volume checks made by fsck on APFS and HFS+ volumes. If your Mac was migrated from a previous model, that may well have copied across logs going back years, and those four folders could well contain dozens of files, and consume several GB of space on your Mac’s startup disk.

What maintains them?

While the Unified log has a dedicated service, logd, that is constantly maintaining the files in /var/db/diagnostics, there’s no app or service that maintains all the others. As they’re considered to be custom features outside the control of the main log, it’s up to the apps that write them to remove old entries when they’re no longer required. Routine system housekeeping used to be performed at regular intervals, run by periodic scripts, but even those have been removed from macOS Sequoia, and before that much of the responsibility still rested with apps that create their own logs.

Logs in /var/log and /var/logs should be well maintained, but for most of logs written outside the Unified log, the answer is that apps responsible for maintaining their own logs don’t, and leave it to the user to clean up after them. Neither do those apps warn the user that they write their own logs but don’t maintain them.

Unified log

Look down through the list of active processes in Activity Monitor and you’ll see two, logd and logd_helper, that are always present from shortly after a Mac starts up until it shuts down. Unlike traditional Unix logs, the Unified log is maintained so its size remains fairly constant, with the main folder of log files, Persist, taking up a little more than 500 MB, and the whole of /var/db/diagnostics coming to about 1.5 GB.

logd does this by working through all the individual entries in the log files and removing those that have passed their retention time. It thus preserves more important entries for longer, and prevents the logs from growing uncontrollably. The unfortunate side-effect of this is that Macs that are making copious log entries have logs that don’t go back long in time. At its worst, that might mean that they may only last a few hours before logd removes them.

Although the Console app provides limited features for accessing your Mac’s log, and the great majority don’t want to go near it, the Unified log has important contents. Among those are reports by Time Machine on its recent backups, and those from XProtect Remediator on its scans to detect and remove malware. It’s also the place where all significant errors and failures should be recorded. Remove the main log files and those all disappear. Apple also may ask for and use recent log records when investigating problems you report to Apple Support or, for developers, using Feedback. Even though you may not want to browse the log yourself, it’s important and not a feature you should throw away.

Housekeeping

Spending a little time every month or so doing the housekeeping on your Mac is an excellent and rewarding investment of your time. When I do this, I like to preserve those logs that might still be significant, including the inevitably copious files written by Adobe’s multitude of services, and anything that still appears to be active. But those that haven’t been modified for the last year or so should be disposable, and any from apps I no longer use can be cleared away without fear. However, I never interfere with the Unified log in /var/db/diagnostics as I know that logd cares for that constantly, and much better than I can.

I’m well aware that many feel that they can delegate the responsibility of housekeeping to an app, now typically one they pay a substantial subscription for. Just as I wouldn’t pay someone to come and work through all my personal papers and records, destroying those that they thought I didn’t need, I will never use an app that does the same with my Macs. If the housekeeping app you use deletes log files, particularly if those could include those of the Unified log, then I’d consider that to be malicious, and proof that the app’s developers don’t understand how macOS works.

And that’s why some discover that those vital logs have been destroyed, and they can’t run the checks built into several of my apps, including the records of XProtect Remediator’s security scans.

Recommendations

Users: perform housekeeping as a routine, particularly on disused and unnecessary logs in ~/Library/Logs. Never interfere with logs stored in /var/db/diagnostics.
Developers: avoid writing your own logs. If your software really has to, ensure that it performs its own housekeeping, and inform the user in its documentation.
Apple: if the Unified log can’t be unified, improve it and make it fit for purpose.