Launching apps in Sonoma 14.6.1: Reduced security

In the first of these articles, I examined security aspects of the process of launching various app configurations in macOS Sonoma 14.6.1, on an Apple silicon Mac with full boot security and other security settings. This article moves on to discover how those change when boot security and security settings are reduced. Full details of how this was done are given in the previous article.

To remind you, the apps used were:

SystHist – notarized, quarantined, moved from its landing folder to avoid app translocation;
SilentKnight – notarized, not quarantined, previously run;
Sparsity – notarized, not quarantined, not previously run;
DelightEd3 – not notarized, signed with a Developer certificate, not quarantined, not previously run;
DelightEd3resigned – not notarized, ad hoc signed, not quarantined, not previously run.

None of the apps run in an app sandbox, and those notarized use a hardened runtime.

This article covers these three variants of the same 14.6.1 VM:

Full Security, with Gatekeeper/XProtect disabled;
Permissive Security, with SIP disabled;
Permissive Security, with both SIP and Gatekeeper/XProtect disabled.

In each VM, settings were confirmed using SilentKnight, which in turn calls standard system tools to determine current security settings, such as those when both SIP and Gatekeeper were disabled.

Gatekeeper disabled

Surprisingly, with Gatekeeper assessments disabled, com.apple.syspolicy.exec still reported that Gatekeeper assessments were made
GK process assessment: <private> <– (<private>, <private>)
Gatekeeper assessment rooted at: <private>
and later
queueing up scan for code: PST: (vuid: 7C5C43BF-A338-4228-B61E-5038F1D93EDB), (objid: 69229), (team: (null)), (id: (null)), (bundle_id: (null))
GK performScan: PST: (vuid: 7C5C43BF-A338-4228-B61E-5038F1D93EDB), (objid: 69229), (team: QWY4LRW926), (id: (null)), (bundle_id: (null))

Following that, XProtect scanned
XPAssessment performAnalysisOnFileImpl continueOnError set to 0
Xprotect is performing a direct malware and dylib scan: <private>
using its standard Yara rules.

CloudKit ticket lookup also proceeded as normal. After a while, though, XProtect announced
Xprotect is skipping executable assessment: <private>

This concluded with
GK scan complete: PST: (vuid: 7C5C43BF-A338-4228-B61E-5038F1D93EDB), (objid: 69229), (team: QWY4LRW926), (id: (null)), (bundle_id: (null)), 4, 4, 0
and
GK evaluateScanResult: 0, PST: (vuid: 7C5C43BF-A338-4228-B61E-5038F1D93EDB), (objid: 69229), (team: QWY4LRW926), (id: co.eclecticlight.SystHist), (bundle_id: co.eclecticlight.SystHist), 1, 0, 1, 0, 4, 4, 0
GK eval – was allowed: 1, show prompt: 1

The normal prompt for user consent was displayed, and handled as expected. Following that, launch proceeded normally.

Similar entries appeared in the checks made on all apps that had undergone Gatekeeper and XProtect assessment when full security was in force. There is nothing in the log entries to indicate that disabling Gatekeeper had any effect on the checks that were made, although as none of these apps failed assessment, it’s possible that any failures would have been ignored.

SIP disabled

When SIP was disabled, the structure of pre-launch assessments changed, and appeared disordered in comparison to those performed under full security and with only Gatekeeper disabled. Most notable, perhaps, was the almost complete absence of log entries from the com.apple.syspolicy subsystem, which in full security is so prominent, although its service syspolicyd did appear in entries.

Although quarantine was recognised, no entry reported the start or conclusion of any GK (Gatekeeper) assessment, nor subsequent XProtect scans. Instead, the XProtect service wrote
Bundle is not apple signed
Bundle size result: 18388222 (YES)
Always scan: YES

Normal ticket checks were made via CloudKit, but shortly after those were completed, XProtect tried to use its standard Yara rules, and ran out of memory doing so, with the kernel reporting
process XprotectService [697] crossed memory high watermark (15 MB); EXC_RESOURCE
XProtectService therefore ran into trouble before it had even started to scan the app. While some entries suggested prompting the user for their consent, that doesn’t appear to have happened. Eventually the app launched in spite of the disorder that had preceded.

When launching a notarized app that wasn’t quarantined, neither Gatekeeper nor XProtect appear to have had any involvement in the approval of the launch.

SIP and Gatekeeper disabled

Results were essentially identical to those obtained with SIP alone disabled, even down to XProtectService exceeding its memory high watermark, and the almost complete absence of log entries from the com.apple.syspolicy subsystem.

SIP and Gatekeeper settings

Prior to examining these log records, I thought I had a clear idea as to what these two controls do. In fact, neither of them does what you’d expect.

Disabling Gatekeeper or XProtect checks doesn’t stop them from occurring, although it might result in macOS ignoring any errors they might find. That would be consistent with the statement in the spctl man page: “Operations that would be denied by system policy will be allowed to proceed; assessment APIs always report success.”

On the other hand, disabling SIP almost completely stops the whole com.apple.syspolicy subsystem, which ordinarily plays a major role in pre-launch checking of apps. This effectively kills both Gatekeeper and XProtect, leaving those checks in disarray. When the XProtectService tries to lend a hand, its attempt to ingest the current Yara rules runs it out of memory, and it appears unable to render any useful assistance to the pre-launch checks.

This may explain why disabling SIP has the effect of shortening the time to launch an app, most noticeably with larger and more complex apps. In return for launching in a shorter time, the app probably isn’t checked against XProtect’s Yara definitions, so could still contain malicious code that would pass undetected.

In the next article I’ll show what does happen when this system encounters live malware.