With iOS 14, Apple improved the built-in logging framework and added many missing pieces. Is OSLog now something that can finally be used?

Does OSLogStore Work Yet?

YES!*

*Apple removed OSLogStore from iOS 14 at the very last moment, but brought it back in iOS 15. As of iOS 15 beta 7 and macOS Monterey beta 5 (21A5304g) OSLogStore finally works. Jump to the latest update “Update: iOS 15”.

Why OSLog Is Awesome

Every developer uses some form of logging, be it NSLog in Objective-C or print in Swift. These helpers are convenient, but they don’t scale — there’s no concept to classify messages.

Apple introduced os_log with iOS 10 and macOS 10.12 in an attempt to provide a better universal logging system.1 It supersedes the ageing Apple System Logger (ASL) and comes with features expected from a modern logging system:

  • Categorization and Filtering — Log levels and grouping via subsystem and category.
  • Privacy — Dynamic strings, collections, and arrays are replaced to preserve personally identifiable information. This can be overridden on a per-parameter basis.
  • Usability — The logging information collects calling information for you, and it’s integrated into the system via Console.app and the log command line tool. If activity tracing is used, logs are automatically correlated.
  • Performance — Logs are stored extremely efficiently and all system logs are in one place.

You might have already stumbled over the log, as you now need a third breakpoint when investigating where a log message is coming from: Next to NSLog and CFLog, you now need a breakpoint to _os_log_impl.

The New Swift Logging Framework

Instead of calling os_log, you can now use the new Logger struct in Swift — at least if you have the luxury of exclusively supporting iOS 14 already:

1
2

let logger = Logger(subsystem: "com.steipete.LoggingTest", category: "main")
logger.info("Logging \(obj.description, privacy: .public)")

This innocent-looking code uses a bunch of new tricks! First off, the new string interpolation feature of Swift 5 is used to make it easy to customize the privacy of data.

Second, Apple modified the Swift compiler to allow compile-time constant expressions to evaluate the string at compile time. This ensures that logging is extremely fast. The technology behind it is fascinating, and it seems it was worth the wait.2 For more example code, see Apple’s excellent documentation on Generating Log Messages from Your Code.

Calling os_log in Swift

If you still need to use iOS 13 or are curious how things worked back in the day, here’s how calling os_log works. Naively, you might try following approach:

1

os_log("foo: \(x) \(obj.description)", log: OSLog.default, type: .debug)

The compiler complains right away:

Cannot convert value of type ‘String’ to expected argument type ‘StaticString’

os_log requires the use of Objective-C style format strings for performance and security reasons.3 At the same time, Apple also strongly recommends to avoid wrapping os_log. So there’s really no way around using the following syntax:

1

os_log("foo: %@ %@", log: .default, type: .debug, x, obj.description)

What’s the Problem with Wrapping OSLog?

OSLog is designed to accept static strings, in order to be fast and to correctly separate between dynamic data (private) and static data (public). Most wrappers simply forward messages via os_log("%{public}@"..., which makes all strings dynamic and public, removing most of the benefits of OSLog.

Apple maintains a separate SwiftLog logging framework for the Swift on Server ecosystem.4 The framework supports plugins, and there’s a third-party swift-log-oslog plugin, which forwards logs to the os_log machinery. Apple does link to it but also explicitly warns about usage:

we recommend using os_log directly as decribed here. Using os_log through swift-log using this backend will be less efficient and will also prevent specifying the privacy of the message. The backend always uses %{public}@ as the format string and eagerly converts all string interpolations to strings. This has two drawbacks: 1. the static components of the string interpolation would be eagerly copied by the unified logging system, which will result in loss of performance. 2. It makes all messages public, which changes the default privacy policy of os_log, and doesn’t allow specifying fine-grained privacy of sections of the message. In a separate on-going work, Swift APIs for os_log are being improved and made to align closely with swift-log APIs. References: Unifying Logging Levels, Making os_log accept string interpolations using compile-time interpretation.

There are also surprising bugs like doubles that aren’t logged correctly, which can be very hard to debug.

Accessing Log Files

Log files are useful whenever users report a bug or a crash is being sent. Most logging frameworks therefore include a rolling file logger, which makes it easy to access logs on demand.

Apple’s solution to accessing logs is either collecting a sysdiagnose or calling log collect --device on a terminal. The latter possibly requires filtering the output via the --start or --last limits and then sending the output manually.

This might work if your audience is macOS developers, but if you’re targeting regular users, it’s much harder to make them follow a tutorial and then explain how to send a multiple-hundred-megabyte sysdiagnose. Furthermore, this will only work if your user runs a recent version of macOS.

In all practicality, this resulted in basically no app using os_log directly, which completely defeats the point. The most used logging frameworks to date seem to be SwiftyBeaver and CocoaLumberjack.

The inability to access logs has been a point of personal frustration; there are multiple forum entries where people are just baffled that this doesn’t exist. I’ve brought it up every year in the WWDC labs except for in 2020. The usual answer is “Apple cares about privacy, so accessing logs is a no-go,” but the alternative is that people use other logging frameworks that have no privacy-preserving features built in.

At PSPDFKit, we tried switching to os_log but hit the same issues: Our customers depend on a custom log handler that integrates our log messages with their infrastructure. To date, this can only be done with OSLog using SPI.5

New in iOS 14: OSLogStore

With OSLogStore, Apple added an API to access the log archive programmatically. It allows accessing OSLogEntryLog, which contains all the log information you’ll possibly ever need.

The new store access is available for all platform versions of 2020. It’s the missing piece in the os_log story that finally will get us the best of both worlds. Let’s look at how this works:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

func getLogEntries() throws -> [OSLogEntryLog] {
    // Open the log store.
    let logStore = try OSLogStore(scope: .currentProcessIdentifier)
    
    // Get all the logs from the last hour.
    let oneHourAgo = logStore.position(date: Date().addingTimeInterval(-3600))
    
    // Fetch log objects.
    let allEntries = try logStore.getEntries(at: oneHourAgo)
    
    // Filter the log to be relevant for our specific subsystem
    // and remove other elements (signposts, etc).
    return allEntries
        .compactMap { $0 as? OSLogEntryLog }
        .filter { $0.subsystem == subsystem }
}

The code is fairly straightforward; however, the above version has various issues, e.g. it works on macOS but doesn’t work on iOS.

Swift Overlay Issues

Apple forgot to add the Swift overlay shims on iOS, so we need to use some tricks to access the Objective-C enumerator (FB8518476):

1
2
3
4
5
6

#if os(macOS)
    let allEntries = try logStore.getEntries(at: oneHourAgo)
#else
    // FB8518476: The Swift shims for the entries enumerator are missing.
    let allEntries = try Array(logStore.__entriesEnumerator(position: oneHourAgo, predicate: nil))
#endif

Since the non-sugared version also works on macOS, it’s not necessary to do an if/else dance at all. However, this is a good pattern to document how it should work, assuming the bug gets fixed before the GM.

Predicate Issues

The next issue is that the entries enumerator has a built-in way of filtering via the predicate. However, this doesn’t seem to work (FB8518539):

1
2
3
4

// TODO: How to format the predicate? See "PREDICATE-BASED FILTERING" in `man log`.
// Things I tried unsuccessfully:
// - NSPredicate(format: "subsystem == %@", "com.steipete.LoggingTest")
// - NSPredicate(format: "subsystem == \"com.steipete.LoggingTest\"")

Credit for the syntax goes to Ole Begemann, who started investigating OSLogStore early on and maintains an OSLogStoreTest sample project.

iOS Entitlement Issues

While it’s fairly easy to work around the first two bugs, the third one is a real showstopper. While accessing the log store works great on macOS AppKit and macOS Catalyst, it fails with an entitlement error on iOS:

1
2

Error Domain=OSLogErrorDomain Code=9 "Client lacks entitlement to perform operation"
UserInfo={NSLocalizedDescription=Client lacks entitlement to perform operation, _OSLogErrorInternalCode=14}

Why this wouldn’t work isn’t documented, and neither is what entitlement is required. There’s anecdotal evidence on Twitter that Apple’s intent is to allow this on iOS, however, we’re slowly nearing the end of Apple’s beta cycle and there hasn’t been an official announcement or a fix.

If you feel strongly about this, please also report a radar and engage in the discussion on Apple’s Developer Forum.

Digging Deeper

How does OSLogStore access logs in the first place? The store is part of the OSLog framework, which includes a small XPC service. At initialization time, the store opens a synchronous XPC request to com.apple.OSLogService, the service included in the framework.

OSLogService ensures logs are filtered for the current process and then accesses OSLogEventStore. OSLogEventStore is implemented in the private LoggingSupport.framework. Here, you can see it connects to logd and also captures the failure condition, which ultimately produces the entitlement error.

If we keep digging and also take a look at logd, we find references to various private entitlements:

  • com.apple.private.logging.admin
  • com.apple.private.logging.diagnostic

The admin one is required for accessing the log store. If we attach to both our app and logd, we can modify the flow at runtime to trick it into believing the entitlement is there, and voila, logging works on iOS!

Full credit goes to Khaos Tian, who took the time to figure out the details. Attaching LLDB to logd requires SIP to be disabled, since logd is hardened and doesn’t include the get-task-allow debug entitlement.

Streaming OSLog

It’s often desirable to stream log messages as they come in. This is a feature of many popular analytics and logging frameworks. Of course, OSLog also needs this feature.

LLDB is doing exactly that; it listens to OSLog messages and prints the streams as they’re created. Thankfully, LLDB is open source, so we can look at DarwinLogCollector.cpp, which implements this feature.6

The SPI we need is in ActivityStreamSPI.h, which is part of LLDB. (It contains various structs which cannot be redefined in Swift; this file needs to be plain C.) The main functions that are important to us are:

1
2
3
4
5
6
7
8
9
10
11
12
13

// Get an activity stream for a process.
typedef os_activity_stream_t (*os_activity_stream_for_pid_t)(
    pid_t pid, os_activity_stream_flag_t flags,
    os_activity_stream_block_t stream_block);

// Start the activity stream.
typedef void (*os_activity_stream_resume_t)(os_activity_stream_t stream);

// Stop the activity stream.
typedef void (*os_activity_stream_cancel_t)(os_activity_stream_t stream);

// Get a formatted message out of an activity.
typedef char *(*os_log_copy_formatted_message_t)(os_log_message_t log_message);

Given these functions, we can write a Swift7 class that accesses the streaming log. You can see a reference implementation in my gist of OSLogStream.swift. While this works, SPI is still a private API and you shouldn’t ship this in the App Store (FB8519418).