SpamSieve 3.3.2 is an update of my Mac e-mail spam filter that improves remote training, can export more rule information, and improves uninstallation, help, and more. The most significant news is that Microsoft has changed plans again. They will no longer be preventing users from continuing to use Legacy Outlook and in fact will keep it available for download as part of the combined Outlook app. This gives more time to adapt to their change in plans for AppleScript. Exchange (EWS) support remains an issue. Microsoft is still planning to cut off Legacy Outlook users, and Apple has yet to announce when Mail will support Graph. So far, I’ve not heard that it’s enabled in the macOS 27.2 beta.

We’re participating in a seasonal festival of artisanal software called SummerFest. For a limited time, you can get a 25% discount on new purchases of SpamSieve, EagleFiler, and other top Mac apps from our indie developer friends.

Some interesting engineering issues were:

• Updating NSWorkspace APIs introduced a problem because with the old launchApplication(at:options:), the .withErrorPresentation option is off by default, whereas with the newer openApplication(at:configuration:completionHandler:), the promptsUserIfNeeded option is on by default.

• Previously, Core Data would strip leading BOMs from NSStrings when saving to a SQLite database. With macOS 27, this happens for Swift Strings, too.

• I have lots of unit tests that check for memory leaks. These caught a change where addPersistentStore(type:configuration:at:options:) now seems to return an autoreleased store. Without adding a pool, it looked like there was a leak.

• Also, the tests were finding lots of “leaks” because KVO was retaining NSTabView, which retains a controller. Previously, XCTestCase.waitForExpectations(timeout:) would spin the run loop, and everything would get cleaned up. With Xcode 27 and macOS 27, I need to manually spin the run loop before waiting for the expectations.

• During a software update, macOS may decide to reload the Dock icon. The application package no longer exists because it’s been replaced with a new version. This wasn’t a problem before because SpamSieve had been drawing to an image in memory and setting that as the Dock icon. Now, with the Liquid Glass–imposed removal of custom Dock icons, macOS tries to load the icon from disk, and rather than handling errors gracefully it will crash. I fixed this by setting an image with a memory representation before starting a software update.

• SpamSieve 3.3.1 introduced a crash affecting a small but significant number of users when doing a Core Data fetch of a large number of objects. The private [_PFManagedObjectReferenceQueue _processReferenceQueue:] method was seemingly releasing an object that had already been released. What’s odd is that, as far as I can tell, none of the related app code had changed since the previous version, yet suddenly crashes started happening all the way from macOS 12 through macOS 27. If I had to guess, this was triggered by upgrading from Xcode 26.2 to Xcode 26.6. Maybe Swift’s ARC optimizations are doing something subtly different?

  The crash was triggered by an autorelease pool that I had inserted to promptly clean up any temporary objects after a fetch. (The actual fetched objects were returned and used outside of the pool.) Interestingly, both Gemini and ChatGPT were convinced that this is a well-known exception to standard memory management rules. They said it’s never safe to use managed objects outside of a pool they were fetched in because they depend on internal Core Data objects that the pool will have released. I was not able to find any basis for this “belief,” but it was easy enough to test. As expected, removing the pools (during the beta cycle) just made the same crashes happen slightly later.

  I think there’s probably something wrong in Core Data’s internal bookkeeping, but this does not seem to be a popular crash (and I’ve never seen it on any of my Macs), so my code must be doing something uncommon to trigger it. My current best guess is that it was related to calling refreshAllObjects() after saving. (I bet there’s some fancy bookkeeping to make that efficient.) I was doing this throughout the app to break retain cycles so that unused objects could be deallocated. But it wasn’t necessary in this particular case because the entity being fetched has no relationships. So far, removing the refresh seems to have stopped the crashes, but I won’t know for sure until more people have been using this version for a while.

Previously:

• The End of EWS in Mail and AppleScript in Outlook
• SpamSieve 3.3.1
• SpamSieve 3.2.2

Alex Chan (Hacker News):

Many of these outages were caused by a single bug, deep in SQLite. It took months of intense forensics to track it down.

[…]

This lack of reliable trigger conditions meant we couldn’t reproduce the bug synthetically. Instead, we had to rely on deploying passive, forensic telemetry in our live environment to catch the corruption red-handed. Gathering live diagnostics for a database issue is the last thing we wanted to do, but we had no choice.

[…]

As an additional complication, the corruption didn’t occur on a regular schedule. Sometimes incidents would be hours apart, other times weeks. This made it difficult to predict progress or plan further work, because we were never sure when we’d get our next diagnostic dump. We had a six-week period between October and December when there were no corruption incidents, before they returned as an unwelcome Christmas present.

[…]

In two incidents, our transaction logs failed to replay cleanly. Upon closer inspection, we discovered that data written and committed by one transaction was inexplicably invisible to later transactions. A write had vanished into thin air without raising an error.

SQLite:

The bug is likely present in all version of SQLite from 3.7.0 (2010-07-21) through 3.51.2 (2026-01-09). It is fixed in version 3.51.3 (2026-03-13) and later. Backports of the fix are available for some earlier releases: 3.44.6 and 3.50.7.

The bug only affects databases in WAL mode when there are two or more database connections open on the same file, in separate threads or processes, and when those two connections attempt to write or checkpoint at the same instant.

Alex Chan:

This investigation is a useful reminder: running boring technology in a non-standard way is a risk. The common paths and standard configurations are incredibly well-tested and reliable. Most people use SQLite in a standard configuration and never face this sort of issue. Everything we were doing was a public, documented, supported configuration—but by taking manual control of the checkpointing process and running at our own aggressive pace, we stepped off the well-trodden operational path.

Scott Perry:

I spent the last two years of my time working on SQLite trying to chase this bug down. It was so rare that I never got an internal report; my assessment of the number of customers affected was “about a county’s worth”, some of whom were generous enough to allow their local Genius Bar to send me backups of the affected databases. I spent so many hours in a hex editor staring at the wreckage.

Top three data corruption bug of my career for sure, and the other two weren’t bugs in SQLite.

A former Tailscale employee got in touch and shared that they were checkpointing these databases every 250ms. Multiplied across a fleet of servers that is a fantastic way to surface a super rare bug—they basically built our test rack as a product, but two orders of magnitude more effective (partially due to scale, but mostly because our rack also panics devices at random in order to test the filesystem, NAND controller, etc. and rebooting takes a while)

[…]

13/10 fantastic bug. proof that 100% test coverage is a good start, and that formal methods are needed for critical applications.

Carl Sverre (Hacker News):

I whipped out my phone, and asked Claude to get to work. I had it get SQL 3.51.2 – still buggy – set up in Antithesis, and then instrument the code with a bunch of Antithesis assertions. You can see the instrumented version here.

Then I asked it to write a simple workload which exercised the WAL insert and checkpoint code. Notably, this is a completely generic workload. It just runs writes and checkpoints concurrently – things you’d expect to actually happen in production, all the time. The assertions are also generic to the bug, they’re all standard assertions you’d add to any database, things like “no lost committed writes” and “database is not corrupt” (called integrity check in sqlite).

On my first run, Antithesis caught the bug in 15 mins. Here’s the report.

Of course, it helps that he knew to look in the checkpoint code, but you could imagine using tools like this to quickly investigate hypotheses when you don’t already know the answer.