Thu, 25 August 2011

With the release of Lion, Core Data now makes it very easy to store large chunks of data, e.g. large media files, in the filesystem instead of in the database. This is a quote from the “Core Data Release Notes for Mac OS X Lion”:

“Small data values like image thumbnails may be efficiently stored in a database, but large photos or other media are best handled directly by the file system. You can now specify that the value of a managed object attribute may be stored as an external record—see setAllowsExternalBinaryDataStorage:. When enabled, Core Data heuristically decides on a per-value basis if it should save the data directly in the database or store a URI to a separate file which it manages for you.”

When I read this, two questions came to mind: where are these external files stored and how large does the data have to be before Core Data decides to store it externally? This doesn’t seem to be documented anywhere and I’m sure it’s subject to change, but this is what I’ve figured out.

The files are stored in the same location as the main database file(s), but they’re in a hidden folder. For example, if the application is called MyLionApp and its persistent store is located in the ~/Library/MyLionApp folder, the external files would be stored in ~/Library/MyLionApp/.MyLionApp_SUPPORT/_EXTERNAL_DATA/.

To find out the data size that’s needed for Core Data to store objects externally, I made a small app that creates NSData objects in 100kB increments. At first it seemed that none of the objects were stored externally, even if they were 1GB large, until I figured out that the data store can’t be NSXMLStoreType for external storage to work. After setting the store type to NSSQLiteStoreType I found that the limit is 1MB. Objects that are smaller than 1MB are stored in the database. For objects that are larger, an external file is created and the database just stores a reference to it.

If you want to use this new feature in Lion, all you need to do is check the Allows External Storage in Xcode for each Binary Data attribute that you would like Core Data to store this way.

Xcode screenshot

Blue Crowbar — Apps for iOS and OS X