With the release of iOS 16, I did what I always do-get as many images as I can from my test phones and start ripping them apart to see what is and isn't there anymore. While quick images (iTunes-style backups) were pretty straightforward minus some enhancements to Messages and Safari (here if you're interested), filesystem images have been a bit different. Buckle up folks because this one is going to be a bit long. If you're looking for the TL;DR, scroll to the bottom to get a quick review of the good news/bad news. However, if you're curious in seeing how the new "biome" data structures are playing into things, keep reading.
Finding the Missing Knowledge
While working through the file system I wanted to check on some of our favorite pattern of life activity such as PowerLog and KnowledgeC data. While PowerLog is still mostly intact (and still a mess to parse through), I noticed that KnowledgeC was missing several of my favorite artifacts. If you're not familiar with KnowledgeC.db, where have you been?! Jokes aside, I recommend reading up on it from the talented mind of Sarah Edwards over on her blog Mac4n6. Sarah's work into KnowledgeC and her tool APOLLO has been amazing for helping to deal with KnowledgeC work and the basis for a lot of our understanding of pattern of life information over the past few years.
So what was missing? The easiest way I can show this is using a tool from my day job, Magnet AXIOM to show you a list of what was present from a device running iOS 15 to iOS 16. Now, take the number of recovered records with a grain of salt as the iOS 15 device was used for a longer period of time versus the iOS 16 one. What is key is taking note of the missing categories entirely.
I'm sure that we all have our favorite artifacts within KnowledgeC but I quickly noticed several of my personal favorites such as Application Focus, Safari History, Install History, and many of what I call "nexus events" such as Device Orientation and Plugged-In States missing too. Yikes! Thinking that it was possible that the stream name of these events within KnowledgeC had changed, I dove into the database to see what was up.
Looking in the ZOBJECT table of the database at the ZSTREAMNAME column, the app/inFocus key is missing entirely. Digging around the file system I found that there were some hits of the same data but in a totally different place. One such place is a folder called "_DKEvent.App.InFocus" which lives in the path /private/var/db/biome/streams/restricted. Biomes? Streams? What does all this mean?
Understanding Biome Data
Let's face it Apple, you're not great at documentation when it comes to the outside world. We still haven't seen all the documentation when it comes to KnowledgeC and some of our other favorite artifacts, and now Apple's throwing us another curve ball! Biome data has been around for a bit in both macOS and iOS but is getting used a lot more now. Biome seems to mean less around "biome
trics" and more around "suggestions." For more information on what one of the best macOS resources I've ever seen has to say on the topic, head over here
to Howard Oakley's Eclectic Light blog on the topic.
I would tend to agree with a lot of what Howard has talked about with biome data as it tracks with what I've found for a while now in iOS. You can see some examples of that in this post here when talking about iOS messages within AppIntents and other locations.
From looking at iOS 16's file system information you can actually find several biome folders:
/private/var/mobile/Library/Biome and /private/var/db/biome. You're not going to find either of these outside of a file system image in my experience. In these folders, you're going to find a "streams" folder with data separated into either "public" or "restricted" folders. Within each of these directories the data is then tracking a specific service or function with other subdirectories and binary files stored within. The age of the binary files seems to be approximately 30 days from my previous testing, but additional digging will be done to see if iOS 16 has changed that.
The binary file is often referred to as a "SEGB" file by several developers and while its name looks like a timestamp, it seems to be tied to a serial number for that process. There is also a tombstone folder which contains expired versions of these binary files.
Within the biome folders, you can see several points of interest in the following screenshots. Several of them match what we USED to see in the KnowledgeC database as well.
Breaking Down the Biomes (Part 1-App.inFocus)
To be honest each of these folders need to be broken down in depth and will be in due time. However, because of the incredible amount of valuable data found within these folders, I think it's important to get the data out there to the community for some of the really crucial ones. Plus it gives me an excuse to turn this into a whole series and keeps me busy!
The best way we can attack these datapoints is to get a better understanding of these "segb" binary files and how they are structured. Luckily, while the records within the files are going to change depending upon what the binary file is tracking, the overall general structure is the same.
There's some great additional information on these files available here
as they relate to AppIntent data from BlueCrew Forensics. I'll take a higher level look at these binary files here as they pertain to a lot of the more generalized storage here in iOS 16.
Let's start with one of my favorite KnowledgeC events that have moved over here, Application Focus. What used to track as app/inFocus in the KnowledgeC.db file can now be found within the _DKEvent.App.inFocus directory's binary files.
Let's start with the "metadata" plist file outside of the binary file. This file exists for each subdirectory in the biome data and helps us better understand how long we can expect this data to exist. These NSKeyedArchiver style plist files can show us the max age of the data within the binary file.
The maxAge value (2419200) appears to be seconds, and if it is converted to days, shows us 28 days which lines up with previous biome experiences.
The first 56 bytes of the binary files are referring to the header of the file itself. It will end with "SEGB" in ASCII.
Immediately following the file header is the first record. These records will have two parts, a 32 byte record header and a variable length record stored as a protobuf. The 32 byte record header's most important part (in my opinion) is the the first four bytes. These four bytes inform us how long the protobuf record actually is. This value is stored in Little Endian format.
From the above screenshot this particular record in question is 264 bytes long. There are also 2 timestamps stored in Mac Absolute Time at offset 8 & 16 of the record header which seem to correspond to when these records are written to these binary files. At the end of the record header starts the protobuf record which in this case is our 264 bytes. If we sweep and extract that data we can open it up in a protobuf viewer such as using something like CyberChef. Note: If you're going to use CyberChef your recipe should be "From Hex"-->"Protobuf Decode."
I personally use a great tool built by a developer friend of mine for reviewing protobuf data that I'll include some screenshots of. In the following example, two records from the SEGB data that are right after another are being displayed. One thing to understand about these protobuf records is that even though the record header tells us how long they are, the next record header may not start immediately. Records may have some 0x00 padding and the next record header will start at next offset divisible by 8.
The "Transition Type" as mentioned above have several observed types, but I'm sure there are going to be more we find.
Some other Biome Info
Something else that should be noted about this biome data is that there are several clues that inform us that this data may be syncing to iCloud. First, in most of the files that are being examined, a record GUID is being given for each recorded action. There's also information within the main Biome folder (private/var/mobile/Library/Biome) which shows that the data is uploading to iCloud and a single device may have records synced from OTHER iOS/macOS devices. Other information within the KnowledgeC.db shows there are records syncing up and down to iCloud as well!
Using the sync.db found within /private/var/mobile/Library/Biome/sync/ you can see within the DevicePeer table records of devices that are syncing with your target device. Each device is assigned a GUID value as well as showing the last synced date in Unix numeric time.
The GUID value listed within the database will also match subdirectories found within the "remote" folder of each biome's service folder. This way we can track where these additional records are coming from when parsing the remote biome data points from the local.
The TL;DR and Wrap-Up
Whew that felt like a lot. Okay, here's the good news/bad news. Bad News: KnowledgeC.db doesn't seem to have as much as it did previously. Good News: That data has moved to another location that while requires a bit more parsing, seems to have some amazing data stored within. In addition, we'll be able to track data from devices we might not have in our possession.
This research is going to be ongoing for a while but I'll continue to break down additional biome files and their protobufs over the next few weeks or so.
Thanks for your work ! Very interesting to read you. So many hours of testing that I have not spared thanks to youReplyDelete
The proposition may generate somewhere within the mid-hundreds of tens of millions annually for the state, but doubtless no more than $500 million per yr, in accordance with estimates by the Legislative Analyst’s Office. California has a free hotline folks can call to get help and get connected to resources. There’s also Gambler’s Anonymous, which is a help group program, like Alcoholics Anonymous, as well as|in addition to} Gam-Anon, a help system for friends and family members of gamblers. California also has a program that provides free 다파벳 remedy — including counseling, outpatient, and residential therapies — to gamblers or folks affected by problem gambling, which incorporates spouses and members of the family.ReplyDelete