Hunter Biden vs. the drive forensics

Hunter Biden’s laptop was a 2017 MacBook Pro. That model (A1708) used a non-standard connector for its internal hard-drive, a connector not used before or since. In order to take a forensics image of such a drive, you need to insert it into a MacBook Pro of similar vintage. You can’t find anything else that’ll read the drive.

I mention this because of this wrong tweet by (the otherwise awesome) @emptywheel that claims the FBI was “insane” for buying laptops in order to do forensics on the drive. She even asserts that all the techies she’s talked agree with her.

All these techies are wrong.

This blogpost is about why techies are wrong. It’s not because they lack the expertise, it’s because of the way non-techies interface with techies trying to confirm their bias.

The background

The underlying story is this FBI timeline describing the events of acquiring Hunter Biden’s laptop from the repair shop, then doing forensics on it as part of a tax investigation case. Several times, they mention how they had to buy an extra MacBook in order to do the forensics.

This story by @emptywheel cites this as the strangest of many oddities in the FBI’s examination of Hunter Biden’s laptop. It’s not odd. By extension, nothing in that timeline is odd: while I can’t explain it all, I don’t find anything unexplainable. I think everything I don’t understand myself could be easily explained by others.

We know Hunter’s laptop is a 2017 MacBook Pro (Model 1708) because of the serial number FVFXC2MMHV29 on the FBI’s court order. The laptop had a dead motherboard, so we can’t use the original laptop to read the drive. To read it, we would need to buy another MacBook that used that connector. This video demonstrates why this needs to be done. You can see the unusual connector on the drive that fits nothing else, not SATA, not NVME, not anything.

Furthermore, when taking a forensics image, you do’t simply boot the laptop as normal. You’d instead boot from an external drive, or boot it in target disk mode (shown in this video). You wouldn’t boot from the internal drive firstly because that would change contents (like logfiles), and secondly, you still can’t access it without knowing the password.

Experts rightly have a concern about this: if the alternate boot fails, files change. You need to hold down the T key in order to enter target disk mode, or the option ⌥ key to boot from a different drive. If you fail at this, then the computer boots normally, which changes many files on the drive (mostly, logfiles). But, if you do it right, you can prove you did it right: if there are no newer timestamps, then you didn’t boot the drive.

For reading this as an external disk, software write blockers can be used. I use Disk Arbitrator for this purpose.

Once a forensics disk image is taken, copies can now shared with others, as the timeline describes. First the DoJ forensics experts create the image (and do some analysis), then they pass it to the FBI’s forensics experts.

When techies go bad

Marcy (@emptywheel) claims that all techies agreed with her that this (buying a laptop) was insane, that no forensics works this way. The techies were all wrong, but it’s not necessarily their fault.

Firstly, experts are limited. Most have never experienced taking a drive from a dead 2017 MacBook. Repair shop technicians have, because in their experience, most computers they see are dead. Cybercrime forensics experts haven’t, because most computers they see are still working. The DoJ/FBI techies probably have more experience than anybody else taking forensics images of hard drives, and it’s still likely they never before encountered this situation of a dead 2017 MacBook Pro.

It’s situation that’s totally normal from one point of a view, and completely novel and bizarre from another point of view. When an expert tells you that something is inexplicable and insane, you must suspect a hole in their knowledge.

Secondly, they haven’t done work. I started with the serial number of FVFXC2MMHV29 of Hunter’s laptop and looked it up online to get to the exact model number. That’s how I know for sure it’s a 2017 model with that oddball hard-drive connector.

It’s the same reason lawyers won’t answer your legal questions on Twitter — the only answer they can give is “while it seems simple, I’d have to spend time researching the exact details of your case”. In law, as in computer repair, small details matter, where two nearly identical situations have different outcomes because of a tiny detail. They aren’t going to spend that much time for free, so no free legal advice.

It’s the same reason doctors don’t want to opine on that rash. It’s probably just an allergy, but it might be cancer. Nobody wants to be in the position of “you told me it was just allergies and now I’ve got stage 4 cancer”.

Asking techies to opine on things, without doing the work, leads to the same situation, conclusions worth only the tiny effort they spent thinking about the situation.

We see that in the Trump-Alpha Bank story, where I see people who I know personally to be experts in DNS making crazy claims that no such expert would make. The reason is that they haven’t done the work, spent days analyzing the DNS logs themselves, but who have only looked at things superficially, under the (many false) assumptions the reporter has given them.

Thirdly, there’s a game of telephone going on. The description is being passed through multiple layers, with each each layer not understanding all the issues, both computer technology and criminal investigation procedures. Perfectly normal things get distorted by confusing descriptions.

A good example of this is @emptywheel’s misunderstanding of “de-dupe”. It doesn’t mean how she uses it in her story. Instead, it means “de-duplication”. A typical hard-drive has duplicates of many files, especially large files like videos. A forensics tool like Celebrite can be instructed to remove duplicates — such that a record of the all the original files exist, but they all point to the same data inside the forensics container. This makes the forensics container much smaller. In the FBI timeline, this fact is used to explain why the DE4 data extraction was smaller than the first 3 extractions.

The point of all this is that when you ask techies to confirm things are oddities, you are getting confirmation bias. I get interviewed a lot by journalists, and I see them make such mistakes in their questions. I do my best not to confirm their biases. There should be training for journalists on how to avoid asking such leading questions.

A more important point is that this leads to conspiracy-theories. It’s what the conspiracy-theorists do, look for oddities that are unexplained, which then become proof of their theory. The unexplained, the bizarre, the insane-looking should never be used to confirm a theory. The only evidence of a theory are those things that can be positively explained.

Conclusion

The FBI timeline shows what looks like to me the normal forensics/investigative process for a hard-drive. There are certainly “oddities” here, but they aren’t significant deviations from the norm — the are normal oddities you see in any investigation, the more people who are involved, the more oddities you get, stemming mostly from miscommunications.

The biggest problem is that we are demanding that timeline answer questions now that they weren’t really thinking about then. The public copy that is being passed around has many alternations, most of which are innocent but some of which may be malicious (trying to implicate Hunter Biden with fake data). This gets especially crazy because some of the “frothy right” are making the assumption that if Giuliani makes a change in his copy of the laptop drive, that somehow this will cause a change in the FBI’s copy. The entire issue has gone completely bonkers.

Once the FBI grabs the laptop, no changes to the laptop drive itself will be made. Even as changes/corruptions spiral out of control with public copies, the FBI’s copy will be pristine.

But the laptop was in possession of the repair shop for months. It’s unlikely changes were ever made to the drive, but that’s not the same as saying it’s guaranteed.

The easiest way the FBI can look for changes is timestamps. Every file has a timestamp indicating when it was last modified/created. Many files have internal timestamps, like metadata on photographs indicating when they were taken, which will be different than the file timestamp of when it was uploaded to the computer.

The FBI can simply look at all these timestamps and see if there are any timestamps newer than when the laptop was dropped off.

But this still isn’t proof of no changes. It’s easy changing the timestamp for the computer backwards so that files you copy have the old timestamp. It’s easy changing the metadata within files to have older timestamps. Very few files have cryptographic signatures that can’t be forged.

It would be nice if the FBI would provide us with a list of the newest items on their forensics image, the newest file modification time, the newest image, newest Word document, newest email, and so on. But they typically don’t spend much effort trying to dispel conspiracy-theories, because doing so would only increase the conspiracy-theories as they find some unexplained oddities about those timestamps.

BTW, while her article on FBI oddities isn’t very good, most all content from Marcy (@emptywheel) is awesome. In context with my “effort” comments above, she spends far more effort researching these things than I do. If you see me disagree with her, you should probably take her side.

---

Update

One of the comments claims I am “over my head” because they think I’ve made mistakes. Among the my “mistakes” was is the claim that macOS has an extra “birth” or “created” timestamp that’s harder to change.

That’s true, but it’s still easy. The following screenshot shows a file with all timestamps set to 11:11:11 11:11:11.

The command-line program ‘stat’ works as it does on Linux, and shows you the traditional Unix/Linux timestamps associated with a file. The standard Unix/Linux program ‘touch’ can change these values.

For macOS specific features, like a separate create/birth timestamp, the command-line tools ‘GetFileInfo’ and ‘SetFile’ can be used. I think these come with developer tools, so may not be on the default macOS installation.

QED: changing the timestamp backwards in time is pretty easy, all the timestamps.