Master Boot Record (MBR) Corruption and PGP Whole Disk Encryption (WDE)

Today is a big deadline in work. The point when weeks of work flow from development to production (via numerous sprints and UAT releases). The culmination of the development team and my efforts over the last two months. As with most IT projects the last two weeks have been a bit fraught, testing us as we prepare to launch a new suite of Cognos reports.

Today is also the day my Thinkpad has decided to corrupt its master boot record (MBR). Preventing my laptop from loading its operating system. Thankfully Microsoft (and third parties) provide bootable utilities to repair MBRs. However…………

My MBR failure is complicated by IBM (sensibly) requiring the use of Symantec’s PGP Whole Disk Encryption (WDE). PGP’s WDE protects our laptops, and any sensitive data on them, in the event laptops are lost or stolen. As a mobile worker, and someone regularly on the move, WDE is a nice saftey blanket. Yes, if someone really wants the data on my laptop they will get it. But for additional security and if the laptop is lost it provides some reassurance. It’s also company policy, there is no point in fighting it.

Update: I was wrong about PGP being crackable. IBM Helldesk are able to deal with a forgotten password because we run a PGP support server. Since PGP version 9.7 there are backups in place to recover a forgotten password. But there is still no known way to crack PGP WDE. E.g. If you can’t decrypt the hdd, the data is lost. On one hand this makes me feel safer about the loss of a laptop and on the other it makes me glad most of my data/files are backed-up. A reminder I need a better backup solution for a hdd failure.

With the whole hard disk drive (hdd) encrypted I can’t use utilities to fix the MBR. The utilities require you to boot from them, and in so doing skip PGP’s BootGuard. BootGuard lets the OS use the encrypted hdd.  Therefore until the hdd is decrypted the utilities can’t access the MBR. Due to its encryption the hdd doesn’t even appear. Thankfully PGP provide recovery CDs, downloaded from here:

http://www.symantec.com/business/support/index?page=content&id=TECH149679

It’s key to know which version of PGP you have. The recovery CDs are version specific. If you can boot into PGP’s BootGuard screen it’s easy to find out: Selecting ‘advance’ instead of ‘continue’ from the options displays the version, and other options to assist in recovery. Since a similar failure in 2009 I keep a note of my PGP version (including any service packs). Just incase PGP’s BootGuard also fails to load. It’s not unheard of for both MBR and BootGuard to be corrupt at the same time. Not knowing which version of PGP I’d installed, combined with the bad sectors that caused the HDD to fail, resulted in my old drive being scrap.

Symantec provide a guide for how to resolve that situation here:

http://www.symantec.com/business/support/index?page=content&id=TECH149345

With the matching version of the recovery disk in place I booted off the recovery CD and tried to let Windows boot itself. In rare cases it’s possible that using the Recovery CD instead of BootGuard lets Windows boot and recovery itself. Sadly this wasn’t the case, I still had an MBR issue. Back to the drawing board.

The next step is a longer one: Rebooting off the Recovery CD, entering my password and then pressing ‘D’ to decrypt the entire hdd. We’re now at 90% having started at 9am this morning. The laptop hdd is 250gig capacity, of which 80gig was in use. I’m hoping the first 80gig takes the longest to decrypt. Ideally the final170gig will be a lot quicker, as it’s empty disk space. I’ll leave it over night and then all being well use Microsoft’s MBR fixer tomorrow. If anything goes wrong or the laptop is disrupted during the decrypt, all data is lost. Not the most relaxing situation to be in but I have 90% of my data backed up. All my work is stored on IBM’s cloud and I only stand to lose emails recently archived locally. The main loss will be time in having to rebuild my Thinkpad. As a worst case this isn’t too bad, but fingers crossed I can fully decrypt the hdd and recover my current MBR.

Update: Sadly my 80gig and free space decrypting quicker theory has been disproved. It’s now at 38% left to go and hopefully will be sorted in the early hours of Tuesday morning (3.5 days to decrypt 250gig). Keeping everything crossed it keeps going and finishes, allowing me to fix the MBR and recover all my data / Laptop. Decryption takes a fraction of the time if the hdd is mounted as a slave on another system. Lesson learnt! From now on I’ll run two hdd and regularly clone (more on this to come in another post).

Before starting a decrypt via the recovery CD I googled alternative options. If you have a second machine with the same version of PGP installed you can plug the hdd in as a slave (via a USB caddy) and use PGP on the local machine to decrypt the hdd. This is the fastest way, sadly I don’t have another machine with PGP installed.

Update: Plugging the hdd in via a USB caddy / as a slave in a second machine is a lot faster because the Recovery CD is limited to 16 bit processing. If in Windows / Linux or OSX the decryption process can be run at 32bit /64bit and takes a fraction of the time. With hind sight waiting for SC to get home and pinching her work laptop would have been a better bet. It’s at 83% now with a very slim chance of being finished by Sunday. At least it’s still going. No physical hdd errors, yet!

I used to backup an image of my machine but Windows 7 made this harder and since upgrading I’ve taken to using IBM’s could to backup all of my work and accepting that if I had a failure I’d need to get an additional machine from IBM and rebuild it. Having now tested this theory it doesn’t work!

The new plan
The Plan comes in two flavours: 1. Get a smart phone and 2. improve my laptop and return to weekly backups.

1. Smart Phone: 99% of my work calls are handled by VOIP but I’ve been toying with getting a smart phone for work as a backup access to my work email, calendar, instant messaging and terminal services. Four key components of my day job that I’m currently without due to my laptop decrypting. As a result I’ve bitten the bullet and ordered the Asus Fonepad. It’s not the best spec but a Galaxy Note II is out of the question at the moment. I hardly make calls on my work phone thanks to VOIP. If I did have to make a call I always have my iPhone5 with free minutes to make emergency work call while out and about. The concept of a 7inch tablet that doubles as an emergency phone for £180 delivered was too good to get hung up on the negatives (slower processor and you’d look like a sketch from Trigger Happy TV if you tried to make a call in public). I’ll post more on this in another blog after some usage.

2. SSD and Weekly Cloning: My boss has an SSD drive and the boot times + smoothness of operation have always appealed. I’ve been waiting since for a reason to rebuild the laptop to get one and this is it. I’ve ordered a Kingston Value 120gig drive after reading several reviews like this review:

http://www.hardocp.com/article/2013/01/28/kingston_ssdnow_v300_120gb_ssd_review/#.Uc1wrj7F1SU

The time it takes to boot my Thinkpad always frustrates me. Even since upgrading from 4 to 8 gig of RAM it’s still sometimes hangs while paging and under heavy loads. Hoping the SSD will also prove more reliable. My Thinkpad travel a lot, the one before clocked over 100k miles. Combined with being on 5 days a week, most weeks of a year, it’s no wonder hdd fails / issues like this occur. With no moving parts an SSD should prove more reliable. It also means I can keep my current drive as a spare (if it’s not beyond repair) and regularly clone the SSD as a backup. More on this to come after the SSD swap and hopeful recovery. A new backup strategy is required (feel free to suggest any ideas in comments, or to laugh at my expense).

For now the Thinkpad is slowly chugging away decrypting and I’m off out to recover and watch Knee High Perform: http://www.kneehigh.co.uk/show/tristan_yseult.php

Thanks to my teams’ efforts and with lots of phone calls the release has gone to UAT and we’ll go live first thing Monday morning. Wish me luck and for a working Thinkpad asap :D.