How to use both FileVault2 and Bitlocker simultaneously on a dual-boot Mac running bootcamp

I am the proud owner of a beautifully noisy drive.

A noisy drive?!?!?!!!! Get your data off that before it dies!!!

Thanks for the concern but I’m not taking about that kind of noise.

Oh, did you get a HDD stepper to play music??

No but I think the imperial march and still alive are my favourite step-step songs.

That fine, if you don’t want to tell me I’ll go to another blog

Ok, Ok. This post is about something I have been wanting for ages and didn’t think was possible. This shows how to use both FileVault 2 and Bitlocker simultaneously on a multi-boot mac (running bootcamp). The noise I’m referring to is the pseudorandom noise of an encrypted drive (FDE) :D

What’s so special about that? Is this not an easy thing to do?

No, generally it is not possible to do this due to some good design choices both Apple and Microsoft (and other encryption providers) have employed mixed with a silly one that Apple have made.

The Microsoft and Apple boot providers require a boot loader to remain on  an unencrypted volume which then provides an mechanism to access the protected , encrypted, partition. So to use FDE on either of these Operating Systems you need 2 primary partitions. Legacy partition tables use MBR to describe the  layout of the partition scheme. This partition scheme has a limit of 4 primary partitions.

Can we not just use those four primary partitions?

Nope as the Mac has a recovery partition that you need to keep intact so there there are 3 primary partitions available. This means that typically only on of these operating system can enjoy FDE (and I’m not using containers in a OS, that’s too leaky…).

Are we going to time travel and change some specs???

Nope, luckily for us the Mac uses 2 partition tables. One is GPT and the other is MBR.

The MBR partition is used to boot the Mac encrypted boot loader which in turn provides access to the encrypted Mac partition from the GPT table.

When loading the Windows/Bootcamp parition the MBR is used to determine the availability of the windows encrypted boot loader which in turn uses the MBR to access the encrypted windows partition.

This means for a dual boot fully encrypted system we only need 3 partitions listed in the MBR. Normally the MBR partition is just a clone of the GPT one which is where the problem lies.

Happily we can use some ancient tools to edit these manually and allow dual boot of fully encrypted operating systems, while keeping our beloved restore partition.

Ok, how?

The process is as follows:

<disclaimer: this guide is provided as is, and has no warranty. If you  suffer data-loss, damage etc then this is your responsibility so be a mature person and accept that>

  • Partition the disk using the Disk utility in OS X
  • Enable Filevault
  • Reboot
  • Get a list of partition parameters from the GPT partition table
  • Erase and recreate the MBR table, including only the windows partitions and the Mac encrypted loader partition
  • Run the windows installer
  • Edit the windows recovery/encrypted loader partition
  • Install windows
  • Allow TPM-free bitlocker use
  • Enable bitlocker
  • Smile

    Partition the disk using the Disk utility in OS X

    Reduce the size of the OS X partition to make room for windows, here I am giving OS X 98GB

    1

    Then create 2 new partitions:

    • One for the windows partition
    • One for the Bitlocker boot loader (around a 200 MB – 1gb)

    2

    Enable Filevault and Reboot

    Go to System Preferences > Security & Privacy > [Filevault] and turn it on. Remember to responsibly store your recovery key, mine is on an encrypted backup…. not stored with Apple.

    3

    Then reboot to enable/verify the encryption.

    Get a list of partition parameters from the GPT partition table

    Open the terminal and run the following command to get the details of the GPT table.

       1:  sudo gpt -rv show -l disk0

    5

    Take note of the start point, size and type of partition.

    Erase and recreate the MBR table, including only the windows partitions and the Mac encrypted loader partition

    The windows partitions and mac encrypted loader are the items at indexes 1, 4 and 5. On your system this may differ but index 1 should remain the same.

    Go to a terminal and open fdisk in edit mode

       1:  sudo fdisk -e /dev/disk0

    Erase the MBR table

       1:  erase

    Add a new table and then a new entry for the mac bootloader

       1:  add 1

    Edit this partition to match the OS X encrypted loader partition, the details are from index 1 above but should be identical to this (as of OS X 10.9 in a standard scenario)

       1:  edit 1
       2:  Partition id : EE
       3:  CHS mode : no
       4:  Partition offset: 1
       5:  Partition size:  409600

    Then add the first windows partition, I got the offset and size from index 4 above – this will be different on your systems so pay attention to YOUR start and size parameters and ensure its the the correct windows partition.

       1:  edit 2
       2:  Partition id : 07 
       3:  CHS mode : no
       4:  Partition offset: 193085424
       5:  Partition size: 296059568

    Now add the second windows partition, I got the offset and size from index 5 above – this will be different on your systems so pay attention to YOUR start and size parameters and ensure its the the correct windows partition.

       1:  edit 3
       2:  Partition id : 07 
       3:  CHS mode : no
       4:  Partition offset: 489407136
       5:  Partition size: 827576

    Then write the MBR table using the command

       1:  write

    When you issue the print command a table similar to the following will be displayed (the exact details will be different as appropriate for your disk layout)

    6

    Now that the tough stuff is done we only need to install windows, pop in a windows install disk and do the following.

  • Reboot the machine and run the windows installer
  • Edit the windows recovery/encrypted loader partition – format it
  • Install windows
  • Allow TPM-free bitlocker use

    Open the group policy editor by running gpedit.msc

    In the group policy editor open:

    Computer Configuration > Administrative Templates > Windows Components > BitLocker Drive Encryption > Operating System Drives.

    Then open the entry for Require additional authentication at startup

    This will bring up an editor for this policy where you can enable the option by clicking the Enabled radio button and then on the options panel click the  Allow BitLocker without a compatible TPM  checkbox.

  • Enable bitlocker
  • Smile

    Enjoy your dual-boot, encrypted work-capable machine :)

  • Java–Serial Killer

    Its about time a public mourning took place for people who are allergic to coffee…

    Although I do feel pity for the people who are allergic to coffee (what a horrible existence… i would hate not being able to have the best addiction)  this is not the topic of this post.

    Ahhh so are you taking about the mass deprecation that takes place in Java?

    Although the deprecation of components in Java does annoy me (Java 7 Deprecations, J6 Deps) it is a necessity in most cases  but this is not the choice of topic.

    I get it, you are taking about all the nuclear facilities that had faulty Java control software in place which caused a meltdown or 2!!

    Eh no, the java agreement requires that “You acknowledge that Licensed Software is not designed or intended for use in the design, construction, operation or maintenance of any nuclear facility”

    Im taking about the state of communicating with RS232 devices from Java.

    That’s simple just use java.comm

    Exactly what I thought too but it appears that the write once run anywhere promise of Java has been broken further, back in the day it was possible to write windows Java apps that could use Java.comm but it appears that it is no longer possible to use this functionality outside of Linux, Mac OS X and Solaris.

    I recently discovered this when I developed a web service component that uses a RS232 sensor on linux where it worked fine and I was a happy coder but when I brought it over to windows I was left astonished and searching for a solution.

    RXTX to the rescue

    After doing some searching around I found a beautiful LGPL product called rxtx which provides platform agnostic RS232 support through a native library and jar class file combination.

    The native library goes into the the /bin sub folder of both the Java runtime and Development Kit folders and the .jar file goes into the companion /lib subfolders of those directories.

    The API is easy to use and clear with a fairly logical structure for example to detect COM ports on a system the following code fragment is used

     1: List <String> list = new ArrayList<String>();
     2: Enumeration portList = CommPortIdentifier.getPortIdentifiers();
     3: 
     4: while (portList.hasMoreElements()) {
     5: 
     6:      CommPortIdentifier portId = (CommPortIdentifier) portList.nextElement();
     7: 
     8:      if (portId.getPortType() == CommPortIdentifier.PORT_SERIAL) {
     9: 
     10:           list.add(portId.getName());
     11:      }
     12: }

    The rxtx project lives at  http://rxtx.qbang.org/wiki/index.php/Main_Page where you can get a variety of downloads and a wealth of documentation

    ReFS

    Re-Implemented FS?

    Re-Designed FS?

    Resilient FS?

    Its actually all of the above.

    ReFS is the new file system that Microsoft’s Steven Sinofsky announced on the 16 Jan 2012, unlike previous attempts by Microsoft to develop a new FS (WinFS, an unholy and aborted RMDBS/OODBMS as a FS that sat on NTFS) this represents a much more logical and less insane evolution of file systems reminiscent in some minor ways to the awesome ZFS (which powers my home server incidentally) developed by the group then called SUN microsystems (they dead, eaten by a Grue/Oracle).

    What can it do?

    After reading the MS posts about this system I can see 3 huge advantages to using this FS.

    1) GREATER INTEGRITY

    The Metadata that the OS uses is verified with a 64-bit checksum which is the product of a mathematical operation of a series of data that creates a fairly unique digest of the contents of that data.

    Simply put a checksum is a fairly unique fingerprint of data which should change if as little as a single bit in a data set is modified for example using the SHA1 checksum the phrase “The name is bond james bond” is digested to “47b5b5cb374b6adf5523aff8b45c742cb03cda48”  but “The name is bond James bond” is digested to “a497f3764a972469d89b4d689eeaf71779b8ec7b” this is used to verify that what we had written to a disk is as intended. This is important as all current storage mechanisms are not 100% reliable and can lead to hidden corrupted files in future.

    This would function something like this:

    [Hold data in RAM] >> [Calculate Checksum of Data: CHK0] >> [Write Data to Disk] >> [Read Data from disk and calculate its Checksum:CHK1]  >> [Compare CHK0 with CHK1, if they differ then attempt to write again]

    Check-summing of the Metadata is automatic but this function is optional for the files contents using “integrity streams”, this makes me unhappy as I prefer to have my data verified as well and so my home server will stick to ZFS.

     

    2) SCALEABLE

    Using B+ trees (more info) exclusively this FS does offers scalable storage capacity and can grow while still offering efficient operation times.

    This means that limits on File size,Volume Size, number of files/directories are now only limited by 64-bit numbers meaning that the maximum volume size is 1 Yobibyte (280 bytes) when using 64KB clusters with a max file size being 16 Exbibytes (260 bytes) for perspective a Terabyte is (212 bytes) so we are talking about some limits that shouldn’t affect us for a while (us as a species, ill likely be dead by then… as will you) .

     

     

    3) FOREVER ONLINE

    Ok that’s a little bit of bait if you turn the system off the disk will not be available but if you wish to perform a low level operation (like a disk check) you shouldn’t need to take the volume offline, no more rebooting to fix corruption (which shouldn’t affect ReFS as much anyway, a little bit of irony here)

    Where does this fit in?

    At the core of windows, deep in the Kernel.

    NTFS.SYS = NTFS upper layer API/semantics engine / NTFS on-disk store engine; ReFS.SYS = Upper layer engine inherited from NTFS / New on-disk store engine

    Above is the image representing the change in FS feature produced by Microsoft, in case you haven’t noticed this is as clear as tar and low on details (what is the bump? Shared Features to be relocated?, Implementation Bloat?, A collection of Ugly Hacks?).

    The blue blocks represent API calls and logic for software to consume the features offered by this FS driver, much like NTFS.sys I would think that these calls are not to consumed by end programmers by instead by the OS Kernel and abstracted into programmer API calls (http://technet.microsoft.com/en-us/library/cc781134(WS.10).aspx), if performed correctly this migration should be transparent to application software.

     

    NTFS Architecture

     

    The Red blocks are where all the exciting changes take place so for example a BlueBox command called writeBytes() could be the same in both NTFS and ReFS but the Red block logic for NTFS could simply write the data to a free area with the ReFS implementation being similar to the write function previously proposed in this article which (for the lazy) is:

    [Hold data in RAM] >> [Calculate Checksum of Data: CHK0] >> [Write Data to Disk] >> [Read Data from disk and calculate its Checksum:CHK1] >> [Compare CHK0 with CHK1, if they differ then attempt to write again]

    Then why is the blue box bigger than the redbox?

    The blue box commands all abstract a combination of common red box commands to produce a goal, e.g a red box open a directory command could be combined with a red box list file/directory function to perform a blue box search function.

    What’s the catch?

    Well there are 3

    1) This is untested

    This is a big drawback, although I’m sure MS have tested every aspect of this system with dog-fooding, Fuzzing and Unit tests there is still the risk that some aspect of this FS will not be implemented 100% correctly and will kill your data and eat your children if given the chance!

    If MS want me to trust this they should move all of their internal systems and Source code repos to  ReFS and inform the world what happened 2 years later…. this wont happen so I will wait until all the bugs have been repaired.

    2) Windows server 8 only and Not backwards compatible

    A ReFS volume cannot be accessed by Windows 7/2008R2 or earlier so if your one Win 8 server dies you may have to wait for a reinstall to get essential data off your SAN, if this delays the payroll processing for your company you better blame a virus or there may be an old fashioned angry mob at your door with cacti.

    3) Not Default and non-bootable

    This FS not the default in Windows 8 server and cannot be booted from, this does provide a lot of information about Microsoft’s confidence in their new baby

    More potential issues

    It appears that MS will be deprecating some NTFS features of these 2 features concern me most: EFS and Hard links.

    EFS

    Is the encrypting File system that NTFS has been incorporated into windows since Win 2000 and is used to secure data in corporate and personal environments (NTFS permissions mean nothing if the ‘protected’ disk is accessed with a live Linux disk or another windows install as they are easily overcome).I think this feature may be the bump in the Red box.

    I hope that the companion storage spaces feature or some other user land tool will add support for EFS as this is an essential feature for backwards compatibility that said bitlocker seems to be the way of the future for windows encryption and so this could be the end of EFS (you willl be moving files to the new volume anyway).

    HARD LINKS

    Removing hard links removes POSIX compatibility, this could mean that systems such as CYGWIN may not function any more in a ReFS volume which may be bad for academic workstations that need GCC and other such UNIX tools in windows (this is essential for system modelling with SPIN).

    OTHER REMOVALS

    There are some other features that are being removed but I personally am not worried as user land tools could easily replace these functions ( and compression is less relevant in the time of 3 TB hard disks)

    Q) What semantics or features of NTFS are no longer supported on ReFS?

    The NTFS features we have chosen to not support in ReFS are: named streams, object IDs, short names, compression, file level encryption (EFS), user data transactions, sparse, hard-links, extended attributes, and quotas.

     

    More info:

    http://blogs.msdn.com/b/b8/archive/2012/01/16/building-the-next-generation-file-system-for-windows-refs.aspx