MacInTouch Amazon link...

APFS / file systems

Channels
Apple, Security, Other, Products
While we're on the subject of Carbon Copy Cloner and file systems, I suspect the answer is yes, but can someone confirm that HFS+ and APFS volumes can coexist on the same physical (spinning) disk?

I have a new hard drive arriving, and I want to create a 256GB APFS partition to back up my internal SSD of the same size. For the time being, I am inclined to leave the rest of the space as HFS+, if that is possible.
 



I've only done it on SSDs, but I'm sure it will also work on spinning disks (which may suffer more from performance problems with APFS).
Thanks, Ric. Yeah, I wouldn't dare run a system from a spinning APFS volume except as a last resort.

On the other hand, that makes me wonder: my source volume is APFS, but (at least until I upgrade to Catalina) is there any compelling reason that my backup partition shouldn't be HFS+?

Why suffer a performance hit if it's not necessary to do so?
 


Yeah, I wouldn't dare run a system from a spinning APFS volume except as a last resort.
FWIW, I'm running Catalina on a mid-2012 i7 MacBook Pro with 8 GB RAM and its original, 5400 RPM spinning hard drive, formatted as APFS. Obviously, an SSD will provide a much snappier experience, but I have to confess that this system does not feel unacceptably slow at all in normal use. Boot time is a little long, though I'm not sure how much of that is APFS, the spinning drive, or Catalina in general. Once the system boots, Safari and other common apps perform just fine, which is not what I expected.

Apple's engineers deserve credit for doing a very good job of tuning APFS performance.

That said, I will be replacing the drive with an SSD and increasing the RAM to 16 GB as soon as I am finished testing Catalina. It's fun to test on a spinning drive, but I do look forward to SSD performance.
 


While we're on the subject of Carbon Copy Cloner and file systems, I suspect the answer is yes, but can someone confirm that HFS+ and APFS volumes can coexist on the same physical (spinning) disk? I have a new hard drive arriving, and I want to create a 256GB APFS partition to back up my internal SSD of the same size. For the time being, I am inclined to leave the rest of the space as HFS+, if that is possible.
... I have APFS and HFS+ partitions on the same disk, in multiple locations and various mediums; which include rotational, SSD and Fusion. And I stress them significantly.
 


Thanks, Ric. Yeah, I wouldn't dare run a system from a spinning APFS volume except as a last resort.

On the other hand, that makes me wonder: my source volume is APFS, but (at least until I upgrade to Catalina) is there any compelling reason that my backup partition shouldn't be HFS+?

Why suffer a performance hit if it's not necessary to do so?
... I just tried backing up my macOS 10.15 Public Beta 2 test Mac to an external HFS+ drive. and it backed up okay. I didn't test restoring it, but it appears to be okay to still use HFS+ for Time Machine backups (at this time - this is a beta, so things may change at a later date).
 


Is there a limit to the number of snapshots an APFS volume may have? With all this talk about Time Machine and its need to use HFS+ (because of its reliance on hard links), it occurs to me that the entire approach may be completely unnecessary.

For example, Carbon Copy Cloner, when cloning to an APFS destination, implements its "safety net" feature using snapshots. So you've got your most recent backup as the main snapshot and all your historic backups as additional snapshots. No need to make extra copies of modified files in a separate location - the snapshot mechanism takes care of that.

It occurs to me that there's no reason why an APFS-destination version of Time Machine can't do the same thing. Instead of keeping a system of hard links to everything, use snapshots. For each backup, make a snapshot and then write the changed files. Keep a database file somewhere to identify which snapshots correspond to each backup run.

The only potential problem here is if there is a limit to the number of snapshots a volume can have. For example, if I've got a Time Machine volume with 5 years worth of backups, then we're looking at about 300 snapshots (52 * 5 = 260 weekly + 30 daily + 24 hourly = 314). If APFS can handle (for example) 1000 snapshots per volume, then that would be enough to support 18 years worth of snapshots - enough that you would almost certainly fill the device and start deleting old snapshots before hitting the limit - then this system should be viable and would be much easier to manage than the HFS+ (hard-link-based) system being used today.

It probably depends on the size of the internal control structures. For example, if they're using a single byte for each block's reference count (for the copy-on-write mechanism) then that implies an upper limit of 255 snapshots. If they use a 2-byte (16-bit) value, then that implies an upper limit of 64K snapshots, which should be more than sufficient for this task. (Of course, there are other control structures beyond the reference count that would almost certainly come into play here as well.)
 


Is there a limit to the number of snapshots an APFS volume may have? With all this talk about Time Machine and its need to use HFS+ (because of its reliance on hard links), it occurs to me that the entire approach may be completely unnecessary.
What you're describing is probably their intended approach. One of the things they added to APFS was sending deltas. We're probably a point release or one major release away from this. The only restriction I'm aware of for snapshots is disk space.
 


Ric Ford

MacInTouch
It probably depends on the size of the internal control structures. For example, if they're using a single byte for each block's reference count (for the copy-on-write mechanism) then that implies an upper limit of 255 snapshots. If they use a 2-byte (16-bit) value, then that implies an upper limit of 64K snapshots, which should be more than sufficient for this task. (Of course, there are other control structures beyond the reference count that would almost certainly come into play here as well.)
Maybe you could decipher Apple's limited documentation better than I could:
Apple File System Reference said:
https://developer.apple.com/support/apple-file-system/Apple-File-System-Reference.pdf

om_snap_count

The number of snapshots that this object map has.
uint32_t om_snap_count;
 


Maybe you could decipher Apple's limited documentation better than I could:

uint32_t om_snap_count;
"uint32_t" is an unsigned integer, 32-bit wide, type. It holds numbers from 0 to 4,294,967,295.

So more than four billion snapshots can exist, assuming no other data – a snap ID, for example – limits the number of snapshots.

You might run out of disk space before that. Then again, remember "640K is enough RAM" and "four billion IP addresses are enough"?
 



Maybe you could decipher Apple's limited documentation better than I could:
The documentation is using a 32-bit int for the count of snapshots, and later on (page 47), I see
Apple File System Reference said:
Object Map Constants

Constants that specify size constraints of an object map.

#define OMAP_MAX_SNAP_COUNT UINT32_MAX

OMAP_MAX_SNAP_COUNT

The maximum number of snapshots that can be stored in an object map.

#define OMAP_MAX_SNAP_COUNT UINT32_MAX
If this document is accurate, then APFS's limit is over 4 billion snapshots - enough that you'll fill the media long before you run out of snapshots.

Of course, we don't know if the current implementation supports this limit, nor do we know if it will run fast/efficiently on a volume with thousands of snapshots.

It's also interesting to note that the superblock (page 48), uses a 64-bit integer for the total count of snapshots. I assume this is done because all of the values there are 64-bit and for possible future expansion, given the above definition that says the limit is a 32-bit value.
 


That's a little scary. Or does user data take precedence over snapshots?
When you make a snapshot of a volume, the disk blocks for every file on the volume get their "reference count" incremented, so when you delete or modify a file, those original blocks remain in use (and therefore consuming free space) by the snapshot.

When a snapshot is deleted, the disk blocks for every file in the snapshot get their reference count decremented. Those blocks whose count goes to zero are returned to the pool of free blocks.

If used to implement a system like the hypothetical APFS Time Machine I described previously in this thread, you're going to find most files' blocks with very large reference counts, so you'll probably have to delete a lot of snapshots in order to free up a lot of space (which is also the way it works with the hard-link-based system used by today's Time Machine on HFS+).

Another approach to snapshots I've seen (e.g. on network appliance file servers) is that snapshots are created and deleted on a schedule. For example, create an "hourly" snapshot every 4 hours, retaining the last 5 hourlies; a "daily" snapshot every day at midnight, retaining the last 6 dailies; a "weekly" snapshot every Sunday at midnight, retaining the last 4 "weeklies". With a system like this, you find that when a file is deleted, it might take as long as a month before its disk blocks are freed (e.g. if you deleted it soon after a weekly snapshot was created).

This can result in a situation where the volume fills up, and deleting files doesn't seem to do anything. When this happens, you often need to get an administrator to run a utility to manually delete some snapshots ahead of schedule. (Of course, if this happens too often, then it means you really need to buy more storage.)

I believe Apple's "local snapshot" facility automatically deletes snapshots if the amount of free space on a volume gets too low, so you might not need to manually delete snapshots, but there may be situations (e.g. snapshots created by other mechanisms, like Carbon Copy Cloner) where they don't get auto-deleted to make room for new files.
 


I hope there's going to be an app to control APFS snapshots. I want the ability to disable them completely. I want the ability to choose how many snapshots occur, how much disk space they will take, and and a frequency I select. I certainly do not want to be faced with my boot disk filling up at random times and getting an alert that there's not enough disk space to copy data to it or save data on it. I do not want an alert that appears during the copy or at/near the end of the copy. The OS should properly calculate at the onset of the copy or save operation as to whether there's adequate free disk space.

I can only imagine the millions of files that APFS operations are going to generate on a boot disk. I sincerely hope that utilities like DiskWarrior will be updated to deal with the inevitable errors that will crop up, due to the complexity of the file system and random acts of sector failures on storage media.

Again, this comes down to me wanting Apple (or a 3rd-party developer) to give users the choice, the option and the control over customizing their computers as they see fit.
 


I hope there's going to be an app to control APFS snapshots. I want the ability to disable them completely. I want the ability to choose how many snapshots occur, how much disk space they will take, and and a frequency I select. I certainly do not want to be faced with my boot disk filling up at random times and getting an alert that there's not enough disk space to copy data to it or save data on it. I do not want an alert that appears during the copy or at/near the end of the copy. The OS should properly calculate at the onset of the copy or save operation as to whether there's adequate free disk space.
Snapshots are not something that are spontaneously created. Some app needs to create them. This may be Time Machine's local backup mechanism, an application you install (like Carbon Copy Cloner) or you might create them yourself - all of which should be configurable (including disabled).

You can manually manage snapshots. Apple's tmutil command will let you create, list and delete them. Carbon Copy Cloner includes a GUI tool for this. I don't think Apple provides a GUI tool, but they should.

With respect to space filling up, snapshots don't spontaneously create files. They may hold on to the disk blocks that are already in use by existing files, but that won't cause all your space to disappear while you're in the middle of something. At least no more than would be created without snapshots.

Ditto for calculating free space. The amount of free space on a volume does not go down when a snapshot is created. It remains exactly the same. A snapshot may prevent free space from increasing as a result of deleting files, but that's not the same thing. If an operation computes the amount of free space, that space will remain free before during and after creating a snapshot. Of course, other processes unrelated to snapshots may write files, but that is nothing new.

As for third-party tools for fixing disk errors, yes, we all want that. I'm sure all the major utility vendors are working on it, but it's hard to do when Apple doesn't provide the low-level documentation necessary for such a tool to get it right.
 


Ric Ford

MacInTouch
There's some very useful information at the Bombich Software (Carbon Copy Cloner) web site about APFS and backups. I'll quote a few tidbits below, but there's much more.
Bombich Software said:
Everything you need to know about Carbon Copy Cloner and APFS

... You also cannot boot a T2 Mac from an HFS+ encrypted volume, so if you have a T2 Mac and encryption of the backup is required, you must choose APFS.

... The macOS installer applies a firmware upgrade to your Mac when you install the macOS upgrade. This firmware upgrade cannot be made part of the cloning process. Only the macOS Installer can upgrade a Macintosh to support APFS. If you attempt to clone an APFS volume to a Macintosh that has not yet received the firmware upgrade from the macOS Installer, that Macintosh will not be able to boot from the APFS volume. Once your Mac has received the firmware upgrade via the macOS Installer, your Mac can boot from a CCC bootable backup on an APFS volume. Note, however, that every major MacOS upgrade may require a new firmware upgrade to allow use of the newer operating system.

Note that this is also applicable to a Macintosh running in Target Disk Mode. If you upgrade one Mac to High Sierra (or later) via the Installer, you cannot boot a second Mac into Target Disk Mode, attach it to the first, then clone High Sierra (or later) to the Mac in Target Disk Mode. The required firmware upgrade cannot be applied to the Mac that is booted in Target Disk Mode, you must run the macOS Installer on that second Mac. Once the second Mac has received the firmware upgrade via the macOS Installer, you can clone the first Mac to the second Mac booted in Target Disk Mode.
 


Ric Ford

MacInTouch
Howard Oakley writes about confusions involved with various RTF file formats and macOS differences vs. other operating systems:
Eclectic Light Co. said:
Rich Text documents: RTF and RTFD

... There’s an immediate problem here: aside from macOS and NeXTSTEP, other operating systems such as Windows and Linux don’t support directories acting as packages. Furthermore, regular RTF was a part of Microsoft Word, whereas the upstart RTFD wasn’t. So ever since then, like two misbehaving children, RTF and RTFD have lived their own separate lives, and getting RTFD to work in Windows has been fairly futile, just as getting Apple to support richer RTF has also failed.

... You’ll sometimes see RTFD packages referred to as ‘bundles’. They’re not: a bundle has quite a tightly prescribed and more complex structure, like an app or other code bundle, and within it is a Property List file named Info.plist. By comparison, RTFD packages are flat and far simpler. You still have to be careful though if you try to manipulate them directly.
 



I need some advice, please, on APFS and free space under a new High Sierra installation, because some files I'm deleting are not freeing space. I thought APFS would be great on an SSD — now I'm worried.

I successfully updated a 2010 Mac Pro from Sierra to High Sierra with a macOS 10.13.6 installer, and it converted my 256GB SSD to APFS (OWC Mercury Extreme Pro SSD, the latest 2015 firmware, TRIM enabled). Before the update, I booted to the shell (via Command-S) and ran fsck -fy It didn't find anything to fix on the drive.

After the update and a few restarts, with 70GB of free space, I copied 35 GB from a different drive, but the Mac froze before the file was finished copying (this was true free space, not purgeable, and SMART reports no problems with the drives). After powering down and booting, I was able to delete the partially copied file from the Finder, and its free space returned, but I thought I should recheck the boot volume.

Booting into recovery mode, Disk Utility would not check the boot volume, because part of it was mounted. (The "Base System" volume is still Mac OS Extended if that's a problem.) So, I booted to the shell tried fsck -fy and it said -f wasn't supported, so I tried fsck -y — nothing appeared wrong, and I rebooted. Now, I have 20GB less space after running that command. Should I avoid fsck in the future, or maybe it's just coincidence?

After rebooting into macOS, I ran Disk Utility, and it found no problems on the boot drive. Next, using Pathfinder 7.6.2 (not version 8), I deleted a 15GB file to free space, and that 15GB has not returned. It makes no sense.

My Time Machine is turned off, and I haven't manually used it for a few days. Using the command tmutil listlocalsnapshots / which I guess is specific to APFS, I see one snapshot, "com.apple.TimeMachine...", dated before deleting those files and after the big update (and only know that because Carbon Copy Cloner listed it, and, no, I'm not using CCC's auto-snapshot creation).

Any ideas on how to reclaim my gigs of space? Clearing all caches with Cocktail didn't help. Apart from having lost 30 GB of space, which doesn't seem great, it appears to be working okay and no more freezes so far.

Should I erase, clean install and force it to HFS, assuming that's safer than APFS? Would the OWC Mercury Extreme 1TB on their Accelsior S card be a good choice for that? Thanks for any help.
 


Ric Ford

MacInTouch
I need some advice, please, on APFS and free space under a new High Sierra installation, because some files I'm deleting are not freeing space. ...
This may not answer all your questions but is probably worth reviewing:
Eclectic Light Co. said:
As far as I know, HFS+ should work OK with High Sierra, though you may need to clone it (e.g with Carbon Copy Cloner or SuperDuper) from an APFS volume to a volume you're pre-formatted/erased as HFS+, because Apple's installer auto-converts HFS+ SSD volumes to APFS during installation.

But I would caution against using an SSD with very little free space in any format, because I think it could contribute to extra wear and tear (even more so with some OWC SSDs that lack Trim support), and I'd try to keep quite a bit of free space available on an SSD, more than you might with a hard drive because of the unique operations involved in SSDs (Trim, garbage collection et al).

APFS, though, is optimized for SSDs, so if you can conquer the free-space "quantum" problem, it may be a better bet than HFS+ for an SSD-based system, and it does have the advantage of supporting snapshots for better recovery options, as well as a higher level of security.

You might want to review other notes in this topic thread, too, starting with some links in the first post.
 


For what it is worth, Dave Hamilton and John F Braun of the Mac Geek Gab podcast have talked about the fact that they have observed an increased number of problems with APFS installations when they are the result of an HFS+ to APFS conversion.

I believe that their recommendation is that you're generally safer either sticking with HFS+ all the way through or else starting with a clean APFS volume (formatted before the install).

It seems likely that the large files you deleted are in that Time Machine snapshot, given the timing that you stated (after big update, before the deletes). If so, it is logical that they are still taking up space in the snapshot, and you won't get that space back until/unless you delete the snapshot. You can find instructions here from MacWorld on how to delete snapshots manually using the Terminal:
Glenn Fleishman said:
How to delete Time Machine snapshots on your Mac
These snapshots can fill up a drive, even though macOS should manage them.
 


Ric Ford

MacInTouch
Howard Oakley has been wrestling with APFS, and it isn't pretty.
Eclectic Light Co. said:
APFS tools suck
APFS is now over two years old, its official release having been in March 2017 with iOS 10.3. Look at its supporting tools today, though, and you could be forgiven for thinking that it’s still in beta.

A couple of days ago, I looked at its limitations in terms of minimum size of containers and volumes, mainly using Disk Utility, and came away completely confused. I resolved to return and this time tackle the problem at the command line, using the all-powerful diskutil. This article is a report on the complications that result from trying to manage APFS using diskutil....
 


I still find no clarity in APFS. The one example that Apple provided (to us non-techies) about improved speed used, as an example, duplicating a file. Certainly, creating a reference to the file would be faster than actually duplicating it, but, if I'm not misconstruing this, such a duplicate, if modified, creates only files containing the changes.

Each time I change the file, another file containing the newest change is created, so changing the file 10 times created 10 new files, plus the reference to the old one. Copy this file (the duplicated/changed one you see in the Finder) to another volume, and macOS will actually create the one real file containing all the changes, using the 11 files. It is only then that the changed file truly exists complete.

Of course, one might argue that HFS+ is already engaged in this sort of practice, as files don't necessarily exist in a contiguous fashion, so reading/writing and, finally, copying to another volume requires a similar sort of Oz-behind-the-curtain magic.

Of course, if the original file (referenced by APFS) dies by bit-rot or whatever, there really is no duplicate, so all you're left with are multiple change files, and you're screwed. (Well, maybe you were smart and did your backups, so you may find that original file and drop it into place. But will that file automatically take the place of the (bad) original file in APFS's catalog?)

Just thought of another thing: If I "Save as..." a new file from within an app, will I have a complete file? I would think that non-Apple apps would make it so, but what about Apple's apps? Do Mojave's (or Catalina's) TextEdit, Pages, etc. engage in this APFS chicanery?
 


Of course, if the original file (referenced by APFS) dies by bit-rot or whatever, there really is no duplicate, so all you're left with are multiple change files, and you're screwed. (Well, maybe you were smart and did your backups, so you may find that original file and drop it into place. But will that file automatically take the place of the (bad) original file in APFS's catalog?)
Yes, when you copy from your backup volume to your main APFS volume, it will be a new file to that volume, not duplicated or modified from the original file, and so it will be completely separate from the original file. It doesn't "take the place" of the original file. The original file will still be there, unless you delete it, or "replace" it by placing the new file into the same directory as the original file and replacing. When I said "duplicated" before, that is only via the Duplicate command in the Finder.
Just thought of another thing: If I "Save as..." a new file from within an app, will I have a complete file? I would think that non-Apple apps would make it so, but what about Apple's apps? Do Mojave's (or Catalina's) TextEdit, Pages, etc. engage in this APFS chicanery?
"Save as" creates a new file, independent from the original, even in Apple applications.
 


... if I'm not misconstruing this, such a duplicate, if modified, creates only files containing the changes. Each time I change the file, another file containing the newest change is created, so changing the file 10 times created 10 new files, plus the reference to the old one.
To be technically precise, the above isn't quite right. Each of the files in your example is complete in itself - it's not that you'd need all 11 of them to exist just to read one. Instead, it works like this:

A 'file' is essentially a bunch of fixed-size chunks of data stored on disk (call them "disk blocks") plus a directory data structure that points to all of the file's data blocks, in order. So the directory might say "to read this file, read blocks 147, 148, 149, 342, 517, and 1031, then you're done."

When a file is duplicated, the duplicate has a new directory, but it points to all of the same disk blocks.

If the duplicate is then modified, only the disk blocks that need to be updated to store the changed data are replaced. So, for example, if I copied the file from above, and then modified the very first bit of the file, the new file's directory might say "to read this file, read blocks 2911, 148, 149, 342, 517, and 1031, then you're done". Block 147 has been replaced by block 2911, but all the other disk blocks are the same for both files.

This is why having 10 different versions of a file doesn't take up 10X the space that having one version does, and it's also why sometimes deleting a file doesn't actually free up any space - all of the disk blocks are also being used by other files, so they're still in use, even though the one file was deleted.

It's also why APFS is really best off when running on SSDs. As you can imagine, after a file has been duplicated and modified several times, the disk blocks aren't very likely to be in a long contiguous order. The example shows this happening - the original file had 147, 148, and 149 in order, but the modified file lost this.

Having to read blocks out of order makes no real difference for SSDs, but it slows mechanical disks down a great deal, because they have to mechanically move the disk head to the new block. The current version of APFS has a 'defragmenting' algorithm that attempts to reorganize files in the background to improve the overall ordering properties, but it's complicated, because there's no necessarily "best" answer — much easier to just not worry about it, which works great with SSDs.
Just thought of another thing: If I "Save as..." a new file from within an app, will I have a complete file? I would think that non-Apple apps would make it so, but what about Apple's apps? Do Mojave's (or Catalina's) TextEdit, Pages, etc. engage in this APFS chicanery?
As Todd says, this is all below the level of applications. Applications see 'files', plain and simple. It's the filesystem implementation that does all the copy-on-write magic - normal applications don't know anything about it at all.
 


Yes, when you copy from your backup volume to your main APFS volume, it will be a new file to that volume, not duplicated or modified from the original file, and so it will be completely separate from the original file. It doesn't "take the place" of the original file. The original file will still be there, unless you delete it, or "replace" it by placing the new file into the same directory as the original file and replacing. When I said "duplicated" before, that is only via the Duplicate command in the Finder.
"Save as" creates a new file, independent from the original, even in Apple applications.
... Based upon your reply, it appears that a defective original file from which an APFS duplicate has been made (said duplicate being only a reference to the original) can't actually be replaced by copying it from a Time Machine backup, because at that point, Time Machine has assembled the pieces (reference plus changes) into a real, complete file for the purposes of the backup. However, if I'm thinking about this correctly, that copied-back file is the complete duplicate-plus-changes, so at that point, all the changes (which are scattered about the source volume) have become irrelevant (as the replaced file is now the complete file), so macOS and APFS should free up that space where the changes had been stored.

... At least, I hope this is the way it works.
 


Ric Ford

MacInTouch
A little more perspective on APFS from Howard Oakley:
Eclectic Light Co. said:
Why isn’t APFS fully supported yet in macOS?
... In a typically bold move, Apple introduced APFS to all iOS devices which were upgraded to iOS 10.3 at the end of March in 2017, and to macOS in High Sierra which followed six months later. Although overtly successful rollouts, there were significant shortcomings in the handling of Unicode normalisation in names, and at the last minute Apple had to remove support for hard disks and its own Fusion Drives.

Tracking build numbers of APFS revealed that it remained very much a work in progress until its release nearly a year ago in Mojave, when it finally did achieve all its major objectives. It now copes well with a full range of Unicode names, hard disks and Fusion Drives, although many users remain unconvinced that it’s the file system of choice on those media.

Support tools for working with APFS have been considerably less impressive, though, and access through developer interfaces or documentation essentially non-existent.
 


Ric Ford

MacInTouch
And then there's Time Machine...
Eclectic Light Co. said:
Are we ready for Time Machine 2.0 yet?
Speculation was rife among users, rather than developers, that last June’s WWDC might have brought announcement of a major update to Time Machine coming in Catalina. Now that all Macs running Mojave have to boot from APFS, and with Catalina’s additional conjuring trick of a read-only APFS system volume, for many systems now the only drives still using HFS+ are those for Time Machine’s backups. So why hasn’t Apple fixed Time Machine so that it can back up to APFS yet?
 


A little more perspective on APFS from Howard Oakley:
[But] APFS is not just a file system; it is also a volume management system. Similar to ZFS, these are mingled together and work "smoother" when you let it manage whole storage devices. They do more than more than just put a directory structure on files — they also manage storage pools (containers). APFS is replacing both HFS+ and CoreStorage.
Howard Oakley said:
Why isn’t APFS fully supported yet in macOS?
... This week I highlighted the fact that, whilst the handling of APFS volumes is well catered for, there is significantly more limited support for APFS containers, which are partitions of the GUID disk.
Things get muddled and detached when trying to equate APFS containers to partitions. An APFS Container is more a replacement for partitioning than partitions themselves.... APFS Containers go into a GUID partition, because... if you want to boot into Windows, Linux, or now-legacy macOS on HFS+ on the same storage drive, then APFS isn't going to manage the whole capacity. APFS Containers are in a partition for the same reason other heterogeneous file systems are in a partition.

Containers are about space sharing, not hard partitions. For example, Apple's KB document on the recommended way for Installing macOS on a separate APFS volume doesn't involve creating a new partition at all. Creating a new partition is the more complex and baroque way of doing the install with the same amount of room on the disk, so it shouldn't be much of a surprise that Apple doesn't have easier tooling in support of doing it the harder way. Shrinking containers only makes them less flexible to use (e.g. decreases space sharing ability). Maximizing the utility of a Volume Management system involves making it cover a larger capacity, not the smallest possible one.

If you don't like that newly installed macOS version, delete the volume. Done. Again, no partition incantations or rejiggering required.

APFS is still growing/evolving so there are some corner cases for developers and specialized folks leveraging archivist versions of macOS to create different containers implemented with different versions of APFS. At the general end-user level though, why would you want to be on an older, less developed version of APFS...?
 


Eclectic Light Co. said:
Are we ready for Time Machine 2.0 yet?
Now that all Macs running Mojave have to boot from APFS...
[But I think] if you use something like Carbon Copy Cloner to clone an APFS internal drive to an external HFS+ drive, it will clone and be bootable. I just booted from such a clone a few minutes ago to double-check this. You may need APFS to install Mojave on an SSD inside your computer, but once there you can copy it to an external and boot from it with no problem.
 


Ric Ford

MacInTouch
... if you use something like Carbon Copy Cloner to clone an APFS internal drive to an external HFS+ drive, it will clone and be bootable
I'm going to take a wild guess that you haven't tried this with macOS 10.14 on a T2-based Mac, such as a 2018 MacBook Pro, a 2018 Mac Mini, or any of the others. If you have, please let us know the details.

In any case, you cannot update macOS 10.14 unless it's on an APFS volume, which I would consider a significant issue, even if you could boot from HFS+ by jumping through these hoops (as you can on a Mac that can run macOS 10.13).
 


I'm going to take a wild guess that you haven't tried this with macOS 10.14 on a T2-based Mac, such as a 2018 MacBook Pro, a 2018 Mac Mini, or any of the others. If you have, please let us know the details.
In any case, you cannot update macOS 10.14 unless it's on an APFS volume, which I would consider a significant issue, even if you could boot from HFS+ by jumping through these hoops (as you can on a Mac that can run macOS 10.13).
No... my iMac is not a T2 iMac, so that's an issue I have not run into. On my 2017 iMac, booting from an external Mojave drive using HFS+ is no problem. That's about all I know. :)
 


[But I think] if you use something like Carbon Copy Cloner to clone an APFS internal drive to an external HFS+ drive, it will clone and be bootable. I just booted from such a clone a few minutes ago to double-check this. You may need APFS to install Mojave on an SSD inside your computer, but once there you can copy it to an external and boot from it with no problem.
This is correct but misses the coming change for Catalina - Carbon Copy Cloner (CCC) will do magic and only produce an APFS bootable backup.

Bombich sayeth, in part, regarding this, in the beta Carbon Copy Cloner Release Notes,
  • CCC will make bootable backups of macOS Catalina startup volumes. For most people, that's all you need to know, and you don't have to make any changes to your current tasks to accommodate the upgrade. The logistics of booting macOS are a bit more complicated in macOS Catalina, but we've risen to the challenge, CCC supports it 100%, and nearly all of these complications are dealt with automatically.
  • macOS Catalina requires APFS, it cannot be backed up to a volume formatted with Apple's legacy HFS+ format. When cloning a macOS Catalina system volume, CCC will inform you of this requirement and request your permission to allow conversion of an HFS+ formatted destination to APFS. When you proceed with the task, CCC will automatically convert the destination to APFS (when possible).
My experience with Catalina Beta 5 and the CCC beta is that it works correctly even when the backup target volume is one of several HFS+ volumes on a backup drive. This means you can continue to place several clone backups on one physical drive. Clones of HFS+ data volumes may remain HFS+. Bootable clones will still be bootable, though changed to APFS for Catalina, etc. (even running CCC on Catalina).
 


I'm going to take a wild guess that you haven't tried this with macOS 10.14 on a T2-based Mac, such as a 2018 MacBook Pro, a 2018 Mac Mini, or any of the others. If you have, please let us know the details.
That may not be a T2-specific thing. Any new Mac model that requires Mojave or later never was intended to boot off of HFS+, so Apple would not necessarily include a HFS+ driver in the firmware. Once macOS is booted, it can have HFS+ drivers, but the boot operating system, EFI, doesn't have to do what the full operating system can. It only needs to read enough to do the handoff. However, if it doesn't hand off to HFS+, then no need to read HFS+.

The T2 also can't boot from an external HFS+ encrypted volume, but I think Apple left some wiggle room in the firmware when the operating system was supported on two file systems across the product line. But once all systems only supported an APFS OS, I wouldn't expect the unsupported mode to appear in future model firmware.

The T2 has a role in that you won't be able to hack the firmware to put something back that Apple took out.
In any case, you cannot update macOS 10.14 unless it's on an APFS volume, which I would consider a significant issue, even if you could boot from HFS+ by jumping through these hoops (as you can on a Mac that can run macOS 10.13).
I suspect this "clone to" HFS+ was in part to support CCC's backup versioning feature (archiving some older copies of files in the clone).... Pragmatically, though... as another response pointed out, this is a 'dead end' feature. macOS 10.15 requires APFS (because all the Macs in 2019 are gong to deploy that way). In addition to the release notes on the subject, there is another note with more detail. APFS is evolving. It isn't just APFS that 10.15 requires; it is the new version of APFS.
Bombich Software said:
Working with APFS Volume Groups
... In macOS Catalina, Apple introduced another new concept to the APFS filesystem: volume groups. This is more of a conceptual grouping of volumes within an APFS container, not a new sub-structure. Apple also greatly expanded the number of roles available for APFS volumes (now there are 16 unique roles). When you upgrade to Catalina, your current macOS system volume is renamed, e.g. to "Macintosh HD - Data", its role is set to Data, and then a new volume is added to your startup disk's APFS container with the System role and simultaneously grouped with the Data volume. The two volumes within that group share special bonds and receive special treatment from the Finder and from each volume's filesystem. From the user perspective, these two volumes are treated as a single, unified volume. If you take a look at Disk Utility, however, you'll see the two volumes as distinct, separate items.

The Read-only System volume
Perhaps the single, largest change in macOS Catalina is the manner in which the System volume is mounted on startup – it's read-only. By mounting the volume read-only, it becomes impossible for attackers to make changes to the content of the macOS System volume. That doesn't mean that your Mac is 100% free from all possible attack vectors, rather it's just another line of defense against them.

The Data volume
You can think of the Data volume as a read-write "shadow" of the System volume. The Data volume contains all of your user data (e.g. your home folder, third-party applications), but also contains a handful of system components that can't reside on a read-only volume. For example, Apple has placed Safari on the Data volume, perhaps so it can be updated more frequently. The current startup disk's Data volume is mounted at a special mountpoint on the system. You can find it if you navigate in the Finder to Macintosh HD > System > Volumes > {Data volume name}. What you'll find there is a replica of the System volume's root-level folders. Within these folders are all of the system components that are still writable. Normally you won't see these items in the Finder, though, because the Finder visually mashes the content of the two volumes together to make them appear as a single volume. Also, the Finder won't list your Data volume alongside all of your other volumes – the Data volume is mounted but hidden.
Being bootable requires the new volume layout and duplicating multiple volumes in a APFS Container.
 


[But] APFS is not just a file system; it is also a volume management system. Similar to ZFS, these are mingled together and work "smoother" when you let it manage whole storage devices. They do more than more than just put a directory structure on files — they also manage storage pools (containers). APFS is replacing both HFS+ and CoreStorage.
Things get muddled and detached when trying to equate APFS containers to partitions. An APFS Container is more a replacement for partitioning than partitions themselves.... APFS Containers go into a GUID partition, because... if you want to boot into Windows, Linux, or now-legacy macOS on HFS+ on the same storage drive, then APFS isn't going to manage the whole capacity. APFS Containers are in a partition for the same reason other heterogeneous file systems are in a partition.

Containers are about space sharing, not hard partitions. For example, Apple's KB document on the recommended way for Installing macOS on a separate APFS volume doesn't involve creating a new partition at all. Creating a new partition is the more complex and baroque way of doing the install with the same amount of room on the disk, so it shouldn't be much of a surprise that Apple doesn't have easier tooling in support of doing it the harder way. Shrinking containers only makes them less flexible to use (e.g. decreases space sharing ability). Maximizing the utility of a Volume Management system involves making it cover a larger capacity, not the smallest possible one.

If you don't like that newly installed macOS version, delete the volume. Done. Again, no partition incantations or rejiggering required.

APFS is still growing/evolving so there are some corner cases for developers and specialized folks leveraging archivist versions of macOS to create different containers implemented with different versions of APFS. At the general end-user level though, why would you want to be on an older, less developed version of APFS...?
I did a test trying multiple volumes inside a single container on macOS 10.14.6 in a 2018 Mac Mini. If I remember correctly, one difference from two partitions was that, when you used First Aid from Disk Utility on one APFS volume, it locked both volumes while doing First Aid. (I'll try to find my test USB SSD to see if I'm remembering correctly.)
 


In any case, you cannot update macOS 10.14 unless it's on an APFS volume, which I would consider a significant issue, even if you could boot from HFS+ by jumping through these hoops (as you can on a Mac that can run macOS 10.13).
This may or may not be a problem, depending on the reason you made the clone.

If you are simply needing a bootable backup, so you can wipe/replace your internal volume, boot the backup and then clone it back to that (APFS-formatted) device, then it doesn't matter if you can install updates.

If, on the other hand, you want to actually run your system from that HFS+ formatted clone, then, yes, that could be a nasty problem.
 


Any new Mac model that requires Mojave or later never was intended to boot off of HFS+, so Apple would not necessarily include a HFS+ driver in the firmware.
What about bootable install media? To my knowledge, bootable installers only work with HFS+.

To test this, I used my MacBook Air Mid-2013 running Mojave 10.14.6 with the Install macOS Mojave app in /Applications also at 10.14.6. I used Disk Utility to erase an SD card to GUID and APFS. When I ran Install macOS Mojave's createinstallmedia, it aborted, saying "APFS disks may not be used as bootable install media."

(Is it possible that createinstallmedia recognizes my older hardware and won't allow APFS? Can someone with a new Mac test this?)
 


Howard Oakley has been wrestling with APFS, and it isn't pretty.
I use a fair number of disk images of all formats, basically as replacements for large folders; but encrypted sparseimages are very important to me for storing sensitive data. Are there any gotchas we need to look out for with disk images and the APFS volume/container dance?
 


Ric Ford

MacInTouch


I use a fair number of disk images of all formats, basically as replacements for large folders; but encrypted sparseimages are very important to me for storing sensitive data. Are there any gotchas we need to look out for with disk images and the APFS volume/container dance?
As of now, there is no reason to migrate user data objects, including encrypted sparseimages, away from HFS+. Through beta 6, Catalina seems to work with data objects on HFS+ just as well as previous macOS versions. Caveat: I don't use FileVault for data-only volumes.
 


Amazon disclaimer:
As an Amazon Associate I earn from qualifying purchases.

Latest posts