I have previously had the misfortune of having formatted an external drive to a wrong format, so that all my ardour projects became un-openable after a linux distro-hop.
I really want this to not happen again, so what is the format I should choose to remformat my external disk to before backing up my projects?
Do you need to have cross-platform access? If you only ever use linux the answer may be different than if you need to move between linux and windows, linux and mac, or linux, windows, and mac.
If you want all three, then exFat is your choice. If Linux only, any number of options are good possibilities, ext4 probably being the simplest, but ask half a dozen people (That know and understand the topic) and you would get a half dozen answers as to what is âbestâ on Linux these days.
plus youâd get a dozen demonstrations at why theyâre telling you there are not much more than one or two people that understand and know the topic
I agree with Seablade, use exFat if you are absolutely sure youâll need quite often (say one medium project per month or one big per year) to cross worlds, or play it cool and easy and use whatever ext4 you prefer and in case of a project needing to cross platform ask the cageyOSes admins to use their best compatibility wondrful tools to read/write ext2/4 filesys or their favorite rosita embedment (note that the good ones will actually do it easily plus being happy to do it )
The problem with all the FAT derivative filesystems is they do not guarantee case sensitive filenames, and I think Ardour needs case preserved.
According to this post by x42 that is the case (ha ha, get it? case?): Ardour sessions are case sensitive
The recommendation from that thread was to work with Ardour sessions on the local disk, using whatever case sensitive filesystem was available (i.e. any Linux or Unix native filesystem, NTFS on Windows) and then zip the project directory to store on an external disk which was FAT32.
The Wikipedia filesystem comparison chart says exFAT is not case sensitive, but is case preserving, so a little bit of a grey area. FAT32 was technically case preserving with an extension, but it seems to have caused problems in the past, so presumably that extension was not universally used.
I think in practice NTFS is practically case preserving, case insensitive (one of those weird Windows caveats, technically the filesystem is case sensitive, but the Windows APIs you use to access normally on Windows are not case sensitive), so maybe exFAT will work fine. Definitely give it some stress testing before you rely on it.
So it is a bit more complicated with ExFat here. The filesystem IS case-insensitive, but filenames can have case. What happens, at least according to a brief search, ais that when searching for files as part of the filesystem operation, the file name searched for is converted to upper case. What this means is that while the file system is case insensitive, since the conversion happens at the filesystem level, Ardour is unaware of this, and it appears that it works fine.
The practical end result, is that so long as all filenames are unique, the session works fine. I did just test this at a VERY basic level by creating a new session and recording a file on my HFS+ case sensitive drive, then copied the session to ExFAT, confirmed it was working, and finally renamed the audio file in the interchange folder, only changing the case of one letter, and the session remained working. Now if I copied the session back to a case sensitive file system AFTER making that change, it would obviously not work, but if I copied it back before then, the case is preserved and the session works.
This is different from say FAT32, which is also case insensitive, but does not recognize case at all in filenames, and what tends to happen is that when copying files, the case gets changed to upper case, so when copying back to a case sensitive filesystem, you have effectively changed the name of the file. That is why FAT32 doesnât work great for an interchange filesystem.
So end result, it works much like your description of NTFS.
And end result of all this is that it tends to work well enough for interoperability functions honestly.
For the record I think it would be more accurate to say it was RARELY used, considering the amount of embedded devices that used FAT32 that didnât bother:)
donât know what happened, but as stated thatâs basically impossible⌠unless you reverted to a much older distro, or a very âcrippledâ one, any Linux distro should be able to access any of the supported file systems (well, unless youâve used a non-native file system, such as NTFS or exFAT, which may be not supported by default by every Linux distro).
if it is for Linux only, neither one. Use a native file system such as ext2 or (better) ext3.
(Iâd rather avoid ext4 instead, as itâs not supported by older Linux systems and many 3rd party add-ons for other platforms).
If you need compatibility with every other system, AFAIK FAT32 is the only one which (to date) is well supported by all platforms, including older ones.
But⌠it does have a lot of limitations (max volume size, max file size, allowed characters and length of file/dir names, etc).
exFAT has less limitations, but it is also less generally supported (also by Linux itselfâŚ). Same goes for NTFS.
ExFAT is a MUCH better choice these days. It is in general supported by Linux and is much easier to deal with than the file issues mentioned abvoe you will deal with in FAT32.
This is true, but I have never had good USB performance with a fat filesystem. If youâre saving a few small things, itâs fine. But thinking about a backup⌠brings tears to my eyes. I stick with ext2. If you need to transfer things to another machine, have a separate fat usb for that. With ext2, if your system dies, you can fire up a live Linux session, plug in your drive and do whatever. I would separate the utility of a âbackupâ drive from a âinter-system transferâ drive. Just my 2 cents.
Speaking of USB, Iâve just tested this with exFAT on a USB thumb drive.
Iâve copied an Arodur session on it, and it loads fine (using Linux).
Edit: I have initially used mkfs.exfat on Linux. The thumb-drive was not readable on macOS (Ventura 13.3.1). I then used Appleâs Disk Utility to create an exFAT system on the thumb-drive. And that worked on both macOS and Linux.
Basically when Windows formats a large-capacity drive in exFAT, it will default to formatting it using a 1024k allocation unit size - this is incompatible with OSX that requires a 128k allocation unit size for exFAT. "
Except in my case it was just a 32GB thumbdrive, not really âlarge capacityâ.
As I said, it is indeed somewhat better than FAT32 (at least, it does not have its severe limitations), but support for it is not as âuniversalâ as it is for FAT32. Do not forget that (just like NTFS) exFAT was born as a fully proprietary file system, and MS have not released its specs until the end of August, 2019. Previously exFAT on Linux was somehow supported (since 2009) only via FUSE (which implies a severe performance penalty per se), and early drivers were based solely on reverse-engineering. Native support have been added only since Kernel 5.4. Latest distros are likely to provide native support for it, but even slightly older ones may not. E.g., IIRC, even Ubuntu 20.04 LTS (and derivatives) supported it only via FUSE.
MacOS supports it only since âSnow Leopardâ, 10.6.5.
So yes, (as I said myself) today exFAT may be a viable option⌠but only if youâre gonna need to access it only from recent versions of the most common platforms. If instead you need to be able to access your files from really any system, including old or âexoticâ ones, than FAT32 still has better chances to be supported anywhere. This is what I meant.
OTOH, if he only need to access that disk from Linux, than a native file system such as ext3 is always a much better choice.
My point is that support for it IS fairly universal at this point (Caveat being what Robin pointed out).
You are correct, it was not released publicly until 2019, however exFAT support existed well before that, even on Linux via as you mentioned a Fuse driver.
Correct but the Fuse solution worked fine in practicality (As i was using it quite a ways back), and is the same way NTFS would be supported for instance for most people. Just because it isnât supported in the kernel doesnât mean it didnât work fine for Linux.
You do realize that is well over a decade old right?
I donât consider a decade old ârecentâ especially when talking about AV production personally.
That was just a general consideration, for completeness, since he was talking about compatibility. Which I generally intend as the possibility to access the files on the external storage from âanyâ machine (not necessarily one where youâre actually gonna do any work on the files).
Actually, for an USB disk which is used for actual work / âproductionâ rather than just for âstaticâ file storage / transfer(*), personally I would not use anything but ext3.
Ext3 is the most âresilientâ file system in case of accidents such as sudden disconnections / power off without proper un-mounting, even among other journaled file systems (let alone the non journaled ones), itâs basically free from fragmentation problems, etc.
(on fixed disks Iâd rather use ext4, which is somewhat faster and more efficient, though a bit less resilient than ext3, or possibly other native *nix file systems, depending on the specific needs).
(*) a removable disk (USB or whatever) is not the best option per se for work/production uses, and not only for audio/video⌠but I understand that sometimes that can be the only viable one.