NA-FS, Not Actually a File System
Written 2010-02-24
Tags:Ext4 Linux ZFS filesystem Mass storage Ubuntu
I've worked with several embedded filesystems, from jffs2, to FAT12/16/32, to hard coded tables of filenames and their containing data. What I would like to propose, is not a filesystem, per-se, but an interface.
I understand that no one filesystem fits every problem. If you need to be able to open a file quickly, FAT won't do it. If you need a tiny driver, don't go with Reiser. If you need flash wear-leveling, your choices are pretty limited to start with. If it only exists in RAM, who wants to bother with a filesystem anyways? My point is that the diversity in these systems is necessary.
However, interfacing with so many filesystems is a nightmare. Even Linux drops filesystem support quite regularly. How can we improve this? By offloading the burden of filesystem support to the device itself. By necessity, the device already has a driver, and this method adds new capabilities.
Real World Example One: A media player acting as USB mass storage. After loading new MP3s, a media player needs to open each file and scan it for meta data such as the band, length, and volume profile. If the media player were the storage back-end directly, every time another device sends it a file, it parses the file as part of the storage operation, preventing a lengthy power-on.
Real World Example Two: Utilizing a system like NA-FS, intelligent memory cards could be swapped with any device supporting NA-FS, without actually changing the backing store. For example, if SD was designed with NA-FS as the interface, then this SD/SDHC/SDXC monkey business would have never been an issue. Small cards could use a compact filesystem like vFAT16, and larger devices could use whatever they needed, from exFAT to ZFS-based RAID.
Technical Example: In Linux, there are filesystems like /proc, which are generated at file-access time. NA-FS would support this intrinsically, as actual file i/o is passed down to the device OS.
Technical Example: Compression - as an intelligent storage device, the bytes on disk don't need to match the bytes at the transaction layer. A smart enough device could compress plain-text documents or use PNG compression for uncompressed images.
Downsides:
- Currently, the media controllers in memory devices are quite limited. Often, they only handle flash wear-leveling, and perhaps block-atomicity.
- No current implementation/standard, just a though I had
- How would features not standard across all filesystems be handled? ACLs come to mind.
- Some features, such as automatic image compressions, would be computationally expensive to implement, for a memory card.
Upsides:
- No device would have to enter 'Mass Storage Mode'. Devices could keep functioning normally, and even use NA-FS to transfer information, much like /proc on linux.
- No device would need to support more than two filesystems: NA-FS, and the backing store of choice.
- A filesystem need never worry about a poor fs implementation on a device corrupting it.