On 14 May 2014 15:20, Ewan Slater ewan.slater@googlemail.com wrote:
I'd do it at the file system level. I would ZFS for compression and deduplication.
The final solution is going to end up on a hosted virtual server somewhere; I've not found any yet that offer ZFS, which is why I was looking at something like FUSE to run something on top of the default filesystem (generally ext4).
I have had a quick play with avfs on my Ubuntu desktop and that looks useful - effectively allowing me to just gzip (or bzip2) the files and have them appear uncompressed to the FTP server, although it takes a bit of messing around (the transparently uncompressed version of ~/foo.gz becomes ~/.avfs/home/mark/foo.gz#, but I can create a symlink in the FTP root to make it appear there as "foo").
fusecompress isn't in the repos so I need to work a bit harder on that one.
Mark