When I have a large set of files, I find compressing them with 7-zip with a larger "solid block size" is much more efficient than compressing them individually. This works because it takes advantage of patterns that exist across the whole set of files (particularly with backups of multiple revisions of files).

Think of how many copies of Windows are installed out there. Online backup services might be storing many, many copies of the same stuff.

Proposed is a cloud backup/file storage service in which all users' files are treated as a solid block.

The only caveat I can imagine is that the entire block would have to be deduplicated on each write. I suppose they could be committed to disk and compressed later.
-- kevinthenerd, Aug 15 2013

Or you could just pay a subscription fee per program installed and add your name to a roster of approved users.
-- RayfordSteele, Aug 15 2013


It's not just programs that are deduplicated, though. With sufficient computing resources, you could deduplicate the recipe collections of two old ladies or of an entire family's photo album (since it's likely they have shared pictures).
-- kevinthenerd, Aug 15 2013



random, halfbakery