When someone wants to back up their computer in a crude fashion, they might simply copy a large folder full of files to their USB hard drive so they can end up with individually readable files on the destination drive without creating some monstrous disk image requiring third-party tools. (It's hard to beat that simplicity and clarity.) An overwrite of everything is unnecessary because most files haven't changed, but if a small piece of a large file HAS changed, standard copy utilities end up copying each whole, changed file, regardless whether only one byte out of a gigabyte changed. (Some good examples include Outlook PST files and VM images.) Proposed is a USB hard drive capable of hashing its contents, communicating just the hashes over the USB channel to save bandwidth, much as rsync does over a network link. In a well-baked implementation, the OS would then be configured to transparently compare hashes in the background to perform implicit synchronization when overwriting existing blocks.
-- kevinthenerd, Jan 05 2017

Is it possible? It sounds possible! I like it. I don't understand it, but I like it.

Seriously though, as a recent victim of catastrophic loss (graphics card went bang, lost all photos of family, major awards, travel junkets, plus the only extant copy of newest novel; backup USB also went bang upon reloading into new computer, so really really lost everything, really), I would appreciate extending the re-write life of backup storage devices. Hash-bit re-writes would do this...yes?
-- Sgt Teacup, Jan 05 2017


Hmmm...Five sentences. Four obvious opportunities for paragraph break.

Do hit the ENTER key twice periodically, will you please?
-- normzone, Jan 05 2017


I think you can already do this with rsync. Yep, I just looked it up, by default when copying to a local drive it transfers whole files, but you can force it to do a delta transfer by specifying --no-whole-file. I suppose it still has to read the drive to calculate the checksums, but reading is generally much faster and lower wear than writing.

[Teacup] Ouch. I think that a lot of work goes into algorithms and special file systems that reduce wear on solid state drives, unfortunately this is followed by manufacturers exclaiming, "Aha, this means we can get away with selling even less reliable media!"
-- mitxela, Jan 05 2017


"I think you can already do this with rsync." The OS still reads all the data from the media; there's no on-media calculation of the hash. With an on-media calculation of the hash, the hash could be cached to keep your flash out of the trash.
-- kevinthenerd, Jan 06 2017



random, halfbakery