Imagine your backup software encounters an unreadable file (permissions, I/O problem, whatever) during the backup process.
What should be the default behavior?
@fribbledom damn, that's a hard one.
@fribbledom halt, catch fire, burn all previous backups, then corrupt the main drive.
I always overlook the obvious choice.
@fribbledom the thing that freaks me out about any backup solutoon is that a corrupted file looks exactly the same as a purposeful modification, and will happily copy over the top of all previous versions silently.
@fribbledom so, an actually unreadable file during a routine backup? things have most likely been fucked long enough for all your backups to be worthless.
@fribbledom what would be good is a backup utility that knows, for instance, DCM_0456.jpeg is not a joeg file this time.
I'm having serious trouble understanding what you want. Yes, if you're using Windows then you're out of luck, but if you use Linux then format your partition to btrfs, or any other checksumming file system. If you're using an Apple product then switch to APFS, which also supports file integrity checks through checksums.
And as a backup drive some external HDD if you want or a cheap online storage.
For the software part, I recommend borg backup.
@KopfKrieg i don’t use windows primarily, i use mac. interestingly, apfs does not do *any* of the things you think it does.
if i were to use linux to back up *to* what would be the point of not setting it up as a NAS?
and your suggestion of a cheap external HHD has me wondering why you think you have the right to speak to me, let alone be smug about it.
@KopfKrieg with shitty backup strategy like this, no wonder you forgot *the very first thing you tooted at me*
@zensaiyuki It's a simple backup solution, that's all. If you want a better backup strategy then use something like 3-2-1 with at least one off-site backup.
@KopfKrieg a simple backup strategy that would make sense to mention if you haven’t read a single thing i have written.
@zensaiyuki Whoops, you're right, APFS uses checksums for metadata only.
A NAS is still not required for backups, though.
Also, I did never say "cheap external HDD".
Oh, and regarding the "right to speak to you": This is the Fediverse. If you don't want people to communicate with you, then make your profile private and don't comment on other's posts.
@KopfKrieg all i ask is that if youugonna come at me with smug snark the least you can do is pass the turing test.
Not sure what you back up to, or with what software, but most of the ones I know support versioning internally: Backintime (Linux), Duplicati (Linux ,Windows, prbly MacOS), Time Machine (Mac only). So if something gets corrupted, you can retrieve older versions of the same file.
@Mr_Teatime @fribbledom i guees i must be weird, because i have never been satisfied with any of those backup solutions. time machine has never finished a backup in less than a week, and that, and everything else doesn’t reallly handle the issue of single very large files changing slightly over time gracefully, which is a thing i run into a lot what with running VM’s and editing photoshop files.
@purple @fribbledom @Mr_Teatime i long ago came to the conclusion that photos, videos, music, large blobs that change slightly over time, code, and documents that get reorganised regularly, each need vastly different solutions .
i don’t have any clue how “normal” people seem to be satisfied with existing solutions.
@purple @fribbledom @Mr_Teatime plus i tend to accumulate stuff over time at a rate regularly exceeding typical backup drive size, so there’s a ton of stuff on old backups that i’d like to have available to look at but it’s on one of the 20 or so external drives i have somewhere, which may or may not be corrupted by now.
@fribbledom Of course it has to drop all of the progress and shit itself, duh. What kinda HDD do you even have that it doesn't let you to get to every file, you... you basic bitch uuuuuuuuuuuuuuuuuuuuuuuuuussssseeeeeeeeeeeeeeeeerrrrr?
@fribbledom report, wait for the user to acknowledge (popup box/press any key), then continue
@email@example.com I think i'd prefer if I was shown a dialog informing me about the error and then getting to decide if I want to continue or not, rather than either of the two immediately happening.
@fribbledom report but continue: partial backup is still better than no backup.
BUT the "report" must be very clear and prominent (even annoying) so ppl know some file was NOT backed up.
This is the default behaviour of DejaDup, btw.
@fribbledom Backups are mostly run on a timer and without someone looking over them. If the erroneous file was the second file looked at and then the backup aborts, you have a very empty backup. Better to try save as much as possible than be pedantic about it being a complete backup
@fribbledom I voted what I know my software (rsync) does, continue and report.
*BUT* ensure that the report is prominent:
1. List and log the failing file(s) and reasons for it at the *END* of the output; otherwise, they will scroll away and be ignored.
2. Print and log the number of failures at the end of the output.
3. On *nix-like environments, exit with a non-zero status that is different from the one caused by fatal-and-can't-continue type errors.
@fribbledom If When I'm using a backup, I'd much rather whatever I can be backed up. You're already going to have an inconsistent state by what has already been copied.
Also, nothing is as infuriating as starting an hours-long process only to come back and discover that it stopped 15 minutes in.
@fribbledom report, continue, but abort if there are hundreds of them, or if more than $threshold% fail?
@fribbledom (i.e. differentiate between "this file failed to back up" and "this backup failed")
@fribbledom Send a cryptic note to an unconfigured mail alias, then disable automatic backups until an undocumented lock file somewhere deep inside /var is deleted.
@fribbledom report and suspend, so operator can do whatever and then resume or abort at their discretion
@fribbledom I think there should be some good retry logic to avoid temporary I/O or whatever problems
Ideally it should be a configurable option; defaults of abort if an interactive session (using gui or attached to tty), report if not (eg timer). The logic being that interactive might be a config issue on a first run and noninteractive could be failing filesystem so save what you can.
@fribbledom Interesting poll! Makes me think of Postgres' fsyncgate a few years back.
@fribbledom skip and maybe retry later, adding the error to a log then exit nonzero. If this happens with cron, mail the log to the user for further investigation.
@fribbledom if i don't catch it in time (i.e. before my previous backups expire), i'd be much more distressed by having no backups at all rather than being unable to restore one file.
@fribbledom report and continue, with an optional flag that would abort if the person doing that backup needed it to do that.
@fribbledom Report, but continue … but it's absolutely essential that the user knows what happened - i.e., a report buried on page 4 of 10 pages of routine gibberish really doesn't count.
@fribbledom skip it and append status to an error log.
@fribbledom Mark the file as needing attention, continue, wait for user intervention once everything else is done.
Server run by the main developers of the project It is not focused on any particular niche interest - everyone is welcome as long as you follow our code of conduct!