Is there a way to do `mv`, in either linux or Perl, that will fail if I try to copy a file over another file?
In the mv manpage, I find a `-n` or `--no-clobber` option which does indeed refuse to replace file A with file B. But it is a silent failure. $? is 0 after I run an unsuccessful `mv -n`.
@mcc would rsync be more flexible for this purpose?
if [ -e dstfile ]; then echo "nope" else mv srcfile dstfile fi
@mcc you could add an -n to the mv to protect against a race condition too if you wanted
@mcc you could also do something similar in perl i think:
perl -e 'rename("srcfile", "destfile") unless -e "dstfile";'
@d6 @mcc Unfortunately common `mv(1)` implementations don't seem to implement this race-free:
https://unix.stackexchange.com/questions/248544/mv-move-file-only-if-destination-does-not-exist/248547?noredirect=1#comment428238_248547
@mcc you could prob use File::Copy module for this in Perl
use File::Copy;
my $source = "doot.txt";
my $destination = "/path/to/doot.txt";
if (-e $destination) {
die "Destination file already exists. Move operation failed.\n";
}
move($source, $destination) or die "Move operation failed: $!";
@rooneymcnibnug I reckon so.
@mcc @rooneymcnibnug Well, not quite atomic (something else could create the file between the `-e` check and the `move`). But it's hard to do better -- the underlying `rename` system call is documented to trample an existing directory entry without checking, so the only way to get genuinely atomic behavior is to guard the whole thing with a lock somewhere else.
@mcc probably an easier way though
@rooneymcnibnug In industrial settings, I'd be nervous about this approach because it is not atomic. But I am not in an industrial setting.
@mcc I suspect a scripting solution is the answer here. You'd just need to do a check before the actual operation. You can do this in perl but that might be overkill. In bash, for example:
if [ -f "$FILE" ]; then
echo "Cannot overwrite $FILE"
exit 1
fi
# proceed with move command here
@gutmunchies @mcc Yeah the example wasn't exhaustive. I think -e would be a better flag to use if you're trying to cover all the edge cases.
@mcc `ln $src $dst && rm $src`
@roguelazer @mcc obligatory "that doesn't work across filesystems" caveat?
@hisham_hm @roguelazer I also wonder what it does in WSL while operating on a NTFS drive.
@hisham_hm @roguelazer (which I am not, today)
@hisham_hm @mcc definitely a reasonable callout
Update to (at least) coreutils 9.2, or use a distro which provides that.
With that version, mv -n will print an error and return 1 if a file is skipped. (Previously, it just printed nothing to indicate it was doing nothing. Insert Drake meme here.)
There is a new `--update none` option to get the old noclobber behavior, if you want it.
tbh I'm surprised 1) they made what could be a breaking change and 2) to my knowledge there has been no "you broke my scripts" outcry. Although probably the second can be found if I look hard enough.
Also, apparently posix only specifies -i and -f. Because, sure.
IMHO good for the coreutils maintainers for making some of this stuff make more sense after all of these years.
But I'll wait for the "they hate Unix!" crowd to show up.
@mattdm @mcc Unfortunately `-n` doesn't seem to be race-free though :/
With that in mind (if still true), I don't see any worth in a `-n` flag (invoking the "it's not UNIX" meme here :D). Then again, while there are many very bad options added by the GNU project if you ask me (GNU/tar's --checkpoint-action flag springs to mind), but this'd be a really useful flag in principle.
@mcc @mattdm Seems to be fixed by now at least:
https://hachyderm.io/@mattdm/112706603255177043
Haven't tested, but I believe that's been fixed sometime after 2015. From the release notes for 8.30 (2018):
``` 'mv -n A B' no longer suffers from a race condition that can
overwrite a simultaneously-created B. This bug fix requires
platform support for the renameat2 or renameatx_np syscalls, found
in recent Linux and macOS kernels. As a side effect, 'mv -n A A'
now silently does nothing if A exists.
[bug introduced with coreutils-7.1]
```
@ljrk @mattdm @soller The one thing that makes me a little worried is the microkernel. Not to say it's a bad idea, just I wonder if it will last. I think of Apple's experience with starting with a microkernel believing the performance difference negligible before eventually realizing when you're writing a kernel your incentives are eventually to move performance encumbrances to absolute zero…
@mcc If they're on the same filesystem so the mv is actually a rename not cp+rm, you can use ln (without -s; hard link) instead. rm the old name only after the ln succeeds.
If it's cross-fs, you can do the same with a temp file on the dest fs.
my copy of `mv` has an `-i` (interactive) option which generates a prompt in the event of a potential overwrite
On Linux, the renameat2 system call takes a flags argument that can include RENAME_NOREPLACE for the behavior you want.
As far as I can tell, coreutils mv will actually use this flag on first try, but then it catches the error and falls back on a hugely complicated path without it.
I can't think of a way to invoke renameat2 from Perl without hardcoding a lot of constants; I wonder if writing a simple C utility might not be the least-bad solution.
#define _GNU_SOURCE
#include <stdio.h>
#include <stdlib.h>
#include <fcntl.h>
int main(int argc, char *argv[])
{
if (3 != argc)
exit(EXIT_FAILURE);
if (0 != renameat2(AT_FDCWD, argv[1],
AT_FDCWD, argv[2],
RENAME_NOREPLACE))
exit(EXIT_FAILURE);
exit(EXIT_SUCCESS);
}
If all else fails, a 10 line bash script to check file exists, move to/use backup file name and or fail otherwise would work.