[Pkg-mc-devel] Bug#592941: Bug#592941: mc: interrupted move causes duplicated files

[Adam Borowski]

On Sun, Aug 15, 2010 at 01:10:54AM +0200, Yury V. Zaytsev wrote:
> Hi!
> 
> > This causes duplicated files, and is especially painful when information
> > about what has been moved and what not is important. All that MC offers is
> > restarting the move from the very start, which is not a good idea for large
> > transfers over slow links.
> 
> Not true. When you start the move again, on the first already existing
> file mc will stop and warn you. There is a number of options available:
> 
> Overwrite all targets? [ All ] [ Update ] [ None ] [ If size differs ] 

The only choice that will complete the move is "All".  The other ones will
leave the duplicates in the source dir.

> The latter is what you need to continue the process properly by leaving
> alone the files that already have been moved and moving only those that
> haven't been moved already.

For a "move" to be done, the source would need to be deleted.  None of the
files at that point have been fully moved, they at most completed the first
step -- copy.

> If you need the information on what has been moved already and what not,
> you need the Compare directories to function properly (recursively).
> There is a ticket for this in upstream bugzilla.

So they'd have to perform a complex action just to get what other file
management programs do by default.  I've looked at nautilus and WinXP's
explorer, and they don't cause duplicated files. 

> > Also, while such postponed deletion does improve performance for small and
> > medium moves, it seriously degrades it when the tree is big enough to not
> > fit in the VFS cache.
> 
> How bad is this performance decrease?

In the case of lots of very small files that do fit into their inodes/
directory tree/metadata blocks, and the admin has foolishly left
atime/relatime on, about a factor of two.

The copy has to read everything, and (if atime is on), write to every inode.
The deletion needs to again read everything, and then write.

> I'd suggest not messing with deletions if the action that the user
> requested has not been successfully completed in full.

The move consists of thousands of actions the user requested, and it's
better to complete _some_ of them instead of none at all.

> Maybe it will give you some (arguable?) performance gain in your very
> specific case, but will break other reasonable usage scenarios.

That optimization is indeed quite rarely useful, but I can't see any use
case where it breaks something.  I proposed only as an addition to fixing
pending deletion not being moved, it is in no way necessary -- it merely
would let us avoid a long pause between an error occuring and returning
control to the user.

It would have a nice side effect, however, by allowing the freed space to be
reused much sooner.

> I think you just need to use "smart" resume strategy (overwrite if size
> differs) and Compare directories when it will be fixed.

Well, but the current behaviour -- could you name any use case when it would
actually be useful?  If for some strange reason the user wanted to ensure
the whole directory is at all times complete on one of the systems, it's a
matter of just F5 enter F8 enter.

> In what concerns a usage scenario that will be broken by your suggested
> change: 
> 
> 1) Imagine that I want to move a big tree to an FTP. In the middle of
> the operation remote server ran out of space and the transfer failed.
> Successfully transferred files have been deleted.

Just as you requested.  In fact, if you selected a bunch of large files, MC
would do that -- it's only moving the contents of a directory which exhibits
this behaviour.

Some consitency would be good.

> 2) I started moving a huge tree to another disk and half-way realized
> that it's the wrong volume. So I abort the move, but screw it, half of
> the collection has already been deleted from the source drive.

How often you do that?  It strikes me as a pretty rare occurence.

And for scenarios broken by the current behaviour, let's start with:

1. You need to free some space.  You did move some big files, but misjudged
the space, and now need to stuff the rest someplace else.  You definitely
don't want to have the data occupy much more than it needs to.

2. Once some stuff has been dealt with, you move it to another place.  You
want to have everything either in the "done" dir or the "todo" one, but not
both.

3. You have a bunch of versions bozos you work with didn't keep under
version control, and want to sort them out.  Naturally, there will be lots
of duplicates, but you need to know which duplicates were there before and
which were not.

> I suggest to close this as wontfix, because I really have to make mc do
> any destructive action before the requested command has been completed
> the way the user wanted it to in full and without problems.

Any given file needs to be completely tranferred before being deleted.
It can't be done in a fully atomic way, but when the failure occurs during
transferring a particular way, MC does ask the user a question: "Incomplete
file was retrieved. Keep it? [ Delete ] [ Keep ]".

If you insist the current behaviour has a purpose, it would be better to at
least give the user an option to clean up one way or the other.

I still that would be redundant with F5 F8, but indeed, in that rare case
you pointed out, the user might have not anticipated that FTP server will
run out of space.

> I hope you can understand my point.

Kind of, but I'm surprised by this inconsitency between MC and other
programs, including MC's own way of moving multiple files.


Cheers and schtuff...
-- 
1KB		// Microsoft corollary to Hanlon's razor:
		//	Never attribute to stupidity what can be
		//	adequately explained by malice.