Moving multiple files in Caja very slow

Development Discussion
,
#1

Hi all,

I’ve noticed that moving multiple files in Caja is really slow.

If I create a bunch of empty files in one folder, and move them to a subfolder of that folder it takes a while.

Here’s how to replicate the problem:

spadge@Jupiter:/media/Rust/home/Move Test$ for ((i=1; $i <= 150; i++)); do touch $i || exit 1; done

This creates 150 empty files named 1 to 150.

Create a folder ‘dest’ in that folder, select all files and drop them onto the dest folder.

Wait.

Keep waiting. You’ll get a ‘preparing to move files’ box with a progress slider pop up.

Eventually it completes the task.

Then back in terminal:

spadge@Jupiter:/media/Rust/home/Move Test$ cd dest
spadge@Jupiter:/media/Rust/home/Move Test/dest$ mv * ../

Instantaneous.

I have folder previews turned off in Caja.

What’s the problem here? Why is it anything other than every bit as fast as the command line?

Thanks

#2

Hi. Have you tried without first copying/moving with caja? Directly in terminal? It may be taking slower time because of buffering or caching. If it takes high time there it wouldn’t be a caja problem.

Is it a laptop or desktop? HDD, SSD, USB? EXT4? i’m not an expert but those details might give an idea to someone else.

1 Like
#4

Hi @SFromley, Good demonstration.

I can think of 2 things that might be involved. 1) File type detection. 2) Undo info.

For this reason alone, it won’t be as fast as commandline. But what is perfectly obvious is it shouldn’t be THAT slow!

I also noticed a second move of files is rather fast, as if there’s a cache involved. Or in the case of file info, it’s already done.

Just my 2 cents.

#5

Nemo is slow too, so is nautilus. I guess the problem lies deeper.

#6

Yeah, I just tried that and saw the same thing.

Reversing the order It’s every bit as slow.

Creating the files, moving them to ./dest on the command line is immediate. Moving them back from ./dest in Caja is every bit as slow as previously. And then moving them back to ./dest in Caja is painfully slow as well.

It’s not a slow system struggling with power supply or memory.

Desktop i5 with 16GB RAM, and it’s barely touching the cpu or memory.

The disk is 1TB of spinning rust formatted with EXT4 and about 80% full.

If I move a single file on it, it’s immediate (as it should be - a filesystem won’t physically shift a file when it’s moved from one place to another within the same filesystem).

Running the same test on the SSD that is / gives the same result.

I never saw this behaviour in Dolphin, although I never ran that on the 16.10 setup.

I’m baffled as to what it is that’s making it take a couple of minutes to move 150 empty files to a subdir in the same filesystem.

#7

An interesting clue it’s struggling with some kind of database Caja must be using.

Once the files struggle to move one time they never will again, even across reboots.

Also, a reboot does not prevent the first time slow move.

It’s as if creating the files with the terminal causes the files to skip over a key step for Caja, even though Caja displays them.

I tested reboot to get system disk caching out of the picture - which it seems to be. It seems purely a Caja database-like thing.

EDIT: Anyone want to wager Caja’s performance/feature trade-off would tip the wrong way if Caja always scanned all displayed directories for such never-before-seen files?

#8

MY MAANNN!!! You really found something!!

OKAY I did some testing:

----PREVIOUS GROUND----
1- Generated a directory and then a directory inside it. I’m calling them, FIRST and SECOND, respectively.
2- Commands to generate the files were:
touch {x1..xn}
3- Commands to remove the files were:
rm {x1..xn}
4- Tested on ext4, NTFS, tmpFS.
5- Ubuntu-Mate with Caja 1.12.7 installed and Ubuntu-Mate with Caja 1.12.7 Live-CD, Porteus Mate with Caja 1.9.1 on two machines. Note that the installed Ubuntu-Mate system was only tested on one machine and on ext4.

----CUT & PASTE TESTING----
1- Generate n1 empty files in FIRST directory.
2- Cut & Paste the files to SECOND directory.
3- Cut & Paste files from SECOND to FIRST.
4- Remove n1 files from last directory (FIRST).
5- Generate n2 empty files in FIRST directory.
6- Cut & Paste the files to SECOND directory.
7- Cut & Paste files from SECOND to FIRST.
8- Remove n2 files from last directory (FIRST).
9- Generate n… empty files in FIRST directory.
10- Cut & Paste the files to SECOND directory.
11- Cut & Paste files from SECOND to FIRST.
12- Remove n… files from last directory (FIRST).

----DRAG & DROP----
1- Generate n1 empty files in FIRST directory.
2- Drag & Drop the files to SECOND directory.
3- Drag & Drop files from SECOND to FIRST.
4- Remove n1 files from last directory (FIRST).
5- Generate n2 empty files in FIRST directory.
6- Drag & Drop the files to SECOND directory.
7- Drag & Drop files from SECOND to FIRST.
8- Remove n2 files from last directory (FIRST).
9- Generate n… empty files in FIRST directory.
10- Drag & Drop the files to SECOND directory.
11- Drag & Drop files from SECOND to FIRST.
12- Remove n… files from last directory (FIRST).

CONCLUSION 1

Only on ubuntu versions and Caja 1.12.7 the system struggles to make it happen while on Porteus with Caja 1.9.1 it was smoothly and immediately. Thus, the problem roots on either Ubuntu or versions after Caja 1.9.1 (1.12.7 tested)

CONCLUSION 2

I lost my Sunday.

3 Likes
#9

Here is the unattended issue https://github.com/mate-desktop/caja/issues/528.
I promess I will post something there. Hopefully it will get some attention.

#10

Hi there,
I have also those sort of issues with Caja. In my case there are a few hundred jpeg files (each between 5 to 10 MB in size) to move from one directory to another on the same disk (Ubunut-MATE Xenial 16.04, MATE Desktop 1.14.). However, I use now Gnome-Commander https://gcmd.github.io/ “GNOME Commander is a “two-pane” graphical file manager for the Linux desktop …”. It is in the Ubuntu repository available and you can install via “sudo apt-get install gnome-commander”. With gnome-commander the same move operations were blazing fast and reliable compared to Caja. The tool itself is easy to use.

#11

Is there any resolution to this “preparing to move” bug?
I have Caja 1.12.7 and any attempt to move more than about a dozen files (from one folder to another within the same EXT4 partition) results in a massive slowdown and if the number of files is greater than about a hundred, a hang requiring forced termination of Caja and a restart.

Apart from this very serious bug, Caja is great.
But why has this bug not been resolved yet?

#12

I ended up getting so fed up with this that I’ve switched my desktop to Mint/Cinnamon last weekend.

I still have Ubuntu MATE on the pi though.

Thanks

#13

Are they doing something silly and dumbed-down, like maybe updating “Recents” for every file?

What i have observed over several progressive debian-based distro releases (from ubuntu 11.10 to debian-jessie and now ubu-mate 16.4) is that the amount of time spent loading shared libraries seems pretty phenomenal. It’s seen clearest imo when looking at startup times for apps like Firefox. But i run a wide variation of processors, from a 32bit Intel Atom on the low end to an i7 quad on the high end, something like a 20-fold speed difference, which makes it easier to localize. Once an app gets started up, its responsiveness is much less variable across different processor speeds; spinny disks make it more clear… and it isn’t just i/o time imo because apps that are using the filesystem are quite fast even on my slower systems.

Anyway my nose is telling me something like they’re updating some fancy dumber-end-user stuff and doing it in a way that causes some library thrashing. It’s a guess.

#14

I’d gone to considerable efforts to stop recent documents from being a thing on the system. Who knows?

#15

One big symptom discounts running/non-running libraries - how the information that speeds up performance survives reboots. This tells me it’s more like a permanent cache than a library in memory. Does that make sense?

Firefox startup is a perfect example of already-in-memory speedup that does not survive reboots. The first Firefox after reboot is always sloooow. But this Caja speedup survives reboots.

Caja has to keep a database that remembers everything from what sorting a directory has been set to emblems a user assigns. I imagine file information (like file types) is also in this cache. And this is exactly the information a commandline action seems to work against - it “surprises” Caja who is forced to catch up when it discovers new info that needs updating.

So in that regard, I’m in total agreement it’s updating stuff you may not need or want. We’re back to basic trade-offs.

The test from @SFromley is very well designed and shows this “surprise” for Caja perfectly, too.

Actually, this phenomena isn’t new. For years, I’ve kept PCManFM on my system just for when a leaner-and-meaner file manager is called for. I don’t use it often but when I do it’s always nice to have. :slight_smile:

#16

It is lean and mean, but I found file transfer rates remain the same.

And about files already being loaded at boot. This is one reason I run preload and I think it helps, but file transfer speed remains the same.

#17

The original post by @SFromley has the test to use. It takes an awful long time for Caja to move just 150 zero-size files. This i7 takes more than a minute on an SSD. 16.04 w/Mate 1.16.

BTW, PCManFM has no problem. Just tried it. Caja needs to handle the files once then it’s fine. Very repeatable.

EDIT: Between other things I’m testing some things and I just found out the zero-size is not involved by making these text files with content “test”$i.

1 Like
#18

There is a thing that used to run on Ubuntu (before i went to debian and back) that tracks recents and other such things, i think it’s called zeitgeist. I tend to turn it off by habit if i notice it in the autostart list. Anyway stuff in the “did this recently” list does seem to survive boots as i recall. fwiw which may be nuffink.

#19

Having also run into this problem, my suspicions lie with /home/user/.local/share/recently-used.xbel*. I had over 700mb of these files. I disabled the file using the method from,
https://alexcabal.com/disabling-gnomes-recently-used-file-list-the-better-way/

So far i’ve not seen the problem since.

1 Like
#20

I also experience the problem, with both Nautilus and Caja.

It’s quite probable that the issue is at least partially caused by upstream problems with gvfs: https://bugzilla.gnome.org/show_bug.cgi?id=757747

See also: https://github.com/mate-desktop/caja/issues/528

#21

Yep, it seems that patching gvfs does indeed solve this issue. A fix will be released shortly: https://bugs.launchpad.net/ubuntu/+source/nautilus/+bug/1133477

1 Like