Regular readers here will recall I've posted several times about the issues I've encountered with duplicate detecting programs, so I decided to go a little deeper. NOTE: I spent one year of my life trying to support a hard drive sales effort and found that drive technology and I don't mix. Still, I decided to try some non-destructive research using debugfs. Since the basic invocation only reports its findings, I issued
sudo debugfs /dev/sda2
Which is the file system containing my data files. After it ran for a while, it gave me the error in the subject line
"Block bitmap checksum does not match bitmap while reading allocation bitmaps"
Uh, OK. What does that mean? Quite obviously debugfs has found an issue, but now what. It's giving me a prompt, but I have no idea what to tell it
One solution that has come to mind is to unmount /dev/sda2 (it's not the boot volume) and run fsck.
Thanks, that's good to hear. OK, I'm going to stop every daemon and process that's acting on files on that system, unmount it let fsck do its thing. Cover me, I'm going in!
Oh, this is fun. Since the volume I want to check is the root file system, I couldn't just unmount it to run fsck. Fortunately, I have a couple of "rescue" USB thumbdrives (note: Rescatux0.74 is a must-have!). So, I booted from a rescue USB, ran fsck which returned a 'clean' status, and booted back to my system.
fdupes is still going into a deadlock state, so I'm exploring other options. One nice tool I've installed is a GUI program called czkawka (which means 'hiccup' in Polish). It's fast, but I haven't yet had time to give it a real workout. On a small data set it found 5 Gb of duplicate files and then allowed me to delete the ones I chose. Nice!
I also realized my 'toolkit' of bootable USB drives is a bit outdated, so I created a "live CD" of Ubuntu MATE 24.04.1. I'm still at 22.04.5 LTS, so if I need newer capabilities, I've got a bootable drive.
Now, I'm going back to checking out duplicate file checkers. There are a few I haven't tried, yet.
OK, a breakthrough and a strong recommendation! I mentioned finding the 'hiccup' (czkawka) program and I'm now putting away all the CLI (x)dupes programs. czkawka) not only is blindingly fast, but it's multi-featured as well. I have now freed up over 300Gb of disk space used by duplicate files, found and deleted over 100 empty directories, identified the largest files on the file system, and located and deleted temporary files and those annoying (Mac) .DS_Store files. My disk utilization has dropped from 77% ro 74%. And czkawka did all the heavy lifting! Cheers!
