NTFS3 kernelmodule corrupted a NTFS-disk again, I'm reverting to fuseblk (ntfs-3g)

Thoughts & Feedback
,
#1

Since I started using the NTFS3 kerneldriver almost a year ago (GPL-driver created by Paragon software) I sometimes end up with corrupted filesystems. This has happened several times since I started using it. :thinking:

I know that several people seem to have issues with it:
see: Ntfs3 keeps corrupting my ntfs partitons - Support - Manjaro Linux Forum
and: Does the Linux NTFS3 Driver Corrupt Directories? - Heiko's Blog

But I tried to sit it out and see how things would develop. ... Until today! :rage:

This time it was a whole 2TB (external) disk filled for 75% with data that corrupted and this time also beyond normal recovery. :confounded:

(To whom it may concern: Kernel 6.5.0-21-generic #21~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Fri Feb 9 13:32:52 UTC 2 x86_64 x86_64 x86_64 GNU/Linux)

It wouldn't mount. It recognized the disk and partitiontable but that was all.
Tried it on a MS-windows OS and it timed out after minutes of trying.
MS-Windows couldn't even recognize the disk itself ! :roll_eyes:

Testdisk would have been able to save a lot though...maybe even everything.

Since I have a duplicate backup I luckily didn't need to go that way. I can just
cp -a /media/xxx/source/. /media/xxx/target/
to a spare disk.

But this is the last time I endured the shenennigans of this driver.
This incident caused me to no longer trust this (now standard) Paragon NTFS3 kerneldriver. I revert back to the userspace fuseblk ntfs-3g driver which is, although slower, more reliable on my setup. :rage:

If you are curious into what I do to revert to ntfs-3g: I use just one command:

echo 'blacklist ntfs3' | sudo tee /etc/modprobe.d/disable-ntfs3.conf

Goodby buggy kerneldriver. :raised_hand:

4 Likes
#2

Epilogue:

  1. After a treecopy from the 2nd backup to the new disk, it turned out that the new copy was about 27 gigabytes smaller than the 2nd backup.

  2. A filecompare (cmp) between the disks showed that all files were identical although the total diskusage was not.

  3. Foldersizes were different although their contents were identical.

I can only conclude that the NTFS3 driver has either a novel way to add low level meta info or was just very sloppy leaving a lot of "dark matter" in its wake potentially corrupting the filesystem.

The corrupted disk was only usable again after wiping the first x-number of sectors. I needed gnome-disks to do that because dd refused (WTF??), so the "dark matter" option mentioned above seems the most plausible explanation.
I'll be wiping the corrupted disk completely just to be sure.

EDIT:
In the long term I might recopy my other disks too. I guess that they all must have been tainted one way or the other by this driver during the past year.

2 Likes
#3

Some months ago I encountered a corrupted directory. It was on the family photos USB drive :scream: (formatted as NTFS so family members can use it on Windows). I noticed when copying files in the file browser that something seemed amiss - files were just, not appearing? :open_file_folder: :ghost:

Windows ironically saved the day. chkdsk was able to fix the corrupted directory before things escalated. A full integrity (with MD5 checksums) against the copies on another drive proved the files were still good. Lucky miss!

Honestly can't remember which NTFS driver it was, but never had corruption like that before! Thanks for the tip on disabling the driver.

This might be to do with how many clusters it's formatted with.

E.g. 100 files of 1 KiB on a 4 KiB clustered disk = 100 KiB logically, 400 KiB physically.

My method of checking integrity is to generate a checksum list and verify that against the other medium:

cd /path/to/source
find . -type f -exec md5sum {} + >> /tmp/checksums.md5
cd /path/to/dest
md5sum -c /tmp/checksums.md5
4 Likes
#4

That is so cool, I love that. :heart_eyes:
It also feels/looks like a more correct way of doing things.

3 Likes
#5

For what it's worth, I was deleting files from a different NTFS partition today and an 'empty' directory got corrupt. Since the blacklist in place, it was mounted with type fuseblk, so I guess that was using ntfs-3g.

Used sudo modprobe ntfs3 and mounted again which reads as ntfs3, so I'll remove ntfs3 from the blacklist and will give the other one a go for the next time I deal with an NTFS file system. Till then, Windows can feel useful by running it through chkdsk again :slight_smile: :window:

2 Likes
#6

Maybe it's time then to switch to exFat since the support on linux is pretty much up to par.

NTFS is extremely fragile, even on MS-Windows (at work) I had sometimes filesystem corruption due to simple actions. For example cancelling a file copy action resulted in a unremovable directory that at the same time didn't exist at all.

I never had a filesystem go bad on me on Linux (until ntfs3 that is ) :thinking:

2 Likes
#7

It happened to me too, on a hard disk with all partitions in ntfs several files disappeared and I could not make copies of other files. I had to use Windows to rescue my files.

2 Likes
#8

Honestly I haven't known a time in using Ubuntu MATE when I can trust NTFS drivers to NOT screw up disks because it's happened too many times to me with things like 12TB disks that are mostly full so I never plug in NTFS disks into Ubuntu MATE, except for a 256GB thumb drive and sadly even THAT has gotten messed up, but I never lose data because of it because I have a strict policy of never deleting data until it is successfully transferred to where I want it to be.

From reading this thread I will remember that if I have folders that disappear when going back to Windows to run check disk. I noticed that same issue where I have folders disappear when bring the disk back to windows.

I run a dual OS system so MATE has the ability to see Win 11 NTFS disks but under the Disk tool I have those disks set to never mount, not be visible and even require additional password. After doing that I've never had a Win11 disk messed with by Ubuntu.

Thanks for the tip on reverting back to a driver that supposedly works. I will at least see if thumb drives don't get screwed up going back and forth.

1 Like