Kernel 5.15, ntfs3, and Caja

I am running UM 21.10. I downloaded and installed kernel 5.15 from Index of /~kernel-ppa/mainline/v5.15/amd64 to check out the new ntfs3 driver. It seems to work fine, and is very fast indeed!

To use it, you have to explicitly specify ntfs3 as the fs type, either in fstab or in the mount command. However, Caja still automounts ntfs partitions using ntfs-3g. Is there any way to get Caja to use the new ntfs3 drivers?

TIA...

As a quick workaround, try moving /sbin/mount.ntfs to /sbin/mount.ntfs.disabled, and adding ntfs3 to the list of recognized filesystems:

sudo mv /sbin/mount.ntfs /sbin/mount.ntfs.disabled
echo ntfs3 | sudo tee /etc/filesystems

Then, if you can, reboot and try to mount an NTFS filesystem. See if you get the new ntfs3 driver!

gordon,

I will try this at first opportunity, and report back. Thanks for the input...

Alas, no joy. 'Unable to mount blah blah blah...' I restored mnt.ntfs to its original state and then Caja was able to mount the partition without issue.

Can it be something hard-coded into mount.ntfs?

BTW, after following your suggestions, ntfs3 was the only entry in /etc/filesystems.

Any more ideas?

It was expected that ntfs3 would be the only thing in /etc/filesystems since that file doesn't exist by default on most Linux systems.

Before I go on, maybe I should start my usual type of diatribe and explain how mounting of filesystems works on a modern desktop on Linux. If you'd rather not read three pages of diatribe, just skip to the TL;DR below and start reading from there.


In the beginning, when you tell Caja to mount a filesystem, Caja will contact a "daemon" (background process) called udisksd to mount the filesystem.

You'd think, "Why can't Caja mount the filesystem by itself?", and the answer is that on most UNIX-like systems (including Linux), only the superuser or a specially-privileged program can mount filesystems.

In ye olden days, filesystems that other users could mount would be listed in the /etc/fstab file with a special parameter, and then any ordinary user could mount the filesystem. The mount command is specially privileged so that, theoretically, any user could mount any filesystem anywhere without first becoming root; however, in practice, mount is programmed to mount only filesystems that were specially marked by the superuser in /etc/fstab ahead of time.

The fstab mechanism used to work, but did not bode well for removable media: What if you inserted a USB stick that you've never used on the system before? There's no entry in /etc/fstab for this USB stick, so ordinary users cannot mount the filesystem. Ultimately, the root user would either have to carry out the mounting operation, or the root user would have to add a line to /etc/fstab so that ordinary users could then mount that filesystem. Either way, plugging in a USB stick means logging in as root and running some arcane command sequence. Not exactly "plug-and-play", and not fun at all. And not really secure either, since you either have to tell everybody who uses removable media the root password, or you have to do all mounting yourself and have everybody hound you whenever they get hold of a new USB stick!

So, among the other good things that came about in 2004-'05, Linux gained all kinds of "plug-and-play" programs and daemons that started with the letter 'U' -- Udev, Udisks, and a few others which I can't remember right now. Anyway, now Udev will know immediately when you plug in a device, be it a USB stick or something else, and will tell Udisks about the device and asks Udisks to mount the device. Similarly, Caja can instruct Udisks to mount a device when you "manually" click on that device in Caja. Either way, Udisks gets a request from some external source and proceeds to mount the filesystem on the behalf of the other party.

Now we're getting down to the meat of the problem. The first thing Udisks does is it uses a library called libblkid (also known as just blkid) to identify what filesystem is on the device in question. blkid has a hard-coded list of known filesystems, and tests the device's contents against the list of properties which identify each known filesystem. Most filesystems contain "magic numbers" or "magic sequences", or at least unique data structures, which enable blkid to identify the filesystem quickly just by looking at select parts of the filesystem data.

Anyway, blkid deliberates and tries a bunch of obscure filesystem types as well as common ones, and eventually it figures out that the device has an NTFS filesystem. blkid then reports the filesystem type as a string reading exactly as ntfs. Udisks then takes this string and attempts to mount the device with a filesystem type of exactly the string returned by blkid, e.g:

mount -t ntfs /dev/sdb10 /media/gordon/mountpoint

I was hoping that by removing the user-space filesystem driver called ntfs (by moving /sbin/mount.ntfs to some other location), that the first attempt at mounting would fail, and then Udisks would go on to plan B, which is to look through each line in the file /etc/filesystems and try each filesystem type one at a time until something worked. Apparently, Udisks just errored out at the first attempt and pouted.


TL;DR: My first try didn't work since /etc/filesystems is not checked unless blkid turns up totally empty-handed, i.e. the filesystem is not recognized as a standard filesystem.


I was going to list off a hack of mine to help you, but I found this for Arch Linux -- it should also work for Ubuntu: AUR (en) - ntfs3-dkms. The link I gave you brings you right to the section you need to look at. There's apparently two files you need to modify. I formulated a procedure based on those instructions, so you don't have to go digging into the details of what needs to be done (see below).

@sgage, I think you're good to follow those directions, since you've got Ubuntu 21.10 and it supposedly has a new enough version of Udisks for the instructions listed above and below to work. But so that nobody else can accuse me of not warning anybody of this caveat, here's my warning:

:warning: WARNING: This hack apparently does not work by default on Ubuntu 20.04 or earlier since such versions have insufficiently new versions of Udisks for one part of the hack to work, though your mileage may vary.

My version of the procedure:

sudo dd status=none of=/etc/udev/rules.d/0000-Use-ntfs3-driver-for-NTFS-filesystems <<EOF
SUBSYSTEM=="block", ENV{ID_FS_TYPE}=="ntfs", ENV{ID_FS_TYPE}="ntfs3"
EOF


sudo dd status=none of=/etc/udisks2/mount_options.conf <<EOF
[defaults]
ntfs3_defaults=uid=$UID,gid=$GID,noatime,prealloc
ntfs3_allow=uid=$UID,gid=$GID,umask,dmask,fmask,iocharset,nohidden,sys_immutable,discard,force,sparse,showmeta,prealloc,noacsrules,acl
EOF

Supposedly, if you do that and reboot, then all will be well and you'll mount NTFS filesystems using ntfs3.

I would indeed rather not read 3 pages of diatribe - I would have preferred 6 pages! :slight_smile: That's some pretty educational diatribe right there... Thanks much for your time and attention to this puzzle...

Alas, although the commands you gave really looked like they were going to do it, Caja still automounts ntfs drives using the ntfs-3g driver.

What next?

1 Like

It looks like changing what udev thinks the filesystem is from ntfs to ntfs3 doesn't work, since Udisks doesn't seem to refer to udev for that information (at least on Ubuntu); as I said above, in my experience Udisks looks to blkid to find that information out. Thus Udisks is probably still using the ntfs filesystem type instead of ntfs3.

Well, I'm running out of ideas. But let's try another one. If a filesystem called literally ntfs isn't supported by the kernel, then the mounting procedure will try to run a program at /sbin/mount.ntfs to mount the filesystem. That's how the old NTFS driver worked -- it wasn't implemented in the kernel, it was implemented as an ordinary program in "userspace", and is thus called a Filesystem in Userspace (FUSE). So with that said, let's keep the above modifications in force, and move the /sbin/mount.ntfs file again, then put a "wrapper" script in its place that mounts the filesystem as ntfs3 after all.

Proposed code:

# Do not revert any of the last changes you made (udev rules, etc.)
# But do run the following commands in addition to what you
# already did in the previous post:

sudo mv /sbin/mount.ntfs /sbin/mount.ntfs.disabled

sudo dd status=none of=/sbin/mount.ntfs > /sbin/mount.ntfs <<EOF
#!/bin/bash
exec mount -t ntfs3 $@
EOF

sudo chmod 755 /sbin/mount.ntfs

Now mount an NTFS filesystem using the GUI (make sure it's unmounted before you "mount it again", obviously!). I hope this works :crossed_fingers:. Keep me posted -- pun intended. :slight_smile:

Well, no luck. When I implemented your suggested script and tried mounting an ntfs partition in Caja, I got an 'unable to mount' error, 'unknown error when mounting /dev/sda1'.

Oh well, again I thank you for your time and attention. This is not really any kind of show-stopper, and I'm sure it will get addressed sooner or later. If you think of anything further to try, I'll be watching the forum. If not, well, you tried!

Cheers,

  • Steve

Try dmesg | tail -25 and see if there's anything in the kernel log related to mounts, NTFS, or sda1. If not, try journalctl | tail -25 and see if anything comes from that.

Usually, more information lay in the logs!

The only error reported is

ntfs3: Unknown parameter 'window_names'

When I put mount.ntfs back to its default link, it still throws that same error, even after a reboot. I reverted the earlier changes - same error. What is invoking ntfs3?

Seriously? So you removed all those files (even the udisks mount_options file) and you're still getting stuff mounted by the NTFS3 driver?

Did you reboot after removing all the files? Because I think you need to!

Try recreating the udev, udisks and stub script files (like you had it before you made your latest post), and change the stub script to:

#!/bin/bash
NEW_PARAMETERS=$(sed -r 's/(,windows?_names)|(windows?_names,)//g' <<<"$@")
exec mount -t ntfs3 $NEW_PARAMETERS

Other than that, if that doesn't work and you want to go any further, we'll have to recompile blkid with some new changes. It won't be a nightmare, but it'll be a pain.

Seriously! Yes, I did reboot. Realio trulio. Upon further review, that exact error msg is kicked out earlier in the boot process - so I reckon it's something to do with the kernel module loading? Subsequently, every action involving mounting an ntfs partition kicks off another of those messages. Successfully mounting via Caja using ntfs-3g emits the same message. Successfully mounting an ntfs partition with ntfs3 explicitly specified emits the same message. It almost seems like some plumbing is in an intermediate stage of development...

Should I still try the new stub script, or does this latest info lead elsewhere?...

All right, now I'm totally stumped. I think I'd have to compile Linux 5.15 myself and try this stuff out and see for myself, because at this rate I've reached a dead end otherwise.

Now if only I could find some NTFS filesystems to try this on... Oh wait, no issue, I have one. I'll see if I can replicate this using Ubuntu MATE 21.10 (which I have installed), the vanilla 5.15 kernel source (which I'm getting right now), and an old NTFS partition whose contents I don't care about anymore.

In the meantime, if you haven't already, revert all the changes we've made, even the udev configuration file. Thanks for everything!

Thank YOU for everything! Anyway, I've reverted the changes and all is well.

(Just FYI, I didn't compile the 5.15 kernel - I just downloaded it from Index of /~kernel-ppa/mainline)

Have fun experimenting!

1 Like

I have no idea on how you might be able to fix udev etc, but... if you're just trying to access a specific partition on the same machine, why not go with your original plan and just mount it in fstab? (or rc.local, or whatever you prefer).
If you're trying to mount a USB stick, then yeah, you're going to need caja to work "better", and I have no idea what sort of gyrations that currently goes through, sorry.

Thanks for replying... Yes, I can mount in fstab, but it's not that important, really. As for USB sticks, I don't think I have any that are formatted ntfs. I will wait and see what happens as kernel 5.15 matures - it is still pretty new, after all!

Hello gordon,

Here is an thread re: ntfs3 and udisks that may be of interest to you: