Pi4 Mate 20.10 Gpart format ntfs partition on SDcard cannot uncheck "run as executable" in Caja

Pi4 8GB, Mate 20.10 fresh install, Sandisk U3 128GB, browser Caja

symptoms on NTFS drive:
"run as executable" cannot be uncheck on file properties
"run as executable" can be uncheck in caja preference but reverts to green minus sign check immediately.

text files cannot be double clicked to open in Caja but do open in Thunar

I created an NTFS partition on my SDcard using Gpart because I wanted to be able to stick sdcard into windows and access content. SDcard is partitioned as:
/dev/mmcblk0p1 fat16 255MB(Boot)
/dev/mmcblk0p2 ext4 16GB(ubuntu Mate)
/dev/mmcblk0p3 ntfs 102GB(file storage)

Everything works correct Except:
Any text file I create in the NTFS drive or text file I copy into NTFS drive has the "Run as executable" checkbox checked. Double clicking such files will not open and you have to right click and select a program to open it. To be clear, a text file that doesn't have "run as executable" checked becomes checked when it is copied into the NTFS drive. This is really strange because I read that you cannot set the executeable flag in NTFS(maybe not by design but obviously not true). I cannot uncheck this, I click on it and nothing happens. File properties under Caja does not allow unchecking it. If I copy any of these files to my ext4 drive they remain as "run as executabe". However, once copied over, I can uncheck it in Caja and the file will open as a text file by double clicking. However, I have scores of text files and I can't keep reseting executable permissions each time I copy a file.

I tried to change the global settings in file manager by unchecking "run as executable" but the minute I reopen settings, it is rechecked again. The check is not a normal check mark but instead a green minus sign. I also tried changing it thru the Caja preference settings but I still keep getting this green minus sign.

This is a brand new Pi4 and I've downloaded Ubuntu Mate 20.10 3 times and tried clean install(formatting drive first) on 2 different SDcard and one flashdrive and it is always the same. After the install the first thing I did each time is make this partition and test, but each time the same results. I also uninstalled and reinstall Caja but that doesn't fix issue. I've installed Thunar and same text files open in this browser but Thunar doesn't have a simple "share" feature which I need to set permissions and access rights for different users and groups. besides, i want to get this fixed rather than a work around - it can't be a general fstab or chmod issue because it works fine in thunar.

I search the web but a clear answer on how to fix Caja is not available. I appreciate any help.

Hey, welcome to the Ubuntu MATE Community, @Harry_Roger! It's good to have you here. :ubuntu_mate:

NTFS and FAT partitions are kind of strange, in the sense of their support on Linux and other Unix-like operating systems. Surprisingly, all the stuff you just described makes perfect sense, once you know the dynamics of how Linux handles these foreign file systems. Here's why:

(NOTE: If you'd rather not read a two-and-a-half page discertation about why the file system is acting wonky, just read the TL;DR section far, far below.)


Neither FAT nor NTFS records Unix-style permissions (read, write, and execute for each of the owner, the owning group, and everybody else); FAT doesn't record any such information, while NTFS records full-fledged Access Control Lists (ACLs) for each file. The ACLs can dictate that one particular user can read and write a file while another, totally different user can only execute the same file. I'm not sure whether Linux uses the ACLs from an NTFS file system (but if I had to guess, I'd guess it doesn't; user IDs on Windows vary from user IDs on Linux, and as such the ACLs would not be very useful anyway, since who knows who would get permissions to access files).

Anyway, the Linux FAT and NTFS file system drivers just fudge Unix-like file permissions on all files on the FAT or NTFS file system, since neither file system stores Unix-like file permissions. In both cases, the default is to give full read, write and execute permissions to the owner of the file system (in other words, the user who mounted the file system in the first place). The executable permission is given so that you can, hypothetically, execute some file on an FAT or NTFS file system without copying it somewhere else first (and in the case of directories, the executable permission is necessary to read the directory at all). In FAT's case, the Linux implementation gives everybody some permissions by default, usually the permissions to read and execute files but not write to them. But in NTFS's case, for reasons which are too complicated to mention here* (see footnote below if you're interested), by default permissions are given only to the owner and to nobody else. Also note that these file systems, as I've said at least three times now, cannot store Unix-style file permissions, and as such when you instruct Caja to change the permissions on any of the files, the file system won't let the change happen; the file system's got nowhere for the permissions data to go.

It's also important to note that Caja's "executable" check box will only be fully checked if everybody has execute permissions. If only the owner has execute permissions, or if only the owning group has permissions, or even if the owner and everybody but the owning group has execute permission, that checkbox will not be checked or unchecked; it will be in the so-called "inconsistent" position, which indicates that the checkbox isn't quite on and isn't quite off. The inconsistent state is usually indicated by a small dash or hyphen mark like you described originally.

So let me make sure I'm clear. I have a FAT-formatted floppy diskette which shows these permissions in Caja:

fat_permissions

So everybody gets read and execute permissions, but only the owner has write permission.

This is Caja's Properties dialog on one of those files, specifically for the item on line 4:

As you'll notice, Caja shows that the file is "executable" by everyone. This matches our observation via Caja's Permissions column above.

Now here's an NTFS file system of mine:

ntfs-permissions

And here's the Properties dialog on the fourth item on that NTFS file system (names have been changed to protect the innocent):

ntfs-file-properties

As you can see, only the owner has execute permission, and therefore the checkbox is in an inconsistent state. What you can't see here is that no matter how many times I try to click that checkbox, I'm never going to be able to unset the execute permission; I'd have to remount the filesystem to remove that tricky execute bit. You can see what's going on if you try to unset the execute bit via the command line; even if you try to do so as root, there's a good chance you'll get the message:

sudo chmod u-x /media/NTFS/file
chmod: changing permissions of '/media/NTFS/file': Operation not permitted

On some versions of Linux, you'll get a different error or no error at all; in the latter case, though, the change will be immediately forgotten, and that's the (somewhat) infuriating part.


TL;DR: Chances are, you won't be able to remove the execute permissions easily. Look into the mount command (specifically the -o noexec,umask=0111 option) if you're really interested. But you can at least tell Caja to ignore the permissions by going to the Caja Preferences and checking:


Assuming you didn't already try that, I hope this works for you.


EDIT: Whoops. I forgot the footnote since I got caught up in that rambling tirade. Here it is:

* The reason for the unusual permissions is a little complicated. Unlike most other file system drivers on Linux, the NTFS file system driver is usually implemented as a "userspace" file system, which means that the file system is a program like any other you'd run. (Most file systems are implemented as a kernel module and are not run like a normal program; they have to be written very carefully to ensure bad people don't do bad things to the file system, whereas userspace file system drivers are far easier to write by comparison.) One of the design goals of userspace file systems was to ensure users could write their own file system drivers and run their own file system drivers, without special privileges; normally, to mount a file system requires special privileges behind the scenes. But the problem with allowing anybody to mount a userspace file system is that if their file system driver isn't written properly, any clueless or evil person on the system can crash or even hack and take advantage of the file system in question; file systems built as a kernel module are almost always thoroughly tested to ensure that they can withstand abuse. So to prevent ill-mannered users from taking advantage of poorly written file system drivers, by default the userspace file system component (called "FUSE") gives full permissions to whoever mounted the file system, but no permissions to anybody else. Paranoid? Maybe. Either way, the NTFS file system driver doesn't override the defaults, and as such only the user who mounted the file system has any permissions to access or use the NTFS file system.

1 Like

Thanks for the reply. I was experimenting with different sdcards and flashdrives as will as trying different combinations of partitions and files systems and I was just about to add my findings when I saw your post. 99% of what you said matches my observations. Both exfat and NTFS have there 'X' bit set and there is nothing that can be done about it. And as you said fat32 has no X set. Strangely, it seems if you had either NTFS or exfat on a partition and format to ext4 without deleteing partition to "unallocated space", you get a permission denied when trying to copy files to drive. Deleteing the partition completely then creating an ext4 drive will fix it and all rwx will be restored. However, I need the drive to be accessable to windows so the storage partition cannot be ext4. I agree that this mismatch creates very strange results.

My current work around is keeping the storage as NTFS and compressing the files before copying to the drive. Then extracting to the ext4 drive keeps the execute bit off. Alos, if I just want to open a text file, I just right click and open in leafpad. Not a clean solution but it gets the job done.

There is one strange thing, Thunar doesn't have this problem. It can open text files from a NTFS drive by double clicking. I'm in the middle of restoring my mate image because I missed it up trying to solve this. lol. So later I'll look in Thunar to see if it has the same permissions as Caja. Also the reason I didn't simply switch to thunar is I couldn't copy a file from ext4 to NTFS - no paste option. I suspect the "w" bit is off.

PS: I am getting into linux because I want to divorce from win10. Had enough. learning to setup Ubuntu Mate on Pi4 as my backup computer and eventually plan to install Arch Manjaro on my PC.

[Update]
I re-installed Thunar to compare against Caja. Very strange results.
Caja pops up the "ask" dialog
[can't post picture, too new]

but Thunar opens the text file

Yet both have the exact same permissions.
caja
[can't post picture, too new]

Thunar
[can't post picture, too new]

Furthermore, I cannot copy -> paste between ext4 and NTFS either way but I can drag and drop both ways When a text file is dragged from ext4 to NTFS, the "X" bit is set but the file still opens in thunar by double clicking and any file dragged from NTFS to ext4 will have it's "X" bit set. Weird and a pain in the butt.

My conclusion is regardless of how the permission bits are set, it's really up to the program to adhere to the permissions(at least for x), it would be really scary if programs could ignore the r and w bits too. I wonder if anyone knows why these two file browsers can behave so differently?

I think I already explained it: Thunar is probably set up to not bother executing text files when double-clicked, whereas Caja is set up to ask first. The execute bit indicates that a file can be executed (theoretically); it does not dictate that the file must be executed. Applications are unable to override the read and write permissions, unless the file system driver in question is really buggy.

To extend on what I said above in my diatribe (I can understand if you didn't read it all): When you instruct Caja or Thunar to copy a file, first they copy the file's contents, and then, if you have the appropriate permissions, they copy the file's metadata, including permissions. Except there's one problem here: Like I said above, the NTFS file system driver on Linux doesn't record file permissions, so Caja and Thunar both fail to set the file permissions correctly after copying the file from ext4 to NTFS. However, ext4 certainly supports file permissions, so when you copy the file back from the NTFS partition to the ext4 file system, your file manager will apply the new NTFS permissions to the ext4 file.

So let me lay out the chain of events here:

  1. You instruct Caja to copy file a.txt from ext4 partition to NTFS partition. Permisssions on a.txt on ext4 partition: Owner can read and write but not execute; owning group can only read; everybody else can only read.
  2. Caja copies contents of a.txt to NTFS partition. Permissions on a.txt on NTFS partition: Owner can read, write and execute; nobody else has permissions.
  3. Caja attempts to change permissions on file a.txt on NTFS partition to: Owner can read and write but not execute; owning group can only read; everybody else can only read. The permissions change operation fails since NTFS can't store Unix-style file permissions.
  4. You now instruct Caja to copy file a.txt from the NTFS partition back to the ext4 partition.
  5. Caja copies the file's contents to the ext4 partition.
  6. Caja now copies the file's permissions from the NTFS to the ext4 file system. Since the NTFS file is readable, writable and executable by the owner only, and ext4 supports Unix-style file permissions, the file on the ext4 file system also gets the NTFS-esque permissions. But at least you can change the permissions on the ext4 file.

I already explained what to do above: Did you see the picture I attached above about the Caja Preferences dialog? It was this picture:

1 Like

Thank you for your response and I did read your reply carefully. My fault for not being able to post the pictures that went along with my text(site only allowed me to post one picture and it was too late at night for me to rewrite everything). I knew from the start "open to view" is available in Caja but I use bash files more than I need to view text files so "execute" is more important than "view". Hence my purpose of jointing this forum and posting was to find a proper fix rather than a work around. I don't need any help to do a Mickey Mouse fix - I'm an experienced Mickey Mouser.

As in my previous post, I am in complete agreement with the permissions bits and that they are set after the file is copied over. Instead I was asking about why the copy behavior and execution behavior are different when the permission bits are clearly identical.

#Caja preference settings
#Double click to open item(checked)
#Run executable files when they are opened(checked)
#file permission wrxwrxwrx
Behavior:
copy-paste=successful
drag-drop=successful
bash file= execution successful
open text= ask popup appears

Vs

#Thunar preference settings
#Double click to activate items (checked)
#file permission wrxwrxwrx
behavior:
copy-paste=unavailable - paste is greyed
drag-drop=successful - ask popup appears
bash file= does not execute & no ask popup
open text= successful

I repeat, I am NOT questioning how the permission bits are set or when they are set.

Since paste is not allowed in thunar, the file was not copied over even though the 'w' bit is set to all users and groups. Now this could be do to issues with the different approach for permissions between windows and linux as you stated earlier, but because simply dragging the file to the folder bypasses any "different approach" and that one program can run bash files while the other cannot leads me to believe the issues I'm concerned about are not with the linux file system or "The lost in translation" between it and windows, but instead how the programs implement the file systems information. In other words, if someone can explain why the two programs act differently even thought they have the exact same settings operating on the exact same file on the exact same drive and file system, then maybe I can figure out a solution to my problem.

All right, now I understand your question. And there's no reason to apologize for not having enough privileges to post two or more images -- I was once a new user, too.

This sounds like a Thunar bug to me. That's really all I can say for now.