Safe to remove USB device?

Support & Help Requests
#21

I believe that it’s due to how the kernel “caches” and “buffers” the drive. It completes the operation sooner (to the program) while the kernel is still writing it to disk. It seems there is a way to turn it off, but I haven’t tested this:

I’m no developer when it comes to hardware, kernel or the file manager, but seeing as 18.04 has the feature, I’m not sure if 16.04 will receive any updates in this area without risking stability as a Long Term Release (LTS).

It could be the kernel version – v4.4 is the one from 16.04 originally, but can be upgraded to v4.10 (and later) with the LTS Enablement Stack.

That said, I’ve yet to have any corruption issues with USB drives, and I’m still rocking 16.04. Instead I observe my system’s resources using indicator-multiload (MATE has a similar applet called System Monitor too) and so if I eject from the file manager, I can usually see there is still some disk write activity and then the RAM (buffer/cache) usage goes down.

If it just happens to one drive, there is a possibility the medium itself is corrupted. I’ve known this with a bunch of old floppy disks :floppy_disk: Even after successfully letting it write and flushing the cache/buffer (without ejecting):

sudo bash -c 'sync; echo 3 > /proc/sys/vm/drop_caches'

The file ended up corrupted, checksums didn’t match.

#22

The link you provided regarding turning off write cache states that you have to enter the command every time the system is rebooted, this isn’t really practical. In relation to using system monitor to monitor the disk activity and ram buffer cache, I cannot see this under system monitor anywhere?

It’s not an issue with the USB stick as the device is brand new, it’s definitely due to the progress indicator that’s effectively useless.

I’m also running kernel 4.17.19. Updating the kernel hasn’t made a difference and as a long term Ubuntu MATE user I can assure you this has been an issue for quite some time, it was present on 15.04 and every time someone complains about it people claim they haven’t experienced it so it’s a non issue - Yet they use methods to monitor buffer status before removing USB storage devices?

LTS or not, this is a fairly sizeable issue resulting in personal data corruption and I’m one of two people to complain of it in a week.

#23

Can you see if this helps File copy to USB appears to finish too fast
Instead of adding the command applet to the mate panel, you could set up Conky.

#24

Mostly as a suggestion to check if disk caching is indeed the culprit. If turning off write cache "solves" the problem on your next writes, then it will help isolate where the problem is for further drilling down. If it improves the situation, they'll be a way to make it permanent in the meantime.

With stock MATE System Monitor's applet, you can only observe graphs with basic tooltips. With indicator-multiload, you can click and configure it to show buffer/cache values and colour code the RAM graph like so:

When ejecting, you can see there's disk writing until it goes back to 0 kB/s. Some disks continue writing 100 MB/s after ejecting and others, 32 MB/s.

Certainly wasn't dismissing this as "non-issue". Considering 16.04 is mostly for security updates, 18.04 contains the latest developments with the new "safe to remove" notification. Thinking about it... I'm not even sure which package triggers that notification.

For all we know, specific hardware causes this behaviour, especially USB controllers and hubs, which may explain why some people experience it and others don't.

#25

The generic USB Storage Device specification leaves a lot to be desired - for example, although essentially all USB thumb drives are flash based, there is no way to apply an Erase or Trim command, which totally screws up the drives ability to do proper wear leveling, and the protocols for controlling and reporting internal cache status are not well standardized .

So even if we have MATE tap into the Linux USB stack, and trip a notification to indicate that the device is no longer being streamed data from Linux’s disk cache (so the USB interface is no longer sending any data to the drive), that drive could still be caching data internally.

To get around these issues I always buy card readers and USB thumb drives which have a write indicator LED, then use the ‘safe remove’ and wait 10 to 20 seconds after the LED stops flickering.

2 Likes
#26

The problem is that even considering the issues relating to flash storage, every other OS seems to be able to adequately detect when data transfer has completely finished and inform the user that the USB stick is definitely safe to remove.

As it stands at the moment, the progress indicator under Ubuntu MATE is totally and utterly out of whack by about 10 mins regarding USB flash drives.

#27

As of today I think that Ubuntu solved the problem, see my post there:


OK it might need more testing BUT the speed and the behavior of the loading bar was very different than usual (both at home and at the office) and matched the blinking of the USB dongle.

1 Like
#28

Very interesting, I’ll have to keep an eye on this myself as I feel it’s a very serious concern.

#29

Take a look at this workaround – you can tweak a config file so the kernel will physically finish writing to disks sooner.

1 Like
#30

Very interesting. Thanks.

I’ll give it a go.