USB Write Speed 10:1 Variance

fey42 · 5 October 2018 17:14

I’m running a freshly install 18.04.
In trying to sort out my USB ports on my HP 8470P I ran into a mess.
I started with a freshly formatted NTFS SanDisk Cruzer Glide 3.0 32GB USB
Changed over to that drive and ran the following commands

fred@Fred-UM:/media/fred/Grey$ sudo dd if=/dev/zero of=./zero.txt bs=1M count=1000
1000+0 records in
1000+0 records out
1048576000 bytes (1.0 GB, 1000 MiB) copied, 5.78271 s, 181 MB/s
fred@Fred-UM:/media/fred/Grey$ rm zero.txt
fred@Fred-UM:/media/fred/Grey$ sudo dd if=/dev/zero of=./zero.txt bs=1M count=1000
1000+0 records in
1000+0 records out
1048576000 bytes (1.0 GB, 1000 MiB) copied, 56.0523 s, 18.7 MB/s
fred@Fred-UM:/media/fred/Grey$ rm zero.txt
fred@Fred-UM:/media/fred/Grey$ sudo dd if=/dev/zero of=./zero.txt bs=1M count=1000
1000+0 records in
1000+0 records out
1048576000 bytes (1.0 GB, 1000 MiB) copied, 57.5003 s, 18.2 MB/s
fred@Fred-UM:/media/fred/Grey$ rm zero.txt
fred@Fred-UM:/media/fred/Grey$ sudo dd if=/dev/zero of=./zero.txt bs=1M count=1000
1000+0 records in
1000+0 records out
1048576000 bytes (1.0 GB, 1000 MiB) copied, 56.9522 s, 18.4 MB/s

I ran all the test back to back using the up arrow to retrieve the appropriate command.
Anybody have any idea of why it exhibits a 10:1 variance in write speed?

Thanks, Fred

pavlos_kairis · 5 October 2018 17:58

Fred, I tried this and get consistent numbers around 1.0 GB/s (my system has an SSD drive)

while true; do dd if=/dev/zero of=zero.txt bs=1M count=1000; rm zero.txt; done

Norbert_X · 5 October 2018 20:39

Dear Fred!

I can recommend to check the speed of USB-ports instead of checking write speed. Speed may vary depending on system load and cache operations.

Ubuntu has special package for USB. It is named usbutils. And it is installed by default.
You can check port speeds with simple command lsusb -t.
Below is output on my system:

$ lsusb -t /: Bus 04.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/4p, 5000M |__ Port 1: Dev 3, If 0, Class=Mass Storage, Driver=usb-storage, 5000M <-- note this /: Bus 03.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/4p, 480M |__ Port 2: Dev 4, If 0, Class=Mass Storage, Driver=usb-storage, 480M <-- note this |__ Port 3: Dev 3, If 0, Class=Human Interface Device, Driver=usbhid, 12M |__ Port 3: Dev 3, If 1, Class=Human Interface Device, Driver=usbhid, 12M /: Bus 02.Port 1: Dev 1, Class=root_hub, Driver=ehci-pci/2p, 480M |__ Port 1: Dev 2, If 0, Class=Hub, Driver=hub/6p, 480M |__ Port 5: Dev 9, If 0, Class=Video, Driver=, 480M |__ Port 5: Dev 9, If 1, Class=Video, Driver=, 480M |__ Port 6: Dev 4, If 0, Class=Wireless, Driver=btusb, 12M |__ Port 6: Dev 4, If 1, Class=Wireless, Driver=btusb, 12M /: Bus 01.Port 1: Dev 1, Class=root_hub, Driver=ehci-pci/2p, 480M |__ Port 1: Dev 2, If 0, Class=Hub, Driver=hub/6p, 480M |__ Port 4: Dev 3, If 0, Class=Vendor Specific Class, Driver=rtsx_usb, 480M
I have connected two flashes - USB 2.0 and USB 3.0 They use usb-storage driver.
Their speeds are shown correctly (480M and 5000M respectively). You can test yours.

fey42 · 5 October 2018 22:28

Tried the same thing and here is what I received:

fred@Fred-UM:/media/fred/Grey$ while true; do dd if=/dev/zero of=zero.txt bs=1M count=1000; rm zero.txt; done
1000+0 records in
1000+0 records out
1048576000 bytes (1.0 GB, 1000 MiB) copied, 5.44285 s, 193 MB/s
1000+0 records in
1000+0 records out
1048576000 bytes (1.0 GB, 1000 MiB) copied, 54.2115 s, 19.3 MB/s
1000+0 records in
1000+0 records out
1048576000 bytes (1.0 GB, 1000 MiB) copied, 5.4261 s, 193 MB/s
1000+0 records in
1000+0 records out
1048576000 bytes (1.0 GB, 1000 MiB) copied, 63.1139 s, 16.6 MB/s
1000+0 records in
1000+0 records out
1048576000 bytes (1.0 GB, 1000 MiB) copied, 5.51158 s, 190 MB/s
1000+0 records in
1000+0 records out
1048576000 bytes (1.0 GB, 1000 MiB) copied, 64.2027 s, 16.3 MB/s
1000+0 records in
1000+0 records out
1048576000 bytes (1.0 GB, 1000 MiB) copied, 5.47892 s, 191 MB/s
1000+0 records in
1000+0 records out
1048576000 bytes (1.0 GB, 1000 MiB) copied, 63.2644 s, 16.6 MB/s
1000+0 records in
1000+0 records out
1048576000 bytes (1.0 GB, 1000 MiB) copied, 5.48504 s, 191 MB/s

Maybe it is because I am writing to a USB

Thanks, Fred

pavlos_kairis · 5 October 2018 22:42

There is a lot of variance in your results so you may try the disks utility (gnome-disks) which allows you to benchmark a device. It will benchmark read, write, and access time. Try it on the usb stick or sdcard and see if you get similar values.

fey42 · 5 October 2018 22:44

I ran lsusb -t and it showed similar numbers.
But that is some theoretical number, not a real world number.
My dd speed shows all over the map from 196MB/s to 14.3MB/s
My system is simply booted up and doing only this, plus all it's background tests.
Next, I don't even understand what it is telling me.
I have four jacks I can plug into. not 20 and that does not tell me which one is in the back left hand corner, which was my whole intent and to see how fast it wrote to a USB3 vs a USB2.

Thanks, Fred

fey42 · 5 October 2018 22:46

Sorry I am so new to this I don't know how to access that.
Please show me.
Thanks, Fred

fey42 · 5 October 2018 22:54

@pavlos_kairis
Found it, but to no avail:

I get the same error on both the SSD and the USB
Now what?
Thanks, Fred

pavlos_kairis · 5 October 2018 23:08

eject the usb and reconnect it. You should not be getting this error.

I placed my 8GB usb drive and started disks, benchmark, see image. Reads speed is blue, writes is red. You can read the info at the bottom part of the image.

fey42 · 6 October 2018 00:09

@pavlos_kairis
It even complained on ejecting, but that solved the problem.

Using dd I saw a range of 10:1, using Disks, I see a range of 4:1 on 10MiB writes, on 100Mib and 1GiB it is essentially flat at 16.4MB/s and 15.7MB/s.

Looking around on the Internet I discovered this USB is an under performer.
SanDisk Cruzer Glide 3.0 USB Device Read 97.72 MB/Sec Write 29.29 MB/Sec.
Disks shows worse.

But it begs the question even more – what is going on?

Thanks for all the help, Fred

Bill_MI · 6 October 2018 10:59

Hi Fred,

There’s a lot going on writing Flash memory. The controller in the stick could be slowing down for 1) heat 2) write-behind 3) buffering reasons to name a few.

But most of these reasons will depend on time. If the stick is idle for a long time (I’m talking maybe a minute or more) is the first measurement always highest? It all may depend how soon after the last test you start a new one.

Your data does look like a distinct throttled rate comes into play. I’ll wager that’s for heat purposes.

fey42 · 6 October 2018 14:23

@Bill_MI

About 80% of the time.

I ran some test on my 2TB WDC My Passport last night and it shows the problem is not unique to USB Sticks. Note there are two different Sample Sizes

Each test was run after disconnecting the drive and plugging it back in to try and remove any after effects of the previous test.

Thanks, Fred

USB 3.0 Port Sample Size = 100MiB

USB 3.0 Port Sample Size = 10MiB

USB 2.0 Port Sample Size = 100MiB

karnemelk · 20 October 2018 08:56

Fred my man that because you use dd without the right flags.

Try this instead with oflag=sync

sudo dd if=/dev/zero of=./zero.txt bs=1M count=1000 oflag=sync

Cheers mate!