Mate 19.10 and Dell Optiplex 390 not a great pairing

Support & Help Requests Hardware
#1

We have been building/running Python based test stations on Raspberry Pi4's but because we had an issue programming old PIC micro-processor's we needed to move over to a i386 based Linux build. Loaded up Ubuntu Mate desktop on a old Optiplex 390 and everything went well until we started having issues accessing our shared Windows drives via autofs. Manually changing the directory to the /mnt/test_station was hit or miss with long stalls on the command line or even failures reading files. After digging in a little we could see that the built in Realtek RTL8111 controller was dropping RX packets(2-3 every second). After a lot of searching we saw that this issue had been seen in the past and that the suggestion was to use a slightly older driver:

We changed the Ethernet driver from r8169 to r8168 by using the link's "sudo apt-get install r8168-dkms" command. Verified that the Realtek was using the newly installed kernel driver by checking with the "lspi -v" command. Unfortunately we are still seeing packets dropping and accessing the shared network drives is very unstable. We also noted that we are not the only ones having seen this issue continue after installing the r8168 kernel drivers:

Currently we are purchasing a Ethernet card to install on the PC in hopes it will resolve this issue. Anyone have a miracle fix or should you make sure you buy a Intel Ethernet card if you want to load Ubuntu on a old Optiplex PC with a Realtek chip onboard?

Thanks

#2

2-3 dropped RX packets every second would indicate a hardware problem -- either cable issue or driver issue at the source. A few dropped packets is normal. I see as much as roughly 2-3 per HOUR. I have 6 systems with Realtek 8168/9 ethernet. All are

Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller [10ec:8168] (rev 06)

and are using

	Kernel driver in use: r8169
	Kernel modules: r8169

UM version 18.04.03
Kernel 5.0.0-37-generic #40~18.04.1-Ubuntu SMP Thu Nov 14 12:06:39 UTC 2019

djb

#3

I think @Dave_Barnes is on the right track. I'd explore hardware issues and change out the cable as my first trouble shooting step.

I've got a OptiPlex 390 as a lab machine and have no issues like you are experiencing with the driver.

Network: Card: Realtek RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller driver: r8169
IF: enp3s0 state: up speed: 1000 Mbps duplex: full

#4

We run the Raspberry Pi4 using the same cable in the exact same test setup without issue. When you say "driver issue at the source", are you saying maybe the Realtek chip itself? We do have another couple systems to try.

Thanks

-----Original kernel setup----------------

03:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 06)
Subsystem: Dell RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller
Flags: bus master, fast devsel, latency 0, IRQ 16
I/O ports at 2000 [size=256]
Memory at e0a10000 (64-bit, prefetchable) [size=4K]
Memory at e0a00000 (64-bit, prefetchable) [size=16K]
Capabilities: [40] Power Management version 3
Capabilities: [50] MSI: Enable- Count=1/1 Maskable- 64bit+
Capabilities: [70] Express Endpoint, MSI 01
Capabilities: [b0] MSI-X: Enable+ Count=4 Masked-
Capabilities: [d0] Vital Product Data
Capabilities: [100] Advanced Error Reporting
Capabilities: [140] Virtual Channel
Capabilities: [160] Device Serial Number 03-00-00-00-68-4c-e0-00
Kernel driver in use: r8169
Kernel modules: r8169

----After new Kernel driver--------------
03:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 06)
Subsystem: Dell RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller
Flags: bus master, fast devsel, latency 0, IRQ 27
I/O ports at 2000 [size=256]
Memory at e0a10000 (64-bit, prefetchable) [size=4K]
Memory at e0a00000 (64-bit, prefetchable) [size=16K]
Capabilities: [40] Power Management version 3
Capabilities: [50] MSI: Enable+ Count=1/1 Maskable- 64bit+
Capabilities: [70] Express Endpoint, MSI 01
Capabilities: [b0] MSI-X: Enable- Count=4 Masked-
Capabilities: [d0] Vital Product Data
Capabilities: [100] Advanced Error Reporting
Capabilities: [140] Virtual Channel
Capabilities: [160] Device Serial Number 03-00-00-00-68-4c-e0-00
Kernel driver in use: r8168
Kernel modules: r8168

---------ifconfig-------
enp3s0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.1.138 netmask 255.255.255.0 broadcast 192.168.1.255
inet6 fe80::7990:ab3b:c414:ac93 prefixlen 64 scopeid 0x20
ether d0:67:e5:0c:b3:27 txqueuelen 1000 (Ethernet)
RX packets 718491 bytes 82511683 (82.5 MB)
RX errors 0 dropped 151520 overruns 0 frame 0
TX packets 37714 bytes 2473611 (2.4 MB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
device interrupt 27 base 0x5000

#5

Ethernet PHY transmitter interface is what I meant by "driver".

What mode (10/100/1000 bit) is the Realtek chip using? If you have a wiring issue, the chip will log errors trying to use gigabit mode. Gigabit ethernet requires all 4 twisted pairs (8 wires). What mode does the Broadcom ethernet on the PI4 use?

I have also seen cases where a flickering florescent light caused enough noise to bother ethernet cabling. Ground loops can cause errors too.

We used to use neon bulbs to protect ethernet devices (chips) from lightning. I've seen ethernet magnetics damaged by a close lightning strike.

Some ethernet PHY interfaces are more sensitive to noise than others. With that said, I see the same error rate (2-3 / hour) with my gigabit Broadcom ethernet stuff.

djb

#6

We took your advice and ended up pulling the hard drive from the original system and plugged it into an identical 390. We still see RX packets being dropped @ ~ 2/sec, but access to our shared windows drive is running well(we had also tried a different Ethernet cable and saw no difference on the original system.) When we saw the shared drive access problems and dropped packets we thought we had our issue. I'll bring the Pi4 up tomorrow to double check the dropped packets.
Is there any way in Ubuntu to see why the packets are being dropped?

Thanks.

#7

The driver accepts a debug parameter, with 0=no debug and 16=max debug. I haven't debugged any drivers in a long time.... I think you would need to create an file in /etc/modprobe.d called r8169.conf with the following contents:

options r8169 debug=1

If debug=1 does give the results you want, try 2. Output probably goes to syslog. The best way to watch syslog for debugging is

tail -f /var/log/syslog

you can use ethtool to look for some of the problems.

ethtool -S eth0

should give some driver statistics but on my system it does not show RX packets dropped at all. I don't see an obvious way to get any useful info for RX packets dropped with ethtool - the r8169 driver won't let you see the ring buffer statistics so you can't check for overruns.

Can you tell whether the r8169 is in 100 or 1000 mode? Full or half duplex?
Look at

"/sys/class/net/{your network device}/speed"

and

"/sys/class/net/{your network device}/duplex"

If you can, look at your network switch to see what mode the switch says your port is. Here is what my BaysStack says:
image

I do have some cheap and very old ethernet cards that do not autonegotiate correctly and exhibit the same problems as you are describing.

djb

#8

Dave, here is the update. We tried every combination of manual/auto Ethernet settings with no success. This morning I started up the Pi4 and it also showed packets dropping. So my guess is that the packets are probably some really old equipment/protocol on our network that the stack is unhappy with but that is not related to the problem with the Windows shared drive . So when we saw dropped packets, windows drive sharing problems and old network card posts, we incorrectly thought they were all the same issue. There is clearly something wrong with our original 390 we used, but using the exact same drive on a different 390 proves that it is some internal hardware issue on our original machine. We will have a secondary Ethernet card in today so I will update this thread with the results.

So to summarize this issue, the Dell Optiplex 390 is a viable option running Ubuntu Mate.

Thanks for all the help resolving this issue.

John C.