So, this is a known issue: Realtek 2.5GBe ethernet on B550 mobo does not work with Linux because reasons.
$ lspci -knn | grep Eth -A3
06:00.0 Ethernet controller : Realtek Semiconductor Co., Ltd. RTL8125 2.5GbE Controller [10ec:8125] (rev 05)
DeviceName: RTL8111E Giga LAN
Subsystem: Gigabyte Technology Co., Ltd RTL8125 2.5GbE Controller [1458:e000]
Kernel modules: r8169
$ dmesg | grep -i r8169
[ 2.931313] r8169 0000:06:00.0: unknown chip XID 641
Arch users have a workaround here which include detailed steps to build the r8125 package, install it and load it up.
I've tried looking for similar solutions in the Ubuntu forums but I couldn't find any with detailed steps to do the same.
I understand that support has been added to the linux-next tree but it'll take a long while before it gets into Ubuntu. (I heard it will be Kernel 5.9, and we're on Kernel 5.4).
Thanks for reading, any help is appreciated.