@tkn @stephematician IT HAS BEEN FIGURED OUT 
Through trials and tribulations I have gotten this 2008 Mac Pro 3,1 A1186 working in many areas. Installed the RX580 ARMOR OC 8G after the RX550 OC 2G. Everything is now working wonderfully, I have found the time and mindset to figure things out. In the past I never had gotten back to putting how/what I found and did back on here for others to find. This time, I will not make that mistake.
I have a Triple Boot setup. I 90% use Mate 24.04 LTS on HD1(sda2), OSX HD2(sdb2), Windows 7 (sda3)(EFI setup currently unfortunately CSM), HD4(sdd4) exFAT (any file transfer). I have to remove windows drive bc of CSM boot @this time. I had originally set up windows with BIOS BOOT/CSM/bootcamp with past Mate Bios boots. The system was extremely slow starting, and response with mate was even slow. Everything is on separate drives for multitudes of reasoning. Linux and OS X are EFI boot.
The biggest hurdles are getting over all three working together for boot/boot display. I have kept 2 backup cards (OEM mac cards) that were both original to either a Pro 1,1 or 3,1. Nvidia GeForce 7300 GT, ATI Radeon HD 2600 XT(RV630). These two have been irreplaceable in the process of graphics. If you are to do anything with graphics and any display anything, I would suggest getting/keeping your mac flashed card before figuring out your next steps. You will need one at some point, i guarantee it.
I have gotten BOTH OSX AND Mate 24.04.3 LTS working TOGETHER at boot and graphics display. YES. This is possible, and through much much much research and time it is possible. “The 2008 Mac Pro cannot work with the RX580” and “SSE 4.2 with CMP 3,1 will not work”. It does, its sitting right next to me. EVEN WITH OS X 10.15. IT WORKS. Non-Mac card, NO SSE 4.2, Mac. Linux does not have such a problem with it, but I’m not using it for anything special here. Just Windows for TSM 2020, this is the whole reasoning I upgraded in the first place with what has become a TREK. Before anything, if you have a Mac Pro or CMP 1,1-5,1 bookmark this page!!!
I have currently gotten the RX580 to boot linux with the HDMI cord plugged in, and to boot. With “quiet splash amdgpu.dc=0” OR amdgpu.dc=1 I would have to boot with the cord UNPLUGGED until i heard the login chime. IF i tried to keep it plugged in, the computer would have a boot hang, never booting, just sitting there like a brick. Without putting all the jabble i found, what currently is working with the cable plugged in and booting is:
GRUB_CMDLINE_LINUX_DEFAULT="amdgpu.dc=0 radeon.modeset=0"
no splash. I have wanted GRUB to show (which it will with the mac card) and this is because of the drivers and handoff. I have tried and tried and tried doing everything i have read. It does not like anything, so we will go to OS X for that.
GRUB_DEFAULT=0
GRUB_TIMEOUT_STYLE=menu
GRUB_TIMEOUT=10 GRUB_CMDLINE_LINUX_DEFAULT="amdgpu.dc=0 radeon.modeset=0" GRUB_DISABLE_OS_PROBER=FALSE GRUB_TERMINAL=console
lspci -k | grep -EA3 'VGA|3D|Display'
02:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere [Radeon RX 470/480/570/570X/580/580X/590] (rev e7)
Subsystem: Micro-Star International Co., Ltd. [MSI] Radeon RX 580 ARMOR 8G OC
Kernel driver in use: amdgpu
Kernel modules: amdgpu
root# glxinfo | grep "direct rendering"
direct rendering: Yes
root# inxi -G
Graphics:
Device-1: AMD Ellesmere [Radeon RX 470/480/570/570X/580/580X/590]
driver: amdgpu v: kernel
Display: x11 server: ``X.Org`` v: 21.1.11 driver: X: loaded: amdgpu
unloaded: fbdev,modesetting,vesa dri: radeonsi gpu: amdgpu
resolution: 1600x900~60Hz
API: EGL v: 1.5 drivers: radeonsi,swrast
platforms: gbm,x11,surfaceless,device
API: OpenGL v: 4.6 compat-v: 4.5 vendor: amd mesa
v: 25.0.7-0ubuntu0.24.04.2 renderer: AMD Radeon RX 580 Series (radeonsi
polaris10 ACO DRM 3.57 6.8.0-88-generic)
API: Vulkan v: 1.3.275 drivers: N/A surfaces: xcb,xlib
So OS X. The 3,1 only goes to 10.11.6…you mentioned 10.15?? YES, yes I did!! Wait…but the 1,1-3,1 don’t have SSE 4,2 CPU set instructions, they “will never work with an RX580”. These posts you will read are from 7-8 years ago, disregard them. Though I would just like grub, and though I never really use OS X It comes to use here and there. I have Macbook 7,1 (running mate 24.04+ OS X 10.13) also but that has given me 20,000 less headaches even with Nividia drivers!!! I found a boot picker that works with linux, with mac startup screen, GOP injection for non-mac card, and with that works with the (it will never work) MSI RX580 ARMOR OC 8G. Meet Dosdude1 and Dortania!!! Dosdude2 uses SSE 4.2 emulation for OS X Sierra-Catalina. Dortania is like rEFInd though over Clover and rEFInd, using OpenCore is easier for boot process keeping it on the Mac EFI HD.
With dosdude1’s installations the RX580 will work with OS X, though there is no boot screen. With dordania’s OpenCore you will get a boot picker, GOP injection(non-mac card), and the boot screen, and YES THIS WORKS with dosdude1’s installation. (i found many answers) but, after his install I installed OpenCore and it works flawlessly. THOUGH, if you have windows with CSM/BIOS boot, it will NOT be found by OpenCore boot picker. At this point, pick EFI only boot, or BIOS only boot. Bootcamp was a waste of my life at this point, and getting windows on a flash drive to boot on this thing is a whole other hurdle, i just stayed to these problems. I could care less, i hate windows and the only use is to play that game. OpenCore wont find it during boot, and Bootrepair wont change grub boot with CSM ON.
efibootmgr is set to 0080,0000. 0080=OSX(hd2/sd1efi), 0000=linux(HD1/sda1efi). Opens Opencore–>boot picker—>pick “EFI” (linux) “Catalina” OSX or “EFI” (USB boot).
A 64bit EFI 1.5 Apple Mac Pro 3,1 2x2.8ghz with a RX580 OS X, Linux, and (sort of) Windows 7.
root# inxi -Fxz
System:
Kernel: 6.8.0-88-generic arch: x86_64 bits: 64 compiler: gcc v: 13.3.0
Desktop: MATE v: 1.26.2 Distro: MATE 24.04.3 LTS (Noble Numbat)
base: Ubuntu
Machine:
Type: Unknown System: Apple product: MacPro3,1 v: 1.0 serial:
Mobo: Apple model: Mac-F42C88C8 v: Proto1 serial: N/A UEFI: Apple
v: MP31.88Z.006C.B05.0802291410 date: 02/29/08
CPU:
Info: 2x 4-core model: Intel Xeon E5462 bits: 64 type: MCP SMP arch: Penryn
rev: 6 cache: L1: 2x 256 KiB (512 KiB) L2: 2x 12 MiB (24 MiB)
Speed (MHz): avg: 2400 min/max: 2400/2800 cores: 1: 2400 2: 2400 3: 2400
4: 2400 5: 2400 6: 2400 7: 2400 8: 2400 bogomips: 44688
Flags: ht lm nx pae sse sse2 sse3 sse4_1 ssse3 vmx
Graphics:
Device-1: AMD Ellesmere [Radeon RX 470/480/570/570X/580/580X/590]
vendor: Micro-Star MSI driver: amdgpu v: kernel arch: GCN-4 bus-ID: 02:00.0
temp: 38.0 C
Display: x11 server: ``X.Org`` v: 21.1.11 driver: X: loaded: amdgpu
unloaded: fbdev,modesetting,vesa dri: radeonsi gpu: amdgpu
resolution: 1600x900~60Hz
API: EGL v: 1.5 drivers: radeonsi,swrast platforms:
active: gbm,x11,surfaceless,device inactive: wayland
API: OpenGL v: 4.6 compat-v: 4.5 vendor: amd mesa
v: 25.0.7-0ubuntu0.24.04.2 glx-v: 1.4 direct-render: yes renderer: AMD
Radeon RX 580 Series (radeonsi polaris10 ACO DRM 3.57 6.8.0-88-generic)
API: Vulkan v: 1.3.275 drivers: N/A surfaces: xcb,xlib devices: 2
Audio:
Device-1: Intel 631xESB/632xESB High Definition Audio driver: snd_hda_intel
v: kernel bus-ID: 00:1b.0
Device-2: AMD Ellesmere HDMI Audio [Radeon RX 470/480 / 570/580/590]
vendor: Micro-Star MSI driver: snd_hda_intel v: kernel bus-ID: 02:00.1
API: ALSA v: k6.8.0-88-generic status: kernel-api
Server-1: PipeWire v: 1.0.5 status: active
Server-2: PulseAudio v: 16.1 status: off (using pipewire-pulse)
Network:
Device-1: Intel 80003ES2LAN Gigabit Ethernet driver: e1000e v: kernel
port: 1020 bus-ID: 07:00.0
IF: enp7s0f0 state: down mac:
Device-2: Intel 80003ES2LAN Gigabit Ethernet driver: e1000e v: kernel
port: 1000 bus-ID: 07:00.1
IF: enp7s0f1 state: down mac:
Device-3: Broadcom BCM4321 802.11a/b/g/n vendor: Apple AirPort Extreme
driver: wl v: kernel bus-ID: 0d:00.0
IF: wls6 state: up mac:
Bluetooth:
Device-1: Apple Bluetooth HCI driver: btusb v: 0.8 type: USB bus-ID: 4-2:3
Report: hciconfig ID: hci0 rfk-id: 2 state: up address: bt-v: 2.0
lmp-v: 3 Sensors:
System Temperatures: cpu: 51.0 C mobo: 42.5 C gpu: amdgpu temp: 38.0 C
Fan Speeds (rpm): cpu: 500 gpu: amdgpu fan: 208
Info:
Memory: total: 16 GiB note: est. available: 13.64 GiB used: 3.43 GiB (25.1%)
Processes: 329 Uptime: 18h 37m Init: systemd target: graphical (5)
Packages: 2559 Compilers: gcc: 13.3.0 Shell: Bash v: 5.2.21 inxi: 3.3.34