Have a look through this thread where a few links are given to assist with this https://forums.macrumors.com/threads/7800gtx-fx4500-working-in-ubuntu-mate.2035433
Especially this part:
Nouveau driver has issues with MSI interrupts, use yaboot parameter
nouveau.config=NvMSI=0You can also add it to your /etc/yaboot.conf and update using sudo ybin -v (see the FAQ for more info)"
https://wiki.ubuntu.com/PowerPCKnownIssues
Basically I had to install initially with the original 6600 card, this worked well except for 2d/3d acceleration. I got these working with the MSI Interrupt above. Then after running updates I tried the 7800GTX/FX4500 and it worked except for a garbled login screen - when I enter the password it is all fine and corrects itself.
So I would try getting 2d/3d all working with your 6600, make sure your system is all up to date - and then try inserting the FX4500 and see what happens (if you get a garbled login screen just enter password, hit enter and see if it sorts itself going into the desktop.