Honestly, I don't really know anything about PulseAudio or ALSA or much about Ubuntu actually, I've just done lots of Googling and eventually found something that worked.
I use VLC and Audacity, and there's an option in both of those programs to use ALSA, so I did that, but a big problem, Firefox doesn't support ALSA. I don't really know what to do about that, switch browsers? As long as VLC and Audacity work in stereo I'm okay with it.