Ah my dear Jim. Keeping politics out of FOSS software, what a pipe dream, have you read RMS' or ESR's blog posts? Richard makes no apologies about being a communist and Eric makes no apologies about being right wing pro gun liberaterian. RMS is the founding father of the Free software movement & GNU and Eric is the founding father of OSS. With these kinds of founding fathers, FOSS is definitely not going to be a knitting club.
When I switched over to Linux and Ubuntu about 10 years ago, I was both shocked and amused by the flame wars and fanaticism over things which most people would consider inconsequential: C vs C++, Gnome vs KDE, Vim vs Nano, QT vs GTK, Systemd vs anything not systemd, X11 vs Wayland, Debs vs Snaps/Flatpaks, etc, you get the picture. Reading through some of the forum posts as a noob was like watching an episode of Jerry Springer, entertaining but thinking to yourself, why would anyone care that much about using one DE over another or why would anyone care that much if an app is based on QT or GTK? Obviously FOSS/Linux purists do. The corporate influence on FOSS has sanitised this a bit and replaced these flame wars with their own corporate approved talking points and flamewars, which could turn on a dime tomorrow.
The latest X11 vs Wayland vs Xlibre vs some other fork, is just the latest episode in the FOSS flame wars, with die hard fanatics on both sides getting in each others faces. Every time I miss Jerry Springer, I go on a flame war forum to get my Jerry Springer fix. The only reason I'm on this forum and use UM is that it's not an episode of Jerry Springer and it should stay that way.
The only reason that most FOSS devs hate on Xlibre is because they hate the politics of the main dev/maintainer, and vice versa, they love the app/project because of the politics of the devs/maintainers. That's the FOSS/Linux "community" in a nutshell and why I generally avoid that "community".
To the average user like myself, I don't care if the display server is X11 or Wayland or Xlibre or Perfect Display Server 1.0, as long as it works and does what it's supposed to do. Wayland doesn't have some of the functionality regarding remote desktop & accessibility, since I don't need these features, I'm not concerned that X11 is being replaced by Wayland. Those that do, should be working on a drop in replacement for X11, whether that's Xlibre or something else. And right on cue, when you post this type of comment on a normal "community" forum, the flame wars start, either defending Wayland as the perfect display server ever made or X11 as the best thing since sliced bread because it has that extra functionality. I can't take that level of fanaticism over a piece of software seriously, it's not even a football team, for God's sake!
When I migrated to Ubuntu 14.04 from Windows 8, I really liked the Unity desktop, it was functional and looked cool and didn't resemble any other DE on the market. I was really upset when Ubuntu moved to Gnome, as you lost most of the cool Unity features with Gnome. That's when I started my search, and found the Mate DE to replicate my Unity DE experience, and I've been a happy Mate user ever since! I use debs and snaps and flatpaks as neccesary to get my work done. I spend a lot of time repurposing old and refurbished hardware into new usuable systems, so I'm not fanatical about what's under the hood, as long as it works.
Getting back to your point about getting politics out of FOSS, that's impossible as FOSS and the GPL/Copyleft are very political by their very nature, just ask RMS. Free software and the GPL are political by design and that's what RMS wanted in the first place.
I'm not a dev, but from what I've seen so far in the FOSS community, is that the best devs have the wackiest views and opinions because all they do all day is code and sit on social media/reddit & vent, so don't really have a life outside of the dev bubble. I have a good friend that works for Microsoft as a low level assembler dev and he's no different. And he's a really good dev, really knows his stuff and even troubleshoots low level python & linux kernel issues for WSL.
The FOSS community just needs to accept this as a fact and just move on. Politics goes hand in hand with FOSS and the best devs are the ones with the wackiest or fanatical or out there views on various things. Linus isn't exempt from this either, and he created the Linux kernel and gave Nvidia the middle finger for not writing proper Linux drivers. These characters make the FOSS community the entertaining & lively, and why FOSS doesn't have the corporate beige box cubicle vibe. Software users in the FOSS community don't value these people enough in my humble opinion.