Switch between Wayland and X11

[arQon]

@tkn

You may have one or two valid points in your rant, but please tone it down a bit.

I've just re-read that post, and I don't see any ranting in it. Some unpleasant truths, certainly, but nothing emotional, rude, or personal - unless you want to count the part that's explicitly spelt out as "the needs of the many" vs "the needs of the few" as personal in some way, but that would be quite a stretch.

@bornagainpenguin doesn't deserve abuse

Obviously. Please point out where you think the abuse is so that I can rephrase or remove it.

Also , he has several valid points too.

Yes, he does. I'm not blaming him for this mess: he's a casualty of it, not the cause.

Listen to what Daniel Stone, one of the developers of X11, has to say about X11 vs Wayland

Thanks, but I've been aware of Wayland for around a decade at this point. I'm very familiar with not just the stated goals but also what's happened with it over the years, and even why some of those things happened when and how they did.
I'm sure you noticed, but the video you linked is from early 2013, nearly 10 years ago. 10 years later, Wayland is not only still unusable for an awful lot of cases, but has only just recently started work on HDR, which is hardly "new technology" that just appeared days ago and took everyone by surprise. I don't think an appeal to authority helps there, as it still presents quite a paradox: either the "authority" was somehow unaware of a topic that had been discussed at length since the mid-90s, in which case they are far from authoritative; or they were aware of it but unwilling or unable to deal with it for 14 years, in which case you can take your pick of any number of options, but none of them are particularly encouraging or flattering.

It's very hard to defend a project like this taking 15 years in the first place, but even that aspect isn't really my main issue with it: there are large chunks of functionality in X that people depend on, which the Wayland team is explicitly opposed to, and their position is "f**k those users".

That attitude is why this saga is still ongoing and shows no signs of ever ending well. It's also putting developers (which is my trade) in the impossible position of having to either burn resources that they don't have supporting two backends, or being harassed over "Wayland support". Developers of DEs are especially bearing the brunt of this, because Wayland goes out of its way to maximize the development costs for everyone that isn't Red Hat.

I find the victims of this cynical behavior much more sympathetic than the people who instigated it. You're free to feel differently.

[arQon]

@bornagainpenguin

It's entirely possible I misjudged your post and completely misunderstood it.

It's also entirely possible I wrote it poorly, since I tend to not have time for forums etc until late in the evening.
Still, let's at least get the first piece out of the way first:

XWayland is "the thing that makes most X programs work on Wayland". Think of it like WINE, if you're familiar with that. That's what the "you'll be running some form of X for a very, very long time yet" part means: the majority of X-based software will simply never be rewritten for Wayland, so without that Wayland would cause even more damage to "the Linux desktop" than it already has.

How would I know? I'm not a programmer.

Yeah, that's fair - and you shouldn't have to be.

Windows 11 seems to have support for doing this. ChromeOS has support for doing this.

That's much too long a topic for me to cover tonight, but, maybe this will help clarify:
On one hand, you have "Continue development of AnBox". On the other, you have "Continue development of AnBox, AND rewrite a chunk of it to use Wayland instead of X". Programmer or not, I'm pretty sure you're not going to have a hard time figuring out which of those involves more work.

I do have what I consider a valid use case though and I dislike that being summarily rejected as being just a 'toy' so I took it harder than perhaps I should have.

Ah, no - and this, I suspect, is the point of confusion. While I'm obviously at fault for not being clear enough in my reply, I want to point out that the "toy" aspect predates your post. I was talking about the handful of examples used to showcase Wayland in the same way that Compiz's "wobbly windows" etc was used to showcase how "cool" a Linux desktop could be a decade ago.

Was I disappointed? Sure

I'll bet. It's almost as if Canonical's desktop ambitions had left them over-extended, and when those were brought up short there was enormous collateral damage that hurt the FOSS community as a whole...

Of course, it would be crazy to even suggest that those efforts were deliberately sabotaged by Intel, of all people - a company that's famously never abused its wealth and position - to defend their still-floundering Wayland project against a competing system that had come out of nowhere and was already shipping: unlike Wayland, despite the head start. I mean, that's conspiracy theory-level stuff, you know?
(Obviously this wasn't the death knell for Mir, but it certainly sent a clear message).

I wasn't trying to stomp on your toes though

I never imagined you were.

I am willing to be as patient as it takes. Hell I've been waiting almost ten years, what's a few more years?

Very much the attitude that anything Wayland-dependent needs to have.

If you have proof that Wayland isn't actually necessary, I'm all ears.

Sorry, no. This part is clearly my fault: I was discussing the history that led to the problem you now have, i.e. Wayland in general; not your problem itself.
AFAIK, you're pretty much screwed w.r.t. Waydroid, for reasons that I think I've at least mostly covered already. It's not that there's anything within Wayland inherently needed for Forked-Anbox to work, it's that because Waydroid depends on it you can't have it on anything but Wayland. I really don't have the energy to go into the details of that part, but it wouldn't help you anyway, unless you had a giant pile of cash to throw at the problem.

But please don't just dismiss anyone waiting patiently as someone who just wants to play with a toy.

That wasn't really the intent, but yeah, I did not phrase that well, sorry. As I said at the time, there are significant negative consequences to adopting Wayland, and for you to get what you want a much larger group will have to suffer those consequences. The responsibility for that lies entirely with the Wayland developers: not you, not me, not the MATE team, and not Canonical.

If you felt dismissed by what I wrote, I'd like you to imagine what your response would be to the outright "Yeah, we literally could not care less that you have to rewrite all your code, or that you're dependent on a piece of X11 functionality that we're getting rid of" attitude of the Wayland team.
It's even worse for those trapped between a rock and the hard place in this fight, who have to make the impossible choice I already covered: either give up months of your own time to rewrite for Wayland, screwing over a lot of your existing users along the way; or deal with at best constant questions along the lines of "Why no Wayland version yet?", and more commonly, outright hostility and abuse. (Which, to be very clear, nobody is accusing you of - but you, and this community in general, are very much the minority).

Incidentally despite how far off topic we seem to have drifted, you mentioned that Wayland is stealing time away from better options. Would you care to mention some of those so we can see what they are?

Arcan is (IMO, at least) the most interesting one: but it's a passion project, not something with a billion-dollar corporation behind it. Despite that, it's come further in 5 years (with a much smaller team) than Wayland has in 15, and from a technology standpoint it's practically science fiction by comparison.

Most of what I was getting at though is that Wayland is inherently stealing time from thousands of other projects - one of which is MATE - by requiring massive rewrites just to coexist with it. That time could be spent on bugfixes that would benefit every MATE user, which is a group that all of us are in. It could also be spent adding new features, or improving performance, or on any number of - to borrow a phrase - "other uses that neither of us can think of right now".

Even if Wayland's incompatibility had been "accidental" or "unavoidable", it would still have been harmful. That it was a deliberate choice which explicitly benefited two companies that were already enormously wealthy and influential, at the expense of their smaller competitors, was cynical in the extreme. That it also placed even more of a burden on independent developers, who are already giving up their time to do unpaid work that they share with everyone, was beyond callous; and has had a significant cost to you even if you're solely a beneficiary of that work, let alone what it's cost the developers themselves in time and lost opportunity.

Like I said, I'm aware this is just tilting at windmills. Nothing I can do or say is going to change that, nor is it going to help you with your Waydroid problem. "Can be messy as hell and take a while", though not inaccurate, massively downplays the real cost and enormous negative impact of this fiasco. If someone had actively set out to sabotage desktop Linux, they still wouldn't have been able to do this much harm over this much time - and it's still nowhere near over yet.

[tkn]

I've just re-read that post, and I don't see any ranting in it. Some unpleasant truths, certainly, but nothing emotional, rude, or personal [...]
Please point out where you think the abuse is so that I can rephrase or remove it.

If you want me to point it out, I will sent you a personal mail.
This forum is not the place for that.

I'm sure you noticed, but the video you linked is from early 2013, nearly 10 years ago. 10 years later, Wayland is not only still unusable for an awful lot of cases, but has only just recently started work on HDR, which is hardly "new technology" that just appeared days ago and took everyone by surprise.

Gimp changed their pixelengine from fixed 8bits (A)RGB to variable up to 32 bit FP because demand. Good move.
So if Wayland incorporates High Dynamic Range in the standard because demand. Good move.
So what exactly is your point ?

I don't think an appeal to authority helps there, as it still presents quite a paradox: either the "authority" was somehow unaware of a topic that had been discussed at length since the mid-90s, in which case they are far from authoritative; or they were aware of it but unwilling or unable to deal with it for 14 years, in which case you can take your pick of any number of options, but none of them are particularly encouraging or flattering.

You are definitely wrong with this last sentence

It is obvious to anyone versed in this matter that it would take at least 10 to 20 years to get a full or at least general switch to a wayland implementation deployed and tested and above all integrated in desktopsystems worldwide.

The "authority" in the video was completely aware and willing and able to deal with it for many years and many years to come. You completely missed the point or you meant to say something completely different than what I understood from your writing which would imply that you haven't watched the video yet

The reason it takes such a long time has several root causes:

1. Code for the compositors is not written by the Wayland developers
   The Wayland developers have only designed the protocol* (both compositorside and clientside). because Wayland is only a protocol (and nothing more).

*They actually published a reference library and a reference-compositor many years ago.
These were intended as examples more than actual implementations, although they do work well as demonstrated by several small distros (AFAIK Rebecca Black OS was the first to do that).

The implementation is up to the compositor builders:
Gnome has created mutter (a.k.a. stutter, a well earned nickname) and they took their sweet time
KDE has Kwin extended with the wayland protocol (and it supports serverside decorations)
MATE will adopt the MIR compositor which fully supports wayland (thank deity...no mutter)
and Enlightenment has its own wayland enabled EFL compositor.

If we are being honest: The loud part of the internet is never looking further than gnome,
and if mutter was the reference, I would be not only sceptical but also downright depressed.

A lot of changes have been done in the kernel, like moving the drivers kernelside, the introduction of libinput, kdbus etc. It is much more than simply exchange Xorg for another compositor and just bullying users.

2. The implementation is slow because, contrary to the GNOME/GTK devs, most don't want to break userspace. Also, only lots of use in the real world is capable of revealing designflaws which is another reason to take it slow. In this case it is better to have a perfect API late than a crappy API early because you can not do this sisyphus-undertaking a second time. So it better be good when the API is frozen.

If, as a project, Gnome3 would be as serious as most wayland implementing devs about not breaking userspace it would have been much much slower in developement. Timewise, implementation of GNOME3 in this case would easily exceed the implementation time of Wayland twice (probably much longer even)

But Gnome3 can get away with breaking userspace every time because it is a toy . That means: Contrary to wayland, Gnome3 is not relevant for the industry.

Wayland is important for industries, especially embedded applications.
Actually, the desktop-world is the slowest in the uptake of wayland just as it is the slowest in the uptake of Linux as a whole.

Wayland is also important to get a working networked desktop back (without hundereds of roundtrips and corresponding latencies up to sometimes minutes). Ever done a remote thunderbird session ? It is a very frustrating experience.

X11 (the network protocol) has been hell on wheels for the past 15 years and outlived its welcome because every application (through toolkit-libs) sends its output as uncompressed bitmaps instead as drawing commands and we are already on 4k screens. Network transparent ? It hasn't been that for more than 15 years. Network capable ? Theoretically: yes. Practically: Hell no!

A client-side wayland buffer could easily be video-compressed (H263/H264) and send over network (and libinput can remote) which will make remoting snappy again. To transport that over 'ssh' would be a no brainer. That is not only more network capable than X11, it is also better for integration.

Many remote serversolutions at this moment work around the limitations by implementing the clientside rendering on the serverside (which is IMHO pretty horrendous but the only way to work around this kludge) and send compressed bitmaps already.

Wayland straightens out a lot of the kludge that X has become. A lot of applications do implement a lot of workarounds to cope with that kludge, and reverting to something sane takes time.

3. There are still a lot of people that think that wayland is the graphical-backend (just like Xorg)

It's very hard to defend a project like this taking 15 years in the first place,

I just did that

but even that aspect isn't really my main issue with it: there are large chunks of functionality in X that people depend on, which the Wayland team is explicitly opposed to, and their position is "f**k those users".

And they are right. Most of the demanded functionality that X offered doesn't belong in the displayserver in the first place. They don't want to repeat the same featurecreep they had with X which absolutely helped a lot to land us in this mess in the first place.
(and don't get me started about the idiots that don't care at all about the security issues of X)

People can be egotistical jerks and rightout refuse to understand what's at stake. So yeah "F**k those users". If you need that extra functionality, do it in a (preferrably desktop agnostic) GUI toolkit where it belongs.

There were also large chunks of GTK that applications/people depended on which were kicked out from under their backside ( I have several GTK applications that no longer work on GTK3. Even GTK3 applications that after a GTK3 update no longer worked ) but nobody complains because that is obviously allright because 'progress'. I call that 'double standards'

(There are also numerous python 2 apps that don't work anymore because python 3 is not backwards compatible and a lot of python 2 stuff is just bitrotting away. Again, most people don't seem to care.)

The impossible position of having to either burn resources that they don't have supporting two backends

No, people who need X have the Xwayland libraries as needed.
You, as a developer, can keep on targeting X as long as you like.
You don't have to support two backends.

Being harassed over "Wayland support" ?
You already support it by means of an install dependency on either X or Xwayland.
To the people who keep on nagging because "it's not real wayland (sob sob cry cry)"
it's either "pay up" or "shut up".

Only applications that depend on X but simultaneously work around X because of shortcomings in X will have trouble. Those are the applications you are talking about.

You, as a developer, are usually targeting a graphical toolkit so you don't have to target the backends.
You might, however, have to target several toolkits like GTK, Qt, Efl or Tk.
Even in that case you can choose to use WxWidgets (which will adapt to any toolkit that the system offers)

Developers of DEs are especially bearing the brunt of this,

I absolutely agree, and that is why it takes so long, that is why everyone working on it is feeling the pain. But everybody also knows that the user-demands will rise to a level that Xorg won't be able to cope with in the near future so it must be done.

It is a system wide structural change with repercussions everywhere. Nobody really likes that, neither OS-devs nor users. But it is crucial. It's not something to hurry up and get it over with.
Actually, the application-devs that use GUI-toolkits have the least of the brunt.

because Wayland goes out of its way to maximize the development costs for everyone that isn't Red Hat.

I highly disagree with that.
RedHat is 'Gnome' and therefore 'mutter' and therefore something to get depressed about.
But Wayland is not a RedHat product. There is no central institution buiding 'the backend'.

It is the single threaded stuttery implementation of that protocol by Gnome/RedHat, together with their intentional breaking of applications (again) with every GTK update that is maximizing the development costs for everyone. That has little to do with the wayland protocol itself.

My opinion:
Wayland is highly needed, X will survive another ten years, but if devs don't work on wayland implementations now (which at the moment means getting the last wrinkles out), we will be stuck without a suitable graphical backend when those years passed us by because user demands as well as application demands as well as hardware demands keep on rising.

Take some time to work this out and let it sink in:
Compare the state of the art from 20 years ago to now and extrapolate that to what it will be 20 years from now.

Scary isn't it ?