Stability VS Stability - Annoying words

I dunno if I'm the only one frustrated by this, or if I'm missing a way to distinguish these two ideas when searching the net but... Does anyone know where I can find analysis of the Stability (as in bug/crash frequency) of Desktop Distributions and Environments?

All I can ever find are threads that devolve into "You don't know what STABILITY means. It means less CHANGE."

Yes... I understand that perfectly well. But I want to know how Desktop Distros are doing on the "other" Stability metric. How often they bug out or crash.

3 Likes

I can't find much in the desktop space, but I found that for servers some related concepts are called reliability, availability, and serviceability. The middle one is the closest match to what you're interested in (from Reliability, Availability and Serviceability (RAS) — The Linux Kernel documentation):

availability is the probability that a system is operational at a given time.

I'm keen to know if anyone has benchmarks for desktops that are similar. I tried searching online but had no luck (even tried duck.ai as a last-ditch effort).

3 Likes

As you said Stephen, from a bizops standpoint, RAS tends to be the focus because management doesn't really want to get into the nitty-gritty, as long as the business is "operating smoothly", meaning the non-IT functions can "produce the goods" and the revenue keeps growing the back account balance at the rate that is expected. In that context, the "nitty-gritty" is for the most part ignored as "background noise". :frowning:

Given that Garrett is a student, his"time investment" and "creation investment", as harvested, congealed, and captured as text/graphics in "transient RAM", are critical one-time, never-to-be-fully-duplicated investments whose loss can be ill afforded by encountering a bug preventing it from being saved properly, or a crash preventing it from being saved at all!

For that reason, I completely applaud Garrett's disciplined approach in trying to "get to the bottom of things" by trying to get his hands on the actual stats for a fact-based decision regarding

  • crash events, frequency and trends
  • bug events, frequency and trends

but I would throw my own spin on things by suggesting that bug events need to be broken into sub-groupings as follows:

  • OS-core software
  • OS-support software
  • non-OS non-GUI software
  • non-OS GUI-based software

and this last grouping broken down even further into

  • business productivity supply-chain
  • business productivity financial
  • business productivity engineering
  • business productivity creative
  • business productivity communications (email, multimedia groupware, multimedia presentation, instant messaging, social media, etc.)
  • other business productivity non-financial

Keeping in mind, personal/home productivity is no different than business productivity, just at a different scale, using scale-appropriate tools. :slight_smile:

Given that, I too "rummaged" to see what I could find, and came up dry for what I think Garrett was really looking for. :frowning:

I fear the only approach available is to visit the "issue management facility" (bug enhancement tracker) for each of the main distros, and somehow distill what is presented into a sensible form. The only alternative to that is for each Distro's problem tracking "Manager" to have compiled the desired type of stats of their own volition, recognizing that such would be most attractive to anyone trying to make that decision to commit to their distro based on "demonstrated stability".


As an aside global total "change count" is a false statistical reference because different Distros have different approaches regarding the scope specification for a code-related issue; what might constitute only 1 issue for resolution might constitute 3 separate issues (more finely divided) for another system.


A further aside, this document is great to "clear one's head" about what to look for in "reported" statistics! :slight_smile:


Regarding security-related issues, you have the following resources "Master Index":

For documented Ubuntu-specific instances, those are summarized here:

As an overview on the Common Vulnerabilities and Exposures (CVE) Program, this presentation offers a good backgrounder:

3 Likes

These in particular I assumed must be measured in some capacity for a distribution team to assess the hotspot issues in their system. While everything else you listed would be fantastic to see analysis of, it seems mostly sensible that monitoring stability among the core and supported software distributed would be of interest.

3 Likes

As far as I can tell, it all boils down to "time crunch", availability of resources to collect and keep on top of those statistics on an ongoing basis. :frowning:

3 Likes

I have never seen any mentions of such a metric and/or analysis.

Stability, reliability, availability, serviceability... IMO 'IT boffins' and 'IT suits' often invent too much of terms balancing on the edge of being fuss or simply nuts.

While administering servers in datacenter I had the only metric. Namely, how long systems do not serve their purpose. I.e. we calculated post factum percentage of time required to perform planned maintenance and to recover from unexpected faults and crashes. Provided 24x7 regime, the target metric was 0.5%. I.e. about 44 hours a year per service. Availability was 99.5%.

Ok, now back to my Ubuntu Mate DE experience since 2016. There was a period of time I'd name as irritating and buggy one. During a month or so I had to reboot my frozen system 2-3 times a day. A reboot took about a minute. Let's assume 4 hours of using laptop a day. Availability was approx. 99.4%.

P.S. I have had no bug/crash related problems with Ubuntu Mate except for the mentioned month.

6 Likes

If you know the package names for specific components (for example, different file managers: caja, nautilus, thunar, etc), this could be a useful tool to get metrics on how many crash reports get sent in:

Not sure if it's actually useful to analyse the "stability" between desktop environments. nautilus is Ubuntu/GNOME, so there's more users and more varied bug reports. I'm not sure that concludes it's less stable then caja.

More on the rationale behind that service:

4 Likes