No login after upgrading to 22.04.3LTS

Hi there,

I just upgraded from 20.04LTS to 22.04LTS. After the required reboot, I get the login screen, but only guest user is shown. My home folder is encrypted, so isn't mounted when starting in recovery mode.

Now what I tried:

  • my usual username has not changed UID and is still 1000
  • From recovery mode, I made sure to complete installation of possible broken packages, and none are reported.
  • In the recovery console, switching from the root console to my username console does work, but running ecryptfs-mount-private does not appear to work and gives an error "mount: No such file or directory" when I enter my login password.
  • Trying to start the GUI with startx gives a timeout because .Xauthority can't be found (Normal, since the home folder isn't mounted)
  • I can mount my encrypted home folder to a temporary folder

Other, maybe unrelated symptom: the splash screen isn't centered anymore since the upgrade.

Can someone explain what bug may have appeared in what is usually a trouble-free operation?

Hi, @Cubytus :slight_smile:

I've done some web searches and I'm afraid the short version seems to be that "ecryptfs" was deprecated in Ubuntu.

See, for instance, the following topic, in the web site "Ubuntu Forums" - https://ubuntuforums.org - started by the user "nader-amadeu" on 7th September 2022:

... and particularly the following answer in that same topic, written by the user "TheFu" on that same day (7th September 2022):

https://ubuntuforums.org/showthread.php?t=2478859&p=14111343#post14111343

September 7th, 2022#2

TheFu

" Re: home dir disappears after upgrade (ecryptfs)

encryptfs has been deprecated since 2018.

I bet there was a clear announcement in the release notes too.

Deprecated means it won't work at some point in the future. Welcome to the future.

You might be able to gain access to the encryptfs data manually, assuming it wasn't deleted. I think it is in ~/.private/ , but since I only used it a few days, don't trust me. It had already been deprecated when I first used it for a number of important security and performance reasons, in favor of using LUKs.

Worst case, just restore from the backups you made before choosing to allow the upgrade. They backup all the files you don't want to lose, do the upgrade, and restore those files.

If you didn't make a pre-release upgrade backup set, it will be harder, but I suspect booting from a 20.04 in "Try Ubuntu" mode will allow manual access, with some magic commands, to the encryptfs files ... again, backup all the files you don't want to lose, take them to the 22.04 system and restore.

Probably not what you wanted to hear, but it shouldn't be any surprise. "

3 Likes

You cannot login unless your home directory has sufficient space available for GUI processes to start; thus your issue is insufficient space in $HOME; or more likely issues mounting your encrypted system (being the reason you don't have space available).

I for sure would read the response from @ricmarques

I had two systems that use(d) encrypted home, my primary PC did (it was a 17.10 install at first, and I used the option provided by ubiquity to encrypt my home partition), and that system was release-upgraded thru all releases every six months until its PSU finally died end of 2022, so it'd got me thru jammy (22.04) even kinetic (22.10) without issue.

I still have that box, as I had hoped to get a replacement PSU for it, and use it for QA testing; alas I've not found a PSU (at a price I'm willing to pay) that fits its case. I can't explore it (no working PSU), and on my replacement box (purchased early 2023) I didn't bother with encryption. (The second laptop I still have but rarely use)

My point above is I used an encrypted home on Ubuntu 22.04 & 22.10 for months without issue, thus it was possible, at least on my setup.

What may differ however is kernel stack. Which are you using??

My (LTS) installs have both GA & HWE installed; and given that install of mine only survived until end 2022; I'd have used only the 5.15 & 5.19 Linux kernels. The HWE stack of jammy or 22.04 is now 6.5 (from 23.10) which is far newer than what my [encrypted home] box ever got to... If it's related to kernel stack; maybe reverting to use the GA kernel stack maybe a solution?? (When you release-upgrade from 20.04 to 22.04; your 22.04 system inherited the default from your 20.04 system; what did it use?)

I don't know, but my 2c thoughts.

FYI: I have also non-destructively re-installed onto a encrypted home system, however as I'm no longer using encrypted home directories; I stopped QA testing it some time back.

4 Likes

Hm, from what I could find, this MATE release was upgraded from 14.04.2 LTS up to 22.04 LTS, iteratively. I have a backup made with Déjà Dup at some point before the upgrade, dating from early 2023 (this was not my main computer and besides, the tool never worked unattended). I can log in as a guest in 22.04, connect to the backup, but I don't know what purpose restoring from there would serve if I can't upgrade. Or, in other words, if I should go through a clean install and just restore my user home folder from the backup.

About the kernels, how would I know if I have HWE or GA? The current, from the broken 22.04 is 5.15.0-94 generic, and the previous one is 5.4.

@ricmarques like in the linked topic, I can mount my home directory from a user's console (called from within a root session) by running ecryptfs-mount-private.

All my files are there, sure, but what to do next? BTW, even if that works from a console, I still can't switch in the desktop environment.

Hi, @Cubytus :slight_smile:

This reply of mine is about your following question:

You can check if you're running a HWE (Hardware Enablement) kernel or a GA (General Availability) one by running the following command:

hwe-support-status --verbose

In the case of this laptop that I'm using right now to write this reply, I'm running Ubuntu MATE 22.04.4 LTS ("Jammy Jellyfish") which began as an Ubuntu MATE 20.04 LTS ("Focal Fossa") installation that I later upgraded to Ubuntu MATE 22.04.1 LTS. If I run that same hwe-support-status --verbose command in this computer, then I conclude that I'm NOT using an HWE Kernel and that the running kernel version is 5.15.0-105-generic:

ricmarques@mylaptop:~$ hwe-support-status --verbose
You are not running a system with a Hardware Enablement Stack. Your system is supported until abril 2027.

ricmarques@mylaptop:~$ cat /etc/os-release 
PRETTY_NAME="Ubuntu 22.04.4 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"
VERSION="22.04.4 LTS (Jammy Jellyfish)"
VERSION_CODENAME=jammy
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=jammy

ricmarques@mylaptop:~$ uname -a
Linux mylaptop 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

On the other hand, if I run the same commands in a Ubuntu MATE 22.04.4 LTS ("Jammy Jellyfish") virtual machine (VM) which is the result of a fresh install where I used the Ubuntu MATE 22.04.4 ISO to install it, then I conclude that VM is running an HWE Kernel version and that the running kernel version is 6.5.0-28-generic:

ricmarques@myvm:~$ hwe-support-status --verbose
Your Hardware Enablement Stack (HWE) is supported until abril 2027.

ricmarques@myvm:~$ cat /etc/os-release 
PRETTY_NAME="Ubuntu 22.04.4 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"
VERSION="22.04.4 LTS (Jammy Jellyfish)"
VERSION_CODENAME=jammy
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=jammy

ricmarques@myvm:~$ uname -a
Linux myvm 6.5.0-28-generic #29~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Thu Apr  4 14:39:20 UTC 2 x86_64 x86_64 x86_64 GNU/Linux

I hope this helps :slight_smile:

2 Likes

Hm, in that case I get:
You have packages from the Hardware enablement Stack (HWE) installed that are going out of support on 2027-04-30

There is a graphics stack installed on this system. An upgrade to a configuration supported for the fill lifetime of the LTS will become available on 2024-04-15 and can be installed by running update-manager in the Dash.

I was able to start my own session in recovery mode after mounting ecryptfs home directory, but what next?

Hi again @Cubytus :slight_smile:

You wrote:

I don't have any recent experience with "ecryptfs" (the last time I worked with "ecryptfs" I believe it was before 2016). I also don't use the "Backups" tool of Ubuntu MATE (which is "Déjà Dup", as you've correctly mentioned in you first post). So, please bear that in mind.

First, and based on @guiverc (Chris Guiver) first paragraph of his reply in this same discussion topic (which he posted on 18th February 2024):

"You cannot login unless your home directory has sufficient space available for GUI processes to start; thus your issue is insufficient space in $HOME; or more likely issues mounting your encrypted system (being the reason you don't have space available )."

... I would start by checking how much free space you have in your filesystems, by running the following command:

df -hT

Could you please reply with the output of running that command in your system?

1 Like
 df -hT
S.ficheros             Tipo     Tamaño Usados  Disp Uso% Montado en
tmpfs                  tmpfs      785M   1,9M  783M   1% /run
/dev/sda6              ext4       123G    76G   41G  66% /
tmpfs                  tmpfs      3,9G    52K  3,9G   1% /dev/shm
tmpfs                  tmpfs      5,0M   4,0K  5,0M   1% /run/lock
tmpfs                  tmpfs      785M   192K  784M   1% /run/user/1000
/home/usuario/.Private ecryptfs   123G    76G   41G  66% /home/usuario
/dev/sdc1              exfat      239G    97G  142G  41% /media/usuario/Samsung2561
tmpfs                  tmpfs      3,9G      0  3,9G   0% /run/qemu

Samsung256 is an SD card. Otherwise, there's plenty of free space.

Other critical malfunction (especially on a laptop): trying to go to standby turns off monitor, but CPU, fan, ssd stay on. Worse, the computer can’t be “woken up” and stays frozen with no video output

Hi again, @Cubytus :slight_smile:

Thanks for having provided the output of df -hT

My suggestion - with NO guarantees of any kind! - is to run the following in your recovery environment (please note that I have NOT tested this procedure myself!):

1. - First, I'm assuming that you have NOT also encrypted your swap! Please, check your /etc/fstab and /etc/crypttab to make sure that you only have a regular swap partition / file (or that you're not using swap at all). If you see that you have an encrypted swap, then please do NOT proceed any further!

IF you're NOT using encrypted swap:

2. - Start by doing a good backup of your unencrypted / decrypted data - the one that got mounted inside your /home/usuario folder when you ran ecryptfs-mount-private - to some kind of external storage, formatted as ext4, as ext4 is the filesystem format that you're using now. For instance - assuming that you already have an external hard drive, plugged with an USB cable to the computer, which has a single partition that you had already formatted as ext4 and that you see that partition has been recognized as /dev/sdx1 - then you would write the following commands:

sudo mkdir -pv /media/externalbk

sudo mount -v /dev/sdx1 /media/externalbk

sudo cp -av /home/usuario /media/externalbk/usuariobk

3. - Check that your /media/externalbk/usuariobk folder has all the data that you need and that you can open files and folders there without any issues. After doing that, unmount it:

cd /

sudo umount -v /media/externalbk

3.1. - Optional but Strongly Recommended: plug that USB hard drive to another computer that you have which also has Ubuntu installed, to see if you can browse and open the files and folders without any problems in that other computer.

4. - Returning to your problematic computer: the next step would be to copy all your unencrypted / decrypted data that is in /home/usuario folder to another folder under /home

I will take here a somewhat similar approach to the following answer by Erick in AskUbuntu:

4.1 - First, check how much data you are using in your /home/usuario folder with the following command:

sudo du -csh /home/usuario

4.2. - If the previous output returns more than 46 GB, then you have a problem because that means you have more than 46 GB used by /home/usuario BUT, according to the output from your df -hT command, your free space is "only" 46 GB . If that's case, please STOP here!

4.3. - If the previous output returns less than 46 GB used, then we can continue. Now, we'll do a copy from /home/usuario to a new folder under /home in the same internal disk of your computer, with the following command:

sudo cp -av /home/usuario /home/usuarionew

5. - After that copy has completed:

5.1. - Check that you have everything that you need in /home/usuarionew (all the files and folders)

5.2. - Unmount the original /home/usuario :

cd /

sudo umount -v /home/usuario

5.3. - Rename /home/usuario to /home/usuarioold :

mv -v /home/usuario /home/usuarioold

5.4. - Rename /home/usuarionew to /home/usuario:

mv -v /home/usuarionew /home/usuario

5.5. - Confirm that the /home/usuario folder and its subfolders and files are owned by the user usuario and by the group usuario (they should be). IF they are not, then make it so with the following command:

sudo chown -Rv usuario:usuario /home/usuario

5.6. - Remove the .ecryptfs folder from under /home/usuario:

sudo rm -rfv /home/usuario/.ecryptfs

6. - Reboot and login to see if the system boots normally and that you can normally log in to your system.

7. - If everything looks OK at this point, I think it should now be safe to remove your old directory - that I remind is now located in /home/usuarioold - and to uninstall the ecryptfs tools since you shouldn't be needing those anymore:

7.1. - Remove the old directory:

sudo rm -rfv /home/usuarioold

7.2. - Uninstall the ecryptfs tools:

sudo apt remove ecryptfs-utils libecryptfs0

8. - Reboot and login again to see if the system still boots normally and that you can still normally log in to your system.

I hope this helps! :slight_smile: Please, keep us posted. And good luck!

1 Like

Well, I really do appreciate the effort you obviously put in your answer @ricmarques and the attention you put in making sure nothing gets lost. What I understand, though, is that there's no "decrypt-in-place" process that would make the whole thing acceptable. After all, the encrypted home directory can actually be mounted, yet for some reason the user doesn't appear at the login screen. Given how rarely I'm using this computer, a full reinstall may be simpler. Seems pretty minor.

Otherwise I'd need:
An external, dedicated HDD. I don't have that, plus this computer not having USB3.0 makes it painful to rely on external HDDs. (Can't a Déjà Dup backup cut it?)

Not need but recommended:
Another computer with a working Ubuntu or another Linux installed. That I don't have.


As for the procedure:
1. There's a swap partition (at least something is working on that system) identified in /etc/fstab. There is one in /etc/crypttab, but with a different UUID and commented out, so I believe it's inactive.

2 to 3.1 As I mounted the home directory in recovery mode, I made a Déjà Dup backup. Will that suffice?

4.1: currently using 54G, which is more than the 46G available space. That can be reduced somewhat if I clean up some things.

To be clear, this is not a case where the home directory can't be found, rather one where the user isn't visible at the login screen.

2 Likes

You're welcome @Cubytus and thanks for your kind words of appreciation about my previous reply :slight_smile:

Unfortunately, as I've mentioned before, I don't have experience with "Déjà Dup" backups (the native Ubuntu MATE "Backups" application) and so I don't know if a "DéJà Dup" backup is a reliable option for your scenario. Maybe someone else, here in the Ubuntu MATE Community, can answer your question?

It's also unclear to me where you are saving your "Déjà Backups" to? Is it to that Samsung256 SD card that you've mentioned that appeared as /dev/sdc1 in your df -hT output? If so, it seems to me that it would be great if you can browse that SD card in another physical computer or Virtual Machine (VM) with Ubuntu (MATE) 22.04 installed, so you could see if you could browse the contents and read some files from that "Backup" in that physical computer or VM. Obviously, if you decide to try to restore the contents of your backup to another computer or VM that has data that you consider valuable, take care to do it in a way that will NOT overwrite that data!

Another information that may be useful is if the output of your /etc/fstab (assuming it doesn't have information that you consider to be private).

Speaking of fstab, another note of warning: please be aware that /dev/sdX device names (such as /dev/sda , /dev/sdb, /dev/sdc) may change between reboots and point to a different device! See, for instance the following reply in the "Unix & Linux Stack Exchange":

1 Like

The SD card has nothing to do with anything. It's just a portable, additional storage space. Incidently, it has enough free space to hold data, but is formatted as exFAT and I can't change that.

As for the other backups, I didn't strictly confirm they are readable, but left Déjà Dup check them automatically, which succeeded. Again, losing the files on this system sure would be annoying, but not as annoying as not knowing how to keep a working encryption method and not being able to use "decrypt in place" like modern OSes can. By the way, if not using the integrated tool, are you making your own backups with rsync?

As for /etc/fstab:

$ cat /etc/fstab
# /etc/fstab: static file system information.
#
# Use 'blkid' to print the universally unique identifier for a
# device; this may be used with UUID= as a more robust way to name devices
# that works even if disks are added and removed. See fstab(5).
#
# <file system> <mount point>   <type>  <options>       <dump>  <pass>
# / was on /dev/sda5 during installation
UUID=6c7e6416-1213-40c0-9ede-d284bc1ffa8d /               ext4    errors=remount-ro 0       1
# swap was on /dev/sda6 during installation
#UUID=49dc9498-e922-424f-bc36-226315a4b94b none            swap    sw              0       0
#UUID=70ac3feb-ea9e-4cf2-b4be-939dc097b5d2 none            swap    sw              0       0
UUID=84c1b4ba-485d-4ccc-bb00-27c3085328ac none            swap    sw              0       0
#/dev/mapper/cryptswap1 none swap sw 0 0
LABEL=Win10Home /mnt/Win10Home auto nosuid,nodev,nofail,x-gvfs-show,remove_hiberfile 0 0

UUIDs are all correct. I'm not sure how you thought /dev/sdX devices would have switched places, and how is that relevant to my issue.

Hi again, @Cubytus :slight_smile:

You wrote:

OK. So it's NOT the SD card that you're using to save your backups. Understood. Thanks for clarifying.

OK. My worry is that IF you are also saving your Déjà Dup backups to your ecryptfs /home/usuario folder then it may become quite troublesome to "undo" your failing ecryptfs configuration, because those backups would also be located inside it. That's why I asked before if you were saving your Déjà Dup backups to some kind of external hard drive or to your SD Card. Or maybe are you saving those backups to another internal storage drive or to another partition of your internal storage drive?

Indeed, my backups are simple and home-made, done by a shell script that I've created and that uses "rsync" commands to save the files to an ext4 filesystem located in a partition of an external hard drive, connected via USB cable / interface. It may not be a great system, but it works well enough for me :slight_smile:

The rsync commands that I use inside that script are simple and look like this:

rsync -avv  --delete "${home_initial_directory}/${subfolder}/" "${destination_home_directory}/${subfolder}"

Typically, I put those rsync commands in loops that iterate over an array of folders and my shell script code for that effect looks something like this:

(...)
declare -a array_subfolders=("Pictures" "Music" "Videos" "Documents" "Public" 
"Downloads" "snap" "Desktop" ".ssh" ".thunderbird" ".config" ".mozilla" ".local")
for subfolder in "${array_subfolders[@]}"
do
    echo $subfolder
    rsync -avv  --delete "${home_initial_directory}/${subfolder}/" "${destination_home_directory}/${subfolder}"
done

(...)

That's great! :slight_smile: I'm sorry that I wasn't clear enough earlier: I didn't know if you were using UUID or not and I was afraid that you might not be using UUID but that you could eventually be using /dev/sdX device names in your fstab and mount commands and that it could have unintended consequences, given those sdX device names are not guaranteed to be the same / point to the same devices after a reboot. But, as you've now explained, you are indeed using UUID, exactly as you should.

1 Like

Indeed!
It wouldn't make any sense to backup on the system SSD to begin with… It's not good practice to save on a system drive, whatever it might be. Mine are actually on an SMB or WebDAV server (don't remember exactly as the NAS has three or four protocols enabled).

As for the backups, I think Déjà Dup uses rsync internally, plus some additional flags so as to generate true timestamped snapshots, plus some "magic" to determine . It would be more interesting to know what is the exact command line it uses, just in case the user lost access to the desktop. But I digress…

[quote="ricmarques, post:13, topic:27164"]
I didn't know if you were using UUID or not
[/quote]Isn't using UUIDs standard on Linux distros since about a decade? My setup didn't deviate from what was standard back from 14.04 LTS…

So, all-in-all, are you reasonably confident that I could

  1. re-create a non-encrypted username in the free space
    AND
  2. copy the non-automatically-mountable home directory content to the new user
  3. Erase the non-automatically-mountable home directory
    OR
  4. Erase the non-automatically-mountable home directory
  5. Restore content previously made with Déjà Dup or copy the home directory content to the new user

Or simply make a clean reinstall and just restore the existing backup?