Memory and disk usage UM 24.04

[Alarik]

Judging by the app and the fact that pCloud is cloud storage service provider I'm guessing that the computer app is in sync with cloud so that might be the case if the user uploads to the cloud and forgets to remove the copy from the device it might get bloated because of that

My suggestion would be to look at the files and folders saved on cloud and compare with ones saved in /home directory and remove all the duplicated files from the device or migrate them to separate external drive if possible/necessary

[Watford]

Ironfoot

Here for the 1TB:

W

[Watford]

re Thunderbird
There seems to be duplication there. I agree.
All this memory problem appeared after an upgrade

re cloud storage
For years my use of the cloud storage was as a 'remote drive' not syncing on my laptop. I'll check with them, but no mod made at my initiative.

re install
When I installed I did a clean install (sweeping the disk) and let the install run automatically (recommended) without playing with the partitions myself. After that there was several upgrades which I rolled in automatically (for any new version). As a simple daily user I didn't dig into the disk usage until I got these mem issues recently - before that the system monitor was indication a maximum mem use of 43% of the whole 1TB.... then now I seem to be maxed out. I can't figure what and when something happened.
Q: Is it possible that an upgrade caused that problem since duplicating and partitioning ?
Q: Is there a way to 'clean this up' without having to sweep clean and reinstall ? Which ever way would be the less tedious.....

I want to thank you guys for your answers (the above and any future ones). This great forum and great members is my sole 'lifeline'.

[ricmarques]

Hi again, @Watford

I may be wrong, but it seems to me that the "MATE Disk Usage Analyzer" is getting confused by your 511 GB LUKS-encrypted container and it's counting that 511 GB space twice (and so it is wrongly showing "Total filesystem capacity: 1.0 TB", leading you to believe that your SSD is 1 TB in size, when it really has only about 512 GB).

Please, run the following command that will show the disks and other drives that you have and their corresponding sizes:

lsblk | grep 'disk'

... and then please run the following command which will show the partitions of every (internal and external) disk / drive:

sudo parted -ls

[Watford]

OK 512 GB...which is way enough for my use

sudo parted -ls:

---

Error: /dev/mapper/nvme0n1p3_crypt: unrecognised disk label
Model: Linux device-mapper (crypt) (dm)
Disk /dev/mapper/nvme0n1p3_crypt: 511GB
Sector size (logical/physical): 512B/512B
Partition Table: unknown
Disk Flags:
Model: Linux device-mapper (linear) (dm)
Disk /dev/mapper/vgubuntu--mate-swap_1: 1028MB
Sector size (logical/physical): 512B/512B
Partition Table: loop
Disk Flags:
Number Start End Size File system Flags
1 0.00B 1028MB 1028MB linux-swap(v1)
Model: Linux device-mapper (linear) (dm)
Disk /dev/mapper/vgubuntu--mate-root: 510GB
Sector size (logical/physical): 512B/512B
Partition Table: loop
Disk Flags:
Number Start End Size File system Flags
1 0.00B 510GB 510GB ext4
Model: INTEL HBRPEKNX0202AH (nvme)
Disk /dev/nvme0n1: 512GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:
Number Start End Size File system Name Flags
1 1049kB 538MB 537MB fat32 EFI System Partition boot, esp
2 538MB 1305MB 768MB ext4
3 1305MB 512GB 511GB
Error: /dev/nvme1n1: unrecognised disk label
Model: INTEL HBRPEKNX0202AHO (nvme)
Disk /dev/nvme1n1: 29.3GB
Sector size (logical/physical): 512B/512B
Partition Table: unknown
Disk Flags:

---

As I said, previous to that mem problem, the 'system monitor' was indicating a maximum mem use of 43%... so 154 GB...
At 512GB it is also an indication that the cloud storage doesn't use space, since my data on PCloud is just over 1.03 TB. In any case, used as remote drive, with no sync.
So it brings us down to the basic issue. How can I 'clean this up'.....

W

[Alarik]

Hmm I thought about the whole situation and you said you've clean installed everything - as @ironfoot had stated in earlier comment about duplication of thunderbird maybe start with locating that duplicate, check the files and if it is all the same delete one of the folders and that should already clear 188GB of space

Try and find those two instances of thunderbird mentioned earlier and compare the files there first so that you can remove at least the most obvious duplication so as you don't get into an issue of filling the whole memory up at least until somebody of us here in the forums don't think of next steps haha

EDIT: one more thing I remembered - there might be the case that somehow your home directory folders were saved and connected to the old username and are now appearing as though are "hidden" but are still on the disk (Idk if it is possible in the case when you don't have separate partitions or not because I only ever had separate home and root partitions and each clean Install when I changed username I would need to look up those hidden folders and bring them to the new username home directory but again I don't think this is the issue because I don't know if it is possible but just stating what I recall can happen in some cases)

[Watford]

Alarik

Well...maybe and to be sure, I should go 'all out' and simply uninstall/wipe out all what is Thunderbird related...
Then reinstall anew - the emails being google related would update themselves anyway.

But yes, it would only address what seems to be the TB duplication part.

p.s. Also since TB being a snap, there is a thunderbird.tmp created in Downloads. There appears a duplicate of any email attachment opened. This is also data duplication.

W

[ricmarques]

@Watford :

• your Thunderbird original profile ( stored at /home/user/.thunderbird/88360zp9.default-release ) was 141 GB in size
• your Thunderbird snap-based profile - stored at ( /home/user/snap/thunderbird/common/.thunderbird/88360zp9.default-release ) is 188 GB in size

Both those sizes seem to me to be way more than the typical Gmail accounts. I would start by BACKING UP those TWO folders to an "ext4" filesystem of an EXTERNAL disk / drive / storage before deleting anything!

[Alarik]

In this link there is an extensive guide on how to fix duplicates

Simple Methods to Remove Duplicate items from Mozilla Thunderbird

I also found there was an issue in Debian 12 with duplicates in Thunderbird so also check this out: Duplicate folders after Thunderbird upgrade in Debian 12 - Super User

[Watford]

Thanks to you both
I'll play safe and try these re duplicates
Will revert with anything being 'weird'

W

[Watford]

Ric
I looked through Thunderbird's Server Settings at the local directory where the messages are stored. One totals 88GB and the other one is 29GB. So, together 117GB. Which has nothing to do with the 141Gb and the 188GB which are indicated otherwise...

[ricmarques]

Hi again, @Watford

You wrote:

@Watford

Ric
I looked through Thunderbird's Server Settings at the local directory where the messages are stored. One totals 88GB and the other one is 29GB. So, together 117GB. Which has nothing to do with the 141Gb and the 188GB which are indicated otherwise...

That's all very nice, but what I do NOT want to happen is that you find yourself in a situation where your Mozilla Thunderbird stops working and/or you lose "stuff" (data and/or other configurations) in your Thunderbird that is currently working.

The following command finds the most recently created or modified file under /home/user/.thunderbird/88360zp9.default-release and it shows the full path of that file, its size and the date and time when it was created or most recently modified:

find /home/user/.thunderbird/ -type f -exec stat -c $'%Y\t%n' {} + | sort -nr | cut -f2- | head -n 1 | xargs -d '\n' -n1 ls -l

Run that command. If its output shows that most recent file is old, I believe that would be a good indicator that Thunderbird was correctly migrated from the .deb version to the "snap" version. Now, please run the following corresponding command to see the same but for the snap-based Thunderbird profile:

find /home/user/snap/thunderbird/common/.thunderbird/88360zp9.default-release -type f -exec stat -c $'%Y\t%n' {} + | sort -nr | cut -f2- | head -n 1 | xargs -d '\n' -n1 ls -l

Now, let's assume that you reach the conclusion that the /home/user/.thunderbird/88360zp9.default-release directory tree has not been changed for quite some time and that is only the /home/user/snap/thunderbird/common/.thunderbird/88360zp9.default-release directory tree that is getting changed. That would seem to me a good indicator that you probably don't need anymore the /home/user/.thunderbird/88360zp9.default-release directory tree. However, I am NOT sure of that, as there could be some kind of symbolic links being used and/or some "bind" mounts.

So, at that point, I would still make a backup of /home/user/.thunderbird/ to some "ext4" filesystem in some external media. AFTER you create that backup, you may try to close Thunderbird and rename /home/user/.thunderbird/ to /home/user/OLD-thunderbird/ using a command like the following:

mv -v /home/user/.thunderbird/ /home/user/OLD-thunderbird/

Then, you would open Thunderbird again. IF - and only IF - everything looks good then, that would be another very good indicator that you don't need /home/user/OLD-thunderbird/ anymore and that you could delete that folder. But again, I am NOT 100% sure of that, so PLEASE back up everything before you take any drastic action!

I hope this helps Please, keep us posted.

[ericmarceau]

Hello Watford,

Getting the "du" report to list all directories may sometimes leads to confusion/misunderstanding, especially if you fed the results thru a "sort -nr" filter (which is what I suspect you did).

To get the clearest picture of usage by Thunderbird, you need the results from these two commands:

du -sh ~/.thunderbird
du -sh ~/snap/thunderbird

Those would give you the results identified by @ricmarques , being

141G /home/user/.thunderbird
189G /home/user/snap/thunderbird

Having said that, all the content of the first one is your personal data: setup data, email, GUI preferences and add-ons.

As for the snap package installation, only the portion under "~/snap/thunderbird/common/.thunderbird" is that same personal data, even though it seems to have bloated up.

To me, that is a sign that either

• you have downloaded an extra 48G of email content (obviously not likely) or

• Thunderbird has changed it software architecture in such a fashion as to "bloat".

That is exactly what should be expected from Thunderbird to conform to Snap expectations of being fully independent of OS libraries to operate.

As for the 1G difference between these two,

189G /home/user/snap/thunderbird/common
188G /home/user/snap/thunderbird/common/.thunderbird

that is the Thunderbird software itself, again bloated to be OS-independent, as far as is possible, to conform to the Snap dictates.

IF your intent is to keep the SNAP-based Thunderbird installation, then make a frozen backup/offline image of the contents of ~/.thunderbird , then purge that entire directory.

On the other hand, IF your intent is to stay with a non-SNAP Thunderbird

            AND

IF you haven't downloaded any email since before your Snap install, then everything under that SNAP Thunderbird is "throwaway".

Just make a backup of your ~/.thunderbird before cleaning up the Thunderbird install as non-snap.

One last observation.

You have directories

ImapMail/imap.gmail.com
ImapMail/imap.gmail-1.com

That tells me that, under the old non-snap Thunderbird,

• you may have created a second email profile for Gmail in your Thunderbird, or

• you did some kind of restore/repair of your old email account at some point but did not finish with the cleanup.

If you check which of those two,"imap.gmail.com" or "imap.gmail-1.com" is the path being pointed to by your Thunderbird client by checking

you can then decide which is your "master" email account and which is your "old" or "defunct" ... or possibly "alternate personality" for accessing GMail.

If not that last one, then the non-master account is OLD and can simply be purged, as long as you are sure that you reclaimed all email from that during your previous recovery attempt.

If you need to review the contents of that older account, I will lets someone else guide you thru the steps of setting that up to perform that in a way to allow you to move files from one account to the other.

If none of the above applies to your case, let me know and I will purge my reply to avoid confusing others.

[Watford]

To the two 'find' command I get this:

find: ‘/home/user/snap/thunderbird/common/.thunderbird/88360zp9.default-release’: No such file or directory

I'll go sleep on it

W

[Watford]

Eric
I have two separate gmail related account. A personal one and G business email linked to the company I am associated with
But yes I'll go through the clean-up.
Thanks for such a guidance.

W

[Alarik]

After the information you've provided about thunderbird only taking 88 and 141 GB I have a feeling something doesn't add up somewhere because when you've run the command @ricmarques suggested:

sudo du -h --one-file-system / | sort -rh | head -n 25

You got listed, and I quote: "189G /home/user/snap/thunderbird/common" so something somewhere is hiding in plain sight it seems

What I would suggest is that you take a look inside thunderbird directly and try and see do any e-mails seem duplicated because 141GB and 188GB really is quite a lot of memory used and there might be some visual clues if old mails indeed are doubled in any way and check for both mails to be double sure

Also what I remembered asking is - did you by any chance try using "sudo apt autoremove --purge" and would this help at least free some space in any way - does it offer an option to clear up some space ?

[ricmarques]

Hi again, @Watford

You wrote:

@Watford

To the two 'find' command I get this:

find: ‘/home/user/snap/thunderbird/common/.thunderbird/88360zp9.default-release’: No such file or directory

Well, that probably means that your home directory is NOT actually /home/user but maybe it is /home/watford or something similar. No problem. In that case, please use instead the following command to see the most recently created / changed file of the original Thunderbird profile / configuration:

find ~/.thunderbird/ -type f -exec stat -c $'%Y\t%n' {} + | sort -nr | cut -f2- | head -n 1 | xargs -d '\n' -n1 ls -l

... and use the following command to see the most recently created / changed file of the "snap-based" Thunderbird user profile:

find ~/snap/thunderbird/common/.thunderbird/88360zp9.default-release -type f -exec stat -c $'%Y\t%n' {} + | sort -nr | cut -f2- | head -n 1 | xargs -d '\n' -n1 ls -l

I hope this helps Please, keep us posted.

[Watford]

Hi Ric

To those two new command I get this:

find: ‘/root/.thunderbird/’: No such file or directory
and
find: ‘/root/snap/thunderbird/common/.thunderbird/88360zp9.default-release’: No such file or directory

For the other Find commands you gave me yesterday, the error was mine. SO I tried them again and got this:

-rw-r--r-- 1 user user 327680 Oct 3 21:19 /home/user/.thunderbird/88360zp9.default-release/history.sqlite
and
-rw-rw-r-- 1 user user 319 Oct 6 09:28 /home/user/snap/thunderbird/common/.thunderbird/88360zp9.default-release/session.json

W

[Watford]

re autoremove -- purge

Yes did. And it says "0 upgraded, 0 newly installed, 0 to remove and 4 not upgraded"

W

[Watford]

Alarik
I use the 'remove duplicates add-on' and check regularly. It says there are no duplicates.

W