As Oliver Hardy would say to Stanley Laurel, "Well, that's another fine mess you've gotten us into!"
In chasing down the 'routined quit unexpectedly' error message, I may have made a discovery. An unpleasant one at that.
Quick background: I have a Mac that has three HDD attached (necessary because of connectors) and these are shared to my Ubuntu 22.04.5 LTS server via SMB. Or I should say "WERE" connected. After running my diagnostic and repair attempts, they no longer attach (I have a cron @reboot entry that automounts them). Nor can I manually attach them. I get either a permissions error, or some other indecipherable message.
I decided to open a terminal on the Mac and run fsck against a 5TB volume. Oops, errors reported. fsck can't repair them but reports the volume as clean. Mac's Disk Utility program is useless, as it barfs when it encounters an error.
This problem may be related to the routined crashing, so the problem now is how to copy off 4.5TB of data. Worse, my MATE server has only about 1.5TB free. I first thought rsync might be the answer, but that won't free up any space. More than likely, I'll have to buy a fresh drive and copy from one volume to another. I've not used rsycnc before, but since it has a resume feature, that could be handy, as transferring that much data between drives could take hours, and one >burp< kills the whole thing.
For usage of rsync, I can only offer up references posted regarding my own usage and coding for rsync-based backups. In my case, I have logic to perform that as independent jobs on each of the partitions on my 2 TB drive (each partition about 300 GB).
Log of typical backup script session is included below.
Note that the posting has a full English translation at the bottom of the posting, after the code listings.
LOG of backup session:
root@OasisMega1:/Oasis/bin# quickRSYNC.sh
NOTE: This script will ONLY perform synchronization backup (applying differences).
It does NOT perform a FULL COPY backup.
For FULL COPY backup, use 'OS_Admin__partitionSecondary_Mirror.sh' responding 'y' at the appropriate prompt ...
Forcing backup of all files. Option '--keepNewer' is available to keep backup version of files if newer ...
Checksum comparisons is disabled. Option '--compareFileData' is available to force this check by rsync ...
Following block devices have been identified:
sda 8:0 0 1.8T 0 disk
sdb 8:16 0 465.8G 0 disk
sdc 8:32 0 3.6T 0 disk
sr0 11:0 1 1024M 0 rom
External BACKUP disk is online ...
==============================================================================================
================================== PARTITION BACKUP START ==================================
==============================================================================================
SOURCE= /DB001_F1
MIRROR= /site/DB005_F8/DB001_F1 ...
Proceed with backup of 'DB001_F1' ? [y|N] =>
'DB001_F1' skipped ...
==============================================================================================
================================== PARTITION BACKUP START ==================================
==============================================================================================
SOURCE= /DB001_F2
MIRROR= /site/DB005_F2/DB001_F2 ...
Proceed with backup of 'DB001_F2' ? [y|N] =>
'DB001_F2' skipped ...
==============================================================================================
================================== PARTITION BACKUP START ==================================
==============================================================================================
SOURCE= /DB001_F3
MIRROR= /site/DB005_F3/DB001_F3 ...
Proceed with backup of 'DB001_F3' ? [y|N] =>
'DB001_F3' skipped ...
==============================================================================================
================================== PARTITION BACKUP START ==================================
==============================================================================================
SOURCE= /DB001_F4
MIRROR= /site/DB005_F4/DB001_F4 ...
Proceed with backup of 'DB001_F4' ? [y|N] => y
Using previously determined bandwidth limit for rsync buffer setting ...
Will apply parameter to limit flooding of I/O, memory and swap ==>> --bwlimit=65536
Mon 20 Jan 2025 06:54:07 PM EST |rsync| Start DB001_F4 ...
Background 'rsync' working ...
Expected Log files:
/site/DB005_F4/Z_backup.DB001_F4.DateSize.out
/site/DB005_F4/Z_backup.DB001_F4.DateSize.err
Use 'OS_Admin__partitionMirror_Monitor.sh' to monitor rsync process.
Imported LIBRARY: INCLUDES__TerminalEscape_SGR.bh ...
Mon 20 Jan 2025 06:54:17 PM EST
PID 19287 is RSYNC child process ...
PID 19286 is RSYNC child process ...
PID 19285 is RSYNC MASTER process ...
RSYNC backup process under way ...
root 19285 19281 0 18:54 pts/0 00:00:00 rsync
--bwlimit=65536
--one-file-system
--recursive
--outbuf=Line
--links
--perms
--times
--group
--owner
--devices
--specials
--verbose
--out-format=%t|%i|%M|%b|%f|
--delete-during
--whole-file
--human-readable
--protect-args
--ignore-errors
--msgs2stderr ./ /site/DB005_F4/DB001_F4/
Scanning at 10 second intervals ...
...............
RSYNC process (# 19285) has completed.
Mon 20 Jan 2025 06:56:50 PM EST
root@OasisMega1:/Oasis/bin#
Images of tail end of log with code-driven colouring:
The allure of using the short-form is seductive, but I prefer to explicitly enumerate all the flags used so that I don't have to remember, or guess at, what the features of "-a" would be.
Having just gone back to review the "-a" option, I've decided to "upgrade" my script command to include
--atimes
--crtimes
I guess I missed those during my initial script development because I thought they were implicitly included already.
Personally don't know of the value of --acls for Desktop. I can see that being used more in a Corporate environment.
Thank you Eugene. I am aware of that being outlined, as you've described, in the man pages. For some reason, my brain thought those two were already included. As I said earlier, reviewing the man page again today, I realized they weren't.
If you already rsync'ed that data onto the server once before, and your files are not monster files/directories, then you should be able to safely perform your rsync, as long as your largest directory does not contain more than 1.3 TB to be on the safe side. If not, things will get tricky. Just make sure you are using the "--delete-during" flag, and NOT the "--delete-after" flag.
Thank you. I am familiar with SIP and disabling it (which I need to do again after resetting PRAM). I am also familiar with launchd and launchctl but my Catalina OS is older than the Ventura mentioned in your links. But I'm going to give it a try. I'm still trying to figure out how I'm going to offload the data on the HDD. It's the old "trying to stuff 10 pounds of manure into a 5 pound bag."
All good questions. This goes back to my question regarding fdupes (and rdfind and jdupes, which all have exhibited problems sniffing out duplicate files.