First I should explain how I handle disks on my home machine.
I have four drives, as follows:
The /u1 drive contains all of my personal data. Projects, photos, my home directory and so forth. I have a script that runs rsync every night to perform backups to /u2 and /u3 which thus end up being mirrored copies of /u1. At one time I had just /u1 being mirrored to /u2, but as these two 2T drives began to nearly get full, I added /u3 which is a 4T drive. The idea was that some fine day I would purchase a second 4T drive, make the original 4T drive my /u1, pull out both of the 2T drives, label them and put them on a shelf, and make the new 4T drive /u2 and continue with the pair of 4T drives in the system.
That day has come. I just ordered a refurbished Western Digital "black" 4T drive on Amazon for $58 and when it arrives later today, I intend to swap the drives. Amazon gives a 90 day return period on such items, and that should be plenty of time to decide if the refurb is a dud or not. My view on refurbished drives is that there is really nothing that they can do to "refurbish" it. They just check it out to find out if some moron returned a perfectly good drive, then sell it again if so.
I have noticed something curious. When I run "df" I see the following:
Filesystem 1K-blocks Used Available Use% Mounted on /dev/sdc3 1864425176 1572180800 197513980 89% /u1 /dev/sdd3 1905239480 1680975084 127413756 93% /u2 /dev/sdb3 3829165828 1680986700 1953597952 47% /u3Why does a mirrored copy on /u2 take up more space than the original on /u1. After some investigation I discover the reason. Hard links. Rsync does not preserve them. What was a single file with two links on the original disk becomes two separate files in the copy. I was never aware of this. Does it matter?
The difference is 108,794,284 blocks of 1k, i.e. 109G of space. Nothing to sneeze at, although on a 4T drive we could ignore wasting about 2 percent of the drive in this way. But maybe there is a way to redo the copy and maintain the links. A search indicate that this is a well known issue, but how to use the options to rsync to do something about it are less than clear.
One tip was to use "cp -aH". I did a quick experiment using this and found that if the experimental copy was on the same filesystem as the original, it make links in the copy to the original files. Not at all what I wanted! However if I do this:
cp -aH orig /u1/copyI get exactly what I want when the directory "orig" is on a different filesystem than "/u1/copy" So, what I can do is to nuke my /u3 disk (perhaps just reformat it) and then use cp -aH to do the copy via:
su umount /u3 mkfs -t ext4 /dev/sdb3 mount /u3 cp -aH /u1 /u3The copy takes a very long time. (It takes on the order of 5 hours). When it is done, I see this:
Filesystem 1K-blocks Used Available Use% Mounted on /dev/sdc3 1864425176 1572180836 197513944 89% /u1 /dev/sdd3 1905239480 1680975084 127413756 93% /u2 /dev/sdb3 3829075716 1572080476 2062414064 44% /u3Looking at the "Used" column, we have now achieved a slight bit of compression! But why the compression? The amount is 100,372 blocks of 1K (100M).
The above command gives me /u3/u1 which is annoying when I first consider it, but it avoids unpleasant possible conflict involving the lost+found directory. It is easy enough to fix as follows:
cd /u3/u1 rmdir lost+found mv * .. cd .. rmdir u1To try to sort out what the "compression" is about, I do this:
cd /u1 ; ls -lR >/home/u1ls cd /u3 ; ls -lR >/home/u3ls cd /home diff u1ls u3lsThis yields quite a lot of strange business, usually in directory sizes, but nothing really illuminating.
As a final note, there is an obscure option to rsync that is intended to help with this business of hard links. This is the "--link-dest=DIR", where "DIR" is supposed to be the target directory. This is not what I want, as it creates links when files are unchanged. It looks like the sort of thing you might use to create a time machine backup scheme.
What I do is to create /u1/home. Then I do this:
su mkdir /u1/home cd /u1/home mv /home/tom . cd /home ln -s /u1/home/tomNote that my SSD has a /boot partition, then a "Linux LVM" for the bulk of the disk. Somehow the Fedora install set up a / and /home inside this LVM and arranges to mount them in a way that I am not familiar with. Each is about 55G in size. The way I am now rearranging things, I might as well not have the /home paritition as it will be mostly empty, but such is life.
I also have a /u1/tom directory, but this is old stuff and not the same as /u1/home/tom. It once was (somewhere in the distant past) my /home directory. To avoid confusion, I decided to rename it to /u1/tom_archive.
Once I am all done and have used "cp -aH" to regenerate the contents of my disk mirrors, "df" shows me:
Filesystem 1K-blocks Used Available Use% Mounted on /dev/sda1 999320 232440 698068 25% /boot /dev/mapper/fedora-root 51474912 38161584 10675504 79% / /dev/mapper/fedora-home 57989140 121956 54898404 1% /home /dev/sdc3 1864425176 1660434964 109259816 94% /u1 /dev/sdd3 1905161656 1660325396 147985620 92% /u2 /dev/sdb3 3829075716 1660374896 1974119644 46% /u3The next thing will be to pull and label /u1 and /u2, make /u3 to be /u1 and install my new 4T as /u2, partition and gemerate the mirror copy on it.
Everything boots up without issue and the new drive shows up as /dev/sdc. I use fdisk. The first command I give is "g" to use a GPT partition table (otherwise I only could use 2T of the 4T drive). Then I use the "n" command to create the 3 partitions. I use "t 2 19" to set the type of partition 2 to swap. When done I see:
/dev/sdc1 2048 100351 98304 48M Linux filesystem /dev/sdc2 100352 33654783 33554432 16G Linux swap /dev/sdc3 33654784 7814037134 7780382351 3.6T Linux filesystemThen I do this:
su mkswap /dev/sdc2 mkfs -t ext4 /dev/sdc3After this I edit /etc/fstab to add the 2 lines for swap and /u2, then:
swapon /dev/sdc2 mount /u2 cp -aH /u1 /u2 cd /u2/u1 rm lost+found mv * .. cd .. rmdir u1I run "tail -f /var/log/messages" in a window to watch for error messages from the new disk. This $58 refurbished disk looks fine so far. No smoke or loud noises anyway. I installed it at 4PM on 5-1-2022. Files all copied by 9 PM.
Filesystem 1K-blocks Used Available Use% Mounted on /dev/sdb3 3829075716 1663676512 1970818028 46% /u1 /dev/sdc3 3827989484 1663669704 1969793840 46% /u2
Adventures in Computing / tom@mmto.org