I have a 2TB drive I want to copy to another 2TB drive. They're attached as USB devices to my Ubuntu desktop. I don't want to shut down and use a boot CD (eg Clonezilla) because I want to keep using my PC in the meantime.
Assuming I want to copy from /dev/sdX to /dev/sdY, I know I can use dd if=/dev/sdX of=dev/sdY .. but it'll be slow because it'll copy loads of empty space.
I know I can use partclone, which "understands" various filesystems and can skip empty space, but that won't clone the partitioning itself, nor I assume will it copy any boot sector stuff?
What's the bit I'm missing, and/or is there a better way?
Mark
On 27/09/16 14:47, Mark Rogers wrote:
I have a 2TB drive I want to copy to another 2TB drive. They're attached as USB devices to my Ubuntu desktop. I don't want to shut down and use a boot CD (eg Clonezilla) because I want to keep using my PC in the meantime.
Assuming I want to copy from /dev/sdX to /dev/sdY, I know I can use dd if=/dev/sdX of=dev/sdY .. but it'll be slow because it'll copy loads of empty space.
I know I can use partclone, which "understands" various filesystems and can skip empty space, but that won't clone the partitioning itself, nor I assume will it copy any boot sector stuff?
What's the bit I'm missing, and/or is there a better way?
Mark
I imagine that dd or partclone would be quicker overall than other methods, depending on how much unused space there is, but, if you want to keep using the machine, then they're probably out of the question. I imagine the reason for using dd or partclone is to get and exact or quick copy, but...
a simple cp or rsync will copy a file structure and file permissions and ownership if you get the right options. It won't copy blank space. It won't bit and exact copy (as in file "a" won't be on inode "235"), but all the data should copy. So why not do that?
This page is one of many giving ideas how to do it. http://superuser.com/questions/307541/copy-entire-file-system-hierarchy-from...
Good luck Steve
On 27 September 2016 at 16:15, steve-ALUG@hst.me.uk wrote:
I imagine that dd or partclone would be quicker overall than other methods, depending on how much unused space there is, but, if you want to keep using the machine, then they're probably out of the question. I imagine the reason for using dd or partclone is to get and exact or quick copy, but...
I'm not sure I was very clear...
The 2TB drive is a boot disk from another PC that is misbehaving. I want to image it to a new disk, using my main desktop PC. Therefore the disk being imaged (and the destination) are not mounted and I can use dd etc on them while I use the PC for everything I would normally do.
dd looks like it'll take 35hrs, at least in part because of the blank space (mostly because of the fact it's working over USB2 but I can't help that).
partclone should work once the partitions are there, and once everything outside the partition structure (eg boot sectors) are copied.
My guess is that simply using dd to copy the start of the disk (although I have no idea how much of it) will take care of boot and partition table, and I can then use partclone for the rest. But guessing isn't a great plan when I'm trying to retain data!
a simple cp or rsync will copy a file structure and file permissions and ownership if you get the right options. It won't copy blank space. It won't bit and exact copy (as in file "a" won't be on inode "235"), but all the data should copy. So why not do that?
I don't care about inodes so that would be OK, but I would expect the result to be bootable, which I didn't think would happen with cp/rsync.
This page is one of many giving ideas how to do it. http://superuser.com/questions/307541/copy-entire-file-system-hierarchy-from...
Thanks, I'll take a look.
[Full story: The disk has three partitions; a Windows one, a Linux one, and a binary data one holding a VeraCrypt encrypted drive. Windows helped itself to an upgrade and rebooted, and won't come back up. I may at some point try to fix it but for now I just want a copy of the disk so I can reinstall. I want to reinstall over the existing 2TB drive though, because that's an SSHD drive which will give me better performance than the 2TB HDD that I want to image to. There's no way around the binary image of the veracrypt partition, but the NTFS and Ext4 partions make up well over half the drive and could be copied a lot more quickly by partclone than dd.]
On 27/09/16 19:59, Mark Rogers wrote:
[Full story: The disk has three partitions; a Windows one, a Linux one, and a binary data one holding a VeraCrypt encrypted drive. Windows helped itself to an upgrade and rebooted, and won't come back up. I may at some point try to fix it but for now I just want a copy of the disk so I can reinstall. I want to reinstall over the existing 2TB drive though, because that's an SSHD drive which will give me better performance than the 2TB HDD that I want to image to. There's no way around the binary image of the veracrypt partition, but the NTFS and Ext4 partions make up well over half the drive and could be copied a lot more quickly by partclone than dd.]
Ah, if you want it to boot, you're right, cp & rsync won't do. Just bite the bullet and dd it, all of it.
But add the bs option at the end. e.g. bs=4M This sets the block size, or how much to copy in one go. I think the default block size is way smaller than that so that should speed things up.
If you have an up-to-date version of dd, you can add |status=progress to show the progress of the copy.
Regards Steve |
On 28 September 2016 at 00:37, steve-ALUG@hst.me.uk wrote:
Ah, if you want it to boot, you're right, cp & rsync won't do. Just bite the bullet and dd it, all of it.
The frustrating thing is that if I booted from a clonezilla boot disk then it would handle this more intelligently using the same tools I can use from my desktop if only I knew how to work them!
I left it imaging yesterday in case dd was the best option so I'm about 50% complete now, so I guess I'll just have to leave it to finish until tomorrow.
But add the bs option at the end. e.g. bs=4M
Yep, I've actually got bs=100M although that's probably overkill. I'm getting about 1GB/min across USB2 though so I reckon that's about as good as it'll get (and both disks are in a 4-disk enclosure on that one USB connection, so it's not even using two ports).
If you have an up-to-date version of dd, you can add |status=progress to show the progress of the copy.
I didn't know that, but I'm using "sudo killall -USR1 dd" periodically to check progress that way.
Thanks for the suggestions.
On 27 September 2016 at 14:47, Mark Rogers mark@more-solutions.co.uk wrote:
I have a 2TB drive I want to copy to another 2TB drive. They're attached as USB devices to my Ubuntu desktop. I don't want to shut down and use a boot CD (eg Clonezilla) because I want to keep using my PC in the meantime.
Assuming I want to copy from /dev/sdX to /dev/sdY, I know I can use dd if=/dev/sdX of=dev/sdY .. but it'll be slow because it'll copy loads of empty space.
I know I can use partclone, which "understands" various filesystems and can skip empty space, but that won't clone the partitioning itself, nor I assume will it copy any boot sector stuff?
What's the bit I'm missing, and/or is there a better way?
Mark
Can you set up the same partitions on the 2nd drive the use rsync to only copy the files?
Cheers, James.
-
James Bensley -
Mark Rogers -
steve-ALUG＠hst.me.uk