The Linux dd command is one of the most powerful utilities for Unix and Unix-like operating systems. It can be used in a variety of ways.
In this tutorial, we learn about the dd command on Linux with practical examples
.
- Prerequisites Terminal access
- A user account with sudo privileges
- Familiarity with the command line
.
.
. dd command
on Linux dd
command is mainly used to copy and convert data, therefore it stands for data duplicator
.
Using the dd command we can do:
- Backup and restore an entire hard drive or partition.
- CD or DVD backup images called ISO
- Copy regions of raw device files, such as MBR (master boot record) backup.
- Conversion of data formats such as ASCII to EBCDIC
- Convert lowercase to uppercase and vice versa.
Create a virtual file system and
files
.
Only the superuser can run the dd command.
You should be very careful while using this command as improper use can cause huge data loss. Therefore, some people consider dd tool as disk destroyer. Unknowingly or being careless can cause destruction of partition and block device metadata.
The
basic use of the dd utility is fairly easy because only two arguments are needed: if= to specify the input file and of= to specify the output file. The arguments for those options can be files or locking devices. However, I wouldn’t recommend using dd to copy files because cp does it in a much simpler way. However, you can use it to clone a hard drive.
Syntax:
dd if=<source file name> of=<target file name> [Options] dd
command Examples
We will learn several options as we review examples of dd commands
.
1. Backup and restore
an entire disk or partition
It is possible to save all data from an entire disk/partition to another disk/partition. It is not a simple copy as a cp command, but a block-sized copy.
a. Backup
the entire disk to disk
You can copy all data (entire disk) from the /dev/sda disk to /dev/sdb. dd knows nothing about the file system or partitions; It will simply copy everything from /dev/sda to /dev/sdb
.
You must indicate the size of the block to be copied at once with the bs option. Therefore, this will clone the disk with the same data on the same partition.
sudo dd if=/dev/sda of=/dev/sdb bs=4096 conv=noerror,sync Output 97281+0 records in 97280+0 records 99614720 bytes (100 MB) copied, 2.75838 s, 36.1 MB/s
This only works if the second device is as large or larger than the first. Otherwise, you get truncated and worthless partitions on the second one.
Where
,
if it means input file, de represents the output file
bs represents the block size (number of bytes to read/write at a time). Be sure to use block sizes in multiples of 1024 bytes, which is equal to 1 KB. If no ‘t specifies block size, dd use a default block size of 512 bytes.
The conv value noerror parameter allows the tool to continue copying data even if it encounters errors. The synchronization option allows you to use synchronized I/O.
b. Creating a dd disk image
(
archive image) You can create
an image of a disk or a file image. Backing up a disk to an image will be faster than copying the exact data. In addition, the disk image makes restoring much easier.
sudo dd if=/dev/sda of=/tmp/sdadisk.img
You can store the output file wherever you want, but you must give an extension to a file name that ends with .img. Instead of /tmp/sdadisk.img, you can store it, for example, in /sdadisk.img if desired.
c. Creating a
Compressed Disk Image
Because dd creates the exact contents of an entire disk, it means that it takes up too much size. You can decide to compress the disk image with the following command
sudo dd if=/dev/vda | gzip -c >/tmp/vdadisk.img.gz
The pipe | operator makes the output of the left command become the input of the right command. The -c option writes the output to the standard output and keeps the original files unchanged.
d. Backing up a partition
or cloning one partition to another
Instead of an entire disk, you can only back up a simple partition. You just need to indicate the name of the partition in the input file.
sudo dd if=/dev/sda1 of=/dev/sdb1 bs=4096 conv=noerror,sync
This will synchronize the /dev/sda1 partition with /dev/sdb1. You should verify that the size of /dev/sdb1 must be larger than /dev/sda1. Or you can create a partition image as below
sudo dd if=/dev/sda1of=/tmp/sda1.img e. Restoring a disk or partition image
Saving a disk or partition helps to restore all data, if there is a problem with our original drive
.
To restore, you must reverse the input file with the output file indicated during the
backup operation: sudo dd if=/tmp/sdadisk.img
of=/dev/sda
It will recover the data that was present before the
backup operation and not after of operation
f. Compressed Image Restoration
You must first indicate the compressed file and the output file which is the compressed disk before
. sudo gzip -dc /tmp/vdadisk.img.gz | dd of=/dev/vda
The -d option here is to unzip. Note the output file. You can mount the restored disc to view the contents.
Arabic numeral. Creating virtual file system images/backing up CDs or DVDs
as ISO files
You may need to create a virtual file system on Linux for some reasons, such as creating a virtual machine on your Linux host. You may also need to create a backup iso image of a CD or DVD
.
Creating
a virtual file system A virtual file system is a file system that exists in a
file, which in turn exists on a physical disk. You may need it to create, for example, an additional swap or loop device or virtual machine. We need /dev/zero, which is a file used to create a file with no data but with the required size (a file with all zeros). In other words, this will create a data file with all zeros in the file that will size a file.
sudo dd if=/dev/zero of=/file bs=1024K count=500 Output 500+0 records in 500+0 copied 524288000 byte (524 MB) registers, 1.21755 s, 431 MB/s
The option count refers to the number of input blocks to copy. Combined with the block size value, it indicates the total size to copy. For example bs=1024k and count=500 give a size=1024K*
500=524288000 bytes=524MB
Now let’s check
the size of our file ls -lh /file Output -rw-r-r- 1 root root 500M May 17 18:57 /file
You can see that we have our virtual file system created with the indicated size. Now you can use it to create a loopback device or a virtual disk or anything else.
b. Modify the first 512 bytes of a null
file
If during the operation you indicate an existing output file, you will lose your data. For some reason, you may need to replace a block size of the output file.
sudo dd if=/dev/zero of=file1 bs=512 count=1 conv=notrunc
The notrunc option refers to not truncating the file, only replacing the first 512 bytes, if it exists. Otherwise, you will get a 512-byte file
c. Creating a CD
or DVD Backup ISO Image
You may wonder why not simply copy the contents of your CD to a directory. How would you handle the boot sector of a CD? You can’t find that as a file on the device because it’s only the first sector. Because dd copies sector by sector, on the other hand, will copy that information as well.
sudo dd if=/dev/cdrom of=/mycd.iso
You should know that you should use the -o loop option, which allows you to mount a file like any normal device. So
, to mount /mycd.iso in the /mnt/cd directory, do the following sudo mount -o loop /mycd.iso /mnt/cd
d. Create a bootable USB drive
Suppose we have downloaded the AlmaLinux iso image here to this directory “
~/Downloads/”. sudo dd if=~/Downloads/AlmaLinux-9-latest-x86_64-dvd.iso of=/dev/sdb bs=1M if=~/Downloads/AlmaLinux-9-latest-x86_64-dvd.iso – the input file is in the path ‘~/Downloads/AlmaLinux-9-latest-x86_64-dvd.iso’ of=/dev/sdb – the
output file is in the
path ‘/dev/sdb’
bs=1M – Read from ‘~/Downloads/AlmaLinux-9-latest-x86_64-dvd.iso’ and write to ‘/dev/sdb’ 1 Megabytes of data at a time
.
3. MBR
Backup and Restore
The GRUB bootloader is most commonly stored in the MBR of the boot drive. The MBR makes up the first 512 bytes of the disk, allowing up to 466 bytes of storage for the boot loader. The additional space will be used to store the partition table for that drive. If MBR gets corrupted, we won’t be able to boot into Linux.
a. MBR backup
Because the MBR makes up the first 512 bytes of the disk, we only need to copy that block size sudo dd if=/dev/sda of=/tmp/sdambr.img bs=512 count=1
With count=1 and bs=512, only 512 bytes will be copied, which corresponds to the size of our
MBR. You can display the
saved MBR
with the od command that dumps files in octal and other formats as shown below
sudo od -xa /tmp/sdambr.img Output 0000000 bf52 81f4 8b66 832d 087d 0f00 e284 8000 R ? T SOH F VT – ETX } BS NUL SI EOT B NUL NUL 0000020 FF7C 7400 6646 1D8B 8B66 044D 3166 B0C0 | del nul t F f vt gs f vt M eot f 1 @ 0 –
a option selects named characters and -x selects hexadecimal units of 2
bytes b. Backing up MBR boot data
excluding partition table The
512-byte MBR data is located in the first sector of the hard disk. It consists of 446 bytes bootstrap, 64 bytes partition table and 2 signature bytes. This means that we can exclude the partition table and byte signature while backing up the MBR while retaining only a block size equal to the size of the bootstrap.
sudo dd if=/dev/sda of=/tmp/sdambr2.img bs=446 count=1 c. Restoring MBR from the MBR
image
You can restore your MBR as shown in the previous commands with
sudo dd if=/tmp/sdambr.img of=/dev/sda
3. Converting Data Formats
If an input file uses a character set that is not the host computer’s native character set, the import operator must perform a conversion. For example, if ASCII is the native format for strings on the host computer, but the input data file represents strings using EBCDIC, you must convert EBCDIC to ASCII and vice versa.
to. Convert the data format of an EBCDIC file to ASCII If there is an ebcdic file with you,
mostly recovered from mainframe systems, then, you would like to convert them to ASCII to make modifications using text editors on UNIX servers
sudo dd if=textfile.ebcdic of=textfile.ascii conv=ascii
The value parameter conv is now ascii because we convert from EBCDIC
to ASCII b
. Convert the data format of a file from
ASCII to EBCDIC
After modifying the ASCII version and once done, you can convert it back to EBCDIC to be used by your application
.
sudo dd if=textfile.ascii of=textfile.ebcdic conv=ebcdic
The conv value parameter is now ebcdic because we convert from ASCII to EBCDIC. If you are only replacing the particular number of bytes with an equivalent number of bytes that have different characters, the conversion would be smooth and the application reading the file should not have any problems.
4. Converting case of a DD command file
can also be used for something amazing. You can convert all text (alphabets) in a file to uppercase or lowercase and vice versa. For the following example, we will have a file for testing.
cat file10 test dd convert a. Convert a file to uppercase
Because our text file example is lowercase, we’ll convert it to uppercase
sudo dd if =~ / file10 of =~ / file20 conv=ucase
The command will create the new file indicated. See that now the conv option takes the value ucase. Let’s check the result
cat file20 TEST DD CONVERT b. Convert a file to lowercase Now we will do the reverse operation that will convert to lowercase dd if=~/file20 of=~/file30 conv=lcase
See that we use lcase of the conv option
to convert from uppercase
to lowercase
. cat file30 test dd convert
dd command does not convert the file names, it is only the contents of it
. 5.
Show progress
dd utility is packaged with Coreutils. Versions earlier than 8.24 may display a progress bar while copying.
You can check the version of coreutils by checking the command dd -version.
The following time displays a progress bar showing the amount of data copied and the elapsed time.
sudo dd if=/home/ubuntu/data.txt of=/home/ubuntu/backup/backup.txt status=progress By default, the dd command
will not display the progress bar you need to add the status=progress option
.
Conclusion
In this tutorial, we learn in depth about the dd command on Linux. We have illustrated with examples of dd commands to understand the options.
Please add your comments and suggestions to the comments section below.
