Quite often, system administrators have to copy various binary data. For example, sometimes you may want to back up your hard drive, create an empty file filled with zeros to organize swap space or another virtual file system.
To solve all these tasks, the linux dd utility is used, which simply copies data from one place to another at the binary level. It can copy a CD/DVD disc, a partition on a disc, or even an entire hard drive. In this article, we will look at what the dd linux command is, its main options and parameters, and how to use it.
First you need to understand how the dd command works and what it does. In fact, this is an analogue of the utility for block data only. The utility simply transfers one block of data of the specified size from one place to another. Because Linux treats everything, including devices, as files, you can move devices to files and vice versa.
Using various utility options, you can influence the block size, and this, in turn, already affects the speed of the program. Next, we will consider the main options of the utility and its capabilities.
dd command
The syntax of the utility is quite unusual, but at the same time very simple, after you remember it and get used to it:
$ dd if= copy_source of= destination parameters
With the if parameter, you need to specify the source from which the blocks will be copied, it can be a device, for example, / dev / sda or a file - disk.img. Next, using the of parameter, you must specify the destination device or file. Other parameters have the same syntax as if and of.
Now let's look at the additional options:
- bs- indicates how many bytes to read and write at a time;
- cbs- how many bytes should be written at a time;
- count- copy the specified number of blocks, the size of one block is specified in the bs parameter;
- conv- apply filters to the data stream;
- ibs- read the specified number of bytes at a time;
- obs- write the specified number of bytes at a time;
- seek- skip the specified number of bytes at the beginning of the device for reading;
- skip- skip the specified number of bytes at the beginning of the output device;
- status- indicates how detailed you need to make a conclusion;
- iflag, oflag- allows you to set additional operation flags for the input and output device, the main ones are: nocache, nofollow.
These were all the basic options you might need. Now let's move closer to practice and look at a few examples of how to use the dd linux utility.
How to use dd
Regular users use the dd command most often to create DVD or CD images. For example, to save a disk image to a file, you can use the following command:
sudo dd if=/dev/sr0 of=~/CD.iso bs=2048 conv=noerror
The noerror filter allows you to disable error response. Next, you can create an image of a hard disk or a partition on it and save this image to disk. Just be careful not to save to the same hard drive or partition, so as not to cause recursion:
dd if=/dev/sda of=~/disk.img
A file called disk1.img will be created in your home folder, which you can deploy and restore the broken system in the future. To write an image to a hard drive or partition, just swap the device addresses:
dd if=~/disk.img of=/dev/sda
A very important and useful option is bs. It allows you to greatly influence the speed of the utility. This option allows you to set the size of one block when transferring data. Here you need to specify a numeric value with one of these format modifiers:
- With- one character;
- b- 512 bytes;
- kB- 1000 bytes;
- K- 1024 bytes;
- MB- 1000 kilobytes;
- M- 1024 kilobytes;
- GB- 1000 megabytes;
- G- 1024 megabytes.
The dd linux command uses just such a system, it is complex, but there is no escape from it. It must be understood and remembered. For example, 2b is 1 kilobyte, and 1k is also 1 kilobyte, 1M is 1 megabyte. By default, the utility uses a block size of 512 bytes. For example, to speed up disk copying, you can take blocks of 5 megabytes in size. For this, the following command is used:
dd if=/dev/sda of=~/disk.img bs=5M
The next parameter is count. With it, you can specify how many blocks to copy. For example, we can create a 512 megabyte file by filling it with zeros from /dev/zero or random numbers from /dev/random:
sudo dd if=/dev/zero of=file.img bs=1M count=512
Please note that this parameter does not indicate the size in megabytes, but only the number of blocks. Therefore, if you specify a block size of 1b, then only two blocks need to be taken to create a 1Kb file. You can also use this option to back up the MBR partition table. To do this, copy the first 512 bytes of the hard disk to the file:
sudo dd if=/dev/sda of=mbr.img bs=1b count=1
To restore, use the normal command to deploy the image to disk.
If the disk image is too large, you can redirect all output to gzip's non-stdout output:
dd if=/dev/sda2 | bzip2 disk.img.bz2
You can also use the linux dd utility to copy files, although this is not its intended purpose:
dd if=/home/sergiy/test.txt of=/home/sergiy/test1.txt
As you know, the linux dd command writes data to disk directly in binary form, which means zeros and ones are written. They override what was previously placed on the recording device. Therefore, to erase a disk, you can simply fill it with zeros from /dev/zero.
sudo dd if=/dev/zero of=/dev/sdb
This use of dd causes the entire disk to be completely erased.
conclusions
In this article, we looked at how to use dd linux, what this utility can be used for, and how useful it can be. It is an almost indispensable system administrator tool because it can be used to backup entire systems. And now you know how. If you have any questions, ask in the comments!
Rufus- free and open source software for formatting removable USB storage media and creating bootable disks with various operating systems. The program is easy to use, high speed and supports multilingual interface.
You can download the program on the developer's website. The page contains links to download the standard version Rufus, and portable Rufus portable, which do not differ in anything, except for the name of the executable file and the location where the settings are stored. The program settings include the language used and the settings for checking for updates. The standard version stores these settings in the registry, while the portable version stores them in a file rufus.ini program directory. Rufus does not require installation on the system - just download the executable file and run it. The program interface is very simple:
In general, the program Rufus is not something unique in the field of tools for creating bootable media and its main advantage is ease of use. In order to create a bootable flash drive with its help, it is enough to have the initial image of the bootable system and be able to click on the “Start” button. All selectable parameters and settings, by default, are already designed for using the program to work on a computer with a standard configuration.
The most simple and convenient to use Rufus to create a bootable flash drive (bootable USB drive) from ISO images of Windows or Linux installation disks, as well as emergency system recovery disks and diagnostic tools.
When creating a bootable Windows flash drive, it is enough to select the device on which the recording will be performed and the bootable iso-image file. The program will substitute other parameters itself.
If there is no ISO image file available, then it can be created based on a physical CD (or from a set of distribution files) using CD / DVD burning programs such as the well-known Nero, Alcohol, or freely distributed CDBurnerXP or ImgBurn.
The procedure for creating a bootable Windows flash drive is as follows:
select the flash drive to which the image will be written. Unlike many similar programs, Rufus displays the volume label, drive letter and size, so if there are several removable drives in the system, it's easy to choose the one to write to.
select partition scheme and system interface type. Rufus allows you to create flash drives for booting in a regular BIOS interface and for booting in a UEFI environment, create boot records for MBR volumes and GPT volumes. The default mode is "MBR for computers with BIOS or UEFI" - the most common mode for bootable flash drives today.
select the file system that will be used on the created bootable flash drive. By default, bootable Windows flash drives use the file system FAT32, but if necessary, you can choose NTFS if you want to use files larger than 4 GB.
set the cluster size. The cluster size is selected by the program based on the image data and file system type, but if necessary, it can be changed.
specify the volume label that will be set for the created flash drive.
set formatting options. It is best to leave these options at their default and simply select the ISO image file. For images created by the program dd on Linux, you need to select the option DD image.
After pressing the button Start the program will format the flash drive, set the active partition flag, write the master boot record and partition boot record, as well as bootable media data from the ISO image. After completion of work Rufus you can boot using the resulting bootable flash drive.
Using virtualization technology to test bootable flash drives. Download links for free and handy programs to simplify the process of creating, debugging and checking the created bootable media.
Be careful, because if you make a mistake in the name of the flash drive in the dd command, you can damage the host's hard drive.
Let's display the designation of sections of all devices
and find among them a flash drive:
sudo fdisk -u -l /dev/sd?
Also, to define partitions on all devices, you can use the command:
sudo cat /proc/partitions
dd command syntax
dd if=/AAAA of=/BBBB bs=CCCC count=DDDD conv=noerror- if: (input file) points to the source, i.e. where we are copying from. Specifies a file, which can be either a regular file or a device file.
- of: (output file) points to the destination file. The same thing, we can write both to a regular file and directly to the device.
- bs: number of bytes to be written at a time. That is, the size of a piece of data that will be read and written at a time. it is recommended to set bs= to the size of the hard drive cache i.e. 8M 16M 32M
- count: how many pieces bs will be copied.
- conv: allows you to connect filters applicable to the data stream. Filter "no error" just disables stopping the program when it encounters a reading error.
Examples
Full disk erasure
So that nothing can be restored on the media - you can fill it with zeros, this command will always end with the error "the media ran out of space"
Dd if=/dev/zero of=/dev/sdX
Create a disk image
dd if=/dev/cdrom of=image.iso conv=noerrorit is also possible in the system
Mount -o loop /PathToImageFile/image.iso /mnt/FolderMount
If something does not work out, the process is divided into 2 levels:
Losetup -e /dev/loop0 /PathToImageFile/image.iso mount /dev/loop0 /mnt/FolderMount
Operations with MBR
The MBR is located in the first 512 bytes of the hard disk, and consists of a partition table, a bootloader, and a couple of extras. byte. Sometimes, it has to be backed up, restored, etc. Backup is done like this:
Dd if=/dev/sda of=mbr.img bs=512 count=1
It's easier to restore:
Dd if=mbr.img of=/dev/sda
Copying with archiving
(using gzip) data from a flash drive to a hard drive.
Dd if=/dev/sdX conv=sync,noerror bs=8M | gzip -c >/PathToSave/sdX.img.gz
and back
Gunzip -c /PathToFile/sdX.img.gz | dd of=/dev/sdX conv=sync,noerror bs=8M
Copying over the network
dd if=/dev/sdX conv=sync,noerror bs=8M | ssh -c blowfish [email protected]"dd of=sdX.img.gz bs=8M"and back
Dd if=sdX.img.gz | ssh -c blowfish [email protected]"dd of=/dev/sdX bs=8M"
To create an image of a hard disk, it is not necessary to use utilities like Acronis True Image or Norton Ghost, a simple dd utility is enough, which is included with most Unix-like operating systems (Linux, FreeBSD, Solaris, etc.) The article describes a simple way creating a hard disk image backup using dd. The first step is to prepare for the backup. In this article, we introduce the following notation:
- /dev/sda - disk to be imaged;
- /dev/sdb - the disk where the image will be written to.
Substitute your own values if necessary.
Preparing to create a hard drive image
The first step is to boot from any available Live-CD that has the dd utility and log into the command line as superuser. Create a mount point for backup.
mkdir /mnt/backup
Mount the hard drive on which you want to save the image.
Create a hard drive image
dd if=/dev/sda of=/mnt/backup/sda.img bs=8M conv=sync,noerror
- if=/dev/sda - copy the entire hard drive sda;
- of=/mnt/backup/sda.img - copy to /mnt/backup/sda.img;
- bs=8M - set the size of the hard disk cache to speed up the copying procedure (otherwise the data will be dumped in small portions of 512 bytes);
- conv=sync,noerror - tell dd to copy bit-for-bit, ignoring read errors.
To reduce the size of a hard disk image, you can compress it with any archiver.
dd if=/dev/sda bs=8M conv=sync,noerror | gzip -c > /mnt/backup/sda.img
Restoring a hard drive image
To restore a hard disk image, you need to follow the procedure reverse to the procedure for creating this image.
dd if=/mnt/backup/sda.img of=/dev/sda bs=8M conv=sync,noerror
When using compression in parallel, you must unzip the image.
gunzip -c /mnt/backup/sda.img | dd of=/dev/sda conv=sync,noerror bs=8M
Migrating the system to another hard drive
To migrate the entire system to another hard drive, you must set the destination of the new drive as the destination.
dd if=/dev/sda of=/dev/sdb bs=8M conv=sync,noerror
Then, if necessary, install the boot from this hard drive. Provided that the new hard drive is larger than the old one, it will remain unallocated area. It should be marked and formatted according to existing requirements.
Copy statistics in dd
The main disadvantage in dd is the lack of a visual representation of the statistics of the execution of the copy procedure. However, this disadvantage can be easily circumvented. It is enough to connect to another terminal.
Determine the process number under which dd is running.
Periodically send this process a kill -USR1 dd_process_number command.
watch -n 5 kill -USR1 dd_process_number
- watch -n 5 - run command every 5 seconds;
- kill -USR1 dd_process_number - show copy statistics.
So, suppose you have downloaded an ISO file with a distribution image and prepared a flash drive.
The terminal command for writing the iso image of the distribution is as follows:
sudo dd if=path to .iso of=/dev/sdb
Where instead path to .iso you need to specify the path to the iso file from your folder Downloads.
sdb at the end of the command, this is the flash drive on which the image will be written. To identify your flash drive, run in a terminal:
And if your flash drive is defined differently, change it in the command.
To correctly enter the path to the file in the command, open the folder with the file, open the nearby terminal and drag the file into it:
The terminal will display the path to the file and the .iso file itself. Copy it without quotes and paste it into the command, and in the terminal press the keyboard shortcut ctrl+c to reset and display a new prompt string.
The easiest way to create a command is in a text editor.
I got the following command:
Now insert the USB flash drive into the port of the computer, copy the received command from the text editor, enter into the terminal and execute (press Enter):
The terminal cursor will blink and it seems that nothing is happening, but in fact there is a recording. Wait for the image recording to finish and at the end you should see something like the following in the terminal:
That's all. Your flash drive has become bootable and you can now install the distribution on any computer.
