rsync Command in Linux with Examples
The rsync
command is one of the most popular and powerful tools in Linux for synchronizing files and directories between different locations. It is widely used for backups, remote file transfers, and directory syncing, offering robust features like incremental file transfers, compression, and file permission preservation.
In this article, we’ll explore the basics of rsync
, along with practical examples to help you master its usage.
Table of Contents
Basic Syntax of rsync
:
rsync [OPTION]... SRC [SRC]... DEST
Where SRC
represents the source directory or files, and DEST
represents the destination.
Commonly Used rsync
Options
-a
: Archive mode, which preserves symbolic links, file permissions, timestamps, etc.-v
: Verbose, to show detailed output of the operation.-z
: Compresses file data during the transfer.-P
: Displays progress during the transfer and allows partial transfers to resume.--delete
: Deletes files from the destination that no longer exist in the source.-r
: Recursively transfer directories.
Examples of rsync
in Action
1. Basic File Synchronization
This simple example copies all files from the source directory to the destination.
rsync -av /source/directory/ /destination/directory/
The -a
flag ensures that all file attributes (permissions, timestamps, etc.) are preserved, while the -v
flag provides a verbose output to show progress.
2. Synchronizing Files Over SSH
rsync
can be used to securely copy files between two remote systems over SSH:
rsync -avz -e ssh /source/directory/ user@remote:/destination/directory/
Here, the -e ssh
option tells rsync
to use SSH for the transfer. This is especially useful for secure backups to remote servers.
3. Excluding Files and Directories
To exclude specific files or directories during synchronization, use the --exclude
option:
rsync -av --exclude 'file.txt' /source/directory/ /destination/directory/
This will copy everything except the file named file.txt
.
4. Resuming Incomplete Transfers
If a file transfer is interrupted, you can resume it with the --partial
option:
rsync -avP /source/directory/ /destination/directory/
The -P
flag combines --partial
and --progress
, which helps track the progress of the transfer and resume partial file transfers.
5. Deleting Files in Destination
To delete files from the destination that no longer exist in the source directory, use the --delete
option:
rsync -av --delete /source/directory/ /destination/directory/
This ensures that the destination mirrors the source by removing any files in the destination that have been deleted from the source.
6. Syncing Files Between Local and Remote Systems
This example synchronizes files from a local directory to a remote server:
rsync -avz /local/directory/ user@remote:/destination/directory/
Similarly, files from a remote server can be synced back to a local system using:
rsync -avz user@remote:/source/directory/ /local/directory/
7. Backing Up a Directory
You can easily create backups of directories with rsync
. This example creates a timestamped backup:
rsync -av /source/directory/ /backup/directory/$(date +%F)
This command will copy the contents of /source/directory/
to a directory in /backup/directory/
with a name based on the current date (e.g., 2024-10-01
).
Complimentary commands
iostat: rsync
can affect disk I/O when transferring large files. You can monitor disk performance during transfers using tools like iostat.
df and du: Before syncing, you might want to check disk space with the df command and analyze disk usage using du.
Conclusion
rsync
is a versatile and powerful tool that simplifies file synchronization and backup tasks on Linux. By using the right combination of options, you can efficiently manage local and remote file transfers while preserving file integrity and security. For secure file transfers over SSH, check out our guide on the scp command in Linux.
scp
: Securely copy files between hosts.cp
: Copy files and directories.tar
: Archive files and directories.
For more advanced usage and options, refer to the rsync
man page by running man rsync
.
Thanks for the post @hydn
I’m curious if you’ve ever used Rsync or Rclone to migrate data from an old server to a new one. If you have, what commands did you use?
I’m asking because I’ve noticed that Rclone or Rsync sometimes don’t set the file permissions properly. Additionally, I believe that we shouldn’t sync all files between the servers, such as /proc or /dev.
What are your thoughts on this?
Yes thats true and i think the best would be to use the dd for the backup and that works very well. You can easily back enormous amount of data and you can have no broken links.
Let me know if that works,
Gaurav
I’ve used “dd” before, but I’ve encountered some issues. What commands do you use for server migration between two different servers? Do you migrate all files from the old server, or do you exclude some?
Thanks in advance.
I mainly use rsync, timeshift, clone for the server migration. If you have to migrate all the files from the one server to the another. Basically, i dont do that.
What i do and what is suggested is to 1. create a root node 2. After creating a root node and then mount the directive to the server. In this way the root node is only having a minimal install and for that a 2.5 HDD disk should be done as the faster booting time.
In case of the mount drives, RAID5-SATA should be done. This will allow the read and drive on the mount drives slow as compared to the booting time but that will not affect the root node. So building a distributed computing as compared to the monolithic system.
Following that connect another drive and after that set a time shift from one drive to another as a cron job. This will allow the complete system back up and at the same time, mounted drives will be safe incase the crash goes on.
Those who keep the data on the root node need to be worrying about the moving of the files and i dont recommend that.
Let me know if you need more information,
Gaurav
Thanks for the reply!
I totally agree with you. However, sometimes, some servers don’t have that separation and they keep their data on the root node. Additionally, the servers must be migrated from one provider to another. In that case, moving the hard drive is not possible either. I was wondering if there is a way to do migration in such cases.