What's new in rsync 3.1.2
Dec 22, 2015
- This is a bug-fix release. It includes a security fix for a transfer from a sender that you don't fully trust.
New in rsync 3.1.0 (Sep 30, 2013)
- OUTPUT CHANGES:
- Output numbers in 3-digit groups by default (e.g. 1,234,567). See the --human-readable option for a way to turn it off. See also the daemon's "log format" parameter and related command-line options (including --out-format) for a modifier that can be used to request digit-grouping or human-readable output in log escapes. (Note that log output is unchanged by default.)
- The --list-only option is now affected by the --human-readable setting. It will display digit groupings by default, and unit suffixes if higher levels of readability are requested. Also, the column width for the size output has increased from 11 to 14 characters when human readability is enabled. Use --no-h to get the old-style output and column size.
- The output of the --progress option has changed: the string "xfer" was shortened to "xfr", and the string "to-check" was shortened to "to-chk", both designed to make room for the (by default) wider display of file size numbers without making the total line-length longer. Also, when incremental recursion is enabled, the string "ir-chk" will be used instead of "to-chk" up until the incremental-recursion scan is done, letting you know that the value to check and the total value will still be increasing as new files are found.
- Enhanced the --stats output: 1) to mention how many files were created (protocol >= 28), 2) to mention how many files were deleted (a new line for protocol 31, but only output when --delete is in effect), and 3) to follow the file-count, created-count, and deleted-count with a subcount list that shows the counts by type. The wording of the transferred count has also changed so that it is clearer that it is only a count of regular files.
- BUG FIXES:
- Fixed a bug in the iconv code when EINVAL or EILSEQ is returned with a full output buffer.
- Fixed some rare bugs in --iconv processing that might cause a multibyte character to get translated incorrectly.
- Fixed a bogus "vanished file" error if some files were specified with "./" prefixes and others were not.
- Fixed a bug in --sparse where an extra gap could get inserted after a partial write.
- Changed the way --progress overwrites its prior output in order to make it nearly impossible for the progress to get overwritten by an error.
- Improved the propagation of abnormal-exit error messages. This should help the client side to receive errors from the server when it is exiting abnormally, and should also avoid dying with an "connection unexpectedly closed" exit when the closed connection is really expected.
- The sender now checks each file it plans to remove to ensure that it hasn't changed from the first stat's info. This helps to avoid losing file data when the user is not using the option in a safe manner.
- Fixed a data-duplication bug in the compress option that made compression less efficient. This improves protocol 31 onward, while behaving in a compatible (buggy) manner with older rsync protocols.
- When creating a temp-file, rsync is now a bit smarter about it dot-char choices, which can fix a problem on OS X with names that start with "..".
- Rsync now sets a cleanup flag for --inplace and --append transfers that will flush the write buffer if the transfer aborts. This ensures that more received data gets written out to the disk on an aborted transfer (which is quite helpful on a slow, flaky connection).
- The reads that map_ptr() now does are aligned on 1K boundaries. This helps some filesystems and/or files that don't like unaligned reads.
- Fix an issue in the msleep() function if time jumps backwards.
- Fix daemon-server module-name splitting bug where an arg would get split even if --protect-args was used.
- ENHANCEMENTS:
- Added the --remote-option=OPT (-M OPT) command-line option that is useful for things like sending a remote --log-file=FILE or --fake-super option.
- Added the --info=FLAGS and --debug=FLAGS options to allow finer-grained control over what is output. Added an extra type of --progress output using --info=progress2.
- The --msgs2stderr option can help with debugging rsync by allowing the debug messages to get output to stderr rather than travel via the socket protocol.
- Added the --delete-missing-args and --ignore-missing-args options to either delete or ignore user-specified files on the receiver that are missing on the sender (normally the absence of user-specified files generates an error).
- Added a "T" (terabyte) category to the --human-readable size suffixes.
- Added the --usermap/--groupmap/--chown options for manipulating file ownership during the copy.
- Added the "%C" escape to the log-output handling, which will output the MD5 checksum of any transferred file, or all files if --checksum was specified (when protocol 30 or above is in effect).
- Added the "reverse lookup" parameter to the rsync daemon config file to allow reverse-DNS lookups to be disabled.
- Added a forward-DNS lookup for the daemon's hosts allow/deny config. Can be disabled via "forward lookup" parameter (defaults to enabled).
- Added a way for more than one group to be specified in the daemon's config file, including a way to specify that you want all of the specified user's groups without having to name them. Also changed the daemon to complain about an inability to set explicitly-specified uid/gid values, even when not run by a super-user.
- The daemon now tries to send the user the error messages from the pre-xfer exec script when it fails.
- Improved the use of alt-dest options into an existing hierarchy of files: If a match is found in an alt-dir, it takes precedence over an existing file. (We'll need to wait for a future version before attribute-changes on otherwise unchanged files are safe when using an existing hierarchy.)
- Added per-user authorization options and group-authorization support to the daemon's "auth users" parameter.
- Added a way to reference environment variables in a daemon's config file (using %VAR% references).
- When replacing a non-dir with a symlink/hard-link/device/special-file, the update should now be done in an atomic manner.
- Avoid re-sending xattr info for hard-linked files w/the same xattrs (protocol 31).
- The backup code was improved to use better logic maintaining the backup directory hierarchy. Also, when a file is being backed up, rsync tries to hard-link it into place so that the upcoming replacement of the destination file will be atomic (for the normal, non-inplace logic).
- Added the ability to synchronize nano-second modified times.
- Added a few more default suffixes for the "dont compress" settings.
- Added the checking of the RSYNC_PROTECT_ARGS environment variable to allow the default for the --protect-args command-line option to be overridden.
- Added the --preallocate command-line option.
- Allow --password-file=- to read the password from stdin (filename "-").
- Rsync now comes packaged with an rsync-ssl helper script that can be used to contact a remote rsync daemon using a piped-stunnel command. It also includes an stunnel config file to run the server side to support ssl daemon connections. See the packaging/lsb/rsync.spec file for one way to package the resulting files. (Suggestions for how to make this even easier to install & use are welcomed.)
- Improved the speed of some --inplace updates when there are lots of identical checksum blocks that end up being unusable.
- Added the --outbuf=N|L|B option for choosing the output buffering.
- Repeating the --fuzzy option now causes the code to look for fuzzy matches inside alt-dest directories too.
- The --chmod option now supports numeric modes, e.g. --chmod=644,D755
- Added some Solaris xattr code.
- Made an rsync daemon (the listening process) exit with a 0 status when it was signaled to die. This helps launchd.
- Improved the RSYNC_* environment variables for the pre-xfer exec script: when a daemon is sent multiple request args, they are now joined into a single return value (separated by spaces) so that the RSYNC_REQUEST environment variable is accurate for any "pre-xfer exec". The values in RSYNC_ARG# vars are no longer truncated at the "." arg (prior to the request dirs/files), so that all the requested values are also listed (separately) in RSYNC_ARG# variables.
- EXTRAS:
- Added an "instant-rsyncd" script to the support directory, which makes it easy to configure a simple rsync daemon in the current directory.
- Added the "mapfrom" and "mapto" scripts to the support directory, which makes it easier to do user/group mapping in a local transfer based on passwd/group files from another machine.
- There's a new, improved version of the lsh script in the support dir: it's written in perl and supports -u without resorting to using sudo (when run as root). The old shell version is now named lsh.sh.
- There is a helper script named rsync-slash-strip in the support directory for anyone that wants to change the way rsync handles args with trailing slashes. (e.g. arg/ would get stripped to arg while arg/. would turn into arg/).
- INTERNAL:
- The I/O code was rewritten to be simpler and do bigger buffered reads over the socket. The I/O between the receiver and the generator was changed to be standard multiplexed-I/O (like that over the socket).
- The sender tries to use any dead time while the generator is looking for files to transfer in order to do sender-side directory scanning in a more parallel manner.
- A daemon can now inform a client about a daemon-configured timeout value so that the client can assist in the keep-alive activity (protocol 31).
- The filter code received some refactoring to make it more extendible, to read better, and do better sanity checking.
- Really big numbers are now output using our own big-num routine rather than casting them to a double and using a %.0f conversion.
- The pool_alloc library has received some minor improvements in alignment handling.
- Added init_stat_x() function to avoid duplication of acl/xattr init code.
- The included zlib was upgraded from 1.2.3 to 1.2.8.
- Rsync can now be compiled to use an unmodified zlib library instead of the tweaked one that is included with rsync. This will eventually become the default, at which point we'll start the countdown to removing the included zlib. Until then, feel free to configure using:
- ./configure --with-included-zlib=no
- DEVELOPER RELATED:
- Added more conditional debug output.
- Fixed some build issues for android and minix.
New in rsync 3.0.9 Pre1 (Jun 28, 2011)
- Fix a crash bug in checksum scanning when --inplace is used.
- Fix preservation of a symlink's system xattrs (e.g. selinux) on Linux.
- Fix a bug with the modifying of unwritable directories.
- Fix --fake-super's interaction with --link-dest same-file comparisons.
- Fix the updating of the curr_dir buffer to avoid a duplicate slash.
- Make daemon-excluded file errors more error-like.
- Fix some issues with the post-processing of the man pages.
New in rsync 3.0.8 Pre1 (Feb 23, 2011)
- BUG FIXES:
- Fixed two buffer-overflow issues: one where a directory path that is exactly MAXPATHLEN was not handled correctly, and one handling a --backup-dir that is extra extra large.
- Fixed a data-corruption issue when preserving hard-links without preserving file ownership, and doing deletions either before or during the transfer. This fixes some assert errors in the hard-linking code, and some potential failed checksums (via -c) that should have matched.
- Fixed a potential crash when an rsync daemon has a filter/exclude list and the transfer is using ACLs or xattrs.
- Fixed a hang if a really large file is being processed by an rsync that can't handle 64-bit numbers. Rsync will now complain about the file being too big and skip it.
- For devices and special files, we now avoid gathering useless ACL and/or xattr information for files that aren't being copied. (The un-copied files are still put into the file list, but there's no need to gather data that is not going to be used.) This ensures that if the user uses --no-D, that rsync can't possibly complain about being unable to gather extended information from special files that are in the file list (but not in the transfer).
- Properly handle requesting remote filenames that start with a dash. This avoids a potential error where a filename could be interpreted as a (usually invalid) option.
- Fixed a bug in the comparing of upper-case letters in file suffixes for --skip-compress.
- If an rsync daemon has a module configured without a path setting, rsync will now disallow access to that module.
- If the destination arg is an empty string, it will be treated as a reference to the current directory (as 2.x used to do).
- If rsync was compiled with a newer time-setting function (such as lutimes), rsync will fall-back to an older function (such as utimes) on a system where the newer function is not around. This helps to make the rsync binary more portable in mixed-OS-release situations.
- Fixed a batch-file writing bug that would not write out the full set of compatibility flags that the transfer was using. This fixes a potential protocol problem for a batch file that contains a sender-side I/O error: it would have been sent in a way that the batch-reader wasn't expecting.
- Some improvements to the hard-linking code to ensure that device-number hashing is working right, and to supply more information if the hard-link code fails.
- The --inplace code was improved to not search for an impossible checksum position. The quadruple-verbose chunk[N] message will now mention when an inplace chunk was handled by a seek rather than a read+write.
- If we fail to connect to an rsync daemon, report all the connection errors (e.g. IPv4 & IPv6), not just the last one.
- Improved ACL mask handling, e.g. for Solaris.
- Fixed an issue where an xattr and/or ACL transfer that used an alt-dest option (e.g. --link-dest) could output an error trying to itemize the changes against the alt-dest directory's xattr/ACL info but was instead trying to access the not-yet-existing new destination directory.
- Improved xattr system-error messages to mention the full path to the file.
- The --link-dest checking for identical symlinks now avoids considering attribute differences that cannot be changed on the receiver.
- Avoid trying to read/write xattrs on certain file types for certain OSes. Improved configure to set NO_SYMLINK_XATTRS, NO_DEVICE_XATTRS, and/or NO_SPECIAL_XATTRS defines in config.h.
- Improved the unsafe-symlink errors messages.
- Fixed a bug setting xattrs on new files that aren't user writable.
- Fixed a bug with --fake-super when copying files and dirs that aren't user writable.
- Fixed a bug where a sparse file could have its last sparse block turned into a real block when rsync sets the file size (requires ftruncate).
- If a temp-file name is too long, rsync now avoids truncating the name in the middle of adjacent high-bit characters. This prevents a potential filename error if the filesystem doesn't allow a name to contain an invalid multi-byte sequence.
- If a muli-protocol socket connection fails (i.e., when contacting a daemon), we now report all the failures, not just the last one. This avoids losing a relevant error (e.g. an IPv4 connection-refused error) that happened before the final error (e.g. an IPv6 protocol-not-supported error).
- Generate a transfer error if we try to call chown with a -1 for a uid or a gid (which is not settable).
- Fixed a forceful delete of a file with --one-file-system.
- Fix the popt arg parsing so that an option that doesn't take an arg will reject an attempt to supply one.
- A couple minor option tweaks to support/rrsync script, and also some regex changes that make vim highlighting happier.
- Fixed some issues in the support/mnt-excl script.
- A few manpage improvements.
- ENHANCEMENTS:
- Added ".hg/" to the default cvs excludes (see -C & --cvs-exclude).
- DEVELOPER RELATED:
- Use lchmod() whenever it is available (not just on symlinks).
- A couple fixes to the socketpair_tcp() routine.
- Updated the helper scripts in the packaging subdirectory.
- Renamed configure.in to configure.ac.
New in rsync 3.0.6 (May 9, 2009)
- Fixed a --read-batch hang when rsync is reading a batch file that was created from an incremental-recursion transfer.
- Fixed the daemon's socket code to handle the simultaneous arrival of multiple connections.
- Fix --safe-links/--copy-unsafe-links to properly handle symlinks that have consecutive slashes in the value.
- Fixed the parsing of an [IPv6_LITERAL_ADDR] when a USER@ is prefixed.
- The sender now skips a (bogus) symlink that has a 0-length value, which avoids a transfer error in the receiver.
- Fixed a case where the sender could die with a tag-0 error if there was an I/O during the sending of the file list.
- Fixed the rrsync script to avoid a server-side problem when -e is at the start of the short options.
- Fixed a problem where a vanished directory could turn into an exit code 23 instead of the proper exit code 24.
- Fixed the --iconv conversion of symlinks when doing a local copy.
- Fixed a problem where --one-file-system was not stopping deletions on the receiving side when a mount-point directory did not match a directory in the transfer.
- Fixed the dropping of an ACL mask when no named ACL values were present.
- Fixed an ACL/xattr corruption issue where the --backup option could cause rsync to associate the wrong ACL/xattr information with received files.
- Fixed the use of --xattrs with --only-write-batch.
- Fixed the use of --dry-run with --read-batch.
- Fixed configure's erroneous use of target.
- Fixed configure's --disable-debug option.
- Fixed a run-time issue for systems that can't find iconv_open() by adding the --disable-iconv-open configure option.
- Complain and die if the user tries to combine --remove-source-files (or the deprecated --remove-sent-files) with --read-batch.
- Fixed an failure transferring special files from Solaris to Linux.