Buffer Copy

Buffer-copy.c copies data efficiently from its stdin to its stdout. It works by spawning two subprocesses, one for reading, and one for writing. That way, both read and write operations can be run in paralell, rather than sequentially. Yes, this does indeed help, although the buffer cache should really take care of this

At Ltnb, we use this to send entire disks over our network for backup purposes. A 10 Gig disk can be backed up in 20 minutes over a 100 Mbps ethernet using this program. Without it (using just dd or cat), it would take at least twice as long (tested with kernel 2.2.10)

Usage: buffer-copy [-i inputbufferSize] [-o outputbufferSize] 
       [-s sleepDelay] [-a amount] [-t targetFile] [-v] <inputFile
-i inputSize the size of the input buffer (that many bytes are read at once from the input file)
-o outputSize the size of the output buffer (that many bytes are written at once to the output file
-s sleepDelay delay, expressed in microseconds, during which the 2 threads wait to synchronize. Mainly intended for testing, it turned out that this only had a minor impact on performance.
Default 100us.
-t targetFile where to write the data to. For efficiency, this is opened with the O_SYNC flag
-a amount number of input buffers copied. Mainly intended for performance testing (if you don't want to wait for the whole 20 minutes to see if a tweak helped or not).
-v verbose. Displays a running count of copied output buffers.

All sizes can be expressed in bytes (b), kilobytes (k) or megabytes (m). Example:

buffer-copy -i 2m -o 2m -v </dev/hda | \
  rsh ltnb5 buffer-copy -i 2m -o 2m -t /dev/hda
Attention: all filesystems on /dev/hda on the source machine should be mounted read-only. Filesystem on /dev/hda on the target machine (ltnb5) shouldn't be mounted at all. Both /dev/hda's on the source and target machine should have the same geometry.

Alain Knaff
Last modified: Wed Dec 18 22:36:11 CET 2002