Gnu Xargs: Examples

Xargs is frequently used along with find. See also: Linux find examples

Xargs is a very useful linux command that enables you to run a command for each input line it's given.

Simplest possible example

E.g. delete all files that end in `".xpto" in current directory and subdirectories:

$ find . -name "*.xpto" | xargs rm

Use custom placeholder to refer to each input line

E.g. copy all files whose name end in ".foo" to directory bar/:

$ find . -name "*.foo" | xargs -I {} cp {} bar/

Run multiple commands for each input file

Use sh -c and add a ; after each command.

This example runs pngquant for each PNG image and then removes the original files.

$ find . -name "*.png" | xargs -I {} sh -c 'pngquant 64 {}; rm {};'

Complex example: Process the output of a command using tr and cut before passing it to xargs

Trivia: this is the command you run on spark to delete state from the checkpoint directory, while not deleting journalled data

Say you want to run ls on HDFS and looks for files that match a certain pattern and delete them

$ hadoop fs -ls /tmp/spark/checkpoint
Found 13 items
drwxr-xr-x   - hadoop hadoop          0 2016-08-12 01:48 /tmp/spark/checkpoint/4a368df6-b096-4fa2-82fe-641ddebb91e1
-rw-r--r--   1 hadoop hadoop      36507 2016-08-12 01:46 /tmp/spark/checkpoint/checkpoint-1470966400000

The actual file names are just the eight piece of information on each line so you need to:

• fetch the result of the command hadoop fs -ls
• pass each output line through tr (replace multiple spaces with a single one)
• and cut (take the eight element of each line) to get the actual file name
• and only then use xargs to run the hadoop command to delete the files that don't (reverse grep) match the string "receivedData":

$ hadoop fs -ls /tmp/spark/checkpoint |\
 tr -s ' ' |\
 cut -d ' ' -f 8 |\
 xargs -I {} sh -c 'if echo "{}" | grep -qv "receivedData"; then echo "{}"; fi'