Home

Run A Command Line Process On All Files In A Directory And Collect The Output Into A New File

July 2023

The Script

This was a test script for setting up a larger process. It reads all the files in a directory, passes them through a secondary process, then puts all the outputs in a single file:

#!/bin/bash

OUTPATH="results file.txt"
: > "$OUTPATH"

for file in input_dir/*.txt
do
  cat "$file" \
  | tr '[:upper:]' '[:lower:]' \
  >> "$OUTPATH" 
done

Details

  • Using quotes around $OUTPATH allows spaces in the path
  • The first `: > "$OUTPATH"bash line clears out the file. From the bash manual, the `:bash "Do nothing beyond expanding arguments and performing redirections. The return status is zero." When you point that to a file with `>bash it clears it out
  • This version uses a `forbash loop. I've seen other stuff that uses `filebash but haven't explored the differences yet
  • The meat of the program runs `catbash on each file to output it then pipes that output to `trbash with `PIPE_HERE`bash` before appending it to the output file via `>[NEO_TODO_ESCAPED_CHARACTER]`bash`
  • The `trbash command changes all upper case letters to lowercase
  • Any command that works in the pipeline can be used in place of `tr`bash`. It's simply what's being used for the illustration

Example

Given these three input files:

Pitch The Straw
TacK iT dOWn
RAISE THE SAIL

The script will produce:

Output:
pitch the straw
tack it down
raise the sail
end of line

References