20+ years later the Unix shell is still the fastest way to get work done on a bunch of files. I'm still regularly combining grep, awk, sort, uniq, etc to do analysis on data.

One common task is doing work for every line of a file.

for f in `cat list`; do
  ls -l "$f"
done

There are a lot of reasons this idiom is broken. The worst problem is that it doesn't work if the lines in the file list have spaces in them and no amount of quoting will fix it. Also if list is large (32k?) it fails because everything's expanded in the limited command line buffer.

The idiom works often enough that I use it all the time. And when I do I have a problem, I'm always left scratching my head to remember the right way. Well, here it is (in bash):

cat list | while read f; do
  ls -l "$f"
done

The read command in bash is a magic builtin. It reads a line from stdin and assigns its contents to shell variables. It also has a return code when EOF is reached, allowing a clean exit from a loop.

read has a lot of options for how it handles the file input. I'm a bit confused that the above sample works, actually, because the bash docs suggest that each line is parsed via IFS and only the first word assigned to the variable f. But in practice that only seems to happen if you have more than one variable. See the docs for options for line delimiters, assigning to an array, backslash handling, etc.