Shell Quoting, Exit Codes and Pipelines

Why this matters

A cleanup script runs nightly for a year, then meets a file named "Q3 report (final).txt" and deletes the wrong directory. Nothing about the script changed; the data did. Shell scripts process adversarial input by default, because filenames, API responses and user input all flow through the same expansion rules. Quoting is not style. It is the difference between a tool and a hazard.

Concepts

The shell expands unquoted variables in three steps: split the value on whitespace, expand globs like into matching names, then pass the fragments as separate arguments. "$file" is one argument no matter what it contains; $file is as many arguments as there are words, plus a directory listing if one of them contains . Double quotes allow expansion but suppress splitting and globbing; single quotes suppress everything. The rule fits in one line: quote every expansion unless you can state why this one must split.

Exit codes are the script's only honest channel. Zero means success, anything else means a specific failure, and if ignores every code by default and marches into the next line. Three guards change that: set -e exits on the first failing command, set -u exits on the first unset variable, set -o pipefail makes a pipeline report the rightmost failure instead of the last command's success. Together as set -euo pipefail they turn silent corruption into an immediate stop with a line number.

Pipelines hide failures by design: $? after a | b | c reports only c. When b fails and c succeeds on empty input, the script reports success and the data is gone. PIPESTATUS holds every stage's code after the fact; pipefail makes the pipeline itself fail. Prefer either over hoping the middle never breaks.

Worked example

A backup loop breaks on spaced names. Before:

for f in $(ls /srv/data); do cp $f /backup/; done

Two faults in one line: $(ls) splits names on spaces, and the unquoted $f splits and globs again. The repair:

#!/usr/bin/env bash set -euo pipefail src="/srv/data" for f in "$src"/*; do [ -e "$f" ] || continue cp -- "$f" /backup/ done

Expected reading: the glob expands to real names instead of parsed text, every expansion is quoted, -- stops cp from treating dash-led names as flags, and any failure (missing source, full backup disk) stops the script instead of copying half a dataset and reporting success. Lab L07 replays this exact class of breakage until quoting becomes reflex.

The common wrong move

Parsing ls output. Filenames may contain spaces, newlines and leading dashes, and ls formats for humans, not for loops. Globs, find -print0 with read -d, and arrays exist precisely because ls parsing cannot be made safe. Any script containing $(ls ...) or backtick ls is broken on arrival; rewrite the loop, not the filenames.

Lab and next step

Lab L07 hands you a script that spaced filenames break and requires the quoted repair plus a failing-then-passing test. Next, lesson 2 makes scripts safe to run twice: idempotency.