Why does -z and -n exist in most shells and /bin/test?

Question

The test command on Unix-like systems provides two special syntax forms for checking whether a string is empty or not:

test -z "$foo" # the length of $foo is zero
test -n "$foo" # the length of $foo is nonzero

These forms are actually redundant, and equivalent to the following:

test "$foo" = ""
test "$foo" != ""

Why are there two different forms? Is there a historical reason or justification for this design decision?

---

Background, as far as I know it. Bash descends from an older shell Almquist shell. The Almquist shell didn't internalize expressions. So you wrote them with using a utility called test which just resolved them to a return code,

if test "$foo" = ""; then echo "Foo has a length of 0"; fi;

That would call the actually utility /bin/test and test that first argument $foo was equal to third argument ( an empty string).

It also provided another option,

if test -z "$foo"; then echo "Foo has a length of 0"; fi;

I don't understand why this syntax exists though.

Answer 1

Because $foo may itself start with a hyphen and look like an option or an operation, which would cause misinterpretation of the command line. Using -z or -n guarantees that no matter the contents of $foo, it will never be interpreted as an option.

The BSD 2.11 man page for test says: The test grammar is inherently ambiguous. In order to assure a degree of consistency, the cases described in the IEEE Std 1003.2 ("POSIX"), section D11.2/4.62.4, standard are evaluated consistently according to the rules specified in the standards document. All other cases are subject to the ambiguity in the command semantics.

So whatever can be done to simplify writing unambiguous expressions, helps.

Answer 2

The argument from consistency:

The majority of test commands are of the form -flag value.

For example, test -e foo.bar - does file foo.bar exist?

test -n "$VAR" fits into that model, and is therefore consistent.

---

The first mention of test I could find in 'man' pages is to this link to the PWB (Programmer's Work Bench) shell aka Mashey shell, it in turn links to this page, which describes a version of test that has both forms: -n str and str !- "". Note that test and if are the same program.

(Curiously, the title of that second page is 'Thompson shell manual')

PWB started on 4th. ed. Unix, and the Mashey shell was derived from Thompson's original sh. Given the focus of PWB, it's probable that this is the first appearance of 'if' and 'test'.

There is also the paper Using a Command Language as a High-Level Programming Language, written by Mashey, that describes if. In that paper, the -n and -z forms are not mentioned, only the string-comparison form.

Two things are possible: (1) at the time of the paper, the full if/test syntax existed in PWB, but the paper is not intended to be exhaustive, or (2) the syntax was later expanded, and/or the 'test' command was added.

Either way, I think this shell is the likely first appearance of 'test' in the shell; plausibly so because it was the explicit intent of PWD to be able to facilitate the programmer's tasks.

We have the first; we can still only guess at the 'why'. But I still think consistency has a lot to do with it, especially if, as appears possible, the string-comparison form existed before the -z/-n form.

Answer 3

Note: This answer is to an extent speculative, and I’m not sure I can find sources for some of the claims I make here. So I cannot say for certain that this is the definitive reason. But I find it pretty plausible.

---

The test command actually provides three forms of (non-)emptiness checking:

1. test "$a"
2. test -n "$a" vs test -z "$a"
3. test "$a" != "" vs test "$a" = ""

In contemporary Unix-like systems, they can be reliably distinguished by counting arguments to the test command: one argument means the first form was used, two arguments mean it’s the second form, and three arguments mean it’s the third form. This method is recommended by SUS and by POSIX (IEEE 1003.1-2017 vol. 3, ch. 4, pp. 3290–3291). Historically though, different algorithms have been used: some implementations of test parsed options like any other utility, matching command line arguments against a list of known options, which caused problems if the string to be tested happened to match one of them. The following transcript, for example, comes from Version 7 Unix:

$ test '' ; echo $?
1
$ test x ; echo $?
0
$ test -r ; echo $?
test: argument expected
255
$ test '' = '' ; echo $?
0
$ test x = '' ; echo $?
1
$ test x = x ; echo $?
0
$ test -r = -r ; echo $?
1

Also, not all of these constructs were available on all systems. Some provided -z, but not -n, and apparently some even provided neither. Furthermore, there were shells in use in which empty string arguments were not passed to the command executed, so that test "$a" = "" could run the test command with just two or one arguments, and trigger a syntax error. For these reasons, in configure scripts generated by GNU Autoconf, emptiness testing is performed via the command test "x$var" = x. (See also a pretty comprehensive blog post by Vidar Holen on this subject.)

Add to that that some shell scripts did not even bother to quote variables: instead of writing "$a" each time, they would write $a, so an empty variable would have resulted in an empty string argument even if the shell supported those correctly. (This remains a common shell programming blunder to this day.)

The -z option may have originated as a workaround for some of these problems:

• Having an explicit operator ensures that tested strings would not be themselves interpreted as testing operators.
• With a buggy shell that doesn’t pass empty arguments correctly, test -z "$foo" may result in test being called:

  • If foo is non-empty, with two arguments: -z, and the contents of foo; this then returns failure, as expected.

  • If foo is empty, with one argument -z, which the test command may then interpret as either:

    • a check whether an empty list of remaining arguments is empty (which it is), or
    • a check whether the -z string is non-empty, which just so happens to have the same truth value as the emptiness of "$foo".

    Either way, the command returns success – as expected.

The -n command may have been added to similarly disambiguate the first form from a missing argument when the string to be tested matches a known test operator, e.g. test -n '-t' vs test -t.