6

The test command on Unix-like systems provides two special syntax forms for checking whether a string is empty or not:

test -z "$foo" # the length of $foo is zero
test -n "$foo" # the length of $foo is nonzero

These forms are actually redundant, and equivalent to the following:

test "$foo" = ""
test "$foo" != ""

Why are there two different forms? Is there a historical reason or justification for this design decision?

Background, as far as I know it. Bash descends from an older shell Almquist shell. The Almquist shell didn't internalize expressions. So you wrote them with using a utility called test which just resolved them to a return code,

if test "$foo" = ""; then echo "Foo has a length of 0"; fi;

That would call the actually utility /bin/test and test that first argument $foo was equal to third argument ( an empty string).

It also provided another option,

if test -z "$foo"; then echo "Foo has a length of 0"; fi;

I don't understand why this syntax exists though.

Improve this question
1
16

Because $foo may itself start with a hyphen and look like an option or an operation, which would cause misinterpretation of the command line. Using -z or -n guarantees that no matter the contents of $foo, it will never be interpreted as an option.

The BSD 2.11 man page for test says: The test grammar is inherently ambiguous. In order to assure a degree of consistency, the cases described in the IEEE Std 1003.2 ("POSIX"), section D11.2/4.62.4, standard are evaluated consistently according to the rules specified in the standards document. All other cases are subject to the ambiguity in the command semantics.

So whatever can be done to simplify writing unambiguous expressions, helps.

Improve this answer
11
  • 1
    Wow, this sounds like a valid argument. So back when test wasn't internalized as a shell command but existed as a stand alone thing, people had to check that the LHS and RHS did not start with -? – Evan Carroll Nov 19 '20 at 7:20
  • 2
    @EvanCarroll The simple cases work (3 operands means that the first and the last are values, and the middle one is the comparison operator). However, the BSD 2.11 man page for test says The test grammar is inherently ambiguous. In order to assure a degree of consistency, the cases described in the IEEE Std 1003.2 ("POSIX"), section D11.2/4.62.4, standard are evaluated consistently according to the rules specified in the standards document. All other cases are subject to the ambiguity in the command semantics. So whatever can be done to simplify writing unambiguous expressions, helps. – Leo B. Nov 19 '20 at 8:04
  • 1
    $foo can start with a dash (or be a !, which is worse) even in something like [ "$foo" = blahblah ]. So if handling that corner case was the main reason for -n, you'd expect to also have a similar prefix operator for equality and inequality testing. – ilkkachu Nov 19 '20 at 16:31
  • 3
    I think the argument count was added by POSIX to provide an unambiguous way of interpreting arguments, whether or not they begin with a -. The fact that the operators do still start with hyphens is to remain compatible with historical usage, not a requirement of the argument parser. – chepner Nov 19 '20 at 17:09
  • 1
    (The fact that boolean -a and -o mess up the argument-count algorithm is used as one rationale for deprecating them.) – chepner Nov 19 '20 at 17:09
7

The argument from consistency:

The majority of test commands are of the form -flag value.

For example, test -e foo.bar - does file foo.bar exist?

test -n "$VAR" fits into that model, and is therefore consistent.

The first mention of test I could find in 'man' pages is to this link to the PWB (Programmer's Work Bench) shell aka Mashey shell, it in turn links to this page, which describes a version of test that has both forms: -n str and str !- "". Note that test and if are the same program.

(Curiously, the title of that second page is 'Thompson shell manual')

PWB started on 4th. ed. Unix, and the Mashey shell was derived from Thompson's original sh. Given the focus of PWB, it's probable that this is the first appearance of 'if' and 'test'.

There is also the paper Using a Command Language as a High-Level Programming Language, written by Mashey, that describes if. In that paper, the -n and -z forms are not mentioned, only the string-comparison form.

Two things are possible: (1) at the time of the paper, the full if/test syntax existed in PWB, but the paper is not intended to be exhaustive, or (2) the syntax was later expanded, and/or the 'test' command was added.

Either way, I think this shell is the likely first appearance of 'test' in the shell; plausibly so because it was the explicit intent of PWD to be able to facilitate the programmer's tasks.

We have the first; we can still only guess at the 'why'. But I still think consistency has a lot to do with it, especially if, as appears possible, the string-comparison form existed before the -z/-n form.

Improve this answer
7
  • Both of these are pretty subjective reasons, and it’s not justified by any source that they were indeed real considerations. The ‘argument from naturalness’ is especially silly, since ‘is empty’ and ‘is equal to the empty string’ are pretty much on the same level on the abstraction ladder; neither is more ‘natural’ than the other. In my answer I mentioned some more pertinent objective reasons for this design. – user3840170 Nov 19 '20 at 15:18
  • 1
    Subjective - like most UI issues. Early Unix design is replete with decisions made because that's what the designers liked. – another-dave Nov 19 '20 at 17:43
  • The design of everything is replete with decisions made because that’s what the designers liked – or sometimes did out of necessity, which only later was retroactively justified, with the (in)famous /usr split as just one example. The question then becomes which is which. – user3840170 Nov 19 '20 at 18:10
  • In some situations where an object will contain a collection of things [strings are a special case of this], testing for whether an object is empty generally means something different from testing whether its contents precisely match those of another collection. If one wants someone to check whether a package has been delivered, would one ask "Is there anything on the porch" or "Does the porch contain precisely the same set of items as an empty porch"? Even if a language doesn't provide an "is empty", I would "empty" constants used in equality tests as "fake constants", since... – supercat Nov 19 '20 at 18:31
  • ...the only situation where an equality test can reliably indicate whether something contains something else is when the first item would otherwise be empty. Comparison against anything other than empty could yield false positives if the comparison mismatch is a result of something having been removed rather than added. – supercat Nov 19 '20 at 18:33
4

Note: This answer is to an extent speculative, and I’m not sure I can find sources for some of the claims I make here. So I cannot say for certain that this is the definitive reason. But I find it pretty plausible.

The test command actually provides three forms of (non-)emptiness checking:

  1. test "$a"
  2. test -n "$a" vs test -z "$a"
  3. test "$a" != "" vs test "$a" = ""

In contemporary Unix-like systems, they can be reliably distinguished by counting arguments to the test command: one argument means the first form was used, two arguments mean it’s the second form, and three arguments mean it’s the third form. This method is recommended by SUS and by POSIX (IEEE 1003.1-2017 vol. 3, ch. 4, pp. 3290–3291). Historically though, different algorithms have been used: some implementations of test parsed options like any other utility, matching command line arguments against a list of known options, which caused problems if the string to be tested happened to match one of them. The following transcript, for example, comes from Version 7 Unix:

$ test '' ; echo $?
1
$ test x ; echo $?
0
$ test -r ; echo $?
test: argument expected
255
$ test '' = '' ; echo $?
0
$ test x = '' ; echo $?
1
$ test x = x ; echo $?
0
$ test -r = -r ; echo $?
1

Also, not all of these constructs were available on all systems. Some provided -z, but not -n, and apparently some even provided neither. Furthermore, there were shells in use in which empty string arguments were not passed to the command executed, so that test "$a" = "" could run the test command with just two or one arguments, and trigger a syntax error. For these reasons, in configure scripts generated by GNU Autoconf, emptiness testing is performed via the command test "x$var" = x.

Add to that that some shell scripts did not even bother to quote variables: instead of writing "$a" each time, they would write $a, so an empty variable would have resulted in an empty string argument even if the shell supported those correctly. (This remains a common shell programming blunder to this day.)

The -z option may have originated as a workaround for some of these problems:

  • Having an explicit operator ensures that tested strings would not be themselves interpreted as testing operators.
  • With a buggy shell that doesn’t pass empty arguments correctly, test -z "$foo" may result in test being called:

    • If foo is non-empty, with two arguments: -z, and the contents of foo; this then returns failure, as expected.

    • If foo is empty, with one argument -z, which the test command may then interpret as either:

      • a check whether an empty list of remaining arguments is empty (which it is), or
      • a check whether the -z string is non-empty, which just so happens to have the same truth value as the emptiness of "$foo".

      Either way, the command returns success – as expected.

The -n command may have been added to similarly disambiguate the first form from a missing argument when the string to be tested matches a known test operator, e.g. test -n '-t' vs test -t.

Improve this answer
6
  • re: the earliest shells apparently provided only forms (1) and (3) -- the earliest mention of 'test' that I found was 7th. Ed. Unix, which already has (2) and (3), not sure of how (1) would be interpreted. Did you find earlier? I suspected PWB would have 'test' but not according to man pages. – another-dave Nov 19 '20 at 17:47
  • I inferred this information from what the Autoconf page said (‘[w]hile "-z" is seen often, even "-n" is rarely implemented.’ – implying either can fail to be implemented); perhaps it is wrong, or I misread it. Maybe they meant some later implementations did not bother to provide -z, -n, or both. – user3840170 Nov 19 '20 at 18:01
  • That's probably correct for some implementations, but I'm guessing there was "a first one"... and then the question is what did that first one do? (-z/-n, =/!=, or both) – another-dave Nov 19 '20 at 18:12
  • Just checked: V7 Unix supports all three forms. However, it doesn’t use argument counting for syntax disambiguation. – user3840170 Nov 19 '20 at 21:59
  • I’ve also found this: in-ulm.de/~mascheck/various/test – might be useful. – user3840170 Nov 19 '20 at 21:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.