12

It just came to me that, the C standard I/O functions fread and fwrite are having 2 size_t arguments because of I guess possibly, that on some systems, there may be more memory of which whose size can be represented in a single size_t type.

With 2 size_ts, 1 for element size, another for elements count, caller will be able to read/write more data than can be measured with a single size_t.

I find it reasonable with some ancient x86 processors with "near" addressing.

Is this thinking right? What's the real history here?

15
  • 10
    calloc also takes 2 size_t parameters. To me these all feel like a nod to the fixed-sized record orientation of file systems of other operating systems that were in existence when UNIX was created.
    – Erik Eidt
    Oct 22, 2020 at 12:00
  • 2
    unlikely. Anything you read must go into a buffer. That buffer can either be a static array (whose maximum size is a size_t ) or be allocated from malloc (whose maximum size is a size_t again).
    – tofro
    Oct 22, 2020 at 12:02
  • What about just the fixed-size record orientation of many data processing algorithms? I think this is more generic than filesystems.
    – Brian H
    Oct 22, 2020 at 14:40
  • 1
    @supercat I'm afraid my comment (or the original question) hasn't been understood: The OP seems to assume that by using more than one size_t-sized argument, the API allows to theoretically transfer SIZE_T_MAX * SIZE_T_MAX bytes. Because you can't define a buffer of that size, that assumption is wrong.
    – tofro
    Oct 22, 2020 at 17:38
  • 2
    Compare unix.stackexchange.com/q/616107/5132 .
    – JdeBP
    Oct 24, 2020 at 0:38

3 Answers 3

8

AT&T's documentation for fread and fwrite that pre-dates size_t is quoted below. But first, to answer the title question:

  1. Both functions are designed for objects, not characters. This is evidenced by the return value being a count of the objects read or written, not the number of characters.

  2. Each function may read/write fewer objects than requested. Well-written code will make another attempt to read/write the remaining objects. (This is also true of the character I/O.) The code to re-attempt reading/writing is a lot simpler if you need to only keep track of the number of objects, instead of the number of characters (which then requires dealing with alignment issues).

  3. The original implementation used the int type for both the number of objects and the size of the objects. When size_t and other _t types were introduced, those two arguments needed to retain the same type, to avoid breaking existing source code and compiled libraries.

It has very little to do with the size of memory.


The use of size_t and other _t data types did not appear until K&Rv2. Before then, the arguments to standard library functions were of int and long types. For example, The C Programmer's Handbook, AT&T Bell Laboratories, February 1984, p. 50 states (C++ style comments are my addition):

BLOCK I/O

Manual Page -- The functions in this section are on the fread(3S) manual page.

fread -- Read specified number of bytes (characters) from stream.

  • Synopsis:
    int fread (ptr, size, nitems, stream)
    char *ptr;
    int size, nitems;    // <--- THEY WERE BOTH INT
    FILE *stream;

fwrite -- Write specified number of bytes to stream.

  • Synopsis:
    int fwrite (ptr, size, nitems, stream)
    char *ptr;
    int size, nitems;    // <--- THEY WERE BOTH INT
    FILE *stream;

size_t was established by the time the second edition of The C Programming Language, Kernighan and Ritchie, 1988 was written. The functions in question are described on p. 247:

B1.5 Direct Input and Output Functions

size_t fread(void *ptr, size_t size, size_t nobj, FILE *stream)

fread reads from stream into the array ptr at most nobj objects of size size. fread returns the number of objects read; this may be less than the number requested. feof and ferror must be used to determine status.

size_t fwrite(const void *ptr, size_t size, size_t nobj, FILE *stream)

fwrite writes, from the array ptr, nobj objects of size size on stream. It returns the number of objects written, which is less than nobj on error.

8
  • 10
    The C Standard specifies that the stride of an array is always equal to the size of the constituent elements. An implementation could make sizeof (struct foo) be three, but only if neither the structure nor any element within it had any alignment requirement. The size of char blob[100][3]; would be specified as precisely 300, and fwrite(blob, 3, 100, someFile) would need to write 300 consecutive bytes. I can think of no way that fwrite could meaningfully allow for padding of objects in memory as you suggest.
    – supercat
    Oct 22, 2020 at 21:29
  • 2
    @DrSheldon: I don't think the fact that an array's stride is equal to the size of the constituent elements has changed since any documented pre-standard version of the language. See page 7 of bell-labs.com/usr/dmr/www/cman.pdf for information about the pointer + operator which forms the basis of array subscripting.
    – supercat
    Oct 22, 2020 at 22:56
  • 3
    I'm not convince by your alignment argument. In particular, let's consider your hypothetical compiler where sizeof(struct foo) is 3, but an array of 100 struct foos takes 400 bytes. Now, what is that compiler going to do with struct bar { char a; char b; char c; }? Is an array of 100 struct bars also going to be padded to 400 bytes? That seems pretty crazy, since each struct is just three chars. But if not, how will fread know which kind of array it's reading data into? Oct 23, 2020 at 3:34
  • 1
    Supercat is right. I used to work at Bell Labs, had the original pre-standard compiler documentation in the department library, and even filed a bug report against the compiler. The C standard documented existing behavior of the K&R compiler, which is quite fundamental to how C compilers do pointer arithmetic. &(z[17]) has to be z+17, and pointer addition in in increments of sizeof(struct foo) bytes.
    – MSalters
    Oct 23, 2020 at 8:47
  • 2
    Concur. sizeof z had better be sizeof (struct foo) * 100. Otherwise sizeof z[0] != sizeof (struct foo), which is crazy talk. Oct 23, 2020 at 11:08
13

The point of fread/fwrite is to write N elements, each of size S bytes. The API is not a simple 'write this number of bytes' interface.

Thus, for example:

struct S { int a, b; float c; };
struct S stuff[92];
fwrite(stuff, sizeof (struct S), 92, stream);

(I would not write '92' in real code, but I want the simple formulation in this example)

Thus it has two size values because it needs them to express the intended operation.

(I don't much like the API, but there it is)

It is not that this is arranged just in case one size_t value can't handle the overall size to read/write, because size_t is by definition large enough to hold the size of the largest possible object (it’s defined as “the unsigned integral type of the result of the sizeof operator”) — and anything fread/fwrite can handle is a single contiguous object. To put it another way, given my above code fragement.

 size_t sz = sizeof stuff;

is guaranteed to be valid.


In the Rationale for ANSI-X3.159-1989 (the document accompanying the standard that explains why decisions were made). it says

size_t is the appropriate type both for an object size and for an array bound, so this is the type of size and nelem.

Now, they were standardizing an existing function, not inventing it, and this just says why the type is now size_t rather than, say, int. But it's clear they were thinking in terms of reading and writing an array.

5
  • 3
    The standard explicitly says that the functions manipulate arrays, so it is indeed clear that the authors were thinking in terms of reading and writing an array ;-). Oct 22, 2020 at 12:40
  • That is indeed a strong hint - not sure why I thought to look in the Rationale rather than the standard itself. Oct 22, 2020 at 12:41
  • @another-dave: Some people would claim that the Rationale is non-normative, and everything other than the normative parts of the Standard should be ignored, but the distinction between normative and non-normative only matters when evaluating requirements. The actual requirements imposed by the Standard, are too weak to be useful for anything. If an implementation can process in Standard-defined fashion at least one--possibly contrived and useless--source text which exercises the translation limits given in the Standard, nothing it does with any other could render it non-conforming.
    – supercat
    Oct 22, 2020 at 18:48
  • 1
    On reflection, the question was 'why', which I think to be the domain of the Rationale. The Standard is for 'what'. Oct 23, 2020 at 0:30
  • 1
    "The API is not a simple 'write this number of bytes' interface." — but it is now if I'm understanding the current POSIX spec for fwrite correctly, which specifies the implementation in terms of fputc calls — see unix.stackexchange.com/questions/616107/… for quote in context of a ± duplicate question I coincidently asked on another SE.
    – natevw
    Oct 26, 2020 at 17:44
3

For STDIO files with no buffering, referring to "raw" devices, like tapes, it is important how the write system calls are issued, because each write system call results in a tape block of the specified size (and to read a raw block, a read system call of size no less than the block size is required).

The fwrite/fread API appears to facilitate writing and reading multiple blocks of the specified size, but various versions of fwrite, including UNIX V7 ignore that distinction.

Could be a conceived but never finalized feature.

11
  • The version of fwrite you referred to is over twenty years old, here’s the current version (which has the same problem). Oct 22, 2020 at 20:48
  • Actually, all of them do, even tuhs.org/cgi-bin/utree.pl?file=V7/usr/src/libc/stdio/rdwr.c How inefficient is it to do single byte system calls when there is no buffering?
    – Leo B.
    Oct 22, 2020 at 22:30
  • @LeoB. If you're referring to the use of getc / putc in a loop, probably not too inefficient, given the way they're defined. Oct 23, 2020 at 3:45
  • @IlmariKaronen They are defined in a way to call _flsbuf every time in the unbuffered mode. That's not what "unbuffered" means.
    – Leo B.
    Oct 23, 2020 at 5:08
  • 1
    stdio just deals with byte streams, it has no notion of device blocks. I don't think the fwrite/fread design is intended to relate to block devices.
    – Barmar
    Oct 23, 2020 at 15:40

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .