Any time you can pass a procedure (function, method) designation (address, reference) as a parameter to a procedure, that is a "callback". The reason why you're passing procedure B to procedure A is so that A can call B at some point, either synchronously or asynchronously.
I think the concept is simply an obvious one which emerges as soon as subroutines (closed subroutines, in the terminology of the time) have been invented.
In many numerical analysis examples (which was one focus of early 'scientific' programming language), the callback is essential. For example, suppose you wish to have a procedure that will determine a definite integral of some (mathematical) function, say by something like Simpson's rule - then the specific function is supplied as a parameter to the generalized integration procedure.
EDSAC
The 'closed subroutine' was invented by David Wheeler for the EDSAC. The seminal book The Preparation Of Programs For An Electronic Digital Computer (Wilkes, Wheeler, and Gill; 1951) describes the technique, which was realized by software convention (the hardware for jump-and-return came later).
Part II of their book, Specification of subroutines, describes routines presumably available at Cambridge. Subroutine G1 solves simultaneous differential equations; it requires an 'auxiliary subroutine' (i.e., provided by the user of G1) to compute values of the first derivative and which is called 4 times for each step in G1.
This seems like a callback, and since it was described simultaneously with the notion of subroutine, by the inventors of the subroutine, it is my proposal to be the first. EDSAC itself became operational in May 1949; the book was published in 1951. The Wheeler Jump (aka subroutine call) was therefore invented somewhere between 1949 and 1951. The subroutine was described in a privately-circulated memo in 1950 so that narrows the timeframe down to 1949--1950.
Early examples - synchronous case
These two aren't presented as contenders for the absolute first, just as examples that it's a long-standing idea.
FORTRAN II for the IBM 7090 supported this sort of callback -- likely it was supported on the 709 as well, but I cannot find a manual for that. The manual I linked to was from 1964 but FORTRAN II dates from 1958.
Algol 60 supported the 'explicit' case of passing procedure (function) parameters in procedure (function) calls, but there was also an implicit case: call-by-name parameters. For call-by-name, the actual argument was evaluated every time it was used in the called procedure, in contrast to call-by-value where the argument is evaluated once before entering the procedure. The general case is implementable by converting the expression as entered for the parameter in the source Algol into a procedure -- termed a 'thunk' at the time it was invented, but which is clearly a 'callback' from the context of the called procedure into the context of the calling procedure.
Asynchronous case
If you think that the essence of a callback is that the callback routine is executed at some future time (after the return from the procedure to which the callback routine was passed), then there are still many early pre-GUI examples.
Any system that supports traps, signals, exceptions, etc., is essentially a callback system. You call a setup function to say "when event E occurs, call procedure P".
The RSX family of DEC minicomputer operating systems used this mechanism for I/O. A system call initiated the I/O operation; the initiate call passed the address of a routine to be called on I/O completion. The system call returned after I/O had been queued to the device. Most async programming would be done in the machine's assembly language. Libraries could and did expose the callback feature to higher level languages such as FORTRAN.
This approach is obviously modelled on what the hardware actually does. You start I/O in progress and later get a completion interrupt. So there's some case for saying the invention of the interrupt was the invention of the callback function. However, as I explained above, my conclusion is that callbacks were invented as an immediate corollary to the invention of subroutines, and thus on EDSAC, prior to the invention of interrupts.
y=fx(x)
graph or calculating a derivative.