It Was Always There
Like with many of today's complex features, it was a non-issue with early mainframes - especially not any kind of software issue, as the hardware was taking care of it. In case of "hibernation", it was built in - as core memory. Core was standard in the 1960s you're asking for.
By pressing STOP on a /360 console the machine stopped and could be powered down, and restarted where it was after powering on again. All states (registers) were part of the scratch memory, which itself was core, either part of main core, or in a dedicated one (for speed).
Having that out of the way:
In the beginning there was no OS
OSes we know them today only arose during the late 1960s and 1970s. Before that they were monitor programs, more like CP/M than Windows or Linux. And without an OS, there could be no OS function doing such very complex stuff.
Those monitors were only a tiny step up from bare metal, providing services more like a function library. After all, even applications did their own hardware access (*1). Some of those services were intended to standardize support for applications.
Why Create them in the First Place?
The ability to restart an application was needed more than just for planned maintenance and super long running scientific jobs. Those machines were unreliable in a way today's PC customers wouldn't accept for a minute (*2).
1970s base line was one non-recoverable memory error per 32 KiB per month. That means even the smallest (full) /360, the model 30, could have every other week an error the machine detects but can not correct, thus stopping execution. If running with multiple applications this meant that the application whose memory contained the error had to be thrown out. If it was inside the monitor, than the whole machine was done for (*3).
Restart Was an Application Feature ...
As a result of those low reliability issues (*4), programs used Checkpoints from very early on. Every now and then, and whenever there was a stable state, they wrote a tape with information allowing to restart itself. Usually dumping some working memory with pointers to the last records processed and soon.
When restarted it would look for a checkpoint tape, read the content and continue from there. It's exactly like today: Libre-Office reopens all files and restores recent changes from a crashed session.
Even More So For Scientific Use
Considering that scientific machines during IBM 70x(x) times sacrificed ECC for more precision, writing such checkpoints was even more important for them.
... Supported By OS
Solely for size reasons it was out of question that an OS would dump a whole application onto some media. Too much, too slow. Having the application decide what's needed was the better option. What the OS could do is improving handling of those checkpoints to enable standardized management for job control. For example, IBM OS/360 introduced the CHKPT macro to write a short record (8..16 Bytes *5) to a specified device (*6).
Not Much But All It Needs
Sounds not much, as the application had still to maintain any additional data on its own media, but a huge step for integration. Now the controlling job could look for a checkpoint on a defined device (*7) and handle restart as needed for that specific application. Support was very much intrinsic (*8).
The huge benefit was that each application could be tailored to its specific needs, while management could be standardized within the OS's script language.
Best of both worlds and able to run on machines as small as 32 KiB :))
Once Popped - Can't Stop
While protecting against hardware failure was a good reason for development, an even better was system management. With those facilities standardized, an operator could stop any job at the next checkpoint and unload it. A situation common in computing centers no matter whether management had a higher priority job for a single-tasking machine, or memory got too low on one running multiple jobs.
Until Virtual Memory systems took over, manually swapping jobs was a standard task. After all, swapping jobs back then usually also meant (un)mounting tapes and disks, so interaction was needed anyway.
Further Remarks
On the PC side it might be notable that DESQview could, even with its first 1984 version, not only run multiple DOS programs in memory, but also swap them out to EMS or disk. This worked on all PCs, including 8088-based machines with as little as 256 KiB of RAM. Although, I'm not sure whether those applications could be restarted after rebooting the system.
Please note that even today, after hibernating the "main" OS on a computer, you can boot a different OS from a removable media, then come back to the saved state of the main system.
That can hardly be called part of the OS nor a function thereof, as booting another system in between is neither done by that OS, nor in any way supported. It's done by external means of the underlying system, such as a hypervisor, BIOS or other boot mechanism.
*1 - Creating the very same issues PC/MS-DOS users had to live with 20 years later.
*2 - Considering the speed of today's CPUs, they might not have to endure it a full minute before crashing:))
*3 - Well, depending on context, user and knowledge, one could try to patch it back to normal and let the machine continue. That was usually with errors in code, where one could simply look up the right code section from a listing - that includes OS code :))
*4 - Don't get that wrong: back then it wasn't seen as something out of the ordinary and painful, but absolute normal. Much like early motorists packed a dozen tires, as a flat was something that simply happens every so often. :)
*5 - For example, a label or a file and some record number. The whole record written was some 600 bytes, as the OS additionally stored information about the program, environment, devices and some other states.
*6 - Jup, device, not file, although that changed in later versions :))
*7 - Imagine them being handled line script/environment variables, so a program could have its default handled by operator assignment.
*8 - In simplified terms one could think of that the job being started at a label whose name was stored with the checkpoint, acting as alternate entry point.
