The early Apple Macintosh computers (original Mac, Mac 512K, Mac Plus) all came with a "Programmer's Switch" installed on the side.
Yes and no. While the switch was there, it was on the inside, so, not really accessible. Only after being 'enhanced' with the so called 'Programmers Key Aid', a snap on after market piece of plastic, griping into the ventilation slots and providing a pin to press the inside switch.
Well, or by using any small object capable of transferring the needed force (~5N).
I know that the switch triggers a CPU interrupt, and that this would be used to immediately halt execution and present a debugger interface.
Pressing that button raised an NMI (*1). Prior to the Mac Plus hitting NMI ended in the System Error Handler reporting a 'No Debugger Installed' error box. That is as long as no NMI handler (like MacsBug) was installed.
How, exactly, was this enabled and/or controlled by the system software?
It could not be disabled. That's the whole point of having an NMI.
How can a user modify this behavior?
Placing a different code (pointer) at the NMI vector.
And how was it possible for software developers to override the switch to prevent "hackers" from gaining access to their assembly source code?
Assembler source code? Serious, who would deliver the source code of an application?
Also, keep in mind, the classic Mac was a real mode OS. Independent of the CPU used, all memory was shared memory, so a programs code (and data) could be read from any other program (with MacsBug being way more comfortable than MicroBug). So no protection to circumvent in the first place
The NMI function was always there with 68k Macs, but it was the Mac Plus ROM that first contained the so called MicroBug, an always available minimalist debugger. In a developer/debugging environment one would install MacsBug, which hooked NMI. Then again, Bugs may as well happen on user machines, or, as we all know only when the debugger is not installed/active. MicroBug offered a built-in help for these situations to at least peek around and gather basic information. Apple describes its usefulness and handling in Technote 1136.
One noteworthy point is that MicroBug was less integrated and more intrusive to the GUI, as it used GUI functions, but could only react to a very limited number of events, whereas MacsBug used his own screen drawing keeping all GUI data untouched.
*1 - NMI stands for Non Maskable Interrupt, an interrupt that can not be disabled, like any other, but will always make the CPU execute the assigned function. Rather handy to revitalize an otherwise dead system.