ENIAC was programmed by physically wiring the "program" on a plugboard.

And since the question is not limited to digital computing, analog(ue) computers were often purpose-built; there is probably nothing more rudimentary than directly building your machine to perform whatever it is you need it to perform.

Or, if they were "programmable", you "programmed" (speaking about electronic ones) them by connecting and adjusting parameters of electronic components, amplifiers etc.

ENIAC was programmed by physically wiring the "program" on a plugboard:

(picture from Wikipedia: https://upload.wikimedia.org/wikipedia/commons/7/7e/Eniac_Aberdeen.jpg)

And since the question is not limited to digital computing, analog(ue) computers were often purpose-built; there is probably nothing more rudimentary than directly building your machine to perform whatever it is you need it to perform.

Or, if they were "programmable", you "programmed" (speaking about electronic ones) them by connecting and adjusting parameters of electronic components, amplifiers etc; often (but not necessarily) using a plugboard:

(picture from Wikipedia: https://upload.wikimedia.org/wikipedia/commons/thumb/6/61/EAI_580_analog_computer_plugboard_at_CHM.jpg/249px-EAI_580_analog_computer_plugboard_at_CHM.jpg)