RSTS-11 (not RSTS/E) ran on the 11/20. It offered multiuser timesharing in BASIC. I think that in addition to the base 11/20 hardware, you needed a clock. You needed to max out the memory (28 Kwords).
V4 was the last RSTS-11 release before it became RSTS/E.
System manager's manual for RSTS-11.
I have not checked to see if there are software kits available.
For a single user disk system, I'd suggest RT-11. You might need a relatively older version for a very small system, I don't know.
You can definitely run RSX-11M unmapped (i.e., no memory management), but I don't know whether it supports the 11/20. My experience was with an 11/40. If you try, I'd suggest V3.1; as I recall 11M got a little fatter after that, at least when running mapped.
On the other hand, configuring an unmapped 11M system is not something I'd recommend to the raw beginner. Better to get familiar with 11M on a larger system first. The unmapped system requires determining during sysgen the physical addresses at which tasks will be loaded.