TL;DR:
So: why use push power buttons?
The same reason that it is used in other appliances: it's convenient.
The Works:
Just think, early (IBMish) PC had their power switches at the right side all the way at the back (*2). One had to to feel one's way all around to flip it - of course with no chance to look at its state. The switch was part of the power supply. This made sure that all high voltage switching only happens within the PSU.

_(Picture taken from Wikipedia)
Putting it up front wasn't really a solution as this would break the encapsulation of high voltage within the PSU. Keep in mind, the PC was a device intended to be opened by any average user, thus exposure to high voltage was a complete no - usually prohibited by most electric codes worldwide. The same reason goes for why it had to switch mains, as only that would guarantee that the PC is completely powered off if switched off but not unplugged - which, depending on workspace setup may not always be possible.
Moving the PSU upfront was not really a great idea, as that would mean having the power cord coming out on the side. Only a PSU reaching all across the case would avoid that.
Using a push switch is about the only (*1) solution, implemented by extending the visible push button by a long 'stick', going all the way back to the PSU, pushing the 'real' toggling power switch. Now the user could operate the power switch from the front of the computer while still having a code-legal encapsulated PSU at the back.
The original AT-class (286) HP Vectra of 1985 was one of the first IBM(ish) PCs to offer this.

(Picture taken from Wikipedia; marking mine)
Long story short: It's all about convenience.
Equally important, adding a certain type of switch is a design decision not necessary based on technical issues, but what look/feel/operation habit a designer wanted to achieve.
Besides, in real life, no user checks the state of a switch before flipping it, when he already assumes the computer being off - after all, he already ignores other easy to check signs like lit LED on keyboard, case or monitor.
*1 - which BTW was the same with the Apple II/III series, undoubtedly the major prototype for the IBM PC, with their enclosed PSU and a switch on the back.
*2 - Of course, like almost always there are other possible solutions. For example using some secondary low-voltage circuit, much like for the later power button, but operated by a switch. Just, this would complicate the PSU considerably, and thus increase the price - not anything compatible manufacturers, usually fighting on cost base, would want.
Equally, if not more important:
It would be a technical divergence from the example set by IBM. IBM had a fully concealed PSU. Having a different design, no matter whether better or worse compared to IBM, has always been used as a point by IBM sales force - and even more in the mind of decision making on buyer side. So it was extremely important for makers of compatibles to withstand that. A compatible had to be the same, or better, without changing anything. It took many years until the market was levered in a way to incorporate new ways.