It wasn't just Windows 95, but Windows 3.x as well, even though they work very differently.
Other answers talk about pre-emptive multitasking, so let's first clarify this:
Window 3.x was using cooperative multitasking where each app would release the cpu for the other apps to use it. Windows 95 uses pre-emptive multitasking where each app is allocated a time slice.
The answer is linked to how the graphic interface works: in a windows graphical app, there is a loop called the 'message pump':
Every event (mouse moved, window got resized, etc) is pushed into a queue. The app is responsible to check if it has messages waiting and, if yes, pull them and process them.
This is at this moment that Windows 3.x was switching to other apps since there was a single point where all apps where going to, but this doesn't apply to windows 95.
What really happens is that, on both OS, you need to process the message loop, but if you want to update something in the background, like a task, a display update, etc, you'd set a timer and the timer would put a message in the queue at a regular interval.
These were better ways to do things on Windows 95, but developers took time to transition from Windows 3.x and many apps were structured the same.
Since the main mechanism was to rely only on the message loop and background operations were done through timer messages, moving the mouse would trigger a lot of messages, move the app up in priority, wake the app up, and get the app to process the background tasks messages. Without moving the mouse, the timer messages would be read up only at a rather slow interval.
The most famous app for this was the disk defragmenter where operations would wait for a message to update the graphic interface! so shaking the mouse would speed up the defrag.