One possible reason for your problem is because, once you start in MS-DOS mode, you are loading certain DOS based drivers.
Typing "exit" will just try to load Windows, without unloading those MS-DOS drivers, which could cause Windows to hang (for various reasons, like not enough memory available, conflicts, etc.)
Mau1wurf1977's suggestion by pressing CTRL+ALT+DELETE is the best solution.
An alternative solution would be to set up a boot menu and boot straight into MS-DOS 7.10 (on which Windows 95b to 98SE runs) when you start up your PC.
That way, you don't have to first boot into Windows & then restart in MS-DOS Mode (after all, if you want to work in DOS, why do you first want to boot into Windows?).
I've done this on all my DOS based PC's and it works much better for me.
You will obviously then have to ensure that you set up your start up files (Autoexec.bat & Config.sys) correctly so that you can load all the relevant DOS drivers (memory manager, CD-ROM drivers, mouse, soundcard, etc.).
You will also have to edit your MSDOS.SYS (read-only & hidden) file in your root directory to enable you to bypass the GUI and boot straight into DOS.
Once you set up your boot menu correctly then, at start up, you can select to either boot into Windows or DOS.
You can set up multiple boot scenario's (for example, to boot with or without EMM386.exe).