Speaking from personal experience, I also used Windows 95 (only from around early 1997) and had various stability issues with it (probably because I was still using some of the earlier releases) - I actually recall one day asking an IT buddy of mine (out of frustration) which version of Windows 95 was the best to use and his short and sweet answer was "the latest version".
I didn't use Windows 95 that long, since Windows 98 was released about a year later and I upgraded to Windows 98FE (later upgrading to 98SE in 1999). I used Windows 98SE up to around late 2002, after which I upgraded to Windows XP.
Subsequently, I've used Windows 98SE on all my retro PC's from Pentium 1 right up to Pentium III PC's and I've found it a very nice OS to use for that era of hardware.
I don't think that all the problems you've experienced could be attributed to the Desktop Update or later versions of IE.
It's possible that the issues you've experienced with Windows 95 or 98SE is not necessarily just due to software bugs, but could be a combination of hardware and software bugs.
A good example of software bugs would be buggy drivers (sound card, motherbard or video card) - it might run fine with Windows 95, but a later revision of the driver (for Windows 98) could have caused crashes, instability, etc.
One thing that comes to mind - since you are running on a Pentium II 450, make sure you install the updated chipset drivers for your motherboard, since Windows 95 could likely still be using the default drivers, which will hamper performance and could also cause stability issues.
From a hardware perspective, it's possible that you could have faulty memory, which doesn't always show up, until you do a specific thing on Windows - in this case, running some diagnostic utilities could help you identify if there is a hardware related problem.
However, since I don't know what issues/problems you've experienced with your Windows versions, I'm purely speculating.
I also use Windows 95, but only for my 486 based retro PC's (since Windows 98SE runs too slow) and then it's only because it makes my life easier to copy and move files, create and delete folders, etc. and not to play Windows games on it (for that I use a faster PC and Windows 98SE).
But, if you feel more comfortable with using Windows 95, then that is your prerogative.
From a retro gaming perspective, if Windows 95 works better for you, then go for it.