How big is the partition on which you install Windows 98SE? It should not be the whole 200GB (without 3rd party patches). Keep it under 128GB. If not it could introduce data corruption and strange problems.

It's partitioned to be a 120GB drive using the standard fdisk found on the Windows 98se installation floppy (the one that asks to boot from CD or HDD).

"every time I put some files on the machine" this sounds suspicously like you moved the drive to another machine to copy those files. I presume you dont actually do that?

You are just copying from floppy or a network share?

I did that earlier. But recent 2 tries I installed the USB driver first and copied it from USB

Maybe a resource problem (IRQ/DMA) as you use 3 sound cards? Else (if there are no hardware issues) it could be a driver problem. Maybe try older/newer drivers. Also you could remove all (sound) cards and try adding one by one until you find the culprit.

What is the motherboard and bios version btw? Did you install chipset and USB drivers if so which versions?

Currently reinstalling Win98SE again. My plan was to keep it a fresh install and then test tomorrow by slowly placing files and installing each driver one by one. And removing the AWE first from device manager (as I don't plan on using it under Win98).
If it has issues tomorrow with a fresh install it's likely a hardware issue. Otherwise I'd probably at some point run into the file or driver that causes it. And if any of that is soundcard related I'll remove the soundcards and slowly reintroduce them in to the system. And if all drivers are successful it might be one of the games and test programs, but I doubt that as they shouldn't be ran on boot.

I had installed the USB drivers on all previous attempts when it got stuck but also other drivers, never installed chipset drivers.

Edit:
Finished installing. Nothing installed, not even USB drivers, a completely fresh install. Now get errors that it can't find the Windows CD, even when I directly point to the location on the CD (because I can browse to the location somehow). And due to that also get a few errors on startup missing files it wants to add, but can't find the disc or the files on the disc.

Many motherboard are unpredictable when using a physical drive larger than 137GB on an LBA28 controller. I think if you use a tool to artificially limit the drive's to 120GB, or use a drive with 137GB or less the problems will disappear.

If you have any CF cards or any other small drives capable of installing windows 98 (even if they may not be large enough to actually use) you can test the theory.