Sounds a lot like overheating. With the P4's overtemperature throttling, users don't notice that their cooling is too low, because the CPU simply shuts off for a while or reduces speed if temperature gets too high. On most other CPUs, the system crashes, so you easily notice that something is wrong.
It also makes sense regarding the symptoms: Lower cycles = lower CPU stress -> longer play time. Other output options = lower CPU stress -> longer play time. Need to wait a bit until it works again -> CPU can cool off. Now some educated guesses: 3D flight mode might use many instructions that are emulated differently and let the CPU run cooler (= longer play time) while briefings might run some input handler in a tight loop that heats up just the wrong part of the CPU (= early slowdown).
Really, I'd bet it is insufficient cooling for your CPU, and you just didn't notice it up to now because of the CPU protection "feature" of the P4. Video card temperature doesn't matter for DOSBox. If this is a selfmade box, you may have installed the cooler incorrectly (too much heat-conductive paste, or none at all, or not thoroughly cleaned the cooler's ground plate and reapplied new conductive paste if you took it off) or have bad air flow inside the case. If it is an off-the-shelf box, it is probably dust buildup as was suggested earlier. In any case, take a look inside (after removing the power cord), vaccum the fans, coolers and rest of the case so that the majority of the dust is gone (no need to be squeaky clean), then close the case and try again. You may also want to check if the fans are still running correctly, since they tend to die over the years.
Heat problems are a common problem, because what was once sufficient gets continuously worse over time due to aging effects, then a few programs start to fail, and some while later, the system gets problems while idle.
BTW: You should reinstall your machine using SP2 and all security updates (start windows update again and again until no updates are left) and then install a virus scanner (like the free and well-known http://www.free-av.com).
Since you are missing the security updates of the past few years and have no virus scanner running, your system must be a biotope of malware by now. A decent virus scanner (i.e., not Norton) and the XP-built-in firewall (i.e., not ZoneAlarm), and if you're using IE or install a lot of software a regular spyware scan (Like the famous Spybot S&D at http://www.safer-networking.org) make your system faster by preventing viruses, spyware and other unwanted background processes invading your system. Those can slow down your system without being seen in the task manager, so you never know about them. A decent slim virus scanner doesn't slow down your system significantly, OTOH. You won't notice a difference while gaming.