UT2003 has benchmark tools, I suppose you can implant them to Unreal Tournament 2004. Or just use UT2003 for benchmarks.

The Serpent Rider wrote on 2025-01-20, 17:47:

UT2003 has benchmark tools, I suppose you can implant them to Unreal Tournament 2004. Or just use UT2003 for benchmarks.

I didn't know that. If I can't benchmark UT2004, then benchmarking UT2003 would be a good compromise, I suppose. Thanks for that, mate.

if you are not just trying to compare your own systems, but also compare with old results from the internet, that would be a somewhat difficult undertaking because you'd have to match game and benchmark versions.

anyway, if umark doesn't run it may not recognize the 3369 .exe, that's what i vaguely remembered from years ago as well. there are some instructions to launch a benchmark here, albeit for the linux version. maybe you can make enough sense of that to launch one of the demos that umark comes with.

alternatively, get the ut04 demo and use this post, that .zip download link works in archive.org. some other people posted results further below, and there might be some more results for the demo elsewhere.

A long time ago I benchmarked UT04 using a brute force method. It took a long time. I used FRAPS on WinXP. I don't know if Fraps still works on Win10.
I chose to do all benchmarking from the first-person view, because that's what you're using when you play the game.
The method was based on recording lots of random framerate samples - enough that the overall average is a well defined result.
The source of error I wanted to avoid was any subconscious bias that would skew the samples I was recording.
This is how I remember doing it (I might forget something, it was a long time ago):

Set up Fraps to log the framerate to a text file for XX seconds whenever you push some key.
Start Fraps.
Start a long botmatch. Be consistent with all settings - for example I found that changing the bots skill level does affect the framerate.

1) Push the key that changes the view to another bot.
2) Immediately after changing the view, push the Fraps button to log/record the framerate for the next 30secs or whatever duration you've set in Fraps.

If you just want to move the camera without recording, then commit to that decision beforehand. Say it out loud if it helps. You do *not* wait until after step 1 to decide on step 2. You are committed to the decision to record before you have seen where the camera is about to go.
Having started the recording, do not abort for any subjective reason - that's human bias. Even if the bot is staring at a wall, just follow the procedure and keep recording the framerate. Because you switched views immediately beforehand, you did not know what the camera would be looking at when you started the recording. So this removes any personal "judgment" that would skew the set of samples you are recording.
If the match ends during the framerate log, I probably aborted and tossed those out - but that's a predetermined policy, it's not subjective.

When the framerate recording is over, repeat. Switch views immediately before each recording as above. Repeat this a few dozen times. 30 is probably enough, but I think I did 40-50. You need lots of samples to average out the randomness so the final result will have a narrow confidence interval. It takes a lot of time, but it does work. It's just a matter of getting lots of random samples.
Exit the game, load the log file. Look at the recording times to make sure they weren't aborted. Enter all the recorded framerates into a spreadsheet. Do a simple average, or whatever other statistics you want.

My mantra in all this was to remove subjectivity from the process, because that will skew the results. That includes no subjective decisions on when to record and which samples to throw out. If samples are thrown out then it should be based on an objective formula (like a "trimmed mean" or similar), *not* subjective judgment.

that's not a good method, apologies for being blunt here. timedemos have been around since doom and both cut down on work, along with offering a repeatable benchmark scenario. that method is a ton of manual work and not a repeatable benchmark.

if a game doesn't offer any kind of benchmark, i'd use the "duke3d method", just spawn, press nothing and write down the framerate. this would work in ut04 because it's possible to disable bots. otherwise hardware sites usually time in-engine cutscenes in that case.

in ut04 it's possible to write "demoplay demoname?timedemo" in the console (without ".dm4"), which acts exactly like quake etc.: plays the demo as fast as possible, then dumps the needed time and average framerate into the console. somehow this tends to be forgotten, including by myself at first. i'll attribute that to the lack of a standard demo supplied with the game. for anything more like min/max FPS umark or RTSS will be needed.

It's an effective method, but it does take time. If the samples are random (not subjectively picked) and you record enough of them then the results are repeatable. Not repeatable from a single sample, but from the mean of a large number of them. In the real world it's routine for statistics to be generated on that principle.
Instead of a large number of 60-90 second framerate logs, it might be just as effective and less laborious to record just 1 huge single framerate log with a very long duration. Like 1 hour from a single bot's POV (making sure the match will last that long). That way you can just hit record once and walk away. I should have tried doing it that way.

Back when I wanted to test UT04, I vaguely recall reading something about that timedemo feature - I forgot about it, thanks for pointing that out.
Back then, I was building a PC for my nephew who I knew would be playing UT04, but I didn't have the game so I was stuck benchmarking the demo version. I wanted to find the impact of faster RAM in that specific system and game (so I wasn't interested in the generic figures from sites like THG/etc). After logging around 30-50 random samples of botmatch gameplay at each timing, I did get a well defined answer.

Spawning into a fixed scene with no bots (like you described as the "duke3d method") is easy and repeatable without needing lots of samples. That's a big advantage for the hardware sites who want to keep everything quick and automated. It's also easier for them to explain to their readers. My criticism of it is that it's a poorer simulation of real gameplay. The method I described would be too labor intensive for anyone doing this in volume or working on a clock. But I agree that a recorded first-person timedemo is good and practical where supported.