You're saying two different things: 1) that you want the best possible image for archiving purposes, and 2) that you don't care about compression and will be uploading to YouTube.

It sounds like you're really focusing on (2).

If you want (1), then you definitely want pixel-level accuracy. That means 4:4:4 chroma and maintaining original resolution and refresh rate.

I'm just bringing this up because you're going to get drastically different responses based on use-case.

I don't have a RetroTink 4K so I can't respond directly, but I'm curious as to what people say about it.

Hmm, sorry if that was confusing. What I’m trying to say is that I want to get the best possible image quality without going overboard or maxing out all the settings just because I can.

I might be wrong in saying that 4:4:4 is overkill. What if it isn’t? That’s actually one of the the main points: I’m not sure whether it makes sense for my specific use case.

I am using a Voodoo 3 for output, Retrotink4k and then the hdmi/dvi signal goes into the datapath4k.

What I am doing on the tink4k:
I upscale everything into a 1080p 60 Hz Signal. On PC skipped frames are not as much of a "problem" as on console or Homecomputer where 50Hz often mean 50 Frames for example. So I don't mind losing a few frames in DOS and you can't put 70FPS/Hz videos on youtube anyways.
The resolution is not maxed out. I ALWAYS do pixel perfect scaling unless the resolution goes too high and it's not possible anymore.
This means I deal in the Pixels as custom resolution and set scaling to integer.

320x200 -> 1280x1000, yes, thats not 4:3 but very close and better than not pixel perfect scaling (big caveat, see below)
640x400 -> 1280x1000, yes, every vertical second pixel is becoming 3 instead of 2 pixels, but again, it's better than the alternative. But this might depend on the game.
640x480 -> 1280x960
800x600 -> From here on, you get into trouble and you will need to use different scaling methods because pixel perfect scaling will not yield decent results anymore

For OBS:
It is essential to "capture" the signal in 4X4X4, even if you just do this for youtube and don't save the actual video in maximum quality. I used a different way of capturing without the RGB part and especially in DOS Navigator with red font on Blue background it looked horrific. So make sure to take in the signal as 4x4x4.

In order to get the best quality on youtube, you should set OBS to 4k, even if nothing you use is 4k. Just Use OBS to 4k, and upscale everything. Youtube will use better renderer and overall better quality for 4k and it even reflects on 1080p footage. Though some of that info might not be up to date anymore.

Anyways, would recommend to set OBS to 4K. Save videos as 4k. I don't even have a 4k Screen. But it comes with a positive side, because you also see what happens if you do it wrong:
The footage you get in your 1080p capture signal should be doubled, resulting in 4k ofc (black borders and your 1280x1000 will turn to 2560x2000 within the 4k signal). The option to double it in OBS is called "point". Area is second best, but Point is pixel perfect duplication (actually you might want to fiddle around with area with these 800x600 signals and up, but I didnt experiment with that in OBS myself)

If you moved this by one pixel. You will see a massive change in the output on a 1080k screen, because now when downscaled, you would "mix" different pixels. (e.g. now its not Pixel1 Red, Pixel 2 Red, Pixel 3 Blue, Pixel 4 blue "downscaled to Red and Blue. Instead you would get some blurry mess where Pixel 2 and Pixel 3 are mixed. I see that in my OBS preview clearly if I move that by just one pixel.

And while I didnt test that with 4k recordings, my guess is that, that this would also affect something like 800x600 if you thought like "ok, but for 4k I could just triple it". It will probably look decent on 4k, but (probably) bad if anyone watches it on 1080p. But this is speculation on my end.

The Big Caveat for 320x200
Now...there is one issue. And this is why all of the above is...not how I am actually doing it for 320x200, but how I would do it if I didnt have the Datapath Vision RGB capture card.
Because you get some bad noise on 320x200 and I havent found any good way to get rid of it via the Tink OR OBS. I linked two videos showing it in this thread. Re: CRT Terminator Digital VGA Feature Card ISA DV1000
This noise is especially visible on dark mono-colored screens. Like the text boxes in Lands of Lore or Eye of the Beholder, Inventory in Might and Magic 3, or just walls in Dungeon Master 2. This also happens on both of my graphics cards (Voodoo 3 and Trio64).
While visible to the eye in preview, the issue gets worse if you compress it with AV1. I was happy to upgrade my graphics cards to a RTX5060ti recently and use the much more effective AV1 coded. However, the compression means that the grain gets more visible. It basically turns into "clouds" which look pretty bad.

Now...there is the option to use the CRT Terminator card. Thread here: CRT Terminator Digital VGA Feature Card ISA DV1000
It's a card you put into your ISA Slot. It snoops the ISA bus and in addition gets plugged into your Graphics cards feature connector, and turns this into a perfect digital signal. No grain.
The Downsides:
-This thing costs 300€ for me if I consider shipping and whatnot.
-For our usecase we wouldnt even use any of the scaling features the cards offers (only passthrough), so I guess a chunk of that strong fpga used would be "wasted". The developer mentioned something along the lines of that a smaller fpga might not help much to reduce costs, because then the order numbers would be less for that small fpga, and in turn increasing the price again
-Your graphics card needs to be compatible. Meaning it needs to be PCI or ISA (not AGP) and the feature connector needs to work (it doesnt on the Voodoo 3)

So for now I did not do that, but still consider the investment.
My workaround is a...silly one, but one you wont have.

Since I use my Datapath for capturing, I have the program VCS. And VCS has a cool feature called "Pixelgate". And this let's me vastly reduce any noise for the cost of a theoretical small delay which I dont notice. The noise turns to "static" non moving noise, reduces file sizes and isn't apparent in the recording. But capturing that is a bit weird. Because how it works is that VCS gets the signal, uses the pixel gate, displays it on the screen. And now I use OBS to capture the VCS window. Which...yes is kinda stupid but it works. However, because everything not on the screen is cut off and VCS comes with this header, I actually forward a 1280x1000 signal to my PC, not the 1920x1080. This gives me 80 pixels for that ugly windows bar and it still fits on the screen (the windows bar is not captured).

I guess this could be done with an OBS filter. But well...someone would make it first. I did post suggestions on Discord and OBS forums but nobody cared, though I guess this would also be super helpful for camera grain. Anyways...guess not going to happen or only happening by coincidence.

For OBS Capture I use
Hybrid MP4 with NVIDIA NVENC AV1
Constant QP with a value of 18. Preset: Slow (good quality), Tuning: High Quality, Multipass: Two passes (Qarter Resolution), Look adead active, 0 B-Frames.