Not sure it helps, but I actually just use the Geforce Experience software on my PC to record pretty much anything. The only disadvantage is that it can't really record just a Windowed application and records the full screen only. But since I usually play DosBox in full screen mode, that isn't really a problem for me. The software doesn't really know when I have DosBox fired up, but I can set screen recording and broadcasting at anytime with Macro keys on the keyboard that are still active when DosBox is running.
My main issue with the GFE software is that it doesn't always record the audio properly and so I have to fiddle with it each time. My system doesn't like my sound card being in the second PCIE slot on my motherboard and would prefer to only operate from the PCIex1 slot. Problem is the sound card won't fit there with my video card below it and my AIO water cooler radiator also. So yeah....
Midi Modules: MT-32 (OLD), MT-200, MT-300, MT-90S, MT-90U, SD-20