uematsufreak wrote on 2022-03-29, 04:09:
I just finished a stream testing things out and have nothing new to report. Things seem to work really well. I encountered a couple glitches, but I was doing glitchy stuff anyway during those playthroughs and I'm certain I triggered them all. 😜
Nice! 😀 I'm savvy enough with the game to get to 225 by memory, but nowhere near enough to speed-run it or know what glitches to exploit. It takes me probably a good 2~3 hours for a start to finish run through. Do you stream publicly?
I've gotten intimate with RTZ resources a bit over the years (using tools built elsewhere to extract PRJ contents including images and PCM audio) and have been messing with the MPEG videos for fun.
I've never looked inside the project file aside from a quick inspection in a hex editor, but have heard rumors that there may be some unused scenes left in it. I'm curious: Which tools do you use for this? Have you inspected the different project file versions like DOS Floppy vs DOS CD-ROM vs DOS MPEG vs Mac CD-ROM vs Mac MPEG? I had have an item on my bucket list to completely reverse-engineer the game (well, at least the DOS CD-ROM version) someday. It would be cool to recreate all the C source code used to build the MADE engine, though it would probably be more effort than it's worth.
Also, I've been thinking about the ML video reconstruction stuff Bobbytoz was using for The Horde. Imagine using that same technology to remaster all the RTZ MPEG assets in 4k resolution. The actual ReelMagic board can not obviously play a 4k video, but the DOSBox ReelMagic emulator currently can (tested working). 😉 Why stop there? Could also update the project file to have remastered assets too. The CD audio tracks from the DOS CD-ROM version would pair nicely with the 4k MPEG video. Of cource the MADE engine would require a few patches to accept larger sized project assets and VGA modes as well as combine the CD audio functionality, but I think it's doable. The CD-ROM image would be a lot bigger than the standard 700MB, but AFAIK there's no reason that won't work with DOSBox. How friggin' cool would that be? 😀 Then again, doing a project like that sound like it might open the door to getting sued if it were shared, so probably not worth the risk. 🙁
I'm not sure how the card actually overlays the image
It's not as complicated as it may initially sound 😀
The actual card itself does the video mixing on the analog side of things. It can place the MPEG feed either in front of or behind the VGA feed. In order to have both MPEG and VGA overlayed like Return to Zork does, the MPEG picture must be placed behind the VGA picture, and a selected VGA pallet color is designated as the transparent/alpha color. Basically, holes are punched in the VGA picture so that you can see parts (or all) of the MPEG picture behind it. The MPEG feed can also be placed in front of the VGA feed, but AFAIK, it does not support MPEG transparency and the only way to see both is to resize the MPEG feed to a smaller "window" so that it only takes up a smaller portion of the screen. Return to Zork does not do this for most scenes though because the mouse cursor (VGA feed) would not be visible in the MPEG regions. (e.g. Ferry Ride, Alexis)
So using Return to Zork as an example, the VGA timing signals are driven at roughly 70Hz for vertical, and roughly 31kHz for horizontal. This means that 70 times per second the monitor will rescan a new picture from the top, and 31,000 times per second a new line is started scanning from the left. Doing the math, this gives us roughly 400 lines at 70 frames per second of visible output on the monitor. Since Return to Zork is using 320x200 on the VGA graphics side of things, this means that scaling up to match 400 lines is a cake walk as everything is simply doubled. However, since the ReelMagic card has 400 lines on the wire to work with, some lines from the MPEG feed are blended and scanned more than once, but each line is scanned at least once, thus no data is lost. In addition to that, add an old CRT/phosphor monitor to the mix and things look pretty decent to the human eye.
The emulator in its current state is bound to the VGA digital dimensions (200 VGA lines when RTZ is running) and that's where the loss comes into play. A while back, I though about hacking in something to upscale to 400 in software like the real hardware is operating at, however, spending too much time on these kind of hacks are counterproductive to actually just doing it right.
Dealing with either missing scanlines on the video or the doubled ones on the overlaid graphics sucks, but I have NO idea how to go about that with my ignorance past maybe scaling both vertically to 1200 then scaling them both back down with an algorithm to 240.
Scaling up to a commonly divisible resolution would certainly work, however, something like that would likely suffer from serious performance problems. The good news is that this problem has already been solved and by using the proper SDL functions that jmarsh mentioned earlier in the thread, I can just give SDL the pictures "as-is" and let it handle the color conversion, scaling, and alpha blending. This is ideal because SDL already implements very good logic for graphics scaling that has been tested working and looking good across a wide variety of applications out there. This also will yield a performance increase too because SDL will likely just hand-off this work to the GPU itself. In addition to that, this opens the door to using OpenGL sharers to cleanup some of the MPEG block boundaries that are more visible on modern displays. Though I will have to figure out how to simulate the VGA alpha channel (punching the holes through the picture) for this to work properly as this is only a concept that exists in the ReelMagic world and is not standard VGA. I do already have a few thoughts on how this can be done and I don't think that it will be very complicated. Unfortunately the bad news is that it will likely be a while before I start work on this as I want to make sure the emulator part is reasonably complete and solid before optimizing things. (e.g. not causing games to crash and burn 😀 )
I have faith a solution will come and I look forward to seeing the future progress, including migrating over to an SVN base. This is seriously AMAZING. I can't state that enough.
Glad you like the project and thanks for your help and feedback testing this thing out! 😀
Uka wrote on 2022-03-29, 07:35:
I have checked all the games again:
As always Uka, thank you very much for help on testing this 😀
6. "Man Enough" - the minor green blocks are now gone! Other things are the same as before - almost everything is fine except a very few garbled videos.
Cool cool glad to see things are improving! Do you happen to have a save game file at a point where there is a garbled video?
7. "Space Ace" - no changes: the videos are fine, but pressing any key freezes the game.
For this game, it looks like the hack I did to make Return to Zork work without actual DMA streaming emulation is corrupting this game's memory. It's likely that multiple runs of this game in the emulator will yield different crash/lockup behavior. Unfortunately, until I actually get the board's DMA engine fully emulated, this game will not run without crashing early on. 🙁 Adding further insult to injury, I can't start on the DMA emulation code until I figure out a permanent fix for the f_code/magic MPEG stuff. The current magical MPEG / f_code hack quick scrubs the MPEG file when a game is loading a video and inspects a handful of the P and B pictures to figure out exactly what is needed to correct the videos on-the-fly so that they are playable in a standard MPEG-1 decoder. With the DMA streaming interface, the game is now in the driver's seat of feeding the MPEG decoders and there is no longer a file that I can peek at behind-the-curtains or even know about, just a raw continuous stream of MPEG bytes that the game gives me. I'll eventually figure this out, especially with all the recent work I have done to uncover how the DMA engine actually works, but it looks like this one is gonna take some time.
9. "The Lord of the Rings" - everything seems to be fine, except that pressing Space + Esc in the intro still freezes the game.
Space + Esc during the intro also locks things up on the real PC here with the Maxima card. Does this also happen on your setup (Lite or Maxima)?
2. "Dragon's Lair" - well, the videos are fine, but it still seems that the game does not react to pressing any keys. However it is all different now! With the previous version of the emulator, it was like a non-interactive demo with hero successully passing all scenes automatically - and now he dies at the very first scene. There are some more comments below!
Awesome good to know this is improving!
Well, I can confirm that "Dragon's Lair" DOES work and reacts to keys pressing on my 386SX with ReelMagic Lite. It is just very difficuly to play!
Thanks for confirming this is good to know!
In fact, it is probably the very first RealMagic game created - my disk is dated 22 November 1993 - and maybe it is just too raw or maybe CPU speed is relevant? I have this game in jewel case with the manual, but no system requirements are listed. And I have heard that there was an updated version of "Dragon's Lair MPEG" released in 1994, but I do not have it.
Interesting to know. Possibly CPU speed related, though I did have the same thought last night and cranked the CPU cycles down to 200 in DOSBox and yet it was still blowing through that part so it's unlikely the CPU. My current running theory is that it could be related to the get subfunction 206h stuff I just implemented. Likely, the game is using the MPEG bytes decoded counter as means to determine when to accept user input. If this is true, this is a TERRIBLY BAD software design decision in the game because this could change depending on all sorts of factors such as the model of ReelMagic card, or even just based on what hardware peripherals are installed in the PC! (Hence why I asked about this working on your 386SX/Lite 😀 ) Though that would explain the game being re-released in 1994. As I mentioned in my last post, the buffer sizes of the MPEG decoder for the emulator are likely much larger than the original hardware and it's possible that the MPEG decoder swallows a single buffer that is much larger than the game was designed for and tested on, then blasts past the stream byte position for what the game considers as "too late" for accepting any user input. The good news is that if this theory is correct, then adjusting the emulator to simulate and report smaller byte offsets is simply a matter of tuning.
I attach some hints from the manual about using the control keys and passing the first scene - can you please try it with both your Maxima and DOSBox-RM?
Thanks this is very helpful... Looks like I was pressing the wrong keys! (Who the hell designed this game's controls!? 😀 ) I'll give this a try later on the Maxima here to see if I can get it to progress, then on the Maxima with the data recorder / spy, then I can use that to compare what is happening differently on the emulator.
4. "Entity" - now I have found something! The subtitles are fine (at least the English ones in the intro), but - I now see where those video messages should be displayed. Sadly the DOSBox crashes there... And it is because of the MPEG videos, as everything is fine in the non-MPEG version. Just near the start of the first level there is a scroll on the ground: you can pick it up by pressing 'down' arrow. Maybe the LOG files would tell what is wrong there in the MPEG version?
Thanks for the screenshot showing exactly what to do... This is very helpful as I was looking in the wrong place (main menu) at the video messages section. I will certainly give this a try! We can see from your logs that right here is the point where things go wrong:
Entity.log Snippet
1570019809: IOCTL:NUL:WRITE
1570019876: INT10:Set Video Mode 11
1570019918: FILES:file open command 0 file FNT.BIN
1570038882: REELMAGIC:Reset
1570038882: REELMAGIC:driver_call(0Eh,00h,0h,0h,0h)=0h
1570039028: REELMAGIC:driver_call(09h,00h,409h,280h,1E0h)=0h
1570039174: REELMAGIC:driver_call(09h,00h,408h,0h,0h)=0h
1570039323: FILES:file open command 0 file ¹
1570039323: FILES:Makename encountered an illegal char ¹ hex:B9 in ¹!
1570039323: FILES:Makename encountered an illegal char ¹ hex:B9 in ¹!
1570039323: FILES:Makename encountered an illegal char ¹ hex:B9 in ¹!
1570039323: REELMAGIC:DOS File: Open for read failed: DOS:¹
1570039323: REELMAGIC:Zeroing out INT return registers due to exception in driver_call(01h,00h,1h,174h,1A2h)
1570039419: EXEC:Execute t¢ 0
1570039419: FILES:file open command 0 file t¢
1570039419: FILES:Makename encountered an illegal char hex:1 in t¢!
1570039419: FILES:Makename encountered an illegal char hex:1 in t¢!
1570039419: FILES:Makename encountered an illegal char hex:1 in t¢!
1570039462: SBLASTER:DSP:Reset
1570308192: IO:Writing 00 to port 1076
1570308192: IO:Writing 00 to port 1077
1570308197: IO:Read from port 1076
1570308197: IO:Read from port 1077
1570308201: IO:Writing 00 to port 0FC6
1570308207: CPU:Illegal/Unhandled opcode 63
1570308208: CPU:Illegal Unhandled Interrupt Called 6
1570308210: CPU:Illegal/Unhandled opcode 63
1570308211: CPU:Illegal Unhandled Interrupt Called 6
1570308213: CPU:Illegal/Unhandled opcode 63
1570308214: CPU:Illegal Unhandled Interrupt Called 6
1570308216: CPU:Illegal/Unhandled opcode 63
From that, I can see that things go bad somewhere between when the game ("ENTITE.EXE") executes "FMPDRV.EXE" and right before the point where the game (again, just "ENTITE.EXE") loads the first MPEG file. The game's memory which contains the filename of the MPEG asset looks like it is corrupt, and then because of that, things just cascade and go to hell from there. I will get some extra probes in there and run some tests at this point in the game to see what is going on. Thanks! 😀
I have checked only the English subtitles - everything is fine there, exactly like on the real machine!
But that's only for the intro, of course - I am now more concerned about video messages not appearing. Yes, the game does freeze when picking scrolls on 386SX with ReelMagic Lite as well. As if it tries to play some files from the disk and cannot do that. Maybe some more ISO modification is needed?..
Yeah I only checked the intro as well. One hypothesis is that using the RTZ "FMPLOAD.COM" file renamed as "FMPDRV.EXE" is creating an issue because the "FMPLOAD.COM" file is a non-relocatable "COM-format" executable disguised as an EXE. Then when that COM file is invoked from inside "ENTITE.EXE", it stomps on some memory belonging to "ENTITE.EXE" that it is not supposed to. Just speculating though. I can try creating a true Zbykowski formatted relocatable DOS EXE using my new handy-dandy ReelMagic SDK to test if this has any impact on the crash.
llm wrote on 2022-03-29, 10:28:the dosbox staging project forsters a dosbox SVN git: https://github.com/dosbox-staging/dosbox-stag … /tree/svn/trunk
this branc […]
Show full quote
the dosbox staging project forsters a dosbox SVN git: https://github.com/dosbox-staging/dosbox-stag … /tree/svn/trunk
this branch is the staging base line and always up to date with dosbox SVN -> its the merge base for staging
even if Staging is not a thing to discuss here loudly, the Staging-SVN-git is an up-to-date and clean Dosbox Svn conversion to github - still better then having x other unmaintained svn-git forks around on github/gitlab
so im not talking about using Staging as a base - only the svn/trunk tag as a base
Thank you this is good to know! 😀
Dominus wrote on 2022-03-29, 11:04:
I honestly have not kept up with which fork adds savestates, as I don't really need it and prefer the vanilla DOSBox experience.
As far as I understood, saving the state means you have to do that for each emulated hardware hence my comment that it would need extra work for this fork to function.
Oh man... Implementing the ability to snapshot and restore the state of the MPEG handles, players, decoders, buffers, and DMA engine sounds like barrels of fun! Good thing it's not a core DOSBox functionality. 😀