Uka wrote on 2022-04-09, 05:17:There is only one minor issue: after quitting the game in the middle with F10, DOSBox crashes instead of leading us back to DOS. It works fine after the game is completed though. I attach the logs.
This bug is now fixed, by the way 😀
The attachment contains an example of sound artifacts in the Horde game.
It appears in some video files (usually large).
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Games used to be better!
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
I've now completed "Crime Patrol 2 - Drug Wars" as well - everything is fine!
I got another one: https://github.com/jrdennisoss/dosboxrm/relea … /tag/2022-04-19
Changes:
Current Tested Game Compatibility:
Known Issues:
I've been kicking the can down the road on doing this change for a while because it is very high-risk as it changes the core functionality of the way the emulator unlocks the "magical MPEG" video files, but I think now is the right time to start rolling this stuff out. This is not a complete implementation of doing "magical MPEG" the right way, but rather a step in the right direction to the way the original hardware does this. This change eliminates the code performing heuristics on the MPEG asset files to guess the state of the MPEG encoder when these files were generated. Instead it now respects the "magic key" value provided from inside the virtual machine and sets up MPEG decoding accordingly. I still do not know the exact relationship between this "magic key" value and how it supposed to be directly applied to the decoding process, however since there are only a limited amount of values that I have ran into (one for "The Horde" and one for everything else that uses "magic MPEG"), I use this value as means to determine which P and B MPEG pictures contain truthful decompression information and reconstruct all MPEG picture headers accordingly. Hopefully I didn't break anything 😀
Uka wrote on 2022-04-19, 06:26:So, for now only one DOS game (out of 10 available for the moment) is not yet supported, "Space Ace".
Yup getting close! Unfortunately getting "Space Ace" to work is a bit tricky as it requires some serious, but well needed changes to the emulator. Eventually I'll figure something out though. I want to soon look into getting things ported to DOSBox SVN trunk using SDL for the video mixing too which should help clean up the video quality a bit. Then after that, I will probably look into seeing what it takes to get things up on Windows 3.1 because I am itching to try out Psychotron. 😀
Uka wrote on 2022-04-19, 06:26:I had one minor issue there: the default sound volume in MPEG videos was too quiet - and making the general volume higher made the sound effects that are not from MPEG videos too loud. I have un-commented audiolevel=100 line in CONF file - and setting it to about 250 made it rather fine! What do those audiolevel settings actually mean? That is, what number is, say, 100% volume?"
Good question... That number is the audio volume percent for the MPEG file being decoded. If you put 100, then the audio sample data in the MPEG file will not be modified and it will be played as-is. Going higher than 100 will amplify the MPEG sound and can certainly help with the exact scenario you described. However, if you go too high then the sound can start to distort with certain games. For example, if an MPEG file uses the full-range of volume and you put a value higher than 100 for that parameter, then the sound will start to distort. For games like Crime Patrol and Entity, it would appear that the MPEG audio tracks don't use the full-range of volume and boosting this value can certainly help out the experience. When debugging some of the Crime Patrol issues, I noticed that there are some extra bits provided to the FMPDRV.EXE play function which I am currently ignoring. My current running hypothesis is that these bits have some impact on the audio stream. I will investigate further and have an issue to track: https://github.com/jrdennisoss/dosboxrm/issues/119
Uka wrote on 2022-04-19, 06:31:This bug is now fixed, by the way 😀
Uka wrote on 2022-04-19, 06:26:I have played and completed "Crime Patrol" - all the videos are fine, no artifacts at all!
Uka wrote on 2022-04-19, 18:00:I've now completed "Crime Patrol 2 - Drug Wars" as well - everything is fine!
Awesome! Thank you so much Uka for helping test! 😀
Bobbytoz wrote on 2022-04-19, 06:46:The attachment contains an example of sound artifacts in the Horde game.
It appears in some video files (usually large).
Thanks Bobbytoz! I will take a look at this later. 😀
Bobbytoz wrote on 2022-04-19, 06:46:The attachment contains an example of sound artifacts in the Horde game.
It appears in some video files (usually large).
Thanks for the video pointing out EXACTLY where this is Bobbytoz! 😀 I guess I never really noticed it before, but yeah if I compare it to playing the video in VLC I do hear some clicks and pops. I'll look into this.
jrdennisoss wrote on 2022-04-19, 18:53:[*] Cleans up major video artifacts in Man Enough.
Awesome 😀
jrdennisoss wrote on 2022-04-19, 18:53:Hopefully I didn't break anything 😀
I have briefly re-checked all the 10 games - everything seems to be fine!
That is, just those good old known bugs. Like "Space Ace" not working, video not progressing in "The Horde" etc.
I think one bug is not yet listed in "Known Issues" though:
[*] The FPS sometimes drops to as low as 3. I thought that was because of using dosbox_heavydebug.exe, but no - it is the same with regular dosbox.exe. That happens for me in the first logo intro of "Man Enough" and in some scenes in "Return to Zork" (for example, when the doggie first appears on the farm).
jrdennisoss wrote on 2022-04-19, 18:53:I want to soon look into getting things ported to DOSBox SVN trunk using SDL for the video mixing too which should help clean up the video quality a bit. Then after that, I will probably look into seeing what it takes to get things up on Windows 3.1 because I am itching to try out Psychotron. 😀
Great! Anxiously waiting 😀
jrdennisoss wrote on 2022-04-19, 18:53:Good question... That number is the audio volume percent for the MPEG file being decoded. If you put 100, then the audio sample data in the MPEG file will not be modified and it will be played as-is. Going higher than 100 will amplify the MPEG sound and can certainly help with the exact scenario you described. However, if you go too high then the sound can start to distort with certain games.
Oh, indeed! I tried playing "The Horde" with audiolevel=250 - and the sound was distorted! I bet those are the same sound artifacts Bobbytoz reports about. Everything is fine though if I re-comment the audiolevel line. And it is still quiet or uneven in other games...
I think I will later try to play all the available parts of "Man Enough", but here is what I already see in the beginning of it:
1) The large artifacts are gone, but there are still a couple of distorted lines at the bottom of the picture. Do you think they could be cleaned up?
2) The sound volume is certainly incorrect in this game! That is, when I first meet Jerry, her first video ("Welcome", the one that did not have artifacts before) sounds fine, but the second video ("I am not buying that story", the one that had artifacts) is way too loud! The third video is fine again. Of course, if I lower the volume then the second video is fine, and the first one is too quiet etc.
jrdennisoss wrote on 2022-04-19, 18:53:[*] No Subtitles
Are you sure it is an issue? I mean, it is all the same on the real hardware, and maybe there are even no subtitles to be displayed there at all? Of course, if there are some subtitles, so if it is not a 'feature' but an original bug, it would be very nice to fix it - but it is still not as critical as the crashes in "Entity".
The Horde
I converted the original INTRO file and made a replacement.
The sound artifacts are gone.
I think the problem is in the audio track.
Although the playback of the original MPG files in Windows 10 (FFPlay) does not give artifacts...
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Games used to be better!
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Uka wrote on 2022-04-20, 05:08:I have briefly re-checked all the 10 games - everything seems to be fine!
Awesome sauce! 😀
Uka wrote on 2022-04-20, 05:08:The FPS sometimes drops to as low as 3. I thought that was because of using dosbox_heavydebug.exe, but no - it is the same with regular dosbox.exe. That happens for me in the first logo intro of "Man Enough" and in some scenes in "Return to Zork" (for example, when the doggie first appears on the farm).
I'll look into this... Thanks for the heads up!
Uka wrote on 2022-04-20, 05:08:1) but there are still a couple of distorted lines at the bottom of the picture. Do you think they could be cleaned up?
If the original hardware setup does not do this then yes probably it can be cleaned up. I'll look into this.
Uka wrote on 2022-04-20, 05:08:2) The sound volume is certainly incorrect in this game! That is, when I first meet Jerry, her first video ("Welcome", the one that did not have artifacts before) sounds fine, but the second video ("I am not buying that story", the one that had artifacts) is way too loud! The third video is fine again. Of course, if I lower the volume then the second video is fine, and the first one is too quiet etc.
Dully noted.
Uka wrote on 2022-04-20, 05:08:Are you sure it is an issue? I mean, it is all the same on the real hardware, and maybe there are even no subtitles to be displayed there at all? Of course, if there are some subtitles, so if it is not a 'feature' but an original bug, it would be very nice to fix it
No I'm not sure this is an issue. I think I may have misunderstood you as I thought you said the original hardware had subtitles. If not, then great! 😀 One thing to note about "Man Enough" is that there IS a lot of user data in the picture headers and this very well might be subtitle data. I'm planning to look at this in detail one of these days.
Uka wrote on 2022-04-20, 05:08:- but it is still not as critical as the crashes in "Entity".
So about that... I took a look at the ENTITE.EXE file using the handy-dandy DOSBox debugger. I was able to pinpoint the location where things get screwed up. (attached theproblem.png file) The cause of the problem is a function which seems to just blow away most of the program's the data segment causing a crash not long after. I did not go into depth to see what this function does exactly, but it does not appear to be related to anything for the ReelMagic stuff. However, the function ONLY gets called if the "FMPDRV.EXE" file is present in the D: drive which is required for the game to play the MPEG videos. I created a quick patch to workaround this issue (attached theworkaround.png file). The patch is to overwrite three bytes in the "ENTITE.EXE" file at byte offset 0x44c9 with: E9 11 00, then replace this file on the CD-ROM. I have attached the patched EXE file for convenience.
Generally it is not a good idea to workaround problems like this by manually patching an EXE file because it can introduce unforeseen problems in the game. That being said, I have only tested the first video message, but things seem to work OK. Let me know what you find Uka! 😀
Bobbytoz wrote on 2022-04-20, 09:15:I think the problem is in the audio track.
Maybe, but I'm optimistic that it can possibly be fixed in the emulator. 😀
Have you tried LOADFIX? It can cure many forms of seemingly inexplicable program misbehavior. 😀
ripsaw8080 wrote on 2022-04-21, 03:47:Have you tried LOADFIX? It can cure many forms of seemingly inexplicable program misbehavior. 😀
You mean, for fixing that crash with "Entity"?
Sadly no - LOADFIX does not help there. And the game's behaviour is the same on the real hardware - as we already mentioned it here before, the CD image of this game is not original and could be dubious, or this edition could be full of original bugs with the MPEG part not being tested enough (or both).
jrdennisoss wrote on 2022-04-21, 02:40:If the original hardware setup does not do this then yes probably it can be cleaned up. I'll look into this.
Yes, it is all clean on the original hardware! But the artifacts are seen when opening files with a video player, like with those MPEG videos you call 'magic'.
jrdennisoss wrote on 2022-04-21, 02:40:Dully noted.
Well, now I see I was probably wrong: the volume difference is the original bug. You can open files like 4001.mpg and 4010.mpg in any video player - and the second one would be much louder than the first one. I doubt that relates to some "MPEG magic"?
jrdennisoss wrote on 2022-04-21, 02:40:That being said, I have only tested the first video message, but things seem to work OK. Let me know what you find Uka! 😀
Great, thanks!!!
Yes, that first video message now plays fine, and I see no other bad things - at least in the intro, in the menu and in the beginning of the first level.
Unfortunately, as I have already noted, I am not able to test this game to the end or at least to some more levels... But I see no reason to suppose that other messages would not appear, if the first one does work fine now.
However, I have to add that there are no subtitiles for that first video message for some reason. Even in the French version of the game. And, as you know, there are subtitles for the intro video even in the English version. I have not tried it yet on the original hardware (as it would require to re-burn the disk), but I do not think it is the emulator's fault.
jrdennisoss wrote on 2022-04-21, 02:40:Uka wrote on 2022-04-20, 05:08:1) but there are still a couple of distorted lines at the bottom of the picture. Do you think they could be cleaned up?
If the original hardware setup does not do this then yes probably it can be cleaned up. I'll look into this.
To me that looks like very typical VHS head switching noise (from the source video used to create the MPEG), if it's not present when using original hardware I would suspect it's being cropped off somehow rather than being a decoding error.
Uka wrote on 2022-04-21, 05:52:ripsaw8080 wrote on 2022-04-21, 03:47:Have you tried LOADFIX? It can cure many forms of seemingly inexplicable program misbehavior. 😀
the CD image of this game is not original and could be dubious, or this edition could be full of original bugs with the MPEG part not being tested enough (or both).
Yeah that's pretty much why I'm trying the last ditch effort of manually patching the EXE file 🙁 to work around or at least hopefully try to understand the original conditions (if there are any) required to get this game to work. I would hate something this remarkable to be lost in history.
Uka wrote on 2022-04-21, 05:52:Yes, it is all clean on the original hardware! But the artifacts are seen when opening files with a video player, like with those MPEG videos you call 'magic'.
These are indeed "magical MPEG" files. However, the ugliness at the bottom is likely due to something else, not the "magic" aspect of the MPEG files as running these files through the decoder now yields zero errors, but rather the emulator currently ignoring certain commands from the game. Running one of the assets through the "unlock_the_magic" tool (specifying "f_code" of 4) and playing in VLC also yields the same output as the emulator. (attached) I am optimistic that this is fixable. I need to do some tests on the real hardware though to understand some things a bit better.
Uka wrote on 2022-04-21, 05:52:Well, now I see I was probably wrong: the volume difference is the original bug. You can open files like 4001.mpg and 4010.mpg in any video player - and the second one would be much louder than the first one. I doubt that relates to some "MPEG magic"?
Audio stuff for sure is not related to "magical MPEG" as this garbage only happens in the video elementary stream. However, what I meant is that there are some bits in a command that is being sent from the game to the ReelMagic driver that currently the emulator is ignoring and I think these bits might be related to something with audio. It could be the game telling the ReelMagic stuff to do something with volume adjustment or something related. I need to play around with this on the real setup to further understand though.
Uka wrote on 2022-04-21, 05:52:Great, thanks!!!
Yes, that first video message now plays fine, and I see no other bad things - at least in the intro, in the menu and in the beginning of the first level.
Unfortunately, as I have already noted, I am not able to test this game to the end or at least to some more levels... But I see no reason to suppose that other messages would not appear, if the first one does work fine now.
Awesome! We'll it's a start 😀
Uka wrote on 2022-04-21, 05:52:However, I have to add that there are no subtitiles for that first video message for some reason. Even in the French version of the game. And, as you know, there are subtitles for the intro video even in the English version. I have not tried it yet on the original hardware (as it would require to re-burn the disk), but I do not think it is the emulator's fault.
The function that the EXE patch skips over could very well be doing something like loading the subtitles into memory. I may have to dig further into this.
Don't burn it just yet... Looks like when running from the CD on the real hardware it only plays the first few seconds of the video message then things stop responding 🙁 , but game resumes instead of crashing locking up. Going back to the DOS prompt after the first video messages, doing a DIR on the D: drive no longer responds... I wonder if it's sending down a bad MSCDEX command or something like that because I saw some MSCDEX stuff also occurring when I was poking around that area in the DOSBox debugger. I'll have to dig further.
jmarsh wrote on 2022-04-21, 09:51:To me that looks like very typical VHS head switching noise (from the source video used to create the MPEG), if it's not present when using original hardware I would suspect it's being cropped off somehow rather than being a decoding error.
This whole game screams 90s VHS 😀 Yeah I suspect that's exactly what is happening because there are commands from the game that the emulator is currently ignoring that I think tell it to "zoom in" / resize things. Some of the values are '640', '320', '200,' '240' so definitely something related to resolution / picture size. I just need to generate some test MPEG assets with "size markers" in the picture and try giving some test values to see exactly what it does to the picture.
There has been no news for a long time, is everything okay with us?
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Games used to be better!
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Uka wrote on 2022-04-21, 05:52:And the game's behaviour is the same on the real hardware - as we already mentioned it here before, the CD image of this game is not original and could be dubious, or this edition could be full of original bugs with the MPEG part not being tested enough (or both).
OK, here is the true story! It turns out that "Entity" was never officially released on CD: Loriciel used to work on the CD/MPEG version after making the floppy version in 1993 - it was planned for May or June 1994, but the company has closed its doors just then and that version was never published by its authors.
However, in 1997 a French magazine called "PC Collector" somehow got hold of all the materials - and released the new version of "Entity" on one of its companion CDs together with two more games. Hoagie from ABF in 2010 has dumped his disk - and made a custom ISO by removing two other games. That is, there are actually no missing files there, but the game - and especially the MPEG version that "PC Collector" people probably could not test and fix in 1997 - is not exactly 100% finished. So that explains all those bugs...
jmarsh wrote on 2022-04-21, 09:51:To me that looks like very typical VHS head switching noise (from the source video used to create the MPEG), if it's not present when using original hardware I would suspect it's being cropped off somehow rather than being a decoding error.
jrdennisoss wrote on 2022-04-21, 15:43:This whole game screams 90s VHS 😀 Yeah I suspect that's exactly what is happening because there are commands from the game that the emulator is currently ignoring that I think tell it to "zoom in" / resize things. Some of the values are '640', '320', '200,' '240' so definitely something related to resolution / picture size. I just need to generate some test MPEG assets with "size markers" in the picture and try giving some test values to see exactly what it does to the picture.
Yes, I am also quite sure that is indeed the case!
You can see my LOGs - they talk about some 'unoptimized resize' of MPEG 352x240 -> VGA 320x240.
I have now tested "Man Enough"! Well, the available part of it , that is - sadly you can't complete it as only disks 1 and 3 are available. Strangely enough, probably 4/5 of all the game is disk 1 - and then it asks for the disk 2 that is missing for now.
Still, everything is fine! Except for that bottom lines not being resized, of course.
I have also met a bug - it is not critical and it is the original bug, i.e. it happens on the real hardware as well. But I write it here in case it would be possible to fix it!
Sometimes during the gameplay that consists of mostly slide-show pictures and line selecting (with a few MPEG videos here and there), a video with Jeri comments pops up randomly - and when that happens, the previous screen with the subtitles (if you turned them on, of course) is not being refreshed for some reason - so that old screen is visible against a newer screen of lines to choose and even more subtitles. Please see the screens attached!
Bobbytoz wrote on 2022-05-01, 20:04:There has been no news for a long time, is everything okay with us?
On my side: Unfortunately my day job as of recently has been very demanding; to the point where at the end of the day and on the weekends I am just too burned out to focus on something like this. 🙁 Hopefully in less than a month things should die down to the point where I can hack at this project some more though.
The next thing I want to do is get this rebased on to DOSBox trunk and eliminate the "video mixer" by using the proper SDL functions. This should help with both video quality and allow me to implement proper MPEG surface positioning and scaling. Right now, I am just ignoring these values when games pass them down.
Uka wrote on 2022-05-02, 03:07:OK, here is the true story! It turns out that "Entity" was never officially released on CD: Loriciel used to work on the CD/MPEG version after making the floppy version in 1993 - it was planned for May or June 1994, but the company has closed its doors just then and that version was never published by its authors.
However, in 1997 a French magazine called "PC Collector" somehow got hold of all the materials - and released the new version of "Entity" on one of its companion CDs together with two more games. Hoagie from ABF in 2010 has dumped his disk - and made a custom ISO by removing two other games. That is, there are actually no missing files there, but the game - and especially the MPEG version that "PC Collector" people probably could not test and fix in 1997 - is not exactly 100% finished. So that explains all those bugs...
Thanks for the history on this Uka! So is it likely that with that patched EXE and running in the DOSBox ReelMagic emulator this is the most this game has actually ran? When I get more time I would love to dig more into this and actually get it to the point where it can stably run on the real HW too 😀
Uka wrote on 2022-05-02, 03:07:Yes, I am also quite sure that is indeed the case!
You can see my LOGs - they talk about some 'unoptimized resize' of MPEG 352x240 -> VGA 320x240.
The "unoptimized resize" messages are a warning from the ReelMagic emulator code that I put in there as a reminder that this code is not the best way to implement "video mixing" and needs to be redone. (e.g. using the proper SDL APIs)
Looking at yer logs, it's likely this that is causing the problem:
1390683763: REELMAGIC:FMPDRV.EXE Unimplemented 09h: handle=1 subfunc=0409h param1=3201390683763: REELMAGIC:driver_call(09h,01h,409h,140h,C8h)=0h
The "FMPDRV.EXE Unimplemented" is a warning from the ReelMagic emulator code telling us that it is ignoring something the game is telling it and it does not currently know what to do with this information. As you can see, the game is giving us two values for media handle #1 via function 09 / subfunction 0409h: 320 (140h) for param1 and 200 (C8h) for param2. Definitely something to do with display dimensions of 320x200. 😉 When I play with these values on the real HW (Maxima card) the video output dimensions change on screen. Once I do the rebase onto DOSBox SVN trunk, implementing this should be pretty straight forward... I think. 😀
Uka wrote on 2022-05-02, 03:07:I have also met a bug - it is not critical and it is the original bug, i.e. it happens on the real hardware as well. But I write it here in case it would be possible to fix it!
Sometimes during the gameplay that consists of mostly slide-show pictures and line selecting (with a few MPEG videos here and there), a video with Jeri comments pops up randomly - and when that happens, the previous screen with the subtitles (if you turned them on, of course) is not being refreshed for some reason - so that old screen is visible against a newer screen of lines to choose and even more subtitles. Please see the screens attached!
Good to know thanks!
jrdennisoss
I'm glad to hear from you!
It's a pity that you're loaded up to your ears...
but the main thing is that you're in business!
It doesn't matter how long it takes.
The main thing is to keep us informed of your affairs...
I think many support your efforts in this matter!
Take your time, but be productive)
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Games used to be better!
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
jrdennisoss wrote on 2022-05-02, 05:24:So is it likely that with that patched EXE and running in the DOSBox ReelMagic emulator this is the most this game has actually ran? When I get more time I would love to dig more into this and actually get it to the point where it can stably run on the real HW too 😀
Yes, sure - you are indeed the only hope this game ever had 😀
jrdennisoss wrote on 2022-05-02, 05:24:Definitely something to do with display dimensions of 320x200. 😉 When I play with these values on the real HW (Maxima card) the video output dimensions change on screen. Once I do the rebase onto DOSBox SVN trunk, implementing this should be pretty straight forward... I think. 😀
Great, thanks - please take your time!
Hi, maybe stupid request ?
Can you make dosbox option on scale videos (mpeg) using bilinear filtering like OpenGL do it, and maybe a 2d games too on top of reelmagic videos.
I mean when i play mpeg directly from cd it looks better and from dosbox its looks like neighborhood (nearest ) pixelated scaling, and it seems mpeg that played from reelmagic and 2d games has a same upscaled resolution and it like 2 layers on one game mpeg + 2d dos game.
I am sorry i very struggle to explain it.
Just a heads up that for me, and I assume at least for some other people, the most significant and historical breakthrough would be getting Entity fully working. Everything else is certainly a breakthrough in emulation, but with Entity its literally bringing a new version of a game into existence. Hopefully the patch has fixed all crashes.
