DOSBox ECE (for Windows & Linux)

Developer's Forum, for discussion of bugs, code, and other developmental aspects of DOSBox.

Re: DOSBox ECE (for Windows)

Postby AlexRK » 2018-11-11 @ 15:23

krcroft wrote:Unfortunately SDL_Sound's FLAC and MP3 backends have their own issues beyond the seek and get duration calls. (see here for a similar FLAC attempt that Yesterplay80 added back in 2016 but had be backed out of the ECE build due to issues: viewtopic.php?f=32&t=49999)

Here's a working and up-to-date patch if you're interested in FLAC support: viewtopic.php?f=41&t=62203

You also mentioned keeping dosbox simple and elegant, which this patch does by eliminating the need for libFLAC, libMp321, libVorbis, and libVorbisFile, resulting in a much smaller statically linked DOSBox binary.


Frankly, I'm presonally only interested in supporting the WAV format (ISO/WAV/CUE, to be exact). All my mixed-mode CDs dumped exactly in that format - because of the need for accurate copying of audio tracks (I use Exact Audio Copy for this). At present time I cant use my dumps to play in DOSBox directly - I need to turn them into BIN/CUE using special software.

As far as I understand, the WAV format has no additional issues besides Seek/Duration, and does not require additional external libraries.

In my humble opinion, support of the playback of individual tracks, along with support of the single image in the BIN/CUE format, looks quite organic and consistent - because it is, in fact, the same data, but presented in slightly different ways.

It should be noted that dosbox allows using the ISO/OGG/CUE format. But not ISO/WAV/CUE format. :( It's a little weird I think.
AlexRK
Newbie
 
Posts: 13
Joined: 2018-10-29 @ 21:52

Re: DOSBox ECE (for Windows)

Postby krcroft » 2018-11-11 @ 15:41

AlexRK wrote: ...
it should be noted that dosbox allows using the ISO/OGG/CUE format. But not ISO/WAV/CUE format. :( It's a little weird I think.


The referenced working patch also addresses this.

It transparency passes through the CUE's audio filetypes to the matching codec's backend, as opposed to expecting everything to be "MP3".

What happens behind the scenes in the existing DOSBox code is the audio file is force-parsed by SDL's MP3 decoder, regardless if it's actually a wav or ogg file. The MP3 parser eventually fails to handle the file, passes control back to SDL, which then enters a fallback mode where it tries all of its decoders (midi, au, spx, aiff, shn, ...) on the file, in round robin fashion, failure after failure, until a successful hit is found.. or none is found and it gives up.

This patch will correctly match your wav files directly with the WAV decoder from the get-go, provided you construct your CUEs with the proper "WAV" type (as opposed to hardcoding every as the MP3 type).
User avatar
krcroft
Member
 
Posts: 268
Joined: 2017-4-29 @ 15:07
Location: Ogden's Retreat

Re: DOSBox ECE (for Windows)

Postby AlexRK » 2018-11-11 @ 17:06

krcroft wrote:The referenced working patch also addresses this.

It transparency passes through the CUE's audio filetypes to the matching codec's backend, as opposed to expecting everything to be "MP3".


You are talking about this patch, right? viewtopic.php?f=41&t=62203
If I understand correctly, this patch is not incorporated into the DOSBox ECE (yet)?

The latest version of the DOSBox ECE when using the ISO/WAV/CUE format shows these strange numbers:

iso+wav.png


While using the ISO/OGG/CUE format all looks correct:

iso+ogg.png



krcroft wrote:This patch will correctly match your wav files directly with the WAV decoder from the get-go, provided you construct your CUEs with the proper "WAV" type (as opposed to hardcoding every as the MP3 type).


Now my CUE files look like this (example):

FILE "HEROES2X.bin/track01.iso" BINARY
TRACK 01 MODE1/2048
INDEX 01 00:00:00
FILE "HEROES2X.bin/track02.wav" WAVE
TRACK 02 AUDIO
PREGAP 00:02:00
INDEX 01 00:00:00
FILE "HEROES2X.bin/track03.wav" WAVE
TRACK 03 AUDIO
INDEX 01 00:00:00
FILE "HEROES2X.bin/track04.wav" WAVE
TRACK 04 AUDIO
INDEX 01 00:00:00
....


Is this correct CUE representation?
AlexRK
Newbie
 
Posts: 13
Joined: 2018-10-29 @ 21:52

Re: DOSBox ECE (for Windows)

Postby krcroft » 2018-11-11 @ 18:40

AlexRK wrote:You are talking about this patch, right? viewtopic.php?f=41&t=62203
If I understand correctly, this patch is not incorporated into the DOSBox ECE (yet)?


Yes to both.

AlexRK wrote:The latest version of the DOSBox ECE when using the ISO/WAV/CUE format shows these strange numbers:

iso%2Bwav.png

While using the ISO/OGG/CUE format all looks correct:

iso%2Bogg.png



Those WAV/CUE track start times (Minute-Second-Frame, or MSF values) look invalid: typical CDROMs are 70 minutes yet you've got 'M' (minute) values exceeding 200 minutes. No fault of your own.. the code is just broken.

As for the OGG/CUE MSF values: although these are sane and valid (and more than likely happily accepted by your game), I anticipate the OGG/CUE MSFs still won't match the MSFs reported if you mount a BIN/CUE (which for a perfect CDROM dump, will match those of the physical CDROM tracks). Then again, exact or "correct" MSF's are more of a curiosity than a necessity though; I'm only aware that Destruction Derby requires MSFs that match the CDROM.

AlexRK wrote:Now my CUE files look like this (example):
Is this correct CUE representation?


Assuming the above patch is in-place, the WAV/CUE would looks like this (note the "WAV" type):

FILE "HEROES2X.bin/track01.iso" BINARY
TRACK 01 MODE1/2048
INDEX 01 00:00:00
FILE "HEROES2X.bin/track02.wav" WAV
TRACK 02 AUDIO
PREGAP 00:02:00
INDEX 01 00:00:00
FILE "HEROES2X.bin/track03.wav" WAV
TRACK 03 AUDIO
INDEX 01 00:00:00
FILE "HEROES2X.bin/track04.wav" WAV
TRACK 04 AUDIO
INDEX 01 00:00:00
....

And if you want (or need) your WAV/CUE MSF's to match that of the original BIN/CUE, then take the INDEX 01 MM:SS:FF values from the BIN/CUE and drop them in your WAV/CUE in place of the INDEX 01 00:00:00's.
(just an example with numbers I made up):

FILE "HEROES2X.bin/track01.iso" BINARY
TRACK 01 MODE1/2048
INDEX 01 00:00:00
FILE "HEROES2X.bin/track02.wav" WAV
TRACK 02 AUDIO
PREGAP 00:02:00
INDEX 01 08:14:22
FILE "HEROES2X.bin/track03.wav" WAV
TRACK 03 AUDIO
INDEX 01 10:37:59
FILE "HEROES2X.bin/track04.wav" WAV
TRACK 04 AUDIO
INDEX 01 15:21:09
...
User avatar
krcroft
Member
 
Posts: 268
Joined: 2017-4-29 @ 15:07
Location: Ogden's Retreat

Re: DOSBox ECE (for Windows)

Postby AlexRK » 2018-11-11 @ 19:32

krcroft wrote:Assuming the above patch is in-place, the WAV/CUE would looks like this (note the "WAV" type):

And if you want (or need) your WAV/CUE MSF's to match that of the original BIN/CUE, then take the INDEX 01 MM:SS:FF values from the BIN/CUE and drop them in your WAV/CUE in place of the INDEX 01 00:00:00's.


Cool, thanks. I would like to ask is there a possibility to implement of the acceptance of the standard "WAVE" type? Or it is requires a lot of work (and modification of the legacy code, maybe)?
AlexRK
Newbie
 
Posts: 13
Joined: 2018-10-29 @ 21:52

Re: DOSBox ECE (for Windows)

Postby krcroft » 2018-11-11 @ 20:29

AlexRK wrote:
krcroft wrote:Assuming the above patch is in-place, the WAV/CUE would looks like this (note the "WAV" type):

And if you want (or need) your WAV/CUE MSF's to match that of the original BIN/CUE, then take the INDEX 01 MM:SS:FF values from the BIN/CUE and drop them in your WAV/CUE in place of the INDEX 01 00:00:00's.


Cool, thanks. I would like to ask is there a possibility to implement of the acceptance of the standard "WAVE" type? Or it is requires a lot of work (and modification of the legacy code, maybe)?


It's a small change that I can drop into my next revision. You can see the history of it on line 54 here: https://hg.icculus.org/icculus/SDL_soun ... ders/wav.c

"WAV", NULL will become "WAV", "WAVE", NULL.

This also looks valid because both wav and wave are used somewhat interchangeably with the format, extensions, and mimetypes. So you will be able to use .wav as the extension but WAVE as the type specifier.
User avatar
krcroft
Member
 
Posts: 268
Joined: 2017-4-29 @ 15:07
Location: Ogden's Retreat

Re: DOSBox ECE (for Windows)

Postby AlexRK » 2018-11-11 @ 21:08

krcroft wrote:It's a small change that I can drop into my next revision.


It would be great. After that, it remains only to ask Yesterplay80 to include the patch into the ECE. :)

krcroft wrote:So you will be able to use .wav as the extension but WAVE as the type specifier.


Yes, this is exactly what I would like.
AlexRK
Newbie
 
Posts: 13
Joined: 2018-10-29 @ 21:52

Re: DOSBox ECE (for Windows)

Postby Yesterplay80 » 2018-11-14 @ 14:25

AlexRK wrote:It would be great. After that, it remains only to ask Yesterplay80 to include the patch into the ECE. :)

I went a little ahead and made the small change already, so if you download the current ECE for Windows (Linux will follow soon), it should already work.
My full-featured DOSBox SVN builds (without debugger) for Windows: Vanilla DOSBox and DOSBox ECE (Enhanced Community Edition)
User avatar
Yesterplay80
Member
 
Posts: 419
Joined: 2016-2-23 @ 11:02
Location: Germany

Re: DOSBox ECE (for Windows)

Postby AlexRK » 2018-11-14 @ 21:13

Yesterplay80 wrote:I went a little ahead and made the small change already, so if you download the current ECE for Windows (Linux will follow soon), it should already work.


It WORKS!

Thank you so much Yesterplay80 and krcroft!
AlexRK
Newbie
 
Posts: 13
Joined: 2018-10-29 @ 21:52

Re: DOSBox ECE (for Windows)

Postby krcroft » 2018-11-14 @ 22:26

AlexRK wrote:
Yesterplay80 wrote:I went a little ahead and made the small change already, so if you download the current ECE for Windows (Linux will follow soon), it should already work.


It WORKS!

Thank you so much Yesterplay80 and krcroft!


AlexRK, thanks for the quick feedback!
Yesterplay80, thank you also for the testing and adding support to DOSBox ECE!
User avatar
krcroft
Member
 
Posts: 268
Joined: 2017-4-29 @ 15:07
Location: Ogden's Retreat

Re: DOSBox ECE (for Windows)

Postby James-F » 2018-11-17 @ 05:45

I no longer have CD Audio with ECE (4172) build, works fine in Vanilla.

EDIT: fixed, changed MP3 to OGG in .cue file.
Excellent work btw, I use Opus plenty, but not in dosbox yet.
Thank you very much for adding it to ECE.

Code: Select all
FILE "Quake.iso" BINARY
  TRACK 01 MODE1/2048
    INDEX 01 00:00:00
FILE "Track02.ogg" MP3
  TRACK 02 AUDIO
    PREGAP 00:02:00
    INDEX 01 00:00:00
FILE "Track03.ogg" MP3
  TRACK 03 AUDIO
    INDEX 01 00:00:00
FILE "Track04.ogg" MP3
  TRACK 04 AUDIO
    INDEX 01 00:00:00
FILE "Track05.ogg" MP3
  TRACK 05 AUDIO
    INDEX 01 00:00:00
FILE "Track06.ogg" MP3
  TRACK 06 AUDIO
    INDEX 01 00:00:00
FILE "Track07.ogg" MP3
  TRACK 07 AUDIO
    INDEX 01 00:00:00
FILE "Track08.ogg" MP3
  TRACK 08 AUDIO
    INDEX 01 00:00:00
FILE "Track09.ogg" MP3
  TRACK 09 AUDIO
    INDEX 01 00:00:00
FILE "Track10.ogg" MP3
  TRACK 10 AUDIO
    INDEX 01 00:00:00
FILE "Track11.ogg" MP3
  TRACK 11 AUDIO
    INDEX 01 00:00:00
User avatar
James-F
Oldbie
 
Posts: 1448
Joined: 2015-11-30 @ 04:10

Re: DOSBox ECE (for Windows)

Postby AlexRK » 2018-11-17 @ 11:04

Yesterplay80, krcroft, maybe it is worth including support of the OGG files for "MP3" file type for backward compatibility? I personally do not use either OGG or MP3, but many existing CUE's use exactly such format (OGG file + MP3 file type in the CUE). Or is it not so easy?
AlexRK
Newbie
 
Posts: 13
Joined: 2018-10-29 @ 21:52

Re: DOSBox ECE (for Windows)

Postby James-F » 2018-11-17 @ 15:49

I used Notepad++ to "replace all" the MP3 with OGG with a single click, but indeed I had to do that to all my cd audio games.
Now they are not compatible with the Vanilla build, but I use ECE exclusively anyway.

@Yesterplay80
Do you still add patches manually or you use some script to apply them all and compile with one click?
I'd imagine it takes a lot of your time patching them one by one each new SVN.
User avatar
James-F
Oldbie
 
Posts: 1448
Joined: 2015-11-30 @ 04:10

Re: DOSBox ECE (for Windows)

Postby krcroft » 2018-11-17 @ 17:02

AlexRK wrote:Yesterplay80, krcroft, maybe it is worth including support of the OGG files for "MP3" file type for backward compatibility? I personally do not use either OGG or MP3, but many existing CUE's use exactly such format (OGG file + MP3 file type in the CUE). Or is it not so easy?


Given the baseline DOSBox only properly supported OGG Vorbis file-based tracks, I'm unsure why "MP3" was chosen for the filetype. (DOSBox doesn't even handle MP3 tracks...perhaps there's an interesting backstory?)

This patch correctly supports MP3, OGG, FLAC, WAV, and OPUS so the CUE filetype is now critical to send the file to the correct codec. Setting the "MP3" filetype will actually open the track file with the MP3 codec.

The full list of valid CUE filetypes (FILE "filename.ext" FILETYPE) is:
  • MP3, MP2, or MP1 for MP3 file decoding
  • FLAC or FLA for Ogg Flac file decoding
  • WAV or WAVE for Microsoft WAVE decoding
  • OGG or OGA for Ogg Vorbis decoding
  • OPUS for Ogg Opus decoding

We could include a backward-compatibility hack following this logic: if the file extension is ogg but the filetype is MP3, then open the file with the Ogg Vorbis codec.
User avatar
krcroft
Member
 
Posts: 268
Joined: 2017-4-29 @ 15:07
Location: Ogden's Retreat

Re: DOSBox ECE (for Windows)

Postby Qbix » 2018-11-17 @ 17:17

It might make sense to make it more robust than looking only at the extension.
(it would be unfortunate to break an existing setup, just to add a new feature)
Water flows down the stream
How to ask questions the smart way!
User avatar
Qbix
DOSBox Author
 
Posts: 10893
Joined: 2002-11-27 @ 14:50
Location: Fryslan

Re: DOSBox ECE (for Windows)

Postby AlexRK » 2018-11-17 @ 17:48

krcroft wrote:Given the baseline DOSBox only properly supported OGG Vorbis file-based tracks, I'm unsure why "MP3" was chosen for the filetype. (DOSBox doesn't even handle MP3 tracks...perhaps there's an interesting backstory?)


It's weird, yes.

krcroft wrote:We could include a backward-compatibility hack following this logic: if the file extension is ogg but the filetype is MP3, then open the file with the Ogg Vorbis codec.


I think this is a good and the most safe option.
Or maybe it worth to look at the file extension only, as Qbix suggests.
AlexRK
Newbie
 
Posts: 13
Joined: 2018-10-29 @ 21:52

Re: DOSBox ECE (for Windows)

Postby James-F » 2018-11-17 @ 18:28

A quick bug report with ECE (working fine in Vanilla);
When I unmount an ISO and mount another ISO, CD audio no longer works in the same dosbox ece session.

I use this kind of batch file with each cd audio game:
Code: Select all
imgmount -u e
imgmount e .\cd\quake.cue -t iso
quake
imgmount -u e
mount e f:\ -t cdrom


Basically it mounts the cue+iso then launches the game, when the game is quitted, my real cd drive is mounted back.
Launching the next game will have no cd audio as per this bug report.
User avatar
James-F
Oldbie
 
Posts: 1448
Joined: 2015-11-30 @ 04:10

Re: DOSBox ECE (for Windows)

Postby Qbix » 2018-11-17 @ 18:47

AlexRK wrote:
krcroft wrote:Given the baseline DOSBox only properly supported OGG Vorbis file-based tracks, I'm unsure why "MP3" was chosen for the filetype. (DOSBox doesn't even handle MP3 tracks...perhaps there's an interesting backstory?)


It's weird, yes.

krcroft wrote:We could include a backward-compatibility hack following this logic: if the file extension is ogg but the filetype is MP3, then open the file with the Ogg Vorbis codec.


I think this is a good and the most safe option.
Or maybe it worth to look at the file extension only, as Qbix suggests.


I didn't suggest that, I suggested the opposite ;) Like always try ogg
Water flows down the stream
How to ask questions the smart way!
User avatar
Qbix
DOSBox Author
 
Posts: 10893
Joined: 2002-11-27 @ 14:50
Location: Fryslan

Re: DOSBox ECE (for Windows)

Postby AlexRK » 2018-11-17 @ 19:07

Qbix wrote:I didn't suggest that, I suggested the opposite ;) Like always try ogg


Oh sorry! I read your message incorrectly. :(
Do you mean that a DOSBox should read a file [meta]data instead of just looking at the extension? IMHO, it may be enough to rely on the file extension only - for example, foobar2000 works exactly like this. But maybe I don’t understand something.
AlexRK
Newbie
 
Posts: 13
Joined: 2018-10-29 @ 21:52

Re: DOSBox ECE (for Windows)

Postby krcroft » 2018-11-17 @ 21:31

I've updated the FLAC-Opus-MP3 patch to rev5, posted here:
viewtopic.php?f=41&t=62203&p=695839#p695839

JamesF,
Thanks for the bug report regarding no CDDA audio after unmounting a CUE image followed by mounting a CDROM. This should be fixed in this patch. It was a one-line typo.

Good news regarding using the audio file's extension vs the CUE type (FILE "filename.ext" TYPE) - it turns out the code exclusively looks at the file extension and ignores the TYPE.
The valid extensions that map to codec handlers are listed in my post above. It's a case-insensitive match, so lower and uppercase extensions are handled the same.
I tested CUEs with FILE "filename.ogg" MP3, and the Ogg track was opened without issue. I tried on Linux and Windows and couldn't reproduce the issue (maybe try my binary posted in the link above?).

If it doesn't recognize the extension it falls back to trying each codec, essentially relying on the codec to properly determine if the content is valid or invalid. With the new backends being more robust, a busted CUE containing the following might still work:

FILE "my_valid_mp3_but_without_extension" UNDEFINED

or

FILE "flac_track.bye" anything_accepted_here

(and so on).. but I haven't tested it yet.

I've added detection and reporting of invalid Vorbis streams containing Ogg multi-track Skeleton Metadata, which stb_vorbis cannot handle. The following helpful error will appear in the DOSBox status window:
VORBIS: does not support multi-track streams; please re-encode without the Ogg Skeleton bitstream (error code: 38)


This patch now finally includes high-performance, PCM-exact, seek handling for MP3 tracks. Both CBR and VBR are handled equally well, so have fun playing with the latest LAME 3.100 encoder.
If you do try MP3 tracks, the first time you load your CUE in DOSBox there will be a one-time-only delay while fast-seek positions are calculated for all of the CUE's tracks. These are cached in a "fastseek.lut" (lookup table) file, which is written into the current working directory from where DOSBox was launched, typically the game's top-most directory. On subsequent runs, the fast-seek positions will be loaded from the lookup table to avoid the calculation delay.

Thanks for the all the testing and feedback!
User avatar
krcroft
Member
 
Posts: 268
Joined: 2017-4-29 @ 15:07
Location: Ogden's Retreat

PreviousNext

Return to DOSBox Development

Who is online

Users browsing this forum: gottenspell and 2 guests