AUDIO Patch supporting FLAC, Opus, and MP3 audio tracks

Here you can discuss the development of patches.

AUDIO Patch supporting FLAC, Opus, and MP3 audio tracks

Postby krcroft » 2018-9-02 @ 05:34

This patch adds decoding for Opus, FLAC, and MP3 compressed audio tracks; up from the existing Vorbis decoding which DOSBox currently supports.

This top-level post will include:
  • The latest patch and builds for Windows, Linux, and macOSX
  • Links to the latest encoders
  • Suggested compression settings to get the most compression-bang-for-your-bytes
  • Example CUE files for these new codecs
  • Technical discussion of runtime and source improvements

1. Why should we care? or Ogg Vorbis ought to be enough for anybody!

  • All gamers, archivists, and collectors should reap the benefits these codecs offer in DOSBox
  • Opus and FLAC, by the Xiph organization, are currently best-in-class open-licensed codecs, covering the range of lossy to lossless compression. Vorbis and Speex are deprecated.
  • Opus has consistently beaten AAC, High-Efficiency-AAC (HE-AAC), Vorbis+Speex, and MP3 in objective ABX-based listening tests
  • FLAC typically reduces a WAVs filesize by 50% to 70%, and in some cases up to 80% to maximize lossless space savings
  • CPU usage to decode these (instead of Vorbis and WAV) is unchanged, so there are only gains here - we're not trading off anything

When it comes to handling audio content from the golden-era of PC gaming, these games deserve the best we can offer.

2. What's the goal of this patch?

  • I want to work with the core developers to add these codecs to the core DOSBox source, in whatever final form is best for maintainability and the developers
  • This isn't about me - it's about bringing quality audio codecs to DOSBox so everyone can use FLAC and Opus tracks

3. How will this playout given GOG uses Vorbis; isn't this patch unecessary?

  • Those existing GOG releases will continue to work just fine
  • When the stable release of DOSBox supports Opus and FLAC, I suspect GOG will deprecate Vorbis
  • Users ripping their own CDROMs or BIN/CUE pairs will be able to go straight to FLAC or Opus, and not use Vorbis (which is inferior) or WAV (which is space-wasting)
  • Enterprising users who care about audio quality might re-rip their existing collections

4. I've ripped my CDCROM and have a bunch of WAV files ready to compress - where can I get the latest codec compressors?

Opus 1.3:

FLAC 1.3.2:

Vorbis 1.4:

LAME 3.100:

5. What are some recommended settings I can try?

FLAC (all files):
  • flac -8 -e -p -b 512 -P=4096 -S- --output-name="outfile" "wavfile", followed by
  • metaflac --add-seekpoint=1s "flacfile"
Opus, for mono voice-only tracks:
  • opusenc --speech --bitrate 24 --downmix-mono "wavfile" "outfile"
  • Alone in the Dark 2, tracks 23 to 63
  • Alone in the Dark 3, tracks 24 to 61
  • Battle Chess Enhanced, track 22
  • Big Red Racing, tracks 12 to 29, and 33 to 39
  • Jones in the Fast Lane: all tracks
  • Shadowcaster: all tracks
Opus, for stereo voice-only tracks:
  • opusenc --speech --bitrate 36 "wavfile" "outfile"
  • Action Soccer, tracks 2 to 13
  • AD&D Dark Sun II Wake of the Ravager, tracks 38 and 39
  • Loom: all tracks
Opus for mono non-voice tracks:
  • opusenc --bitrate 54 --downmix-mono "wavfile" "outfile"
  • Destruction Derby, tracks 2, and 4 to 9
  • Stellar 7, tracks 4 and 5
  • Zyclunt, all tracks
Opus for stereo non-voice tracks:
  • opusenc --bitrate 84 "wavfile" "outfile"

6. How should my CUE files look?

Here's a CUE with examples for each codec:

Code: Select all
FILE "data.iso" BINARY
  TRACK 01 MODE1/2048
    INDEX 01 00:00:00

FILE "track02.opus" OPUS
  TRACK 02  AUDIO
    PREGAP 00:02:00
    INDEX 01 00:00:00

FILE "track03.flac" FLAC
  TRACK 03 AUDIO
    INDEX 01 00:00:00
   
FILE "track04.ogg" OGG
  TRACK 04 AUDIO
    INDEX 01 00:00:00

FILE "track05.mp3" MP3
  TRACK 05 AUDIO
    INDEX 01 00:00:00


7. What runtime improvements have been made?

  • Replaces the existing audio callback routine with an efficient chunked circular-buffer audio reader
  • Replaces assumptions that all audio tracks are 44.1 kHz & stereo. The mixer is now fed data at the actual compressed track's rate and channel count
  • Eliminates all SDL locks in the audio code in favour of mixer state control
  • Fixes the CDROM double-load bug (https://sourceforge.net/p/dosbox/bugs/488/), now using ripsaw8080's one-line preemptive fix
  • Queries the codec for track-length instead of using hundreds of iterative decimating seeks to determine track length (loading a 99-track CUE now takes 0.1 user-seconds versus 3+ seconds previously)
  • Adds handling for exact track positions via MSF values if specified in the cue file (https://sourceforge.net/p/dosbox/bugs/489/)
  • The Opus decoder seek within its buffer (if possible) before actually moving the file pointer and re-running the decode sequence to get a different chunk of data; an optimization not found in the other codecs
  • SDL_Sound's buffer-size is now set once and never resized, which eliminates repeated re-malloc'ing in the library
  • Only one seek is performed per-playback sequence followed by sequential decodes, similar to a physical CDROM (current code reads 2352-byte chunks and includes seek calculations for every read)
  • The DOSBox mixer channel is now only active during playback sequences, and is no longer left active and fed with silence, which reduce call overhead and audio latency in the mixer
  • When using Opus audio tracks, and if your dosbox-SVN.conf [mixer] rate=48000, then you will (very likely) achieve sample-exact unadulterated pass-through along your entire audio chain from the decoder, to DOSBox, to your operating system's software mixer, to your sound card driver, to your sound card, to your speakers, or to your digital receiver / USB speakers/headphones / or HDMI TV/screen. This is because almost all modern hardware DACs use a native sample rate of 48000

8. What source-level maintenance improvements have been made?
  • Eliminated external dependence on SDL_Sound and its mess of external library dependencies -- thank you Dominus and DosFreak for recommending this approach.
  • SDL_Sound, Vorbis, FLAC, Opus, and Ogg libraries are not needed on your system nor do you need to build or provide pointers to these in CFLAGS/LIBS/etc..
  • All codec handling is 100% internal DOSBox's to source and "always on"
  • It strips all references to SDL_Sound from configure.ac
  • It strips all pre-processor #ifdef branching for SDL_Sound from the code
  • For Vorbis decoding, we now use the single-header stb_vorbis.h instead of of libogg, libvorbis, and libvorbisfile
  • For MP3 decoding, we now use the single-header dr_mp3.h instead of libmp123
  • For FLAC decoding, we now use the single-header dr_flac.h instead of libogg and libFLAC
  • It adds src/libs/decoders, which holds the top-level opus, flac, vorbis, and wav decoders
  • It adds src/libs/decoders/internal, where the opus-dependencies are automatically fetched, built, and linked as part of DOSBox's internal libraries
  • Fixes all codec compiler warnings; 100% clean builds now
  • Build-tested on Linux, MacOSX (xcode), and Windows (MinGW msys 1.0)

The Makefile in src/libs/decoders/internal fetches Opus's dependencies using curl, so that should be in your PATH.
  • If you're on MinGW MSYS 1.0, I would suggest installing Git For Windows and allowing it to add itself to your PATH, which gives you a modern curl build
  • If you're on MinGW-32/64 or MSYS 2.0, then you might already have curl or can use their package manager to install it
  • macOS and Linux come with curl, so no action needed

9. What about SDL_Sound 1.0.3, can't we sit back and wait for the SDL Developers to add Opus and fix its broken MP3 and FLAC decoding?

SDL Sound 1.0.3 hasn't been touched in over a decade and has many bugs and autotool issues within modern environments. The developers are unyeilding on going back and fixing SDL1 libraries to accomodate issues cropping up in modern environments. For them, it's SDL2 or the highway. Example: https://forums.libsdl.org/viewtopic.php?p=45264 (that was 4 years ago, and we're still at SDL 1.2.15)

Ryan Gordon informed me he will use my Opus decoder to add Opus handling to this new SDL2-based sound libraries, however DOSBox would have to move to SDL2 to get it.

This patch brings audio-codec handling under our own control, strips it down to the bare minimum, and corrects all deficiencies.

If and when DOSBox decides to move to SDL2, we can decide then if we want to move back to dependence on SDL2_Sound.
You do not have the required permissions to view the files attached to this post.
Last edited by krcroft on 2018-11-17 @ 20:59, edited 15 times in total.
User avatar
krcroft
Member
 
Posts: 112
Joined: 2017-4-29 @ 15:07
Location: Ogden's Retreat

Re: AUDIO Patch for FLAC and Ogg Opus support (SDL2)

Postby Qbix » 2018-9-06 @ 12:20

would there be an easy way to remove all formatting changes that you did ? (familiar with git commits/patches)
As those don't belong in this patch.

I have to test the frame sized decoding, as the current ogg decoding can give a lot of stuttering I noticed..
Water flows down the stream
How to ask questions the smart way!
User avatar
Qbix
DOSBox Author
 
Posts: 10680
Joined: 2002-11-27 @ 14:50
Location: Fryslan

Re: AUDIO Patch for FLAC and Ogg Opus support (SDL2)

Postby krcroft » 2018-9-06 @ 13:51

Qbix wrote:would there be an easy way to remove all formatting changes that you did ? (familiar with git commits/patches)
As those don't belong in this patch.

I have to test the frame sized decoding, as the current ogg decoding can give a lot of stuttering I noticed..


Yes, I will tidy up the diff today!
User avatar
krcroft
Member
 
Posts: 112
Joined: 2017-4-29 @ 15:07
Location: Ogden's Retreat

Re: AUDIO Patch for FLAC and Ogg Opus support (SDL2)

Postby krcroft » 2018-9-08 @ 22:45

Qbix,

Here's a non-SDL2, non-SDL_Mixer-X patch that focuses just on the core cdrom_image.cpp changes, so you can test on your side:
  • 16KB circular buffer used to feed the callbacks
  • 4KB chunked reads from SDL_Sound (recommended by the vorbis project)
  • SDL audio locking and unlocking has been eliminated
  • SDL_Sound's buffer-size is now set once and never resized (to avoid repeated re-malloc'ing in the library)
  • Only one seek is performed per-playback sequence; all remaining calls to SDL sound are sequential decodes (similar to a physical CDROM)
  • The DOSBox mixer channel is now only active during playback sequences, and is no longer left active and fed with silence (to reduce call overhead and audio latency in the mixer)

Highlighted diff is here: https://gitlab.com/krcroft/dosbox_audio ... expanded=1
Last edited by krcroft on 2018-11-05 @ 20:09, edited 3 times in total.
User avatar
krcroft
Member
 
Posts: 112
Joined: 2017-4-29 @ 15:07
Location: Ogden's Retreat

Re: AUDIO Patch for FLAC and Ogg Opus support (SDL2)

Postby krcroft » 2018-9-08 @ 22:54

If you comment-in the three 'printStatus' calls in the callback, it will show the behavior of the buffer.

If the 16KB circular buffer doesn't have enough data to satisfy the callback, it attempts to fill itself using chunked-reads from SDL_Sound.

We see four of these reads initially here, followed by lots tiny callbacks that slowly deplete the buffer.

Code: Select all
 25% [|||||||||||||||                                             ] - fill
 50% [||||||||||||||||||||||||||||||                              ] - fill
 74% [||||||||||||||||||||||||||||||||||||||||||||                ] - fill
 99% [||||||||||||||||||||||||||||||||||||||||||||||||||||||||||| ] - fill
 98% [||||||||||||||||||||||||||||||||||||||||||||||||||||||||||| ] - consume
 97% [||||||||||||||||||||||||||||||||||||||||||||||||||||||||||  ] - consume
 96% [|||||||||||||||||||||||||||||||||||||||||||||||||||||||||   ] - consume
 95% [|||||||||||||||||||||||||||||||||||||||||||||||||||||||||   ] - consume
 94% [||||||||||||||||||||||||||||||||||||||||||||||||||||||||    ] - consume
 93% [|||||||||||||||||||||||||||||||||||||||||||||||||||||||     ] - consume
 92% [|||||||||||||||||||||||||||||||||||||||||||||||||||||||     ] - consume
 91% [||||||||||||||||||||||||||||||||||||||||||||||||||||||      ] - consume
 90% [||||||||||||||||||||||||||||||||||||||||||||||||||||||      ] - consume
 89% [|||||||||||||||||||||||||||||||||||||||||||||||||||||       ] - consume
 87% [||||||||||||||||||||||||||||||||||||||||||||||||||||        ] - consume
 86% [||||||||||||||||||||||||||||||||||||||||||||||||||||        ] - consume
 85% [|||||||||||||||||||||||||||||||||||||||||||||||||||         ] - consume
 84% [||||||||||||||||||||||||||||||||||||||||||||||||||          ] - consume
 83% [||||||||||||||||||||||||||||||||||||||||||||||||||          ] - consume
 82% [|||||||||||||||||||||||||||||||||||||||||||||||||           ] - consume
 81% [||||||||||||||||||||||||||||||||||||||||||||||||            ] - consume
 80% [||||||||||||||||||||||||||||||||||||||||||||||||            ] - consume
 79% [|||||||||||||||||||||||||||||||||||||||||||||||             ] - consume
 78% [|||||||||||||||||||||||||||||||||||||||||||||||             ] - consume
 77% [||||||||||||||||||||||||||||||||||||||||||||||              ] - consume
 76% [|||||||||||||||||||||||||||||||||||||||||||||               ] - consume
 75% [|||||||||||||||||||||||||||||||||||||||||||||               ] - consume
 74% [||||||||||||||||||||||||||||||||||||||||||||                ] - consume
 73% [|||||||||||||||||||||||||||||||||||||||||||                 ] - consume
 71% [|||||||||||||||||||||||||||||||||||||||||||                 ] - consume
 70% [||||||||||||||||||||||||||||||||||||||||||                  ] - consume
 69% [|||||||||||||||||||||||||||||||||||||||||                   ] - consume
 68% [|||||||||||||||||||||||||||||||||||||||||                   ] - consume
 67% [||||||||||||||||||||||||||||||||||||||||                    ] - consume
 66% [|||||||||||||||||||||||||||||||||||||||                     ] - consume
 65% [|||||||||||||||||||||||||||||||||||||||                     ] - consume
 64% [||||||||||||||||||||||||||||||||||||||                      ] - consume
 63% [||||||||||||||||||||||||||||||||||||||                      ] - consume
 62% [|||||||||||||||||||||||||||||||||||||                       ] - consume
 61% [||||||||||||||||||||||||||||||||||||                        ] - consume
 60% [||||||||||||||||||||||||||||||||||||                        ] - consume
 59% [|||||||||||||||||||||||||||||||||||                         ] - consume
 58% [||||||||||||||||||||||||||||||||||                          ] - consume
 56% [||||||||||||||||||||||||||||||||||                          ] - consume
 55% [|||||||||||||||||||||||||||||||||                           ] - consume
 54% [||||||||||||||||||||||||||||||||                            ] - consume
 53% [||||||||||||||||||||||||||||||||                            ] - consume
 52% [|||||||||||||||||||||||||||||||                             ] - consume
 51% [||||||||||||||||||||||||||||||                              ] - consume
 50% [||||||||||||||||||||||||||||||                              ] - consume
 49% [|||||||||||||||||||||||||||||                               ] - consume
 48% [|||||||||||||||||||||||||||||                               ] - consume
 47% [||||||||||||||||||||||||||||                                ] - consume
 46% [|||||||||||||||||||||||||||                                 ] - consume
 45% [|||||||||||||||||||||||||||                                 ] - consume
 44% [||||||||||||||||||||||||||                                  ] - consume
 42% [|||||||||||||||||||||||||                                   ] - consume
 41% [|||||||||||||||||||||||||                                   ] - consume
 40% [||||||||||||||||||||||||                                    ] - consume
 39% [|||||||||||||||||||||||                                     ] - consume
 38% [|||||||||||||||||||||||                                     ] - consume
 37% [||||||||||||||||||||||                                      ] - consume
 36% [|||||||||||||||||||||                                       ] - consume
 35% [|||||||||||||||||||||                                       ] - consume
 34% [||||||||||||||||||||                                        ] - consume
 33% [|||||||||||||||||||                                         ] - consume
 32% [|||||||||||||||||||                                         ] - consume
 31% [||||||||||||||||||                                          ] - consume
 30% [||||||||||||||||||                                          ] - consume
 28% [|||||||||||||||||                                           ] - consume
 27% [||||||||||||||||                                            ] - consume
 26% [||||||||||||||||                                            ] - consume
 25% [|||||||||||||||                                             ] - consume
 24% [||||||||||||||                                              ] - consume
 23% [||||||||||||||                                              ] - consume
 22% [|||||||||||||                                               ] - consume
 21% [||||||||||||                                                ] - consume
 20% [||||||||||||                                                ] - consume
 19% [|||||||||||                                                 ] - consume
 18% [||||||||||                                                  ] - consume
 17% [||||||||||                                                  ] - consume
 16% [|||||||||                                                   ] - consume
 15% [|||||||||                                                   ] - consume
 14% [||||||||                                                    ] - consume
 12% [|||||||                                                     ] - consume
 11% [|||||||                                                     ] - consume
 10% [||||||                                                      ] - consume
  9% [|||||                                                       ] - consume
  8% [|||||                                                       ] - consume
  7% [||||                                                        ] - consume
  6% [|||                                                         ] - consume
  5% [|||                                                         ] - consume
  4% [||                                                          ] - consume
  3% [||                                                          ] - consume
  2% [|                                                           ] - consume
  1% [                                                            ] - consume
  0% [                                                            ] - consume
  0% [                                                            ] - wrap
 24% [||||||||||||||                                              ] - fill
 49% [|||||||||||||||||||||||||||||                               ] - fill
 48% [|||||||||||||||||||||||||||||                               ] - consume
 47% [||||||||||||||||||||||||||||                                ] - consume
 46% [|||||||||||||||||||||||||||                                 ] - consume
 45% [|||||||||||||||||||||||||||                                 ] - consume
 44% [||||||||||||||||||||||||||                                  ] - consume
 43% [||||||||||||||||||||||||||                                  ] - consume
 42% [|||||||||||||||||||||||||                                   ] - consume
 41% [||||||||||||||||||||||||                                    ] - consume
 40% [||||||||||||||||||||||||                                    ] - consume
 39% [|||||||||||||||||||||||                                     ] - consume
 38% [||||||||||||||||||||||                                      ] - consume
 36% [||||||||||||||||||||||                                      ] - consume
 35% [|||||||||||||||||||||                                       ] - consume
 34% [||||||||||||||||||||                                        ] - consume
 33% [||||||||||||||||||||                                        ] - consume
 32% [|||||||||||||||||||                                         ] - consume
 31% [||||||||||||||||||                                          ] - consume
 30% [||||||||||||||||||                                          ] - consume
 29% [|||||||||||||||||                                           ] - consume
 28% [|||||||||||||||||                                           ] - consume
 27% [||||||||||||||||                                            ] - consume
 26% [|||||||||||||||                                             ] - consume
 25% [|||||||||||||||                                             ] - consume
 24% [||||||||||||||                                              ] - consume
 23% [|||||||||||||                                               ] - consume
 21% [|||||||||||||                                               ] - consume
 20% [||||||||||||                                                ] - consume
 19% [|||||||||||                                                 ] - consume
 18% [|||||||||||                                                 ] - consume
 18% [||||||||||                                                  ] - consume
 18% [||||||||||                                                  ] - stop
StopAudio -- CDROM audio channel disabled
CDROM audio channel freed


During that second round of fills (which only partially filled the buffer), it only needed two chunks to satisfy the remaining playback duration.
User avatar
krcroft
Member
 
Posts: 112
Joined: 2017-4-29 @ 15:07
Location: Ogden's Retreat

Re: AUDIO Patch for FLAC and Ogg Opus support

Postby krcroft » 2018-9-13 @ 04:41

Windows binaries attached in top-post, including latest version of the audio encoders:
- DOSBox r4160 (latest) with ability to play FLAC and Opus audio tracks (in addition to improved Vorbis decoding) from BIN/CUE files.
- FLAC 1.3.2 encoder
- Opus 1.3-rc and encoding tools 0.1.10-2
- Vorbis 1.3.6 + aoTuV b6.03 + Lancer encoder

in your CUE files, you can now include opus and flac files, with the following example format:

Code: Select all
FILE "data.iso" BINARY
  TRACK 01 MODE1/2048
    INDEX 01 00:00:00

FILE "audio02.ogg" OGG
  TRACK 02 AUDIO
    PREGAP 00:02:00
    INDEX 01 00:00:00

FILE "track03.opus" OPUS
  TRACK 03 AUDIO
    PREGAP 00:02:00
    INDEX 01 00:00:00

FILE "track04.flac" FLAC
  TRACK 04 AUDIO
    PREGAP 00:02:00
    INDEX 01 00:00:00


Attached are the two patches used to create this:
- dosbox-r4160-audio_updates.patch
- sdl_sound-1.0.3-opus_flac_stb_vorbis.patch

The DOSBox patch applies to r4160 (-p1), and includes:
- Lock-free chunked reads from SDL_Audio
- Direct pass-through of the audio-track's channels and sampling rate to the mixer
This allows for interpolation-free playback if your files match your mixer sampling rate.
The channel is adjusted at playback, so it's fine to have a cue sheet with mono and stereo files, as well as different sampling rates.
- Fix for the double-loading of CD image's

The SDL_Sound patch applies to SDL_Sound-1.0.3 (-p1), and includes:
- Ogg Opus decoder routines (new)
- Improved Dr.FLAC decoder routines (ported from upstream)
- Improved STB Vorbis decoder routines (ported from upstream)

Meanwhile, SDL_Sound upstream is being renovated (SDL2.0 exclusive, autotools ripped out and replaced w/ cmake, decoders are being replaced and deprecated).

Changelog:
- rev2: fixed raw BIN audio playback (which doesn't involve audio files).
Last edited by krcroft on 2018-11-05 @ 20:09, edited 4 times in total.
User avatar
krcroft
Member
 
Posts: 112
Joined: 2017-4-29 @ 15:07
Location: Ogden's Retreat

Re: AUDIO Patch for FLAC and Ogg Opus support

Postby Qbix » 2018-9-13 @ 06:55

let me move it to the patches forum.
I have to check the changes in more detail when I finally find that dynrec bug that has been haunting me.
Water flows down the stream
How to ask questions the smart way!
User avatar
Qbix
DOSBox Author
 
Posts: 10680
Joined: 2002-11-27 @ 14:50
Location: Fryslan

Re: AUDIO Patch for FLAC and Ogg Opus support

Postby krcroft » 2018-9-14 @ 04:00

Qbix wrote:let me move it to the patches forum.
I have to check the changes in more detail when I finally find that dynrec bug that has been haunting me.

Qbix, please disregard that first DOSBox patch; it broken BIN-audio playback (not involving audio files).
I've attached "dosbox-r4160-audio_updates-rev2.patch" above which corrects it using TrackFile polymorphism (included in patch in top-post).
All of the improvements made in on the AudioFile playback side now similarly apply when playing audio from a BinaryFile.
Last edited by krcroft on 2018-11-05 @ 20:11, edited 2 times in total.
User avatar
krcroft
Member
 
Posts: 112
Joined: 2017-4-29 @ 15:07
Location: Ogden's Retreat

Re: AUDIO Patch for FLAC and Ogg Opus support

Postby leileilol » 2018-10-12 @ 01:47

A big problem with Opus for this is the fact it only supports 48khz, which really isn't tbe most desirable (or "recommended") way dealing with 44.1khz CD audio.

FLAC support's way more realistic however.
Voodoo2s aren't 100mhz stock
Geforce256 isn't released as a beta on New Years '99 under the Quadro brand
DOS gaming isn't a bilinear 320x200 16:10
DOS PCs aren't better than the Macintosh
DOSBox is not for running Windows 9x
SGL != Glide
User avatar
leileilol
l33t++
 
Posts: 9428
Joined: 2006-12-16 @ 18:03

Re: AUDIO Patch for FLAC and Ogg Opus support

Postby krcroft » 2018-10-12 @ 05:00

When it comes to lossy audio, you can't do better than Opus - it beats AAC, High-Efficiency-AAC, Ogg Vorbis, and MP3 across multiple ABX-based listening tests. The quality-gap keeps widening as new versions are released (we're only months away from Opus 1.3 being released).

There are no drawbacks to encoding your 44.1 kHz DOS audio tracks to Opus; assuming you use this patch. Yes, Opus up-samples all inputs internally to 48 kHz using the SpeexDSP resampler, which introduces a mathematically measurable difference of -90dB, which is beneath the noise floor of a CDROM. So this is a non-issue.

Besides getting the best quality-for-your-bytes, there are other minor technical advantages with this patch, too:

The patch makes DOSBox read compressed audio in large chunks into a 16KB circular buffer; likewise the Opus decoder for SDL1 Sound (https://gitlab.com/krcroft/sdl_sound/bl ... ers/opus.c) reads data in exact Opus-framed chunks (also into a circular buffer); both efficiency improvements combine to result in roughly an order-of-magnitude fewer function calls per second and a modest reduction in CPU usage when I've profile it against the existing DOSBox + Vorbis code.

My Opus decoder is also the only SDL decoder that will seek within its buffer (if possible) before actually moving the file pointer and re-running the decode sequence to get a different chunk of data.

Compare that to the existing SDL1 Ogg Vorbis decoder: it doesn't use a circular buffer and will make repeated read calls inefficiently across Vorbis frame boundaries to fill the requested buffer. Likewise, the existing DOSBox code reads compressed audio in unnecessarily small CDROM Redbook 2352-byte chunks, which does not translate to any compressed stream frame-size (FLAC for example uses 8KB frames).

Another minor advantage is the fact that almost every consumer hardware audio DACs' native sampling rate is 48 kHz: this includes the your audio card chipset, USB headphones, A/V digital receiver, and your HDMI TV input.
- By setting DOSBox's mixer to 48 kHz and using Opus audio tracks, those samples pass-through the entire software and hardware chain as-is.
- However, by using 44.1 kHz audio tracks, you guarantee your audio is going to be re-sampled one or more times, possibly by DOSBox and/or again by your audio hardware.
- But using 48 kHz straight-through, it removes cheap hardware DACs and DOSBox from the resampling equation.
Last edited by krcroft on 2018-10-12 @ 21:40, edited 6 times in total.
User avatar
krcroft
Member
 
Posts: 112
Joined: 2017-4-29 @ 15:07
Location: Ogden's Retreat

Re: AUDIO Patch for FLAC and Ogg Opus support

Postby krcroft » 2018-10-12 @ 05:13

Attached are ready-to-run builds tracking DOSBox's SVN r1468 source (see top-post)

To compress your WAV tracks, grab the latest Opus encoder here: https://hydrogenaud.io/index.php/topic, ... #msg963060

To see the power of Opus in DOSBox:

Encode the Jones in the Fast Lane 373 MB audio track to Opus:

opusenc --speech --bitrate 24 --downmix-mono track02.wav track02.opus

This will produce a 6.4 MB track with little to no discernible audio fidelity difference versus the original.

Set DOSBox's mixer rate to 48000 (although this isn't necessary), and use a CUE file like:

Code: Select all
FILE "data.iso" BINARY
  TRACK 01 MODE1/2048
    INDEX 01 00:00:00
FILE "track02.opus" OPUS
  TRACK 02 AUDIO
    PREGAP 00:02:00
    INDEX 01 00:00:00


Enjoy!
Last edited by krcroft on 2018-11-05 @ 20:11, edited 4 times in total.
User avatar
krcroft
Member
 
Posts: 112
Joined: 2017-4-29 @ 15:07
Location: Ogden's Retreat

Re: AUDIO Patch for FLAC and Ogg Opus support

Postby krcroft » 2018-10-12 @ 05:28

To use FLAC:

Code: Select all
FILE "data.iso" BINARY
  TRACK 01 MODE1/2048
    INDEX 01 00:00:00
FILE "track02.flac" FLAC
  TRACK 02 AUDIO
    PREGAP 00:02:00
    INDEX 01 00:00:00
User avatar
krcroft
Member
 
Posts: 112
Joined: 2017-4-29 @ 15:07
Location: Ogden's Retreat

Re: AUDIO Patch for FLAC and Ogg Opus support

Postby Dominus » 2018-10-12 @ 07:07

Out of interest, is your patch now independent of SDL-mixer X (? - never heard about X before this thread) and SDL-Sound? And if not, what does it patch? The SDL satellite libs and DOSBox or only DOSBox?
I got confused by the posts ;)

(A patch to directly work with mp3/ogg/opus/flac borrowing from the simple headers the new SDL-Sound uses would be perfect IMHO - I never liked either SDL-Sound or -Mixer - in Exult we long ago got rid of SDL-Mixer)
User avatar
Dominus
DOSBox Moderator
 
Posts: 7674
Joined: 2002-10-03 @ 09:54
Location: Ludwigsburg

Re: AUDIO Patch for FLAC and Ogg Opus support

Postby krcroft » 2018-10-12 @ 21:12

Dominus wrote:Out of interest, is your patch now independent of SDL-mixer X (? - never heard about X before this thread) and SDL-Sound? And if not, what does it patch? The SDL satellite libs and DOSBox or only DOSBox?


Sorry for the confusion! Completely understandable given I started the thread talking about SDL2 and Mixer-X.

Yes, the patch is independent of SDL-Mixer-X and SDL2; I've since transitioned back to pure SDL1.

First patch is to SDL1_Sound 1.0.3:
- Updates auto-tools configs to current standards
- Fixes all compiler warnings; 100% clean builds now
- Renames all 'ogg' instances to Vorbis (which was a desired "TODO" item listed by the SDL team)
- Replaces the broken FLAC decoder with the header-only Dr. FLAC - eliminating the need for libFLAC
- Replaces the Vorbis decoder with the header-only STB Vorbis - eliminating the need for libvorbis and vorbisfile
- Adds my Opus decoder, which depends on libopusfile, libspeexdsp, libopus, and libogg
Ryan Gordon mentioned he's pulling my Opus decoder into the new SDL(2)_Sound which he plans to use until a BSD-licensed pure-header only-decoder is developed.
That said, the SDL team hasn't responded to my SDL-Sound mailing-list posts so it seems 1.0.3 is end-of-the-line, and Ryan Gordon is going 100% SDL2 now.

Second patch is to DOSBox:
- Replaces the existing audio callback routine with the the chunked circular-buffer audio reader
- Replaces assumptions that the audio track is 44.1 kHz - calculations now use whatever the rate the track actually is
- Eliminates all SDL locks in the audio code in favour of mixer state control
- Fixes the double-load bug (https://sourceforge.net/p/dosbox/bugs/488/)
- Adds a fast track-length call (determined by configure.ac if SDL_Sound supports "getDuration"), eliminating the need for iterative seeks to determine track length.
(loading a 99-track CUE now takes 0.1 user-seconds versus 3+ seconds previously)
- Adds handling for exact track positions via MSF values if specified in the cue file (https://sourceforge.net/p/dosbox/bugs/489/)

(A patch to directly work with mp3/ogg/opus/flac borrowing from the simple headers the new SDL-Sound uses would be perfect IMHO - I never liked either SDL-Sound or -Mixer - in Exult we long ago got rid of SDL-Mixer)


Completely agree.. it would be ideal to strip down this patched SDL_Sound to its essential Opus-FLAC-Vorbis source files, throw away everything else, and simply package it in with DOSBox and make it our own.
Every build would have Vorbis and FLAC out of the box - we wouldn't need a bunch of #IFDEF's riddling the code to check for SDL_Sound. This would retain compatibility with legacy GOG releases using Vorbis-encoded tracks, while distributions and expeditious users looking for the highest audio fidelity would take advantage of Opus-encoded tracks (or FLAC).
Last edited by krcroft on 2018-11-05 @ 20:11, edited 2 times in total.
User avatar
krcroft
Member
 
Posts: 112
Joined: 2017-4-29 @ 15:07
Location: Ogden's Retreat

Re: AUDIO Patch for FLAC and Ogg Opus support

Postby Dominus » 2018-10-12 @ 21:17

Many thanks for clarifying. Very impressive work. Very
User avatar
Dominus
DOSBox Moderator
 
Posts: 7674
Joined: 2002-10-03 @ 09:54
Location: Ludwigsburg

Re: AUDIO Patch for FLAC and Ogg Opus support

Postby krcroft » 2018-10-12 @ 21:27

Dominus wrote:Many thanks for clarifying. Very impressive work.


Thanks Dominus! Hey - it was your very helpful build-guilds that got me off the ground; thank you!
User avatar
krcroft
Member
 
Posts: 112
Joined: 2017-4-29 @ 15:07
Location: Ogden's Retreat

Re: AUDIO Patch for FLAC and Ogg Opus support

Postby DosFreak » 2018-10-12 @ 21:49

Anything that cleans up the SDL_Sound mess I'm all for. In my guides I encountered some oddities and it would simplify the guide alot, it's also a pain troubleshooting when SDL_Sound can't find the other libraries.

Anyone test to make sure it still compiles with Mingw w/ Msys 1? Had to manually update some of the tools in the Windows ver of Mingw since the Mingw site downloads old ver that have issues when compiling the various libraries DOSBox uses, no issue with Linux since the repos keep those tools up to date. I have an up to date Mingw/Msys 1 on my Google Drive so will give it a shot next week. https://drive.google.com/drive/folders/ ... 3aMO8U9mvV

Reminds me I need to contact the SDL_net devs about adding a def or ver check to remove a dependency added in the last ver of 1.2.8 that most people don't need.
"Added SDLNet_GetLocalAddresses() to query local interfaces"

Even if you use WSOCK instead of ws2_32 it still tries to use a dll it doesn't even need and if your program doesn't use that functionality (DOSBox) it still needs the dependency.
User avatar
DosFreak
l33t++
 
Posts: 9935
Joined: 2002-6-30 @ 16:35
Location: Your Head

Re: AUDIO Patch for FLAC and Ogg Opus support

Postby krcroft » 2018-10-12 @ 22:35

mingw32-msys1.0-gcc.txt
DosFreak wrote:Anyone test to make sure it still compiles with Mingw w/ Msys 1?


Yup, the patched SDL_Sound builds clean under MinGW and Msys 1.0 (attached are my configure, make, and gcc -v logs).

Glad to hear more people want to do away SDL_Sound and all its dependency baggage - I'll start gutting what I have, and folding the remaining useful bits into DOSBox's source.
You do not have the required permissions to view the files attached to this post.
User avatar
krcroft
Member
 
Posts: 112
Joined: 2017-4-29 @ 15:07
Location: Ogden's Retreat

Re: AUDIO Patch for FLAC and Ogg Opus support

Postby krcroft » 2018-10-14 @ 18:24

This patch makes compressed-audio handing internal to DOSBox.
- It adds src/libs/decoders, which holds the top-level opus, flac, vorbis, and wav decoders
- It adds src/libs/decoders/internal, where the opus-dependencies are fetched and built
- It strips all references to SDL_Sound from configure.ac
- It strips all the #IFDEFs for SDL_Sound from the code
- SDL_Sound, Vorbis, FLAC, Opus, and Ogg libraries are not needed on your system nor do you need to build or provide pointers to these in CFLAGS/LIBS/etc..

The Makefile in src/libs/decoders/internal fetches Opus's dependencies using curl, so that should be in your PATH.
- If you're on MinGW MSYS 1.0, I would suggest installing Git For Windows and allowing it to add itself to your PATH, which gives you a modern curl build
- If you're on MinGW-32/64 or MSYS 2.0, then you might already have curl or can use their package manager to install it
- This isn't a requirement though: you can also simply download the gzip's and place them in src/libs/decoders/internal, which the Makefile will happily use

Dominus, DosFreak - thanks for the suggestion.

This is the right move to take an improved SDL_Sound internally given it's now whittled down to down to handful of files and the fact the public package has been unmaintained for over 10 years now. If and when DOSBox moves to SDL2 - then we can drop this.

The patch output gives a nice overview of its structure (see patch in top-post):

Code: Select all
patch -p1 < ~/dosbox-r4168-internal_decoders.patch
patching file configure.ac
patching file Makefile.am
patching file src/dos/cdrom.h
patching file src/dos/cdrom_image.cpp
patching file src/dos/dos_mscdex.cpp
patching file src/dos/drive_iso.cpp
patching file src/libs/decoders/audio_convert.c
patching file src/libs/decoders/docs/copying.txt
patching file src/libs/decoders/docs/credits.txt
patching file src/libs/decoders/docs/license.txt
patching file src/libs/decoders/dr_flac.h
patching file src/libs/decoders/extra_rwops.c
patching file src/libs/decoders/extra_rwops.h
patching file src/libs/decoders/flac.c
patching file src/libs/decoders/internal/Makefile
patching file src/libs/decoders/Makefile.am
patching file src/libs/decoders/opus.c
patching file src/libs/decoders/SDL_sound.c
patching file src/libs/decoders/SDL_sound.h
patching file src/libs/decoders/SDL_sound_internal.h
patching file src/libs/decoders/stb_vorbis.h
patching file src/libs/decoders/vorbis.c
patching file src/libs/decoders/wav.c
patching file src/libs/Makefile.am
patching file src/Makefile.am
Last edited by krcroft on 2018-11-05 @ 20:12, edited 2 times in total.
User avatar
krcroft
Member
 
Posts: 112
Joined: 2017-4-29 @ 15:07
Location: Ogden's Retreat

Re: AUDIO Patch for FLAC and Ogg Opus support

Postby Dominus » 2018-10-14 @ 18:32

Hey wow! That was fast!
What about mp3 decoding? This was working with SDL-Sound, I think.

Qbix: apart from not having tested this, I really agree that this is a good direction!!!!
User avatar
Dominus
DOSBox Moderator
 
Posts: 7674
Joined: 2002-10-03 @ 09:54
Location: Ludwigsburg

Next

Return to DOSBox Patches

Who is online

Users browsing this forum: No registered users and 3 guests