VOGONS


Issues with new video recording commit r4314

Topic actions

First post, by kas1e

User metadata
Rank Newbie
Rank
Newbie

Hi All!

Tested today latest commit with a major rewrite in terms of video recording, and found some issues:

1). The issue happens on x86: on win10, the recorded video can't be played by the usual win10 player. Audio plays, but no video, like no codec.

2). The second issue happens on PPC (at least on my 2GHZ machine): when I record video, video records fine, but audio interrupted every equal second. I tried even on very simple games which does just 5% of CPU loading, and it the same, audio always interrupted every few seconds. Also, that file while can be played on PPC, can't be played on X86 (says broken). There is that example: http://kas1e.mikendezign.com/aos4/dosbox/reco … aladdin_000.avi

Through not sure how it helps if it didn't play on x86.

3). On Linux x86 we get the same broken behavior: video records fine, but there is no sound at all.

Reply 1 of 21, by jmarsh

User metadata
Rank Oldbie
Rank
Oldbie

The posted file is broken and has not been written correctly, probably because your libc is not POSIX compliant:

The fseek() function shall allow the file-position indicator to be set beyond the end of existing data in the file. If data is l […]
Show full quote

The fseek() function shall allow the file-position indicator to be
set beyond the end of existing data in the file. If data is later
written at this point, subsequent reads of data in the gap shall
return bytes with the value 0 until data is actually written into
the gap.

This should not be an issue on linux, windows nor OSX.
Regardless ffplay/avplay can still handle a file that is broken in this way.

Last edited by jmarsh on 2020-02-07, 06:21. Edited 1 time in total.

Reply 2 of 21, by dreamer_

User metadata
Rank Member
Rank
Member

On Windows 10 I had the same behaviour in VLC: audio plays, no video.

I did several more tests on Linux and the situation is more complicated: videos recorded via DOSBox do not work correctly in general (different bugs in different players; bugs exist since 0.74-3 and are still happening in r4314). One of these bugs results in VLC hanging in the background. Hung VLC triggered muted audio when I tested next video, so initially I thought no audio was recorded at all - but it is.

On Windows, I would consider r4314 a regression (previously I could record gameplay and it worked fine in VLC).
On Linux, the behaviour seems to be broken in 0.74-3 and r4314 exactly in the same way:
- VLC does not close after playing the movie; video seems fine; audio seems fine
- Totem closes ok after playing movie; video is broken; audio seems fine

| ← Ceci n'est pas une pipe
dosbox-staging

Reply 3 of 21, by jmarsh

User metadata
Rank Oldbie
Rank
Oldbie

I think you should narrow down exactly what the differences in behaviour are with files created before/after r4314 before you start shouting about regressions, sounds to me like your VLC is just broken.

Reply 4 of 21, by kas1e

User metadata
Rank Newbie
Rank
Newbie
jmarsh wrote on 2020-02-07, 06:07:

The posted file is broken and has not been written correctly, probably because your libc is not POSIX compliant

That very unluckily, as good bunch of software already use it in all ways. Maybe this time is a very special case ?

Can it be same problem which causing those audio interrupts in recorded video?

Regardless ffplay/avplay can still handle a file that is broken in this way.

Yep i can play it with ffplay, but audio strangely interrupts.

As for dreamer's broken vlc, i have no video in default win10 player at all too with latest commit when compile mingw version of dosbox for x86/win. Didnt test if it the same with previous version

Reply 5 of 21, by jmarsh

User metadata
Rank Oldbie
Rank
Oldbie
kas1e wrote on 2020-02-07, 10:06:

Didnt test if it the same with previous version

Then the obvious problem would be that you don't have the ZMBV codec installed.

Reply 6 of 21, by Kisai

User metadata
Rank Member
Rank
Member
kas1e wrote on 2020-02-07, 05:35:
Hi All! […]
Show full quote

Hi All!

Tested today latest commit with a major rewrite in terms of video recording, and found some issues:

1). The issue happens on x86: on win10, the recorded video can't be played by the usual win10 player. Audio plays, but no video, like no codec.

2). The second issue happens on PPC (at least on my 2GHZ machine): when I record video, video records fine, but audio interrupted every equal second. I tried even on very simple games which does just 5% of CPU loading, and it the same, audio always interrupted every few seconds. Also, that file while can be played on PPC, can't be played on X86 (says broken). There is that example: http://kas1e.mikendezign.com/aos4/dosbox/reco … aladdin_000.avi

Through not sure how it helps if it didn't play on x86.

3). On Linux x86 we get the same broken behavior: video records fine, but there is no sound at all.

You probably don't have a codec that works, but I tried your video on my ZMBV64 build and it crashes it with an access violation, which means the problem is the video. Did you not close the video (stop capture) before closing/exiting dosbox? If you don't the video won't be saved properly. VLC does "this file is broken, want to fix it?", and fixing it just looks like it skipped a keyframe.

RIFFPad reports "extra junk at the end of file" and "this file is corrupt"

Not stopping capture before closing dosbox is a long-standing problem, but so is overflowing the 32-bit file size.

Reply 7 of 21, by jmarsh

User metadata
Rank Oldbie
Rank
Oldbie
Kisai wrote on 2020-02-07, 10:14:

You probably don't have a codec that works, but I tried your video on my ZMBV64 build and it crashes it with an access violation, which means the problem is the video. Did you not close the video (stop capture) before closing/exiting dosbox? If you don't the video won't be saved properly. VLC does "this file is broken, want to fix it?", and fixing it just looks like it skipped a keyframe.

The file is broken because padding is missing between data chunks and it throws off alignment of the rest of the file (including the index). You can tell by looking at the size of the RIFF chunk vs. the size of the actual file, bytes that should have been written out have gone missing.

The new code handles unexpected DOSBox terminations better, rather than the whole file being unusable only the index will be missing so it can still be loaded by most apps (but won't be seekable). It also handles switching to a new capture file when the current one is about to exceed 2GB.
(It also fixes a big memory leak in ZMBV that nobody seems to have noticed before.)

Last edited by jmarsh on 2020-02-07, 10:32. Edited 3 times in total.

Reply 8 of 21, by latalante

User metadata
Rank Newbie
Rank
Newbie
kas1e wrote on 2020-02-07, 05:35:

Through not sure how it helps if it didn't play on x86.

for me it plays correctly under linux x86-64, mpv 0.32.0

mpv --video-unscaled=yes aladdin_000.avi (+) Video --vid=1 (zmbv 320x200 70.086fps) (+) Audio --aid=1 (pcm_s16le 2ch 44100Hz) Fi […]
Show full quote

mpv --video-unscaled=yes aladdin_000.avi
(+) Video --vid=1 (zmbv 320x200 70.086fps)
(+) Audio --aid=1 (pcm_s16le 2ch 44100Hz)
File tags:
Comment: DOSBox git
[autoconvert] Converting pal8 -> argb
AO: [alsa] 48000Hz stereo 2ch s16
VO: [gpu] 320x200 argb
AV: 00:00:18 / 00:00:18 (99%) A-V: 0.000

Reply 9 of 21, by kas1e

User metadata
Rank Newbie
Rank
Newbie
latalante wrote:

for me it plays correctly under linux x86-64, mpv 0.32.0

What about audio ? Same little pauses each second or two ?

Reply 10 of 21, by latalante

User metadata
Rank Newbie
Rank
Newbie

plays without any stuttering

Reply 11 of 21, by kas1e

User metadata
Rank Newbie
Rank
Newbie

Doing some normal tests. Making 4 tests, 2 per PPC (BE) on AmigaOS4 and 2 per X86 (LE) on Win10.

Tried just r4313 (so without video-record changes) and r4314 (so with new changes). Stop recording with "ctrl+alt+f5" of course.

First 2 tests, on my Big-Endian PPC machine:

DosBox compiled with GCC 8.2.0

With r4313 (for that one I use with help of Dreamer some byte swapping so to record audio without distortion):

-- I can play video on my big-endian machine. One player gives me "audio-interrupts" (so seems player's issues), but Mplayer port plays that file fine.
-- On win10 with installed ZMBV:
a). The default Win10 player plays only audio.
b). VLCPlayer plays audio and video, but colors wrong.
c). BSPlayer plays video and audio fine, but still, say that something wrong and needs to send a bug report.

So recorded file definitely broken still, but at least something.

There is a video file: http://kas1e.mikendezign.com/aos4/dosbox/reco … n_r4313_ppc.avi

With r4314:

Things start to be worse.

-- I can't anymore play the video with Mplayer on my OS, it just exits after the first frame.
-- On win10 with installed ZMBV :
a). default win10 player say a file is broken and exit, no audio or video.
b). VLC didn't play a file, says it broke and need to build an index, once index build video still fucked up and exit.
c). BSplayer still tries to play video but has artifacts, color swap issues, and the same "send a bug report".

So, the file again is broken, but it's starting to be much worse than with r4313. Like, an index is broken now (while with r4313 is ok).

There is a video file: http://kas1e.mikendezign.com/aos4/dosbox/reco … n_r4314_ppc.avi

Second 2 tests on my Win10/x86

DosBox compiled with Mingw's GCC 7.3.0

Same just r4313 (so without video-record changes) and r4314 (so with new changes). Stop recording with "ctrl+alt+f5" of course.

With r4313:

On win10 with installed ZMBV:

a). The default Win10 player plays only audio.
b). VLCPlayer plays audio and video, but colors wrong.
c). BSPlayer plays video and audio fine, but still, say that something wrong and needs to send a bug report.

And I can play that video on the PPC machine as well (same in Mplayer fine, and audio-interrupts in another player).

There is a video file: http://kas1e.mikendezign.com/aos4/dosbox/reco … n_r4313_x86.avi

With r4314:

On win10 with installed ZMBV:

a). The default Win10 player plays only audio.
b). VLCPlayer plays audio and video, but colors wrong.
c). BSPlayer plays video and audio fine, but still, say that something wrong and needs to send a bug report.

And I can play that video on the PPC machine as well (same in Mplayer fine, and audio-interrupts in another player).

There is a video file: http://kas1e.mikendezign.com/aos4/dosbox/reco … n_r4314_x86.avi

So, as a result of all those tests we can say that:

1). There are bugs in a video recording on any platform. BSPlayer says something broken, VLC produces wrong colors, default win10 player fails to show video (which ZMBV codec installed).
That happens and when I record file on PPC, and when I record file on X86. It does not matter if it r4313 or r4314. Just the same.

2). In terms of Big-Endian (at least on PPC and AmigaOS4), with r4314 things start to be worse: it seems, that building of Index fail now or something, while with 4313 it was ok (still with all issues I mention in 1). ).

Reply 12 of 21, by jmarsh

User metadata
Rank Oldbie
Rank
Oldbie

The only reason you're seeing any changes with r4314 is because your AmigaOS4 libc is broken. Want proof? Build and run this source code, post here the results:

#include <stdio.h>

int main(void) {
FILE *fp = fopen("tmp.bin", "wb");
if (fp) {
fseek(fp, 4, SEEK_CUR);
fprintf(stderr, "filepos: %ld\n", ftell(fp));
fwrite("", 1, 1, fp);
fprintf(stderr, "filepos: %ld\n", ftell(fp));
fclose(fp);
} else fprintf(stderr, "Failed to open file\n");
return 0;
}

The other issues aren't related to r4314; they are problems with other programs (i.e. VLC bugs: #14975, #9940) or user error (failure to install correct 32/64-bit version of ZMBV codec). Install something like LAV filters and use MPC-HC or virtualdub.

Reply 13 of 21, by kas1e

User metadata
Rank Newbie
Rank
Newbie

@jmarsh

The only reason you're seeing any changes with r4314 is because your AmigaOS4 libc is broken. Want proof? Build and run this source code, post here the results:

Compiled for 2 versions of LibC we have (Newlib and clib2), in both cases output looks like this:
filepos: 0
filepos: 1

Compiled it for x64 (Cygwin's GCC and Mingw's GCC), and the output looks like this:
filepos: 4
filepos: 5

Reply 14 of 21, by krcroft

User metadata
Rank Oldbie
Rank
Oldbie

jmarsh, kas1e - very interesting guys.

So seek-beyond-EOF-then-write doesn't work on AmigaOS4.

c11 and C99 describes seek-beyond-EOF-then-write within a POSIX clause; meaning non-POSIX systems don't have to honor it. Sure enough AmigaOS4 does not claim POSIX compliance. Even the venerable MingGW states

"MinGW ... does not, and never will, attempt to provide a POSIX runtime environment"

http://www.mingw.org/

DOSBox itself doesn't require POSIX-compliance, so why would we knowingly write code that behaves differently between POSIX vs non-POSIX systems? Extending a file beyond EOF can be achieved by writing empty data instead of seeking beyond EOF (yes; it's less efficient because the latter requires sparse-file support); but it's robust.

Carnegie Melon's Software Engineering Institute describes other fun POSIX gotchas with 'seek' here ...

https://wiki.sei.cmu.edu/confluence/display/c … +a+regular+file

References:

C11 standard (ISO/IEC 9899:2011): 7.21.9.2 The fseek function (p: 336-337)
C99 standard (ISO/IEC 9899:1999): 7.19.9.2 The fseek function (p: 302-303)
https://en.cppreference.com/w/c/io/fseek

POSIX allows seeking beyond the existing end of file. If an output is performed after this seek, any read from the gap will return zero bytes. Where supported by the filesystem, this creates a sparse file.

and

IEEE Std 1003.1, 2004 Edition (POSIX is an IEEE standard)
https://pubs.opengroup.org/onlinepubs/0096968 … ions/fseek.html

The fseek() function shall allow the file-position indicator to be set beyond the end of existing data in the file. If data is later written at this point, subsequent reads of data in the gap shall return bytes with the value 0 until data is actually written into the gap.

Reply 15 of 21, by jmarsh

User metadata
Rank Oldbie
Rank
Oldbie
krcroft wrote on 2020-02-08, 18:00:

Even the venerable MingGW states

"MinGW ... does not, and never will, attempt to provide a POSIX runtime environment"

http://www.mingw.org/

And yet as you can see from the above, mingw behaves correctly in this case.

DOSBox itself doesn't require POSIX-compliance, so why would we knowingly write code that behaves differently between POSIX vs non-POSIX systems? Extending a file beyond EOF can be achieved by writing empty data instead of seeking beyond EOF (yes; it's less efficient because the latter requires sparse-file support); but it's robust.

Because in this case using fwrite is not correct behaviour. The seek is used to align the next chunk of data; if nothing is subsequently written, no padding should exist in the output file. It also performs an implicit flush of any buffered data.

Carnegie Melon's Software Engineering Institute describes other fun POSIX gotchas with 'seek' here ...

https://wiki.sei.cmu.edu/confluence/display/c … +a+regular+file

Right. That's why the new AVI code only uses ftell to obtain a position that is later passed to fseek. The returned positions are treated as opaque and never examined directly.

Reply 16 of 21, by kas1e

User metadata
Rank Newbie
Rank
Newbie

@Jmarsh
Maybe it possible with not much blood made some ifdefs or something, so this code can be used on amigaos4 & morphos as well? like #ifndef POSIX or maybe just #if defined(__amigaos4__) && defined(__morphos__) ?

Reply 17 of 21, by krcroft

User metadata
Rank Oldbie
Rank
Oldbie

jmarsh, good stuff ( accommodating non-POSIX systems in the video code).

Yeah, my comment was pointing wat that calling someone's non-POSIX OS broken "your AmigaOS4 libc is broken." and then using a POSIX-specific code snippet as proof, only reconfirms that their OS is indeed not POSIX.

Edit: but more constructively, it seems like a variety of video players don't handle the video correctly; while other players - perhaps the more robust ones like mpv and its derivatives, and mxplayer on Android, can handle it. Perhaps the DOSBox video codec is simply too esoteric and rarely used, that some players trip over it?

Kas1e, I wonder what the upstream player maintainers might say, if they have a chance to inspect the video and their player's behavior?

Reply 18 of 21, by jmarsh

User metadata
Rank Oldbie
Rank
Oldbie
krcroft wrote on 2020-02-09, 19:27:

Edit: but more constructively, it seems like a variety of video players don't handle the video correctly; while other players - perhaps the more robust ones like mpv and its derivatives, and mxplayer on Android, can handle it. Perhaps the DOSBox video codec is simply too esoteric and rarely used, that some players trip over it?

That's pretty irrelevant when the main use case for DOSBox recordings is to be edited and/or reprocessed into some other (more mainstream) format. In that sense, ffmpeg and Virtualdub (combined with the ZMBV VFW codec included with DOSBox) work perfectly.
There are practically no other lossless video codecs designed to handle palette-based video.

If you're having a hard time playing them, try using quickview to watch DOSBox recordings from inside DOSBox.

Reply 19 of 21, by krcroft

User metadata
Rank Oldbie
Rank
Oldbie

Ahh; great point jmarsh.. yeah, it's just a lossless precursor format destinated for youtube or other end-point consumption.

You've made that last point a reality as well.. by fixing dynrec, QuickView now works in DOSBox under all OSes 😀.