NewRisingSun wrote:

In the meantime, I have noticed that in addition to Microsoft Decathlon, Adventure in Serenia, the IBM PC port of Sierra's The Wizard and the Princess, is somewhat sensitive to the CGA revision used, assuming that I am applying the algorithms correctly.

Interesting - comparing these to Servo's images on MG, the buildings in his shot look more like old.png, while the road is closer to new.png... then again, capture cards always seem to introduce their own quirks.

By the way, this is irrelevant as far as DOSBox is concerned, but I'm curious about the technique you're using to produce the "defringing" in your images; IIRC, some of the code in Blargg's NTSC libraries came from you - is that the same filtering method seen here?

It's seen like the dark cyan is swaped with light green.

Also, the gray color is darker and darker gray is black.

reenigne wrote:

If we're doing our band-limiting by just averaging over 4 consecutive pixels (as the current DOSBox implementation does, I think) and we limit ourselves to graphics modes we can just enumerate all the colour possibilities directly in the palette. There are only 64 of them

Having thought about this some more, I now think we need 160 palette entries for the low-res mode because every other 4-pixel span includes data about 3 low-res pixels (the right half of the left one, all of the middle one and left half of the right one) so needs 7 bits of data to describe it (1 for its position and 2 for each of the low-res pixels). Then 16 palette entries for the other 2 positions.

How much work would it be to free up another 32 palette entries for the composite decoder's use? I'm sure it's possible since the 256 colour VGA modes have use of all 256 palette entries. What are the lower 128 palette entries reserved for at the moment?

I think using the 128 upper entries is just a convenience, to allow easier index calculations using bit-shifts. The first 16 palette entries are reserved for the RGB CGA colors, but I don't see anything else using other entries in the lower half (someone correct me if I'm wrong).

NewRisingSun wrote:

But it won't necessarily make for particularly pretty colors. In particularly, people will continue to pester the developers about brown being too greenish or red being too pinkish, depending on the hue setting.

NewRisingSun wrote:

See pictures, all at a hue setting of ±0°. You would not need any fancy non-linear color correction either. That's actually the solution I would now favor, as there are no brownie points to be earned from following a 59-year-old standard as closely as possible.

I'm not convinced that there's any transformation which is going to make everyone happy as everyone is going to have their own aesthetic tastes and memories of how those games originally looked. So the best we can do is to make it as objectively correct as possible (according to the relevant standards) and then provide knobs that people can twiddle if they aren't satisfied with the outcome. I don't think the age of the standard has any bearing one way or the other.

NewRisingSun wrote:

In the meantime, I have noticed that in addition to Microsoft Decathlon, Adventure in Serenia, the IBM PC port of Sierra's The Wizard and the Princess, is somewhat sensitive to the CGA revision used, assuming that I am applying the algorithms correctly.

Any game that uses low-res mode will be affected to some degree because the chroma colours are so different between 1501486 and 1501981.

nikiniki wrote:

It's seen like the dark cyan is swaped with light green.

Also, the gray color is darker and darker gray is black.

You're not comparing like with like there - the colours between the CGA composite and the PCjr versions were never the same and were never supposed to be because the sets of colours that the two machines are capable of are completely different. In particular, CGA composite only has one shade of grey (plus black and white) while the PCjr has two, which is why the lines in the castle walls are all black in the CGA composite version.

reenigne wrote:

So the best we can do is to make it as objectively correct as possible (according to the relevant standards) and then provide knobs that people can twiddle if they aren't satisfied with the outcome

Okay, but a hue knob alone won't be enough to do that.

You would at least have to provide the user with the ability to decide the YIQ decoding matrix, preferably in polar coordinate form. NEStopia already does that. And if you're going to do the NTSC-to-709 correction, being able to choose input and output color spaces arbitrarily could also be added.

Of course, that's a heck of a lot of settings, which probably wouldn't be popular with developers to implement, maintain and mostly, explain in a documentation. (Implementation would be the least problem, as code for all aspects already exists.)

nikiniki wrote:

It's seen like the dark cyan is swaped with light green.

Please notice in the two pictures attached how the booter version and the 1987 AGI version differ in that regard. (I have adjusted the "pretty" matrix to make the grass less blueish.)

nikiniki wrote:

Also, the gray color is darker and darker gray is black.

That's inherent with composite color artifacting --- there is only one gray. Actually, there are two grays, but they are of equal brightness. The only way to get a darker gray with composite color mode is to assume a three-line comb filter and alternate dark blue and dark green horizontal lines, which will be filtered to a darker gray by the comb filter. Consider the third attached picture.

NewRisingSun wrote:

Of course, that's a heck of a lot of settings, which probably wouldn't be popular with developers to implement, maintain and mostly, explain in a documentation. (Implementation would be the least problem, as code for all aspects already exists.)

Yep - so it's a classic 3-way engineering tradeoff: attractiveness, adherence to standards, ease of implementation - pick two.

NewRisingSun wrote:

The only way to get a darker gray with composite color mode is to assume a three-line comb filter and alternate dark blue and dark green horizontal lines, which will be filtered to a darker gray by the comb filter. Consider the third attached picture.

Did that actually work on any real monitors? I thought I had debunked it back on page 8 of this thread (CGA uses 228 color carrier cycles per scanline instead of 227.5 which makes a comb filter useless and causes monitors which have them to turn it off). But I always enjoy being proved wrong!

reenigne wrote:

it's a classic 3-way engineering tradeoff: attractiveness, adherence to standards, ease of implementation - pick two.

I'll leave that decision to others.

reenigne wrote:

Did that actually work on any real monitors?

Given the sheer number of Apple II (and a few CGA) games using the technique, I would assume so. It evidently does work wth Servo's TV capture card used to create his MobyGames screenshots. I'll look at some composite monitor schematics later.

reenigne wrote:

CGA uses 228 color carrier cycles per scanline instead of 227.5 which makes a comb filter useless

It's not useless. Using a comb filter reduces fringing even on CGA images, as the attached Ultima III screenshots show. Look at the mountains.

reenigne wrote:

and causes monitors which have them to turn it off

Has that been proven? I find it hard to believe that a comb-filter-equipped TV would bother to implement a detection of non-standard line rates just so it can then turn the comb filter off.

VileRancour wrote:

As Qbix noted, the precalculation produces 80 colors (16 hues * 5 luminance levels for each); the line-drawing handler then determines the palette index for each pixel by doing some averaging over adjacent pixels. If you go back to the Tapper screenshot comparison on page 8, you can see the effect: most areas with colored fringes look acceptable, even though a few scattered errors are visible (for example, the character "1" on the bottom left gets a greyscale transition instead of color blending). So it isn't precise, but for a palette-based approach it seems to work surprisingly well.

Could the boundary condition have anything to do with that ?
At the left boundary I average over

10 0 left left+1 

The 2 zeroes are because "lack" of information, however it might make sense to repeat the left value or even repeat the 2 left most values.

1left left left left+1

or

1left left+1 left left+1

NewRisingSun wrote:

Given the sheer number of Apple II (and a few CGA) games using the technique, I would assume so.

I saw the "Mask of the Sun" example that TeaRex posted at CGA Composite Mode under DOSBOX (Commited r3804) but I think the horizontal lines there are just dithering between two colours which can't appear next to each other horizontally because of the quirks of the Apple II video hardware.

I'd like to see an example of a CGA game which uses it. I'm aware of horizontal stripes in the composite output of some CGA 320x200 games, but I think they arise from doing checkerboard dithering aimed at RGB monitors.

NewRisingSun wrote:

It evidently does work wth Servo's TV capture card used to create his MobyGames screenshots.

Agreed, but TV capture cards do at least some of the NTSC decoding in software so can't be considered authentic.

NewRisingSun wrote:

It's not useless. Using a comb filter reduces fringing even on CGA images, as the attached Ultima II screenshots show. Look at the mountains.

So that's doing delay of 228 color carrier cycles (i.e. blurring each line with the one directly below it). I'm not sure whether real comb filters work by delaying the signal by 63.555us (227.5 color carrier cycles) or by one scanline - certainly the original analogue delay line filters would have done the former, and wouldn't have had the same effect on Ultima II. Digital ones could be done either way though.

NewRisingSun wrote:

Has that been proven? I find it hard to believe that a comb-filter-equipped TV would bother to implement a detection of non-standard line rates just so it can then turn the comb filter off.

They do so for VCRs, which look worse with comb filters because of the instability of the timebase (Source: http://www.extron.com/company/article.aspx?id=ntscdb3 under "Comb When You Can" near the bottom). Also the 90s-era TV I'm using as a composite monitor definitely doesn't use a comb filter on the output of my CGA card, but I haven't yet been able to determine if that's because it doesn't have a comb filter at all, or if it's because it has one and turns it off.

Qbix wrote:

Could the boundary condition have anything to do with that ?

It's not just the boundary - look at the rightmost white vertical line in the lower door in each pair of doors - the same lack of fringing shows up there too.

The boundary values should be the values corresponding to whatever colour is in the border (so 0 for 640x200 mode and whatever colour 0 is in 320x200 mode).

reenigne wrote:

I'd like to see an example of a CGA game which uses it.

Blue Angels: Formation Flight Simulation uses the dark gray I mentioned earlier. I'll post screenshots later.

As for the Apple II, Karateka has a floor that alternates between blue and orange, which a comb filter would filter to gray. The screenshot on the back of the box also shows it gray. It would be a strange coincidence if the box screenshot had been doctored in a way a comb filter would when comb filters had not been available or all failed to work with the Apple II.

reenigne wrote:

I'm not sure whether real comb filters work by delaying the signal by 63.555us (227.5 color carrier cycles)

To work properly, it would have to delay by 63.555 µs, not by one scanline. A comb filter expects, at the same horizontal position, Y+C in line 0 and Y-C in line 1, as is the case with broadcast NTSC. If the comb filter were to be based on actual scanlines, it would in the case of CGA/AppleII find Y+C in line 0 and Y+C in line 1. If it then were to filter in the same way as for broadcast NTSC, it would remove all solid color areas and only keep the fringing! But because it delays by 227.5 color carrier cycles regardless of how long the line actually is, it still finds Y+C in line 0 and Y-C in line 1, as expected. In terms of CGA's 640 horizontal pixels, that means that a pixel's chroma is not averaged with the exact pixel above and below but two pixels to the left and right, respectively (because half a 3.58 color cycle takes two pixels at the 14.34 pixel clock), but otherwise it still works like a charm.

So it's actually the digital filter that samples scanlines that might run into trouble, not the analog filter with the delay line.

NewRisingSun wrote:

Blue Angels: Formation Flight Simulation[/i] uses the dark gray I mentioned earlier. I'll post screenshots later.

I found the ones on MobyGames. It definitely looks much better there than it does on DOSBox at the moment. Between that and Karateka I'm convinced - there were monitors which applied a (227.5 cycle) comb filter to 228 cycle-per-scanline inputs, doing this works fine and produces useful new colours which were used by games developers. So it would indeed be a useful new option for DOSBox.

Blue Angels needs +140° hue offset in DOSBox to look similar to Servo's screenshots; the no-offset default colors are awful. That's a rather large adjustment when most games need little if any...

You can also see the filtered grays in Demon's Forge (link)

Accolade evidently used a Tandy system to test Blue Angel's composite mode. The Tandy has a shift between the color subcarrier and the pixel clock of 135 degrees compared to the IBM CGA.

For what it's worth, as promised earlier, a picture from the game without and with comb-filtered grays.

NewRisingSun wrote:

For what it's worth, as promised earlier, a picture from the game without and with comb-filtered grays.

One more question about comb filters: what happens when you alternate black and white lines?

On that screenshot (and the one on MobyGames) it appears that doing this gives you black and white lines instead of blurring them into mid-grey (look at the S in the word SIMULATOR in the bottom left in particular, and in the compass in all the screenshots). I think that happens because both Servo's capture card and your decoder are sampling the luma on the scanline and the chroma "in between scanlines" (i.e. the chroma comes from the output of the comb filter and the luma comes from the output of the notch/bandpass filter). So you only get a solid colour when the luma is the same in both of the colours that you're alternating between.

But http://www.extron.com/company/article.aspx?id=ntscdb3 suggests that comb filters normally work on both chroma and luma which would make more sense for broadcast video since that gives you improved luma bandwidth and having the chroma offset from the luma by half a scanline would look wrong (though perhaps not noticably so).

Though I suppose if we were adding a comb filter to DOSBox we'd want to use the variation which gave the best results for the rare games that used it rather than the theoretically correct variation.

reenigne wrote:

I think that happens because both Servo's capture card and your decoder are sampling the luma on the scanline and the chroma "in between scanlines" (i.e. the chroma comes from the output of the comb filter and the luma comes from the output of the notch/bandpass filter).

Not quite. See below.

reenigne wrote:

http://www.extron.com/company/article.aspx?id=ntscdb3 suggests that comb filters normally work on both chroma and luma which would make more sense for broadcast video since that gives you improved luma bandwidth

I'm not sure how reliable the information on those pages is. Why use a resonant bandpass filter at all in the comb-filter-less design? Why not just use a notch filter for chroma, then subtract the chroma signal obtained in that manner from the composite signal to obtain luma? That way, your luma signal retains the full bandwidth, and every single frequency component is either in luma or in chroma, never in both. Any artifacts remaining will be from misidentifying chroma as luma and vice-versa.

That's exactly the approach I use in my algorithm when disabling the comb filter, which is why the luma signal still is sharp, so sharp that 1010 gray still remains as such, as seen in the attached picture. In all other pictures, I deliberately blur luma to get rid of the vertical banding; but that has nothing to do with the Y/C separator.

For the comb filter, I would normally go about it in the same manner, subtracting the comb-filtered chroma from composite to obtain luma. Because my comb filter is not yet adaptive however, I don't do that, subtracting the notch-filtered chroma from luma instead to avoid hanging dots, and because luma is sharp enough as it is.

reenigne wrote:

and having the chroma offset from the luma by half a scanline would look wrong

Why is the chroma offset off by half a scanline?

I don't want to derail the technical discussion, but I have to ask :

When were comb filters first introduced in composite color computer monitors or in consumer TVs?
When Wozniak tested his Apple II's video output, he probably used modest consumer TVs from the mid-70s connected with an RF box. I am not sure whether composite color monitors were readily available in those days outside of TV studio work.
When IBM tested its card back in 1980-81, presumably they used a wider range of sets, since they were targeting the card to business and home users alike. So shouldn't one know what was commonly available at the time and then focus on what games would have looked like on it? It would seem to me that for composite color monitors, one should focus on :

Commodore 1701 & 1702, 1080 & 1084 (early)
Tandy TRS-80 Color Monitor
Apple ColorMonitor IIe/IIc

None of them advertise comb filters or advanced processing in their specs, just video bandwidth and sometimes dot pitch.

NewRisingSun wrote:

Why use a resonant bandpass filter at all in the comb-filter-less design? Why not just use a notch filter for chroma, then subtract the chroma signal obtained in that manner from the composite signal to obtain luma?

Don't they amount to the same thing? (Assuming the filters are exactly matched).

NewRisingSun wrote:

Why is the chroma offset off by half a scanline?

Oh, you're using a 3 line comb filter (what Extron calls a 2 line comb filter) right? I was thinking of a 2 line comb filter (what Extron calls a single line comb filter). Yes, if the filter incorporates as much data from above as it does from below then there will be no offset.

Trixter sent me this interesting photo of what Blue Angels looks like on a broadcast quality monitor http://www.reenigne.org/misc/blue_angels_comb_filtered.jpg. This clearly has a very sophisticated (adaptive?) comb filter - on the 25% and 75% grey sections it retains the full luma bandwidth (so you get a cross hatch pattern instead of a solid grey). There's also no vertical colour bleeding, but there are hanging dots (I think... I haven't seen hanging dots before so I might not be using the term correctly).

I wrote a little hack to display all the possible vertically alternating patterns - it'd be interesting to see what different comb filter algorithms make of this: http://www.reenigne.org/misc/cftp.com. My TV gives output similar to that of DOSBox, confirming that it's not using a comb filter.

Great Hierophant wrote:

It would seem to me that for composite color monitors, one should focus on : […]
Show full quote

It would seem to me that for composite color monitors, one should focus on :

Commodore 1701 & 1702, 1080 & 1084 (early)
Tandy TRS-80 Color Monitor
Apple ColorMonitor IIe/IIc

None of them advertise comb filters or advanced processing in their specs, just video bandwidth and sometimes dot pitch.

I'm pretty sure connecting monitors with comb filters to CGA cards (at least comb filters that weren't disabled for 228-cycle-per-scanline inputs)was very rare throughout the lifetime of the CGA, just because there are so few games that take advantage of those extra colours.

I've read elsewhere (while researching a reply earlier in the thread) that the AppleColor monitors did not have comb filters.