Error in hq2x / hq3x

Developer's Forum, for discussion of bugs, code, and other developmental aspects of DOSBox.

Error in hq2x / hq3x

Postby M-HT » 2019-3-25 @ 10:09

There is an error in hq2x / hq3x scalers.
In file src/gui/render_templates_hq.h (in function diffYUV) the lines 44, 49, 54 are the same and look like this:
Code: Select all
mask = diff >> 31; // -1 if value < 0, 0 otherwise

Since diff is an unsigned integer, the result won't be -1 if value < 0, but the result will be 1.
To get the correct result, the lines should to be modified to look like this:
Code: Select all
mask = ((Bit32s)diff) >> 31; // -1 if value < 0, 0 otherwise
M-HT
Newbie
 
Posts: 68
Joined: 2008-9-01 @ 12:55
Location: Bratislava

Re: Error in hq2x / hq3x

Postby M-HT » 2019-3-31 @ 12:50

An example image scaled with hq3x using unfixed code:
test1.png
hq3x using unfixed code
test1.png (7.19 KiB) Viewed 144 times

An example image scaled with hq3x using fixed code:
test2.png
hq3x using fixed code
test2.png (8.14 KiB) Viewed 144 times
M-HT
Newbie
 
Posts: 68
Joined: 2008-9-01 @ 12:55
Location: Bratislava

Re: Error in hq2x / hq3x

Postby Qbix » 2019-3-31 @ 12:55

Thanks!

Will correct it.
Water flows down the stream
How to ask questions the smart way!
User avatar
Qbix
DOSBox Author
 
Posts: 10916
Joined: 2002-11-27 @ 14:50
Location: Fryslan

Re: Error in hq2x / hq3x

Postby Qbix » 2019-3-31 @ 18:06

Both diff and mask are both unsigned. So we are dealing with ~1 internally. Let me update the comment as well.
Commited in 4198
Water flows down the stream
How to ask questions the smart way!
User avatar
Qbix
DOSBox Author
 
Posts: 10916
Joined: 2002-11-27 @ 14:50
Location: Fryslan


Return to DOSBox Development

Who is online

Users browsing this forum: No registered users and 2 guests