VOGONS


First post, by M-HT

User metadata
Rank Newbie
Rank
Newbie

There is an error in hq2x / hq3x scalers.
In file src/gui/render_templates_hq.h (in function diffYUV) the lines 44, 49, 54 are the same and look like this:

mask = diff >> 31; // -1 if value < 0, 0 otherwise

Since diff is an unsigned integer, the result won't be -1 if value < 0, but the result will be 1.
To get the correct result, the lines should to be modified to look like this:

mask = ((Bit32s)diff) >> 31; // -1 if value < 0, 0 otherwise

Reply 1 of 3, by M-HT

User metadata
Rank Newbie
Rank
Newbie

An example image scaled with hq3x using unfixed code:

test1.png
Filename
test1.png
File size
7.19 KiB
Views
533 views
File comment
hq3x using unfixed code
File license
Fair use/fair dealing exception

An example image scaled with hq3x using fixed code:

test2.png
Filename
test2.png
File size
8.14 KiB
Views
533 views
File comment
hq3x using fixed code
File license
Fair use/fair dealing exception

Reply 3 of 3, by Qbix

User metadata
Rank DOSBox Author
Rank
DOSBox Author

Both diff and mask are both unsigned. So we are dealing with ~1 internally. Let me update the comment as well.
Commited in 4198

Water flows down the stream
How to ask questions the smart way!