c1bcb43f7c
The existing lanczos_weight() implementation naively follows the purely mathematical definition for the `x == 0` special case. However, the point of defining that special case is obviously to prevent division by zero in the general case formula. Unfortunately we are still doing some multiplications with `x` before we get to the division step, and our fpmath library loses precision during multiplication. This can lead to edge cases where `x` is not zero but `x_times_pi` later ends up being 0, which causes the division to throw an exception after all. (I guess we've just been lucky to not see this case in practice for now... it requires the output pixel coordinate to be extremely close to but not quite on the next input pixel coordinate, which may be rare in practice with our scaling algorithms.) This patch fixes the issue by implementing the special case later and checking if `x_times_pi` is zero instead. Note that as long as we pass this check, we can be confident that the division cannot fail even though fpdiv() also truncates the divisor: this is because `x_times_pi` was calculated from an fpmul() call with the constant fppi(), which has 34 significant bits. Even if x is the smallest possible non-zero value after scaling for multiplication, the result `x_times_pi` must still have 18 significant bits. That means it can be scaled down a further 16 bits for division without becoming zero. Also add a simple unit test forcing exactly this condition to ensure the code will not regress. Change-Id: I2f212ee5df38252e97ec55aba3d2d25320c4b102 Signed-off-by: Julius Werner <jwerner@chromium.org> Reviewed-on: https://review.coreboot.org/c/coreboot/+/87532 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Yu-Ping Wu <yupingso@google.com> Reviewed-by: Jakub "Kuba" Czapiga <czapiga@google.com> |
||
|---|---|---|
| .. | ||
| coreinfo | ||
| external | ||
| libpayload | ||
| linuxcheck | ||
| nvramcui | ||
| Kconfig | ||
| Makefile.mk | ||