[lib] Move type cast in Fletcher checksum
The int type cast should be on the whole expression
passed to the mod operator. Otherwise it won't work
when/if c0/c1 is unsigned.
Making c0/c1 unsigned makes it possible to use 5802
as MODX value.
diff --git a/lib/checksum.c b/lib/checksum.c
index f6d74d3..3ddde81 100644
--- a/lib/checksum.c
+++ b/lib/checksum.c
@@ -91,7 +91,7 @@
}
/* The cast is important, to ensure the mod is taken as a signed value. */
- x = ((int)(len - offset - 1) * c0 - c1) % 255;
+ x = (int)((len - offset - 1) * c0 - c1) % 255;
if (x <= 0)
x += 255;