[lib] Switch Fletcher checksum back to old ospfd version
* lib/checksum.c: (fletcher_checksum) Switch the second phase of the checksum
back to the old ospfd logic.
The isisd-derived version:
a) is very hard to follow
b) had some kind of subtle bug that caused it be wrong when c0=0 and c1=254
(potentially fixable by doing the mods before adjusting x and y)
Additionally:
- explicitely cast expressions using non-internal variables to int, to ensure
the result is signed.
- defensively change the length argument to 'size_t', to ensure the code
works with that argument being unsigned..
Thanks to Joakim Tjernlund for the investigative work into this bug.
* tests/test-checksum.c: new file to exercise the checksum code.
diff --git a/lib/checksum.h b/lib/checksum.h
index d3ce930..da1d3cb 100644
--- a/lib/checksum.h
+++ b/lib/checksum.h
@@ -1,2 +1,2 @@
extern int in_cksum(void *, int);
-extern u_int16_t fletcher_checksum(u_char * buffer, int len, u_int16_t offset);
+extern u_int16_t fletcher_checksum(u_char *, const size_t len, const uint16_t offset);