ospf6d: review LSA sequence number comparison

It seems that there is a bug in ospf6d in ospf6_lsa_compare(): If LSA A
has sequence number smaller than 0x80000000 and LSA B has sequence
number larger than 0x80000000, ospf6_lsa_compare() returns that B is
more recent than A, although RFC says that sequence numbers should be
compared as signed numbers (0x8000001 smallest and 0x7FFFFFFF largest).

In ospfd, the function ospf_lsa_more_recent() has it right.

The problem appears when Quagga is used together with OSPFv3 in
development version of BIRD daemon ( http://bird.network.cz/ ),
which creates LSAs with maximum sequence number (0x7FFFFFFF)
as a part of flushing/premature aging LSA from OSPF area.

Because both daemons has different idea of which LSA instance
is more recent, it would lead to LSA storm.

1 files changed, 4 insertions, 7 deletions

diff --git a/ospf6d/ospf6_lsa.c b/ospf6d/ospf6_lsa.c
index 07d9f91eb..c1db3741b 100644
--- a/ospf6d/ospf6_lsa.c
+++ b/ospf6d/ospf6_lsa.c
@@ -256,7 +256,7 @@ ospf6_lsa_premature_aging (struct ospf6_lsa *lsa)
 int
 ospf6_lsa_compare (struct ospf6_lsa *a, struct ospf6_lsa *b)
 {
-  signed long seqnuma, seqnumb;
+  int seqnuma, seqnumb;
   u_int16_t cksuma, cksumb;
   u_int16_t agea, ageb;
 
@@ -264,16 +264,13 @@ ospf6_lsa_compare (struct ospf6_lsa *a, struct ospf6_lsa *b)
   assert (b && b->header);
   assert (OSPF6_LSA_IS_SAME (a, b));
 
-  seqnuma = ((signed long) ntohl (a->header->seqnum))
-             - (signed long) INITIAL_SEQUENCE_NUMBER;
-  seqnumb = ((signed long) ntohl (b->header->seqnum))
-             - (signed long) INITIAL_SEQUENCE_NUMBER;
+  seqnuma = (int) ntohl (a->header->seqnum);
+  seqnumb = (int) ntohl (b->header->seqnum);
 
   /* compare by sequence number */
-  /* XXX, LS sequence number wrapping */
   if (seqnuma > seqnumb)
     return -1;
-  else if (seqnuma < seqnumb)
+  if (seqnuma < seqnumb)
     return 1;
 
   /* Checksum */