flightaware / speedtables

Speed tables is a high-performance memory-resident database. The speed table compiler reads a table definition and generates a set of C access routines to create, manipulate and search tables containing millions of rows. Currently oriented towards Tcl.

Home Page:https://flightaware.github.io/speedtables/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

dropping and recreating index can cause panic

bovine opened this issue · comments

dropping and recreating index can cause panic

inflight index indexed
ok {ident orig dest clock lat}
inflight index indexable
ok {ident orig dest clock lat}
inflight index drop lat
ok {}
inflight index create lat
Connection closed by foreign host.

The panic produced was:

Double insert row for field lat
Abort

Which occurs in:

ctables/ctable_search.c
// invariant: prev is always NULL if not in list
    if(row->_ll_nodes[index].prev != NULL) {
        panic ("Double insert row for field %s", ctable->creator->fields[field]->name);
    }

So far as I can see there is no code currently in place to clear the bucket list when dropping an index. All it does is remove the skiplist itself.

Given how long this code has been there, I'd like feedback on this before committing it.

diff --git a/ctables/ctable_search.c b/ctables/ctable_search.c
index a837679..62760d3 100644
--- a/ctables/ctable_search.c
+++ b/ctables/ctable_search.c
@@ -2478,11 +2478,28 @@ ctable_SetupAndPerformSearch (Tcl_Interp *interp, Tcl_Obj *CONST objv[], int obj
 CTABLE_INTERNAL void
 ctable_DropIndex (CTable *ctable, int field, int final) {
     jsw_skip_t *skip = ctable->skipLists[field];
+    ctable_BaseRow *row;
+    int listIndexNumber = ctable->creator->fields[field]->indexNumber;

     if (skip == NULL) return;

+    // Forget I had a skiplist
     ctable->skipLists[field] = NULL;
+
+    // Delete the skiplist
     jsw_sdelete_skiplist (skip, final);
+
+    // Don't need to reset the bucket list if it's just going to be deleted
+    if(final) return;
+
+    // Walk the table and erase the bucket lists for the row
+    // We're walking the main index for the table so it's safe to
+    // clear the index for $field
+    CTABLE_LIST_FOREACH (ctable->ll_head, row, 0) {
+   row->_ll_nodes[listIndexNumber].next = NULL;
+   row->_ll_nodes[listIndexNumber].prev = NULL;
+   row->_ll_nodes[listIndexNumber].head = NULL;
+    }
 }

 //

Does not appear to panic anymore after I applied this patch.

Thanks, I'd still like to get Karl's input before committing it... the indexing is still 99% his code (I've mostly just re-ordered events to support the lockless shared memory) and I might have missed some other place this was supposed to be getting cleared that I need to dig into.

Fix was committed last month, after review. Closing.