search -compare "{!= key value}" produces bogus results for indexed columns

[Jeff Lawson]

The != operator appears to behave incorrectly when there are indexed columns present. In the following example, only one record should be returned.

Commenting out the line "infield index create ident" restores correct behavior.

#!/usr/local/bin/tclsh8.5

package require ctable

CExtension cowseye 1.0 {


CTable Infield {
    key fp
    varstring ident indexed 1

    varstring orig notnull 1 default "moo"  indexed 1
    varstring dest notnull 1 default "moo" indexed 1

    long clock notnull 1
}

}


package require Cowseye


Infield create infield
infield index create ident
infield index create orig
infield index create dest


set myflight [list ident N1615A  fp N1615A-airline-12341252  clock 1253246234]
infield store -nocomplain $myflight

set myflight [list ident N1615A  fp N1615A-airline-12342223  clock 1253446234]
infield store -nocomplain $myflight

set myflight [list ident N857CP  fp N857CP-airline-35412351  clock 12342353421]
infield store -nocomplain $myflight



infield search -compare [list [list = ident N1615A] [list != fp N1615A-airline-12342223] [list > clock 1234]] -array birdflight -code {
    parray birdflight
    puts "----"

    if {$birdflight(ident) != "N1615A" || $birdflight(fp) == "N1615A-airline-12342223"} {
        puts "BAD!"
    }
}

[Peter da Silva]

The problem was that the optimizer was prematurely setting the walk-type to a HASH walk rather than an INDEX walk when evaluating a possible HASH walk, after setting up for an INDEX walk, but before discarding the HASH. So it was performing a HASH walk with a search structure set up for an INDEX walk.

Let me know if this latest update fixes it.

[Jeff Lawson]

Yup, seems to work for my example! Though it's hard for me to say if it regresses anything else, since this type of problem wasn't caught by the current test cases.

You left the debug printing enabled in your commit, so another commit is needed to disable or remove it.

So the problem wasn't specifically due to the != operator, but the types of two indexes being used to optimize the search. This could have a lot of unexpected and wide-impacting behavior.

[Peter da Silva]

You got into that quickly. I did another commit to remove the debug print shortly afterwards.

I'm pretty sure there's no regression, I know what decision I made that caused this one to happen, it was a switch to using indexes rather than hashes for some searches.

[Jeff Lawson]

Closing, since your commit seems to fix it.