Multi-column `.join-table` produces whacky SQL
jonathanstowe opened this issue · comments
Given some code like:
use Red;
my $*RED-DB = database "SQLite";
my $*RED-DEBUG = True;
model Foo {
has Str $.a is column;
has Str $.b is column;
has Str $.c is column;
has Date $.foo-date is column = Date.today;
}
Foo.^create-table;
my Date $today = Date.today;
my Date $yesterday = Date.today.earlier(days => 1);
for Foo.^all.grep(*.foo-date eq $yesterday ).join-model( Foo, -> $a, $b { $a.a == $b.a && $a.b == $b.b && $a.c == $b.c && $b.foo-date eq $today } ) -> $foo {
say $foo;
}
The SQL generated is:
SQL : SELECT
"foo_1".a , "foo_1".b , "foo_1".c , "foo_1".foo_date as "foo-date"
FROM
"foo"
INNER JOIN "foo" as foo_1 ON ((((((((("foo".c != "foo_1".c AND "foo".b != "foo_1".b) AND "foo".a = "foo_1".a) AND "foo".b = "foo_1".b) OR ((("foo".a != "foo_1".a AND "foo".b = "foo_1".b) AND "foo".c = "foo_1".c) AND "foo".a = "foo_1".a)) OR ((("foo".c = "foo_1".c AND "foo".a = "foo_1".a) AND "foo".b != "foo_1".b) AND "foo".b = "foo_1".b)) OR ((("foo".c = "foo_1".c AND "foo".a != "foo_1".a) AND "foo".b != "foo_1".b) AND "foo".a = "foo_1".a)) OR ((("foo".c != "foo_1".c AND "foo".a != "foo_1".a) AND "foo".b = "foo_1".b) AND "foo".a = "foo_1".a)) OR ((("foo".c != "foo_1".c AND "foo".a != "foo_1".a) AND "foo".b != "foo_1".b) AND "foo".a = "foo_1".a)) OR ((("foo".b = "foo_1".b AND "foo".c != "foo_1".c) AND "foo".a = "foo_1".a) AND "foo".c = "foo_1".c)) OR ((("foo".a = "foo_1".a AND "foo".c = "foo_1".c) AND "foo".b = "foo_1".b) AND "foo_1".foo_date = ?)
WHERE
"foo".foo_date = ?
BIND: ["2021-06-26", "2021-06-25"]
Whereas I'd expect it to make something more like:
SELECT
"foo_1".a , "foo_1".b , "foo_1".c , "foo_1".foo_date as "foo-date"
FROM
"foo"
INNER JOIN "foo" as foo_1 ON ( "foo_1".a = "foo".a AND "foo_1".b = "foo".b AND "foo_1".c = "foo".c AND "foo_1".foo_date = ? )
WHERE
"foo".foo_date = ?
Obviously the generated SQL doesn't actually work with real data.
The self join in the example is just for convenience (and reflects my real world code,) it does the same with separate models.
Ah, laid siege to it and worked out how to achieve that:
for Foo.^rs.grep(*.foo-date eq $yesterday ).join-model( Foo, -> $a, $b { ($a.a == $b.a ) AND ($a.b == $b.b ) && $b.foo-date eq $today } ) -> $foo {
say $foo;
}
Does make the expected SQL, thought it's a little non-obvious (I'd forgotten about the special AND. ) Also it suggests that there is a need for (Red::AST, Red::AST) candidate for eq because that doesn't work here (it doesn't affect the result, but that's what most people would go given a Str typed column.)
It seems to be a problem on an Optmizer or the lack of an Optimizer (I mean here). I'll take a look as soon as I have some time...
Odd, eq yes should wait for (Red::AST, Red::AST) but also should ==. And Foo.a should return a Red::Column that IS a Red::AST
What about now? I've added the optimization:
⚙ fernando@MBP-de-Fernando ~/Red master ✚ raku -I. -e '
use Red;
my $*RED-DB = database "SQLite";
my $*RED-DEBUG = True;
model Foo {
has Str $.a is column;
has Str $.b is column;
has Str $.c is column;
has Date $.foo-date is column = Date.today;
}
Foo.^create-table;
my Date $today = Date.today;
my Date $yesterday = Date.today.earlier(days => 1);
for Foo.^all.grep(*.foo-date eq $yesterday ).join-model( Foo, -> $a, $b { $a.a == $b.a && $a.b == $b.b && $a.c == $b.c && $b.foo-date eq $today } ) -> $foo {
say $foo;
}
'
SQL : CREATE TABLE "foo" (
a varchar(255) NOT NULL ,
b varchar(255) NOT NULL ,
c varchar(255) NOT NULL ,
foo_date varchar(255) NOT NULL
)
BIND: []
SQL : SELECT
"foo_1".a , "foo_1".b , "foo_1".c , "foo_1".foo_date as "foo-date"
FROM
"foo"
INNER JOIN "foo" as foo_1 ON "foo".c = "foo_1".c AND "foo".b = "foo_1".b AND "foo".a = "foo_1".a AND "foo_1".foo_date = ?
WHERE
"foo".foo_date = ?
BIND: ["2021-06-27", "2021-06-26"]
Looks good to me 👍 The example is fine, I'll try it on the real code tomorrow,.
I think all that will be much easier with RakuAST...
Okay the original example is fine, but the wheels come off again if there are additional clauses:
use Red;
my $*RED-DB = database "SQLite";
my $*RED-DEBUG = True;
my $*RED-DEBUG-AST = True;
model Foo {
has Str $.a is column;
has Str $.b is column;
has Str $.c is column;
has Date $.foo-date is column = Date.today;
has Int $.d is column = 0;
has Int $.e is column = 0;
has Bool $.f is column = False;
}
Foo.^create-table;
my Date $today = Date.today;
my Date $yesterday = Date.today.earlier(days => 1);
for Foo.^rs.grep(*.foo-date eq $yesterday ).join-model( Foo, -> $a, $b { $a.a == $b.a && $a.b == $b.b && $a.c == $b.c && $b.foo-date eq $today && ( $a.e - $a.d == $b.e - $b.d ) && !$a.f } ) -> $foo {
say $foo;
}
Which results in:
SQL : SELECT
"foo_1".a , "foo_1".b , "foo_1".c , "foo_1".foo_date as "foo-date", "foo_1".d , "foo_1".e , "foo_1".f
FROM
"foo"
INNER JOIN "foo" as foo_1 ON ("foo".b = "foo_1".b AND "foo".c = "foo_1".c AND "foo".a = "foo_1".a AND "foo".e - "foo".d != "foo_1".e - "foo_1".d AND "foo_1".foo_date = ? AND "foo".e - "foo".d = "foo_1".e - "foo_1".d) OR ("foo".b = "foo_1".b AND "foo".a = "foo_1".a AND "foo".c = "foo_1".c AND "foo_1".foo_date = ? AND "foo".e - "foo".d = "foo_1".e - "foo_1".d AND ("foo".f == 0 OR "foo".f IS NULL))
WHERE
"foo".foo_date = ?
BIND: ["2021-06-28", "2021-06-28", "2021-06-27"]
Which is remedied by changing the && !$a.f to AND !$a.f
There's another weirdness in my actual code where it is losing the bind but I think that's something else.
The bind thing may be a difference between SQLite and Pg (where I'm running the actual code,) as otherwise the code is functionally identical.
Not Pg vs SQLite. Weird.
Even weirder, if $today, $yesterday are passed into a sub it loses the bind, if they are created locally it's fine, it appears to be whether the variable is writeable or not. So that's something for another time.
It seems to be more missing optimizations...
I think this is fine now.