Feature Request: Enforce query size limit

Question

Feature Request: Enforce query size limit

lordnynex opened this issue 9 years ago · comments

It seems cassandra has some sort of max query size?

I'm performing a very large WHERE IN () and it appears the query is being truncated consistently which throws a syntax error.

Cassandra returned error (Syntax_error): "line 1:6116 no viable alternative at input ')' (...

I'm not sure what this limit is but it would be nice if the driver knew about it.

Juarez Bochi · Answer 1 · Mon Mar 30 2015 20:23:22 GMT+0800 (China Standard Time)

Hi Brandon,

The frame is limited by the protocol in 256MB. Is it possible that you are
hitting this limit? How many elements and of which type are you using in
the WHERE IN clause?

Brandon B · Answer 2 · Tue Mar 31 2015 04:27:05 GMT+0800 (China Standard Time)

I believe this is a malformed query on my side. In the end the fix was to break the WHERE IN clause into multiple select's. I wish I had more time to chase this down, but the IN clause was around 1000 arguments.

I was thinking, it would be beneficial to my current and future work if I can pass a tailored object to bind params on select. Similar to :execute("INSERT INTO ...(some_set) VALUES(?)", { cassandra.set){}).

Juarez Bochi · Answer 3 · Tue Mar 31 2015 07:12:51 GMT+0800 (China Standard Time)

Sorry, I am not sure I understood your suggestion correctly. Do you mean named arguments like #28?

Could you please elaborate?

Anyway, I will close this issue for now. Please feel free to open it again if you are able to reproduce the issue.

Brandon B · Answer 4 · Tue Mar 31 2015 07:24:17 GMT+0800 (China Standard Time)

I want to use bind params on a large WHERE IN SET. As opposed to writing 'WHERE IN(?, ?, ?, ?, ?........)

I'd like to write something like
c:execute("SELECT id FROM table WHERE id IN(?)", { cassandra.whereIn({ flat table of values}))

Thibault Charbonnier · Answer 5 · Tue Mar 31 2015 07:54:58 GMT+0800 (China Standard Time)

You can already do this @lordnynex. A IN is a cassandra.list iirc.

So you can have

c:execute("SELECT id FROM table WHERE id IN ?", { cassandra.list({"foo", "bar"}))

Brandon B · Answer 6 · Tue Mar 31 2015 07:57:18 GMT+0800 (China Standard Time)

Oh awesome. I'll give it a shot. I was under the impression that cassandra.list was the equivalent of { '', '' } which seemed in-congruent with the where in syntax.

Thibault Charbonnier · Answer 7 · Tue Mar 31 2015 07:58:30 GMT+0800 (China Standard Time)

If not, give a try to one of the other types. I am 100% sure this is possible because I did it in the past, don't remember which type/syntax combination tho.

Juarez Bochi · Answer 8 · Tue Mar 31 2015 21:02:15 GMT+0800 (China Standard Time)

@lordnynex, @thibaultcha is right, you can use cassandra.list for that. There is an example in the specs: https://github.com/jbochi/lua-resty-cassandra/blob/master/spec/functional_spec.lua#L170