Multiple Statements with Single Roundtrip to Postgre Database

[Philipp D. Czernitzki]

Hi I am wondering if it is possible to have a sql`` call with multiple sql statements that will be sent to the database in a single roundtrip. Does postgre's protocol support this? Does the postgre client abstract this feature away? Thanks for the help!

[Rasmus Porsager]

Yes, PostgreSQL calls this pipelining, and Postgres.js does this automatically, which is part of the reason it's fast 😊

[Philipp D. Czernitzki]

Thank you! In the docs I found the following about pipelining:

It's also possible to pipeline the requests in a transaction if needed by returning an array with queries from the callback function like this:

const result = await sql.begin(sql => [
  sql`update ...`,
  sql`update ...`,
  sql`insert ...`
])

Does it mean I need to return an array like this to do the pipelining?
Is it possible to pass the results around because I need reuse an auto generated ID in a following sql statement?

---

I have also come across the distinction of simple and extended. I need to use data from multiple JS objects in multiple statements. It sounds like extended can receive the data when passed in, but not more than one statement. And it seems simple can't do it because it can't accept any query parameters? Does this mean I can't do a single roundtrip?

The postgres wire protocol supports "simple" and "extended" queries. "simple" queries supports multiple statements, but does not support any dynamic parameters. "extended" queries support parameters but only one statement.

Reusing a whole sql query, seems reasonable for what I want to achieve
Let's say there are two tables

CREATE TABLE Variables (
    id bigint generated by default as identity,
    col1 text,
    col2 text,
	PRIMARY KEY (id)
); 
CREATE TABLE Variables2 (
    id bigint not null,
    col1 text,
    col2 text,
	PRIMARY KEY (id)
); 

and I want to execute multiple statements in a single roundtrip

WITH inserted_row AS (
  INSERT INTO Variables (col1, col2) VALUES ('obj1.var', 'obj2.var') RETURNING *
)
INSERT INTO Variables2 (id, col1, col2)
SELECT id, 'obj3.var', 'obj4.var'
FROM inserted_row;

then I would want to do:

await sql`
WITH inserted_row AS (
  INSERT INTO Variables (col1, col2) VALUES (${obj1.var}, ${obj2.var}) RETURNING *
)
INSERT INTO Variables2 (id, col1, col2)
SELECT id, ${obj3.var}, ${obj4.var}
FROM inserted_row;
`

as far as I understand I need to add .simple() because it's multiple statements and I can't do ${} because simple does not support parameters?

[Rasmus Porsager]

CTEs are not considered as independent statements. A quick loose rule is that you have a statement once you put a semicolon 😉

With regards to transactions, then yes if you want to pipeline all the queries in your transaction you can do as mentioned, but if you need any values returned to use in js land you're kind of breaking up pipelining anyway so it doesn't really matter?

[Philipp D. Czernitzki]

CTEs are not considered as independent statements.

now it makes sense, thank you!