Giters

voltrondata / flight-sql-server-example

An example Flight SQL Server implementation - with DuckDB and SQLite back-ends.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Connect to duckdb file from external process

TaoVonQi opened this issue · comments

commented

Is it possible to establish a read/write connection to the same duckdb file to perform some write operations and then close the connection?

It seems that the connection is coupled to the state of the server and it does not re-establish a connection every time to allow other external processes to connect-write-disconnect to the same duckdb file.

I have not yet tried it but I'm very eager to do so soon. I'm sorry if that's not the case. My C++ is very rusty.

commented

Hi @TaoVonQi - I see your point, but why not just connect to the Flight SQL Server and perform the operations you refer to? I assume you are running SQL operations against the database, right?

commented

That makes perfect sense! Thank you! I'm very eager to test this tomorrow with the Grafana flightsql plugin

commented

I actually need to refresh the duckdb database with newly added parquet files.

I was thinking of running a query like so:

CREATE TABLE t1 AS SELECT * FROM 'test.parquet';

Do you think this will work? I mean it's valid duckdb SQL so it should work right?

commented

yep - that should work - you should probably do CREATE OR REPLACE TABLE t1... if you want the same table name. You should be able to run any DuckDB command (I haven't run into any yet that I cannot run).

commented

Wonderful! Thank you!!

commented

hi @TaoVonQi - if you are just adding parquet files to an existing directory (appending data) - you could just have a view pointing to the directory - like:
CREATE OR REPLACE VIEW parquet_view AS SELECT * FROM '/some_parquet_directory/*.parquet';

That way - you shouldn't have to refresh anything...