post_checkout and pre_checkin callbacks
sjones512 opened this issue · comments
Hello,
I'm trying to start up a project that is using db_connection 2.4.3. I would like to know when a connection is checked out/in.
It looks like this behavior is offered in the DBConnection.Ownership pool, but I am having trouble understanding the differences between this pool in :auto mode and the default pool. Is the Ownership pool acceptable for use in a project, or is it only intended for testing?
I also couldn't find any examples of how to use the pool in manual mode. I am still new to elixir and learning Phoenix and I am confused as to which process should be doing the checkout since the db connection processes seem ephemeral. This didn't really matter as :auto seems acceptable for my use case, although I admit I don't fully understand the implications of choosing the Ownership pool in :auto vs the default ConnectionPool.
I also couldn't get the post_checkout or pre_checkin callbacks to call. I think I tracked this down to DBConnection.Manager not passing along the callbacks to the checkout_opts on line 84, but I'm not sure I'm fully understanding the code.
Is there another way I can enable post_checkout and pre_checkin, or would it require a change in db_connection?
Thank you for any help!
You should be able to use listeners for this purpose: https://github.com/elixir-ecto/db_connection/blob/master/lib/db_connection.ex#L419
@josevalim are those called on checkout?
I had tried implementing them and it seemed like they were only called when the connection was added to the pool at application startup, not when the connection is checked out by a process.
Maybe I didn't get the implementation quite right. I will try again.
Was finally able to test some again and, unless I'm missing something it looks like the DBConnectionListener is called once for each connection in the pool at application startup, and not again after that during checkout.
Sorry, you are right. So yes, currently we don't have such an API.
Can you please expand what you are ultimately trying to achieve? Note that you can wrap your own calls to DBConnection to add the callbacks that you need.
I would like to try and SET LOCAL variables in Postgres before each query.
I tried doing it in prepare_query, but it didn't seem to be guaranteed to use the same connection as the subsequent query.
I tried using a Plug that wrapped the Router plug, but then queries that came through the Live Socket/Channel weren't affected.
If I can wrap my own calls to DBConnection, that might work. How would I go about that?
Thanks for your help with this!
I tried doing it in prepare_query, but it didn't seem to be guaranteed to use the same connection as the subsequent query.
Are you using Ecto? If so there is a Repo.checkout function: https://hexdocs.pm/ecto/Ecto.Repo.html#c:checkout/2
I would like to try and SET LOCAL variables in Postgres before each query.
but why? I am not sure you will find an easy way to achieve this :(
I would like to try and SET LOCAL variables in Postgres before each query.
Reading the Postgres docs, SET LOCAL is only valid for the transaction its set in. Then its effects disappear. Maybe what you need is simply wrapping your queries in a transaction with that statement at the beginning?
But similar to Jose's sentiments, whatever you are trying to achieve can probably be done simpler. If you're needing to configure parameters on every query instead of just a small set of unique queries, it feels like a smell.
I appreciate all the feedback, and I'll take it in to consideration.
However, I'm still curious what the role of the Ownership pool and its post_checkout and pre_checkin callbacks is? If these aren't intended to be used or supported perhaps the documentation could be updated to more explicitly state that, and more importantly why that is the case.
Thanks again.
They are used indeed as post and pre checking but the ownership pool has a very specific use case that is mostly related to testing and it is not general purpose.
So are the pre and post checkout methods supposed to work? Because I wasn't able to get them to work without modifying manager.ex to include them in the chekcout_opts in the init function.
I would like to try and SET LOCAL variables in Postgres before each query.
but why? I am not sure you will find an easy way to achieve this :(
I would like to utilize Postgres Row Level Security, and at this point in time the best way I know to achieve this is by setting Postgres variables before a query is invoked.
They should work if you are using ownership. See how Ecto.Adapters.SQL.Sandbox uses them. :)
for the variables, then you will need transactions as noted by @greg-rychlewski. So you can explicitly start a transaction around your queries .
hmm, glancing at the code it seems to confirm what I had said initially.
The callbacks don't seem to work when they are included in the opts during init because they are never added to the state.
I assume they only work when you pass them to ownership_checkout since that is what the sandbox appears to do here:
if Keyword.get(opts, :sandbox, true) do
[
post_checkout: &post_checkout(&1, &2, opts),
pre_checkin: &pre_checkin(&1, &2, &3, opts)
] ++ pool_opts
else
pool_opts
end
pool_opts_overrides = Keyword.take(opts, [:ownership_timeout, :isolation_level])
pool_opts = Keyword.merge(pool_opts, pool_opts_overrides)
case DBConnection.Ownership.ownership_checkout(pool, pool_opts) do```