Giters

elixir-ecto / db_connection

Database connection behaviour

Home Page:http://hexdocs.pm/db_connection/DBConnection.html

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Proposal: Allow different database per connection

greg-rychlewski opened this issue · comments

commented

Context

MySQL tests can't be run concurrently using the Ecto Sandbox because of deadlock issues. This has the potential to dramatically slow down a user's test suite.

One way around this is to ensure each concurrent test runs on a separate logical database.

Proposal

Currently the Sandbox eventually calls the code below, setting up a pool of connections with the same :database option:

children = for id <- 1..size, do: conn(owner, tag, id, mod, opts)

My proposal is to have a new option, working name: :database_pool. This would be a list of database names whose length must match the value of :pool_size. The database names would be passed out to the connections.

Alternatives Considered

I think this is an unconventional request so I wanted to list out the alternatives I could think of and the issues I see with them.

  1. Configure multiple repos and have each test use a different repo.

    • In this case, a test must know which of the repos are being used by other tests. I don't think it's possible to track this accurately in all cases. Say a connection belonging to a pool crashes unexpectedly. I believe the supervisor will just restart the connection and the process we use to track which pools are in use won't be able to find out about it.

  2. Launch multiple instances of the elixir app each connecting to a different db and run different tests on different instances.

    • This seems like a lot of work to set up and also collect the results.

  3. Create a separate package.

    • To keep all the goodies from the Ecto Sandbox/DBConnection this would end up being a copy and paste of the Ecto functionality with a couple of lines changed. I don't believe this option makes too much sense.

More Work

If this happens then Ecto SQL will need to be able to handle a list of databases when doing the migrations.

commented

I wonder if in this case it is better/simple to use mix test --partitions, which is already supported by Phoenix.

commented

It looks like it would help and it might be good enough. The downside I see is getting the equivalent of async: true needs one machine per async test. So it's potentially expensive for CI and difficult to do for local dev.

It's ok for me to say this issue is not worth it. I'm not using MySQL, was just trying to think of a way around its concurrency problem.

commented

Yeah, my reasoning for mix partition is because the migrations and everything work out of the box. So I would say this is not worth it at the moment. :)