Incorrect page count when joining two tables

[Eric Lathrop]

With this query:

    query = from u in User,
      join: r in assoc(u, :roles),
      where: not r.name in ["a", "b"],
      preload: [roles: r]

The sql returns 28 rows, but there's only 20 unique users.

%Scrivener.Page{entries: [...], page_number: 1, page_size: 10, total_entries: 28,
 total_pages: 3}

When I iterate through the items page 1 has 10 items, page 2 has 7 items, and page 3 has 3 items.

I'm pretty new to Ecto, so maybe my query is wrong, but it looks like scrivener isn't recognizing the duplicate rows from the join.

[Drew Olson]

@ericlathrop thanks for submitting the issue, this is definitely a problem. #12 may potentially fix this. I'm going to take a look in the next few days.

[Eric Lathrop]

I realize that the problem is probably impossible to solve for any general query. I think I need to rewrite my query to only return rows for User, then paginate it, then Repo.preload(:roles).

The next problem is that if I use distinct: true, I get the correct number of rows with Repo.all, but paginate excludes the select so it's back to the full number.

I then tried using group_by: [...], which returns the correct number of rows with Repo.all, but explodes with this error in paginate:

** (Ecto.MultipleResultsError) expected at most one result but got 19 in query:

from u in Models.User,
  join: r in assoc(u, :roles),
  where: not(r.name in ["a", "b"]),
  group_by: [...],
  select: count("*")

[Asher Van Brunt]

@ericlathrop I ran into this issue as well, I'm not sure if it is currently solvable with Ecto until they implement subqueries. The issue is here: https://github.com/drewolson/scrivener/blob/master/lib/scrivener.ex#L152

When you run count on the query with group by statement it's going to return multiple counts (and not the total row count), one for each row, but the repo.one line expects there to be only one result.

[Drew Olson]

@exitface If you've got ideas for implementing something like this once subqueries are supported, I'd love to see a PR.

[Asher Van Brunt]

@drewolson I'm very new to elixir and ecto, I solved by own problem by "cheating" and dropping down to SQL by rewriting total_entries to this:

  defp total_entries(query, repo) do
    { sql, args } = Ecto.Adapters.SQL.to_sql(:all, repo, query)
    { :ok, %{ rows: [[count]] } } = Ecto.Adapters.SQL.query(repo, "SELECT COUNT(*) FROM (#{sql}) c", args)
    count
  end

Once subqueries are supported you obviously could do something similar, but more less janky. There's probably a better solution then that though.

[Drew Olson]

I see. PR #12 actually does something very similar. Perhaps I should just pull it in.

[Asher Van Brunt]

That seems to be the most straight forward way that I've seen. Looking at some other pagination implementations they use similar methods.

[Drew Olson]

Folks seeing this issue, please try using master. I've merged in #12.

[Christian Blavier]

Working great, thanks! 👍