no function clause matching in Poison.Encoder.BitString.chunk_size/3
seanwash opened this issue · comments
Hey all,
Thanks for this awesome tool! I've noticed that on certain pages the tool bar doesn't render and I see an error in my console. I'll keep digging to see if I can't find out any more useful information.
Phoenix 1.3
Elixir 1.5
Trace:
[info] JOIN "toolbar:request:pib7tg9s0e4d6e8l6coj9tir8v771ni3" to ExDebugToolbar.ToolbarChannel
Transport: Phoenix.Transports.WebSocket (1.0.0)
Serializer: Phoenix.Transports.WebSocketSerializer
Parameters: %{}
[info] Replied toolbar:request:pib7tg9s0e4d6e8l6coj9tir8v771ni3 :ok
[error] Ranch protocol #PID<0.3060.0> (:cowboy_protocol) of listener DkWeb.Endpoint.HTTP terminated
** (exit) exited in: Phoenix.Endpoint.CowboyWebSocket.resume()
** (EXIT) an exception was raised:
** (FunctionClauseError) no function clause matching in Poison.Encoder.BitString.chunk_size/3
(poison) lib/poison/encoder.ex:127: Poison.Encoder.BitString.chunk_size(<<255, 255, 255, 255, 255, 255, 255, 255, 64, 0, 0, 0, 0, 0, 0, 0, 0, 0, 64, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 131, 0, ...>>, nil, 0)
(poison) lib/poison/encoder.ex:122: Poison.Encoder.BitString.escape/2
(poison) lib/poison/encoder.ex:97: Poison.Encoder.BitString.escape/2
(poison) lib/poison/encoder.ex:124: Poison.Encoder.BitString.escape/2
(poison) lib/poison/encoder.ex:84: Poison.Encoder.BitString.encode/2
(poison) lib/poison/encoder.ex:232: anonymous fn/3 in Poison.Encoder.List.encode/3
(poison) lib/poison/encoder.ex:233: Poison.Encoder.List."-encode/3-lists^foldr/2-1-"/3
(poison) lib/poison/encoder.ex:233: Poison.Encoder.List.encode/3
(poison) lib/poison/encoder.ex:213: anonymous fn/4 in Poison.Encoder.Map.encode/3
(poison) lib/poison/encoder.ex:214: Poison.Encoder.Map."-encode/3-lists^foldl/2-0-"/3
(poison) lib/poison/encoder.ex:214: Poison.Encoder.Map.encode/3
(poison) lib/poison/encoder.ex:232: anonymous fn/3 in Poison.Encoder.List.encode/3
(poison) lib/poison/encoder.ex:233: Poison.Encoder.List."-encode/3-lists^foldr/2-1-"/3
(poison) lib/poison/encoder.ex:233: Poison.Encoder.List.encode/3
(poison) lib/poison/encoder.ex:232: anonymous fn/3 in Poison.Encoder.List.encode/3
(poison) lib/poison/encoder.ex:233: Poison.Encoder.List."-encode/3-lists^foldr/2-1-"/3
(poison) lib/poison/encoder.ex:233: Poison.Encoder.List.encode/3
(poison) lib/poison/encoder.ex:232: anonymous fn/3 in Poison.Encoder.List.encode/3
(poison) lib/poison/encoder.ex:233: Poison.Encoder.List.encode/3
(poison) lib/poison/encoder.ex:213: anonymous fn/4 in Poison.Encoder.Map.encode/3
Hey @seanwash,
Error is happening in toolbar's channel when it tries to serialize the payload. Could you give more details about pages that cause this error? Do they have something in common? (a query, a log, maybe http headers, etc)
Are you serializing a binary? Maybe a UUID? See devinus/poison#66
I'm getting this issue too. The map being encoded includes UUIDs in params of the Ecto logs being collected.
%{ansi_color: :cyan, caller_pid: #PID<0.602.0>, connection_pid: nil,
decode_time: 81001,
params: [[<<100, 31, 228, 56, 162, 105, 77, 149, 180, 17, 110, 24, 193, 73, 176, 217>>,
<<241, 242, 243, 240, 5, 83, 65, 246, 176, 100, 249, 148, 128, 64, 38, 160>>,
<<11, 201, 57, 20, 118, 19, 74, 48, 180, 186, 104, 214, 193, 208, 51, 160>>]],
query: "SELECT...WHERE (l0.\"id\" = ANY($1))",
query_time: 3200002, queue_time: 135000,
result: {:ok,
%Postgrex.Result{columns: [...], command: :select, connection_id: 60492, num_rows: 3, rows: []}},
source: "listings"}Poison is borking on encoding the UUID.
(poison) lib/poison/encoder.ex:139: Poison.Encoder.BitString.chunk_size(<<201, 57, 20, 118, 19, 74, 48, 180, 186, 104, 214, 193, 208, 51, 160>>, nil, 0)
(poison) lib/poison/encoder.ex:134: Poison.Encoder.BitString.escape/2
(poison) lib/poison/encoder.ex:101: Poison.Encoder.BitString.escape/2
(poison) lib/poison/encoder.ex:88: Poison.Encoder.BitString.encode/2Note that it matches the first char as a Unicode control char, then tries to chunk the rest.
hey, if i'm not mistaken, Ecto.UUID is serializable out of the box. How/what do you have there? can you show the schema definition?
That is the map being encoded by Poison, which appears to be the log entries being collected. It would seem that the log entry isn't representing the params as Ecto.UUID, but as just BitString.
All schemas have
@primary_key {:id, Ecto.UUID, autogenerate: true}
@foreign_key_type Ecto.UUIDI am able to work around the issue, but it requires reimplementing https://github.com/devinus/poison/blob/master/lib/poison/encoder.ex#L111
defimpl Poison.Encoder, for: BitStringOne can either load the UUID with Ecto.UUID.load/1 or recreate Ecto.UUID.encode/1, after matching on it, and placing that encoder ahead of https://github.com/devinus/poison/blob/master/lib/poison/encoder.ex#L131
defp escape(<<char>> <> rest, mode) when char <= 0x1F or char == 0x7FThis seems rather heavy handed though, as BitString is a pretty big impl. 🤔
@erikreedstrom, can you give this branch a try fix/collectin_ecto_logs_with_uuid to confirm it fixes the issue in your app?
@erikreedstrom, @seanwash hex package has been updated to fix the issue.
@kagux Thanks so much for the update, I apologize for not chiming back in, it's been a busy week over here.
I've updated to 0.3.15 and Im still getting the same error. I'm looking around for anything common between the requests but I'm not finding anything yet.
@seanwash, I assume you don't use Ecto.UUID, right?
This issue is triggered by a binary value in logs or queries. Can you think of what could it be?
That's correct, I'm not using Ecto.UUID. Im racking my brains over here! I'll make sure to post back as soon as I find something. It only happens on a few pages so it has to be something very specific to them.
is the stack trace you posted in the beginning complete??
Yep!
Here's a screenshot of the queries and the stack https://cl.ly/mGBt
what's the data type of store_images.image field?
The definition is field :image, Dk.Media.BasicImage.Type. I'm using arc_ecto to manage media attachments.
https://github.com/stavro/arc_ecto/blob/master/lib/arc_ecto/type.ex
https://github.com/stavro/arc_ecto/blob/master/lib/arc_ecto/schema.ex
doesn't look like images have any binary values, the field is string.
Would you be able to print out conn inspect on failing controller?
Sure, I checked that as well and I didn't see any binary values. Here it is with the cookie values removed:
%Plug.Conn{adapter: {Plug.Adapters.Cowboy.Conn, :...},
assigns: %{current_user: %{__meta__: #Ecto.Schema.Metadata<:loaded, "users">,
__struct__: Dk.User, bio: "I make web apps and music.",
city: "South Royalton", created_at: ~N[2016-04-15 20:13:13.528119],
current_password: nil, email: "sean@domain.com",
encrypted_password: "",
failed_attempts: 0, first_name: "Sean", id: 4,
image: %{file_name: "Sean_2.jpg", updated_at: nil},
last_name: "Washington", locked_at: nil,
orders: #Ecto.Association.NotLoaded<association :orders is not loaded>,
password: nil, password_confirmation: nil,
reset_password_sent_at: #Ecto.DateTime<2017-04-05 18:32:33.617993>,
reset_password_token: "",
state: "Vermont",
stores: #Ecto.Association.NotLoaded<association :stores is not loaded>,
unlock_token: nil, updated_at: ~N[2017-08-29 18:52:37.657299]}},
before_send: [#Function<0.96292254/1 in Plug.CSRFProtection.call/2>,
#Function<4.13225457/1 in Phoenix.Controller.fetch_flash/2>,
#Function<0.112984571/1 in Plug.Session.before_send/2>,
#Function<1.127499069/1 in Plug.Logger.call/2>,
#Function<0.28837570/1 in Phoenix.LiveReloader.before_send_inject_reloader/2>,
#Function<0.73665496/1 in ExDebugToolbar.Collector.ConnCollector.call/2>,
#Function<0.129890256/1 in ExDebugToolbar.Plug.CodeInjector.call/2>],
body_params: %{},
cookies: %{"_designkollective_key" => "",
"_ga" => "GA1.1.168630523.1503945289",
"_gid" => "GA1.1.217299926.1504156616",
"_knightfoundation_session" => "",
"connect.sid" => "",
"intercom-id-gvmy47vl" => ""},
halted: false, host: "localhost", method: "GET", owner: #PID<0.4238.0>,
params: %{}, path_info: ["join"], path_params: %{},
peer: {{127, 0, 0, 1}, 54013}, port: 4000,
private: %{DkWeb.Router => {[],
%{Plug.Swoosh.MailboxPreview => ["mailbox"], RouterPlug.Router => []}},
:phoenix_action => :join,
:phoenix_controller => DkWeb.StoreOnboardingController,
:phoenix_endpoint => DkWeb.Endpoint, :phoenix_flash => %{},
:phoenix_format => "html",
:phoenix_layout => {DkWeb.LayoutView, "onboarding.html"},
:phoenix_pipelines => [:protected], :phoenix_router => DkWeb.Router,
:phoenix_view => DkWeb.StoreOnboardingView,
:plug_route => #Function<3.42728315/1 in ExDebugToolbar.Plug.Router.do_match/4>,
:plug_session => %{"_csrf_token" => "hG4dHhjMk6HatK0d1rglRg==",
"_navigation_history_default" => "/join|/join|/join|/join|/admin/details|/stores/dacha|/stores|/|/join|/join",
"session_auth" => "d5767fb8-8e54-11e7-b898-f40f240ee217",
"user_return_to" => nil}, :plug_session_fetch => :done,
:plug_session_info => :write,
:request_id => "tbli26hm048us87st2bh98rnfet7at3e"}, query_params: %{},
query_string: "", remote_ip: {127, 0, 0, 1},
req_cookies: %{"_designkollective_key" => "",
"_ga" => "GA1.1.168630523.1503945289",
"_gid" => "GA1.1.217299926.1504156616",
"_knightfoundation_session" => "",
"connect.sid" => "",
"intercom-id-gvmy47vl" => "d5b76513-efe5-463d-bad7-937f7ed358b8"},
req_headers: [{"host", "localhost:4000"}, {"connection", "keep-alive"},
{"cache-control", "max-age=0"}, {"upgrade-insecure-requests", "1"},
{"user-agent",
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.113 Safari/537.36"},
{"accept",
"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8"},
{"dnt", "1"}, {"accept-encoding", "gzip, deflate, br"},
{"accept-language", "en-US,en;q=0.8"},
{"cookie", ""},
{"x-request-id", "tbli26hm048us87st2bh98rnfet7at3e"}], request_path: "/join",
resp_body: nil, resp_cookies: %{},
resp_headers: [{"cache-control", "max-age=0, private, must-revalidate"},
{"x-request-id", "tbli26hm048us87st2bh98rnfet7at3e"}, {"connection", "close"},
{"x-frame-options", "SAMEORIGIN"}, {"x-xss-protection", "1; mode=block"},
{"x-content-type-options", "nosniff"}], scheme: :http, script_name: [],
secret_key_base: "",
state: :unset, status: nil}
looks alright to me as well.
There is an issue #33 to allow printing out debug info on demand exactly for such ocasions. So, one option is two wait till we add that to print out more logs. Or, you could change in your deps {:ex_debug_toolbar, path: "deps/ex_debug_toolbar"} and then update locally deps/ex_debug_toolbar/web/channels/toolbar_channel.ex#L37 with a new line IO.inspect(request) which should print the struct it tries to serialize
hey @seanwash,
Debug mode has been merged into master and will be available in hex in couple days. Update config to enable it:
config :ex_debug_toolbar,
enable: true,
debug: true
Once enabled you'll see detailed output of what's being serialized. I'd appreciate if you could share it. With that we should nail the cause of this issue.
@seanwash, I'll close this issue for now, but should you decide to take a another go with debug mode, feel free to reopen this issue
Hi guys,
this tool is great but I had some problems too: on certain pages the toolbar doesn't render and I found this correlated issue.
I think I have identified the problem, at least in my case, and solved it so I wanted to share: maybe it'll help you and the solution can be merged in the hex package.
I'm on: Phoenix 1.3 and Elixir 1.5 and enabling debug configuration for ex_debug_toolbar print a debug trace very similar to that reported on begin of this post except for the numbers listed in the bitstring.
After many tries I've identified that the problems was only on form pages where my controllers assigned a model changeset that have a "validate_format" validation with a Regex as param.
In this conditions the problem is that ex_debug_toolbar requires to Poison the json encoding of the changeset: it contains the regex param inside changeset.validations and Poison don't know how to encode this regex.
The solution is very simple: you can add this code wherever in you project.
defimpl Poison.Encoder, for: Regex do
def encode(r, options) do
Poison.Encoder.BitString.encode(r.source, options)
end
end
After you'll have to renew the protocols consolidation with mix compile.protocols, next restart your Phoenix server and the issue was resolved!
I think the solution concerns this package and not the Poison package because it's very unusual need encoding regex to json while it's usual to use validate_format while using this package.
Hey @moisella,
Appreciate you spending time investigating the issue!
We'll patch it up following your suggestion. In fact we had to introduce encoders for other types for exact same cause.
Although, now that I think of it, this behavior should only trigger in debug mode. Not a huge fan of introducing encoders by default as they might interfere with main application.
@moissela, I've just published new version 0.4.4 that includes regex encoder
thanks again for your feedback!