PNixx / clickhouse-activerecord

A Ruby database ActiveRecord driver for ClickHouse

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

ActiveRecord::ActiveRecordError: Response code: 400: Code: 26. DB::Exception: Cannot parse quoted string: expected opening quote ''', got '"': while converting '{"value"=>"2", "agent_name"=>"frodo"}' to Map(String, String): at row 0: While executing ValuesBlockInputFormat. (CANNOT_PARSE_QUOTED_STRING) (version 24.1.5.6 (official build))

dorianmariecom opened this issue · comments

I'm getting this error with lago.

You can see the source code at https://github.com/dorianmariecom/lago-api/tree/dorian/clickhouse-activerecord-bug

Also happens with properties: properties and with properties.to_json

      Failure/Error:
        Clickhouse::EventsRaw.create!(
          transaction_id: SecureRandom.uuid,
          organization_id: organization.id,
          external_subscription_id: subscription.external_id,
          external_customer_id: customer.external_id,
          code:,
          timestamp: values[:timestamp],
          properties: properties.transform_values(&:to_s).transform_keys(&:to_s),
        )
      
      ActiveRecord::ActiveRecordError:
        Response code: 400:
        Code: 26. DB::Exception: Cannot parse quoted string: expected opening quote ''', got '"': while converting '{"value"=>"2", "agent_name"=>"frodo"}' to Map(String, String):  at row 0: While executing ValuesBlockInputFormat. (CANNOT_PARSE_QUOTED_STRING) (version 24.1.5.6 (official build))
      # /usr/local/bundle/bundler/gems/clickhouse-activerecord-32cb41de3108/lib/active_record/connection_adapters/clickhouse/schema_statements.rb:124:in `process_response'
      # /usr/local/bundle/bundler/gems/clickhouse-activerecord-32cb41de3108/lib/active_record/connection_adapters/clickhouse/schema_statements.rb:79:in `block in do_execute'
      # /usr/local/bundle/bundler/gems/clickhouse-activerecord-32cb41de3108/lib/active_record/connection_adapters/clickhouse/schema_statements.rb:74:in `do_execute'
      # /usr/local/bundle/bundler/gems/clickhouse-activerecord-32cb41de3108/lib/active_record/connection_adapters/clickhouse/schema_statements.rb:15:in `exec_insert'
      # ./spec/services/events/stores/clickhouse_store_spec.rb:996:in `block (4 levels) in <top (required)>'
      # ./spec/services/events/stores/clickhouse_store_spec.rb:991:in `map'
      # ./spec/services/events/stores/clickhouse_store_spec.rb:991:in `block (3 levels) in <top (required)>'
      # ./spec/services/events/stores/clickhouse_store_spec.rb:74:in `block (2 levels) in <top (required)>'
      # ./spec/spec_helper.rb:33:in `block (3 levels) in <top (required)>'
      # /usr/local/bundle/gems/database_cleaner-core-2.0.1/lib/database_cleaner/strategy.rb:30:in `cleaning'
      # /usr/local/bundle/gems/database_cleaner-core-2.0.1/lib/database_cleaner/cleaners.rb:34:in `block (2 levels) in cleaning'
      # /usr/local/bundle/gems/database_cleaner-core-2.0.1/lib/database_cleaner/cleaners.rb:35:in `cleaning'
      # ./spec/spec_helper.rb:32:in `block (2 levels) in <top (required)>'
      # /usr/local/bundle/gems/webmock-3.22.0/lib/webmock/rspec.rb:39:in `block (2 levels) in <top (required)>'

I fixed it by doing:

    def self.create!(**attributes)
      if attributes[:properties].present?
        attributes[:properties] = attributes[:properties]
          .transform_values(&:to_s)
          .to_json
          .gsub('"', "'")
      end

      super(**attributes)
    end

Actually I did:

# frozen_string_literal: true

module Clickhouse
  class EventsRaw < BaseRecord
    self.table_name = 'events_raw'

    def properties=(properties)
      super(properties.transform_values(&:to_s).to_json.gsub('"', "'"))
    rescue
      super(properties)
    end

    def properties
      JSON.parse(super.gsub("'", '"'))
    rescue
      super
    end
  end
end

Even more of a hack:

# frozen_string_literal: true

module Clickhouse
  class EventsRaw < BaseRecord
    self.table_name = 'events_raw'

    def properties=(properties)
      super(properties.transform_values(&:to_s).to_json.gsub('"', "'").gsub("=>", ":"))
    end

    def properties
      JSON.parse(super.gsub("'", '"').gsub("=>", ":"))
    end
  end
end

@dorianmariecom Can you write the structure of your table?

This issue is related to a fork of the gem that is adding a (limited) support of the Map type.
See: https://github.com/getlago/clickhouse-activerecord

We (Lago) are willing to contribute to this gem with the changes but it's far from ready yet as the current implementation only supports Map(String, String) and is not tested at all