Array(DateTime) columns loading as null

[Ottony Costa]

When there's an array column with DateTime (e.g. Array(DateTime)), it's loaded as nil even if not null.

Example:

-- `array_column` Array(DateTime)

INSERT INTO sample_table (id, array_column) VALUES (1, '[''2022-12-06 15:22:49'',''2022-12-06 15:22:49'']')

> SampleTable.find(1).as_json
 => {"array_column" => nil, "id" => 1}

Removing the OID::Array :datetime subtype or changing to any other like :string, :integer or :DateTime, seems to load the value as a string.

class Array < Type::Value # :nodoc:
  def initialize(sql_type)
    @subtype = case sql_type
      when /U?Int\d+/
        :integer
      when /DateTime/
        :integer
      when /Date/
        :date
      else
        :string
      end
  end
  
  ...
end

After:

> SampleTable.find(1).as_json
 => {"array_column" => ["2022-12-06 15:22:49", "2022-12-06 15:22:49"], "id" => 1}
 
> SampleTable.find(1).array_column.map(&:class)
 => [String, String]

Version:

• rails 6.1.7
• clickhouse-activerecord 0.5.11
• ClickHouse 22.8.6.71

[Sergey Odintsov]

Please write your version of ClickHouse and Rails

[Ottony Costa]

I'm using

• rails 6.1.7
• clickhouse-activerecord 0.5.11
• ClickHouse 22.8.6.71

[Juliana Lucena]

Even after the fix mentioned above, the problem remains.

I added more column types to verify:

CREATE TABLE IF NOT EXISTS sample_table
(
  id UInt32,
  datetime_nullable_column Nullable(DateTime),
  array_nullable_datetime Array(Nullable(DateTime)),
  array_nullable_integer Array(Nullable(Integer)),
  array_datetime Array(DateTime),
  array_integer Array(Integer)
)
ENGINE = MergeTree()
ORDER BY(id)

Example

INSERT INTO sample_table (
  id,
  datetime_column,
  array_nullable_datetime,
  array_nullable_integer,
  array_datetime,
  array_integer
) VALUES (
  1,
  '2022-12-06 15:22:49',
  '[''2022-12-06 15:22:49'',''2022-12-06 15:22:49'']',
  [1, 2],
  '[''2022-12-06 15:22:49'',''2022-12-06 15:22:49'']',
  [3, 4]
)

INSERT INTO sample_table (
  id,
  datetime_column,
  array_nullable_datetime,
  array_nullable_integer,
  array_datetime,
  array_integer
) VALUES (
  2,
  NULL,
  '[NULL,''2022-12-06 15:22:49'']',
  [NULL, 2],
  '[''2022-12-06 15:22:49'',''2022-12-06 15:22:49'']',
  [3, 4]
)

The datetime column loads nicely, but Array(Datetime) does not:

irb(main):001:0> SampleTable.find(1, 2)
  Clickhouse SampleTable Load (3.0ms)  SELECT sample_table.* FROM sample_table WHERE sample_table.id IN (1, 2)
=>
[#<SampleTable:0x00007f6af4149ce8
  id: 1,
  datetime_nullable_column: Tue, 06 Dec 2022 15:22:49.000000000 UTC +00:00,
  array_nullable_datetime: nil,
  array_nullable_integer: [1, 2],
  array_datetime: nil,
  array_integer: [3, 4]>,
 #<SampleTable:0x00007f6af539f460
  id: 2,
  datetime_nullable_column: nil,
  array_nullable_datetime: nil,
  array_nullable_integer: [nil, 2],
  array_datetime: nil,
  array_integer: [3, 4]>]

After applying the monkey patch explained by @ottony, all columns are loaded:

irb(main):001:0> SampleTable.find(1, 2)
  Clickhouse SampleTable Load (9.2ms)  SELECT sample_table.* FROM sample_table WHERE sample_table.id IN (1, 2)
=>
[#<SampleTable:0x00007f7e270a6520
  id: 1,
  datetime_nullable_column: Tue, 06 Dec 2022 15:22:49.000000000 UTC +00:00,
  array_nullable_datetime: ["2022-12-06 15:22:49", "2022-12-06 15:22:49"],
  array_nullable_integer: [1, 2],
  array_datetime: ["2022-12-06 15:22:49", "2022-12-06 15:22:49"],
  array_integer: [3, 4]>,
 #<SampleTable:0x00007f7e1ee55968
  id: 2,
  datetime_nullable_column: nil,
  array_nullable_datetime: [nil, "2022-12-06 15:22:49"],
  array_nullable_integer: [nil, 2],
  array_datetime: ["2022-12-06 15:22:49", "2022-12-06 15:22:49"],
  array_integer: [3, 4]>

Versions:

• rails (6.1.7.6)
• clickhouse-activerecord (0.6.0)
• clickhouse-server 23.3.8.22.altinitystable