Cannot import SQLite array columns for various types into PostgreSQL (`TEXT[]`, `NUMERIC[]`, `BYTE[]`)

[Joey Pereira]

It seems like the pgloader grammer simply cannot handle <TYPE>[], leading to inability to parse these files at all. I was just trying to import a misc app into Postgres, so I'm not exactly sure what the type is (JSON? Something else?) but the application bootstrapped the Postgres tables with columns that had the exact same types so PostgreSQL clearly supports the same thing.

Specifically, the SQLite's column types were text[] defaut '{}'. The postgres bootstrapped tables appear to be text[] NULL [{}] (I didn't check if the former syntax is compatible).

---

• pgloader --version

  pgloader version "3.6.7~devel"
  compiled with SBCL 2.2.9.debian
  

• did you test a fresh compile from the source tree?

  I did not, but I saw no changes on HEAD which would appeared related to this issue.

• did you search for other similar issues?

  • There's a few other issues on the SQLITE-TYPE-NAME parsing, such as:
    • #1533 (NUMERIC[])
    • #1231 (BYTE[])

• how can I reproduce the bug?

# Generating the SQL file
sqlite3 /tmp/test.db \
  "CREATE TABLE foo(id TEXT PRIMARY KEY, broken TEXT[] DEFAULT '{}');" \
  "INSERT INTO foo VALUES ('bar', '{}');" \
  "CREATE TABLE foo2(id TEXT PRIMARY KEY, broken NUMERIC[] DEFAULT '{}');" \
  "INSERT INTO foo2 VALUES ('bar', '{}');" 

# pgloader command (plain ol' import)
pgloader /tmp/test.db postgresql://localhost/testdb

• pgloader output you obtain

2023-12-29T17:52:56.016000Z LOG pgloader version "3.6.7~devel"
2023-12-29T17:52:56.120000Z LOG Migrating from #<SQLITE-CONNECTION sqlite:///tmp/test.db {1005D913E3}>
2023-12-29T17:52:56.120000Z LOG Migrating into #<PGSQL-CONNECTION pgsql://localhost/testdb {1005F57353}>
2023-12-29T17:52:56.156000Z ERROR sqlite: At

  text[]
      ^ (Line 1, Column 4, Position 4)

In context SQLITE-TYPE-NAME:

While parsing SQLITE-TYPE-NAME. Problem:

  The production

    #\]

  does not satisfy the predicate ALPHA-CHAR-P.

Expected:

     the character _ (LOW_LINE)
  or any character satisfying ALPHA-CHAR-P
  or <end of input>
2023-12-29T17:52:56.156000Z LOG report summary reset
       table name     errors       rows      bytes      total time
-----------------  ---------  ---------  ---------  --------------
            fetch          0          0                     0.000s
  fetch meta data          0          0                     0.000s
-----------------  ---------  ---------  ---------  --------------
-----------------  ---------  ---------  ---------  --------------