dimitri / pgloader

Migrate to PostgreSQL in a single command!

Home Page:http://pgloader.io

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

bytes after `0x00` get silently truncated when importing from mysql `text` column to pgsql

n0099 opened this issue · comments

  • pgloader --version
    pgloader version "3.6.af8c3c1"
    compiled with SBCL 2.1.11.debian
    
  • did you test a fresh compile from the source tree?
  • did you search for other similar issues?
  • how can I reproduce the bug?
    LOAD DATABASE
         FROM mysql://user:@localhost/db
         INTO pgsql://user:@localhost/db
    
    WITH quote identifiers, on error stop, workers = 2, concurrency = 1, batch rows = 100, batch size = 1MB, prefetch rows = 1000
    
  • pgloader output you obtain
    2024-04-09T05:26:32.012000+08:00 LOG pgloader version "3.6.af8c3c1"                                
    2024-04-09T05:26:32.016000+08:00 LOG Data errors in '/tmp/pgloader/'                               
    2024-04-09T05:26:32.016000+08:00 LOG Parsing commands from file #P"my.load" 
    2024-04-09T05:26:32.108001+08:00 LOG Migrating from #<MYSQL-CONNECTION mysql://user@localhost:3306/db {1007FA2213}>
    2024-04-09T05:26:32.108001+08:00 LOG Migrating into #<PGSQL-CONNECTION pgsql://user@localhost:5432/db {1007FA2E33}>
    2024-04-09T09:07:46.803223+08:00 LOG report summary reset 
    
  • data that is being loaded, if relevant

    mysql

    CREATE TABLE test(t text);
    INSERT INTO test(t) VALUES (CONCAT(0x020100, '1234'));
    SELECT hex(t) FROM test;
    HEX(t)
    02010031323334
  • How the data is different from what you expected, if relevant

    pgsql

    SELECT encode(t::bytea, 'hex') FROM test;
    encode
    0201

I guessed the reason in EnterpriseDB/mysql_fdw#299, and suggest that there should be a error or warning when inserting bytes contains 0x00 to column with type text.