DPY-3013
sfanghaenel opened this issue · comments
- What versions are you using?
oracledb.version:
Linux-3.10.0-1160.105.1.el7.x86_64-x86_64-with-glibc2.17
True
3.9.18
-
Is it an error or a hang or a crash?
-
What error(s) or behavior you are seeing?
Hallo when i try executmany into a table i got the following error:
DPY-3013 unsupported Python type int for database type DB_TYPE_VARCHAR
I try to insert values into a table.
first i create e Table via:
CREATE TABLE (Name1 NUMBER(10), Name2 NUMBER(2))
everything works fine
after this i try to insert values in batches/chunks (each 10000)
The first iteration is fine with execute many
INSERT INTO (Name1, Name2) values (values for Name1, values for Name2)
Important: all values for Column Name2 are None in the first batch!
In the next batch/chunk:
all values for Column Name2 are None except the last 200 (from 9800-10000) are Integers between 0-6
in the executemany statement i ve got the above error.
What I am doing wrong?
Thanx Sven
python skript
import numpy as np
import oracledb
oracledb.init_oracle_client()
con = oracledb.connect(user=env.USER_MD_WSP,
password=env.PWD,
dsn=env.DNS)
cur = con.cursor()
sql = "CREATE TABLE TB_BUG (NAME1 NUMBER(10), NAME2 NUMBER(2))"
cur.execute(sql)
con.commit()
batch1 = [(n, None) for n in range(10000)]
sql = "insert into TB_BUG (NAME1, NAME2) values (:1, :2)"
cur.executemany(sql, batch1, batcherrors=True)
con.commit()
batch2 = [(n, None) for n in range(9000)]
batch2_1 = [(n, np.random.randint(0, 6)) for n in range(1000)]
batch2_all = batch2 + batch2_1
sql = "insert into TB_BUG (NAME1, NAME2) values (:1, :2)"
cur.executemany(sql, batch2_all, batcherrors=True)
con.commit()
This is not a bug but expected behavior. If you call executemany() and all of the values are NULL, then the driver assumes that the value is of type DB_TYPE_VARCHAR with length 1. When you call it again with some values that are integers the driver complains because you can't assign integers to DB_TYPE_VARCHAR. If you have this situation you need to call cursor.setinputsizes() like the following:
cur.setinputsizes(int, int) # or cur.setinputsizes(oracledb.DB_TYPE_NUMBER, oracledb.DB_TYPE_NUMBER)That will tell the driver that you intend to store integers (numbers) and the value null (None) can still be assigned.
Thanx for your Answer.
This behavior is a little bit strange because i define the Datatype explicit as a Number(2) in the create Table statement.
Some times i have a big Table with many rows and columns (20.000.000 x 200) with different Datatypes. Often are Null values in the first 1.000.00 rows in some columns. Than i run in this error. How can i fix this error?
Do i have to declare the Datytypes before:
e.G. Table with 4 Columns with Datatypes:
Varchar2(20), Number(10), Number(2), Varchar2(5)
cur.setinputsizes(?,?,?,?)
Thanx Sven
When performing an insert there is no way for the driver to know anything about the data unless you supply that information! If the data types are not known by your application and you don't mind incurring a round trip to the database to figure that out you can perform a query against the table you are inserting into and look at the metadata that is returned in Cursor.description, then make a call to Cursor.setinputsizes() as needed. This is necessary when you have so many null values as there is no way to differentiate between None intended for a NUMBER column and None intended for a DATE column and None intended for a VARCHAR2 column!
Is the driver attempting to insert None values as VACHAR2(1)? My naive assumption is that None is the equivalent of NULL, which can be inserted into any non-null column.
Yes, None is the equivalent of NULL which can be inserted into any column. The problem is that if you supply a None value there is no way to know the type of that data -- so the driver simply assumes VARCHAR2(1). You can avoid that assumption by using Cursor.setinputsizes().
I will close this under the assumption that your question has been answered. If not, please feel free to re-open!