neisbut / Npgsql.Bulk

Helper for performing COPY (bulk insert and update) operation easily, using Entity Framework + Npgsql.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Database.SetCommandTimeout not respected

gregtayl opened this issue · comments

I'm experiencing random timeouts when trying to insert a large number of entities (50,000+). I've tried increasing the command timeout via Database.SetCommandTimeout, however, NpgSqlBulkUploader does not appear to respect this setting.

Is there a specific way to set the command/connection timeout?

(Also, this library has been a huge help so far, so thank you for all your effort!)

Code:

using (var context = new DbContext())
{
    context.Database.SetCommandTimeout(TimeSpan.FromMinutes(30));

    var uploader = new NpgsqlBulkUploader(context);
    uploader.Insert(entities, onConflict); 
}

Exception:
Npgsql.NpgsqlException
HResult=0x80004005
Message=Exception while reading from stream
Source=Npgsql
StackTrace:
at Npgsql.NpgsqlReadBuffer.<>c__DisplayClass31_0.<g__EnsureLong|0>d.MoveNext()
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at Npgsql.NpgsqlConnector.<>c__DisplayClass161_0.<g__ReadMessageLong|0>d.MoveNext()
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at Npgsql.NpgsqlDataReader.d__46.MoveNext()
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Npgsql.NpgsqlDataReader.NextResult()
at Npgsql.NpgsqlCommand.d__100.MoveNext()
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Npgsql.NpgsqlCommand.ExecuteDbDataReader(CommandBehavior behavior)
at Npgsql.Bulk.NpgsqlBulkUploader.Insert[T](IEnumerable`1 entities, InsertConflictAction onConflict)

Inner Exception 1:
IOException: Unable to read data from the transport connection: Connection timed out.

Inner Exception 2:
SocketException: Connection timed out

Hi @gregtayl, looks like you have a different problem. SetCommandTimeout sets timeout for command execution, but you seems to have timeout during connection. (Similar problem is described here: https://stackoverflow.com/questions/39455927/asp-net-core-sql-connection-timeout). So to workaround this you probably need to adjust connection timout (not command timeout). To do this you need to append Timeout parameter in your connection string (and set it to higher value, by default it is 15 seconds). More on Npgsql timeout - https://www.npgsql.org/doc/connection-string-parameters.html

You are correct, my apologies! Thank you for the detailed reply.