[Quartz/C3P0] org.quartz.JobPersistenceException: Couldn't acquire next trigger: The statement is closed.
anandbikas opened this issue · comments
C3P0 : 0.10.0-pre2 onwards up-to 0.10.0
Quartz: 2.3.2
Change causing the issue: 6accf55
Stacktrace
`2024-03-29T13:10:13.455+05:30 INFO 30071 --- [SchedulerThread] c.m.v2.c3p0.stmt.GooGooStatementCache : Problem with checked-in Statement, discarding.
com.microsoft.sqlserver.jdbc.SQLServerException: The statement is closed.
at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDriverError(SQLServerException.java:237) ~[mssql-jdbc-12.3.0.jre17-preview.jar:na]
at com.microsoft.sqlserver.jdbc.SQLServerStatement.checkClosed(SQLServerStatement.java:1117) ~[mssql-jdbc-12.3.0.jre17-preview.jar:na]
at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.clearParameters(SQLServerPreparedStatement.java:382) ~[mssql-jdbc-12.3.0.jre17-preview.jar:na]
at com.mchange.v2.c3p0.stmt.GooGooStatementCache.refreshStatement(GooGooStatementCache.java:640) ~[c3p0-0.10.0-pre2.jar:na]
at com.mchange.v2.c3p0.stmt.GooGooStatementCache.checkinStatement(GooGooStatementCache.java:272) ~[c3p0-0.10.0-pre2.jar:na]
at com.mchange.v2.c3p0.impl.NewPooledConnection.checkinStatement(NewPooledConnection.java:326) ~[c3p0-0.10.0-pre2.jar:na]
at com.mchange.v2.c3p0.impl.NewProxyPreparedStatement.close(NewProxyPreparedStatement.java:2997) ~[c3p0-0.10.0-pre2.jar:na]
at org.quartz.impl.jdbcjobstore.StdJDBCDelegate.closeStatement(StdJDBCDelegate.java:3287) ~[quartz-2.3.2.jar:na]
at org.quartz.impl.jdbcjobstore.StdJDBCDelegate.selectTriggerToAcquire(StdJDBCDelegate.java:2624) ~[quartz-2.3.2.jar:na]
at org.quartz.impl.jdbcjobstore.JobStoreSupport.acquireNextTrigger(JobStoreSupport.java:2844) ~[quartz-2.3.2.jar:na]
at org.quartz.impl.jdbcjobstore.JobStoreSupport$41.execute(JobStoreSupport.java:2805) ~[quartz-2.3.2.jar:na]
at org.quartz.impl.jdbcjobstore.JobStoreSupport$41.execute(JobStoreSupport.java:2803) ~[quartz-2.3.2.jar:na]
at org.quartz.impl.jdbcjobstore.JobStoreSupport.executeInNonManagedTXLock(JobStoreSupport.java:3864) ~[quartz-2.3.2.jar:na]
at org.quartz.impl.jdbcjobstore.JobStoreSupport.acquireNextTriggers(JobStoreSupport.java:2802) ~[quartz-2.3.2.jar:na]
at org.quartz.core.QuartzSchedulerThread.run(QuartzSchedulerThread.java:287) ~[quartz-2.3.2.jar:na]
2024-03-29T13:10:13.460+05:30 ERROR 30071 --- [SchedulerThread] org.quartz.core.ErrorLogger : An error occurred while scanning for the next triggers to fire.
org.quartz.JobPersistenceException: Couldn't acquire next trigger: The statement is closed.
at org.quartz.impl.jdbcjobstore.JobStoreSupport.acquireNextTrigger(JobStoreSupport.java:2923) ~[quartz-2.3.2.jar:na]
at org.quartz.impl.jdbcjobstore.JobStoreSupport$41.execute(JobStoreSupport.java:2805) ~[quartz-2.3.2.jar:na]
at org.quartz.impl.jdbcjobstore.JobStoreSupport$41.execute(JobStoreSupport.java:2803) ~[quartz-2.3.2.jar:na]
at org.quartz.impl.jdbcjobstore.JobStoreSupport.executeInNonManagedTXLock(JobStoreSupport.java:3864) ~[quartz-2.3.2.jar:na]
at org.quartz.impl.jdbcjobstore.JobStoreSupport.acquireNextTriggers(JobStoreSupport.java:2802) ~[quartz-2.3.2.jar:na]
at org.quartz.core.QuartzSchedulerThread.run(QuartzSchedulerThread.java:287) ~[quartz-2.3.2.jar:na]
Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: The statement is closed.
at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDriverError(SQLServerException.java:237) ~[mssql-jdbc-12.3.0.jre17-preview.jar:na]
at com.microsoft.sqlserver.jdbc.SQLServerStatement.checkClosed(SQLServerStatement.java:1117) ~[mssql-jdbc-12.3.0.jre17-preview.jar:na]
at com.microsoft.sqlserver.jdbc.SQLServerStatement.setMaxRows(SQLServerStatement.java:1169) ~[mssql-jdbc-12.3.0.jre17-preview.jar:na]
at com.mchange.v2.c3p0.impl.NewProxyPreparedStatement.setMaxRows(NewProxyPreparedStatement.java:2716) ~[c3p0-0.10.0-pre2.jar:na]
at org.quartz.impl.jdbcjobstore.StdJDBCDelegate.selectTriggerToAcquire(StdJDBCDelegate.java:2604) ~[quartz-2.3.2.jar:na]
at org.quartz.impl.jdbcjobstore.JobStoreSupport.acquireNextTrigger(JobStoreSupport.java:2844) ~[quartz-2.3.2.jar:na]
... 5 common frames omitted. . I tried to comment the code for the blow lines after which it was running fine. .
//markEndRequest(pc);
//markBeginRequest(pc);`
.
I am integrating with the latest C3p0, Please help analyse the issue.
@swaldman Could you please take a look?
@anandbikas Hi! Thanks for doing the hard part, tracking the issue down to markBegin/EndRequest!
It looks like your JDBC driver responds to the "hint" that a client session has ended by closing unclosed Connection
s Statements, which breaks the PreparedStatement
cache. The cache presumes and relies upon PreparedStatement
s surviving across client sessions.
Two questions:
-
Can you confirm that if you set (leave) both
c3p0.maxStatements
andc3p0.maxStatementsPerConnection
to 0, that the issue goes away? If my understanding is right, the issue should only appear when you enableStatement
caching. -
Would it be a good solution for you if there were a config setting, something like
c3p0.markSessionBoundaries
with might take on valuesalways
|never
|no-statement-cache
?
(Looking through ms-sql release notes, it looks like the driver can maintain its own statement cache, so it might be worth checking whether you get a performance benefit from also caching at the c3p0 level? But I'm glad to add a parameter like this, so that c3p0's statement cache is always an option.)
(I had left out the bolded word Statement previously, which is really important.)
Thanks @swaldman for a quick reply.
.
I tried option 1 (It was already at default values).
The value of maxStatementsPerConnection defaulting to 120, the same is not getting overridden using the property c3p0.maxStatementsPerConnection.
com.mchange.v2.c3p0.ComboPooledDataSource [maxStatements -> 0, maxStatementsPerConnection -> 120 ]
.
In further investigation found that the default was coming from org.quartz.utils.C3p0PoolingConnectionProvider
public static final int DEFAULT_DB_MAX_CACHED_STATEMENTS_PER_CONNECTION = 120;
.
Quartz has another property for this to override org.quartz.dataSource..maxCachedStatementsPerConnection , setting it to 0 worked.
.
Further queries...
- Do we need to always disable the statement caching after your markBegin/markEnd fix?
- What is the way to keep the statement caching on for C3P0 and MS-SQL server combination?
Hi!
- If I add the fix as proposed above, you will not need to disable statement caching if you set
c3p0.markSessionBoundaries
tonever
orno-statement-cache
. c3p0 will revert to its 0.9.x behavior of not informing the driver with markBegin/EndRequest about user checkouts and checkins, so MS-SQL's JDBC driver will be unable to close cached statements at user checkin. - See Question 1! Under the proposal, if you set
c3p0.markSessionBoundaries
tonever
orno-statement-cache
, you will be able to use c3p0 statement caching in combination with the MS-SQL JDBC driver. (Whether that would be helpful or harmful to performance, relative to using just an MS-SQL-JDBC-internal cache, I don't know. But you could try it both ways and test!)
You would need to explicitly set the c3p0.markSessionBoundaries
config param. It would default to always
, as JDBC 4.3 intends that pools should inform JDBC drivers when client sessions begin and end. So unless I get a lot of reports that it is disruptive, c3p0 will default to providing this information.
Would adding this configuration parameter be a reasonable solution for you?
Thanks!
That would work!
But this looks like a consistent behaviour for me, hence c3p0.markSessionBoundaries
condition should depend on the statementCache configuration. If the statement caching is enabled, the session boundaries should not be performed.
.
But I agree we can wait for others to report for similar issues. For now I have disabled statement caching and using the latest c3p0 version.
That's why we'd include no-statement-cache
as a possible value for c3p0.markSessionBoundaries
, along with always
and never
. (Maybe if-no-statement-cache
would be clearer?)
For your JDBC driver, you want to mark boundaries if you've not enabled the statement cache, but not do so if you have. That would be the intent of this setting.
Other drivers (I've tested mostly with postgres) don't have the issue, so always
is fine. I'll try to test with a mysql driver, and see if markEndRequest(...)
closes statements.
I think as long as the two main OSS JDBC drivers don't have the issue, We should leave the default to always
, because the JDBC 4.3 spec intends sessions to be reported. If mysql has the issue, then we'll default to...
no-statement-cache
? if-no-statement-cache
?
Which do you think?
Hi! Sorry it's taken me a while to get to this.
The solution described above is now implemented, and will be included when version 0.10.1. (I chose if-no-statement-cache
.)
You can try it out as a SNAPSHOT release on my local repository, https://www.mchange.com/repository/ as "com.mchange:c3p0:0.10.1-SNAPSHOT" ( or download the SNAPSHOT jar file directly from https://www.mchange.com/repository/com/mchange/c3p0/0.10.1-SNAPSHOT/c3p0-0.10.1-SNAPSHOT.jar )
(Please don't permanently depend upon SNAPSHOT releases. They are for testing only, and frequently ovewritten.)
Thank you for your help!
@anandbikas The new config param is documented, see SNAPSHOT documentation @ https://www.mchange.com/projects/c3p0-versions/c3p0-0.10.1-SNAPSHOT/
Hi! c3p0-0.10.1 is now a release. The solution proposed above is implemented. I'll close this issue for now. If anything goes wrong, please feel free to submit a new issue or reopen this one. Thanks!
Thanks @swaldman! Good to see that it's a release now.