Our application is deployed on weblogic servers. Our Managed cluster enters in Overload state with all connections of datasource being used up. We get this error on random days and on 2 specific timings. We checked everything and are not sure exactly. Even no Jobs are active during that timestamp. The server load is also normal.
As an immediate solution, we have to restart the server and then it works fine until next time.
We see below errors in logs-
weblogic.jdbc.extensions.PoolLimitSQLException: weblogic.common.resourcepool.ResourceLimitException:** No resources currently available in pool to allocate to applications, please increase the size of the pool and retry..** at weblogic.jdbc.common.internal.JDBCUtil.wrapAndThrowResourceException(JDBCUtil.java:280) at weblogic.jdbc.pool.Driver.connect(Driver.java:154) at weblogic.jdbc.jts.Driver.getNonTxConnection(Driver.java:665) at weblogic.jdbc.jts.Driver.connect(Driver.java:129)
java.sql.SQLException: No JDBC connection can be made because the transaction has timed out
at weblogic.jdbc.jts.Driver.newConnection(Driver.java:922)
Also when server is in overload state, we are unable to connect to datasource (which is obvious), it shows below error- Connection test failed with the following exception: weblogic.common.resourcepool.ResourceUnavailableException: No resources currently available in pool to allocate to applications Either specify a time period to wait for resources to become available, or increase the size of the pool and retry..
Our current configuration of connection pool Max Capacity is 15. With our last deployment, we have 26 JMS queues earlier it was 24. On daily basis 10-12 thousand messages flow. Database mostly has insert operations only.
Help will be appreciated.
We tried to check DB load. But it was normal. We tried to check if any job ran during that time period and caused issue, but that was not the case. We checked datasource monitoring table and could see all 15 connections consumed. We can try increasing connection count, but not sure what will be performance impact.