Open file descriptors held by the java process kept hitting our default limit

[Tejas jain]

I ran into a problem with recent version when under high load (~3k a min). The number of open file descriptors held by the java process kept hitting default limit, which at the time was set to 1024 on centos 6.7 hosts. Once the limit was hit, it seemed to get stuck and we had to restart the process. We increased the limit to 4096, and that helped for a while, but we still hit the limit after a time. Finally, we increased the limit to 50000. So far we haven’t hit that.

While monitoring the # of open FDs after that, we can see the number increase to between 5000 and 6000, but thankfully eventually resets itself and drops back down to 100-200 or so.

In order to increase that limit for our ‘tomcat’ user, we made the following modification to the /etc/security/limits.conf file.

Just add these lines:

tomcat      hard    nofile      50000
tomcat      soft    nofile      50000

The tomcat user may need to re-login for these changes to have effect.

[Dave Moten]

Thanks for the report, that's interesting. Have you thought of strategies in subethasmtp that might help you in this scenario? You can limit concurrent sessions by setting a SessionHandler. An example is at https://github.com/davidmoten/subethasmtp/blob/master/src/main/java/org/subethamail/smtp/internal/server/ConcurrentSessionsBySourceLimiter.java.

[Tejas jain]

Thanks for the example. I will give a try & update you.

[Dave Moten]

Closing due to inactivity. Reopen if you have more to add.