I have been seeing this error in my logs quite frequently for several users after they successfully log in. The application fails to load the start page for the user after this error. The stack trace contains this
org.eclipse.jetty.io.EofException
at org.eclipse.jetty.http.HttpGenerator.flushBuffer(HttpGenerator.java:914)
at org.eclipse.jetty.http.AbstractGenerator.blockForOutput(AbstractGenerator.java:523)
at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:170)
at org.eclipse.jetty.server.HttpOutp...
and
Caused by: java.io.IOException: An existing connection was forcibly closed by the remote host
at sun.nio.ch.SocketDispatcher.write0(Native Method)
at sun.nio.ch.SocketDispatcher.write(Unknown Source)
at sun.nio.ch.IOUtil.writeFromNativeBuffer(Unk...
A similar issue was reported a couple of years ago here https://forum.mendixcloud.com/link/questions/8890. Looking at the links here is suggests that this might be caused by either TCP errors or running out of connections to the database. My application is hosted on premise using MS SQL Server.
Can someone please check my reasoning here about what might be the issue... The Database is configured for 0 (unlimited) connections, so I don't think the DB configuration is the issue. The Mendix runtime has no special configuration options set, so I think this defaults to a pool of 50 connections?? So, I'm going to add the configuration option ConnectionPoolingMaxActive and set it to a higher number. Is this a reasonable thing to try, and what number should I set, and are there any performance effects on the runtime?
A second observation about this trying to monitor the number of DB connections through MS SQL Management Studio:
SELECT DB_NAME(sP.dbid) AS the_database,
COUNT(sP.spid) AS total_database_connections
FROM sys.sysprocesses sP
GROUP BY DB_NAME(sP.dbid)
ORDER BY 1;
When I run this against an idle application instance (just restarted) I have one or two connections to the database, presumably for scheduled events, etc. When the first user logs in I get an increase of 20 or more database connections established - is this normal?
When the user logs out the connections are not closed. Should I set the configuration option ConnectionPoolingTimeBetweenEvictionRunsMillis and ConnectionPoolingSoftMinEvictableIdleTimeMillis to something less than 5 minutes to release the connections more frequestly, or is this a different configuration option? Again, what are the impacts for making this change?
Thanks for any advice
Issue resolved:
I had rewrite rules (IIS front end) and SSL installed. Some users were accessing through http:// + portnumber, some using http:// using the rewrite rules, and some using https:// with rewrite rules. The users having the issues seemed to be in the first group. I closed down all unnecessary ports and only allowed access through https:// with the rewrite rules and that seems to have fixed the issue.