Error logs in the server. Leading to device disconnects

vishalv20506 years ago

I have been seeing this kind of very very frequent error logs in the server

Apr 25, 2019 5:41:19 PM org.glassfish.jersey.server.ServerRuntime$Responder writeResponse                                                                 [284/1805]
SEVERE: An I/O error has occurred while writing a response message entity to the container output stream.
org.glassfish.jersey.server.internal.process.MappableException: org.eclipse.jetty.io.EofException
        at org.glassfish.jersey.server.internal.MappableExceptionWrapperInterceptor.aroundWriteTo(MappableExceptionWrapperInterceptor.java:92)
        at org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:162)
        at org.glassfish.jersey.message.internal.MessageBodyFactory.writeTo(MessageBodyFactory.java:1130)
        at org.glassfish.jersey.server.ServerRuntime$Responder.writeResponse(ServerRuntime.java:711)
        at org.glassfish.jersey.server.ServerRuntime$Responder.processResponse(ServerRuntime.java:444)
        at org.glassfish.jersey.server.ServerRuntime$Responder.process(ServerRuntime.java:434)
        at org.glassfish.jersey.server.ServerRuntime$2.run(ServerRuntime.java:329)
        at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271)
        at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267)
        at org.glassfish.jersey.internal.Errors.process(Errors.java:315)
        at org.glassfish.jersey.internal.Errors.process(Errors.java:297)
        at org.glassfish.jersey.internal.Errors.process(Errors.java:267)
        at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317)
        at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:305)
        at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1154)
        at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:473)
        at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:427)
        at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:388)
        at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:341)
        at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:228)
        at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:812)
        at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:587)
        at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221)
        at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
        at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
        at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
        at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
        at org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:52)
        at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
        at org.eclipse.jetty.server.Server.handle(Server.java:499)
        at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311)
        at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
        at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:544)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
        at java.lang.Thread.run(Thread.java:748)
Caused by: org.eclipse.jetty.io.EofException
        at org.eclipse.jetty.io.ChannelEndPoint.flush(ChannelEndPoint.java:192)
        at org.eclipse.jetty.io.WriteFlusher.flush(WriteFlusher.java:408)
        at org.eclipse.jetty.io.WriteFlusher.completeWrite(WriteFlusher.java:364)
        at org.eclipse.jetty.io.SelectChannelEndPoint.onSelected(SelectChannelEndPoint.java:111)
        at org.eclipse.jetty.io.SelectorManager$ManagedSelector.processKey(SelectorManager.java:641)
        at org.eclipse.jetty.io.SelectorManager$ManagedSelector.select(SelectorManager.java:612)
        at org.eclipse.jetty.io.SelectorManager$ManagedSelector.run(SelectorManager.java:550)
        at org.eclipse.jetty.util.thread.NonBlockingThread.run(NonBlockingThread.java:52)
        ... 3 more
Caused by: java.io.IOException: Broken pipe
        at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
        at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47)
        at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93)
        at sun.nio.ch.IOUtil.write(IOUtil.java:65)
        at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:492)
        at org.eclipse.jetty.io.ChannelEndPoint.flush(ChannelEndPoint.java:170)
        ... 10 more

Only gt06 devices are connecting to my server
As for other symptoms, I can see in the logs that I don't get anything from a device for some time and then I see a sudden splurge of data from it (the devices are configured to ping every 10 seconds)
In this duration the device status shows as disconnected. This has started happening fairly recently and majority of devices are getting affected.
There have been no changes from my end on the server, hardly any new devices added as well

I have not been able to see anything like this elsewhere in the forums and any help will be really appreciated

Anton Tananaev6 years ago

Why are you marking your log as Java code?

vishalv20506 years ago

My bad, does not look like I can edit the post.

Anton Tananaev6 years ago

This error should not affect devices. What port are you using for devices?

vishalv20506 years ago

5023, gt06 devices

Anton Tananaev6 years ago

Then this error definitely has nothing to do with your issue.

vishalv20506 years ago

I saw these errors popping up every few minutes or so, and at the same time notices the device issue. Thought they were connected.
What can cause these errors ? Have you seen such behaviour before ?

Or the case where we dont see any data from a device in the logs then a sudden flurry.