Help with IndexOutOfBoundsException in TeltonikaProtocolDecoder

Simo12 days ago

Hi, I'm encountering an IndexOutOfBoundsException in the Traccar logs with the following error message:

2025-01-21 15:59:17 WARN: [U4eba76f4] error - readerIndex(0) + length(2) exceeds writerIndex(1): PooledUnsafeDirectByteBuf(ridx: 0, widx: 1, cap: 2048) - IndexOutOfBoundsException (... < TeltonikaProtocolDecoder:711 < *:689 < ExtendedObjectDecoder:73 < ... < StandardLoggingHandler:62 < ... < NetworkMessageHandler:34 < ...)

It seems to be related to the TeltonikaProtocolDecoder, and I suspect there is an issue with how the incoming data is being processed, possibly due to device configuration.

How can I identify which specific device is causing this error? Is there a way to correlate the error message or trace the device responsible? I want to check the device's configuration to ensure it's sending the correct data format.

I've tried reviewing the logs, but it's unclear how to pinpoint the device responsible. Any guidance or suggestions on how to track this down would be greatly appreciated!

I'm using the latest Traccar version 6.6.

I have many teltonika (FMB ) devices running, and when I set the logger level to "info," I get too many logs. I’m not sure which one is related to this error.

Anton Tananaev12 days ago

You have to provide more information from logs, specifically the hex sample that caused the problem.

Simo12 days ago

Full Error :

2025-01-21 16:14:15  WARN: [U0ca0504e] error - 
java.lang.IndexOutOfBoundsException: readerIndex(0) + length(2) exceeds writerIndex(1): PooledUnsafeDirectByteBuf(ridx: 0, widx: 1, cap: 2048)
    at io.netty.buffer.AbstractByteBuf.checkReadableBytes0(AbstractByteBuf.java:1442)
    at io.netty.buffer.AbstractByteBuf.readShort(AbstractByteBuf.java:749)
    at io.netty.buffer.AbstractByteBuf.readUnsignedShort(AbstractByteBuf.java:765)
    at org.traccar.protocol.TeltonikaProtocolDecoder.decodeUdp(TeltonikaProtocolDecoder.java:711)
    at org.traccar.protocol.TeltonikaProtocolDecoder.decode(TeltonikaProtocolDecoder.java:689)
    at org.traccar.ExtendedObjectDecoder.channelRead(ExtendedObjectDecoder.java:73)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
    at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93)
    at org.traccar.handler.network.StandardLoggingHandler.channelRead(StandardLoggingHandler.java:62)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
    at org.traccar.handler.network.NetworkMessageHandler.channelRead(NetworkMessageHandler.java:34)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
    at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1357)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
    at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:868)
    at io.netty.channel.nio.AbstractNioMessageChannel$NioMessageUnsafe.read(AbstractNioMessageChannel.java:97)
    at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788)
    at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724)
    at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650)
    at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562)
    at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
    at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
    at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
    at java.base/java.lang.Thread.run(Thread.java:1583)
Simo12 days ago

If I set the logger level to "info" to see the hex data, I receive hex data from other devices, and I can't identify which one is responsible for this error.

Here’s a simple log example:

2025-01-21 16:16:58  INFO: Started Server@7d088a93{STARTING}[11.0.24,sto=0] @3790ms
2025-01-21 16:16:59  INFO: [U439381d7: teltonika < 105.66.128.247] ff
2025-01-21 16:16:59  WARN: [U439381d7] error - readerIndex(0) + length(2) exceeds writerIndex(1): PooledUnsafeDirectByteBuf(ridx: 0, widx: 1, cap: 2048) - IndexOutOfBoundsException (... < TeltonikaProtocolDecoder:711 < *:689 < ExtendedObjectDecoder:73 < ... < StandardLoggingHandler:62 < ... < NetworkMessageHandler:34 < ...)
2025-01-21 16:17:00  INFO: [U439381d7: teltonika < 105.66.134.23] ff
2025-01-21 16:17:00  WARN: [U439381d7] error - readerIndex(0) + length(2) exceeds writerIndex(1): PooledUnsafeDirectByteBuf(ridx: 0, widx: 1, cap: 2048) - IndexOutOfBoundsException (... < TeltonikaProtocolDecoder:711 < *:689 < ExtendedObjectDecoder:73 < ... < StandardLoggingHandler:62 < ... < NetworkMessageHandler:34 < ...)
2025-01-21 16:17:00  INFO: [U439381d7: teltonika < 105.67.1.54] 007acafe016a000f3335373037333239303734363531368e010000019489702a2800faebeb8f1338a003015200000f003500000010000800ef0100f00100150300c80000450100010100b300007151000600b5000900b600060042340300430f4900440000000900ac000200f10000ebf20010070ff0fa0000000001
2025-01-21 16:17:00  INFO: [U439381d7: teltonika > 105.67.1.54] 00050000016a01
2025-01-21 16:17:00  INFO: Event id: 357073290746516, time: 2025-01-21 16:17:00, type: deviceOnline, notifications: 0
2025-01-21 16:17:00  INFO: [U439381d7: teltonika < 105.71.132.97] ff
2025-01-21 16:17:00  WARN: [U439381d7] error - readerIndex(0) + length(2) exceeds writerIndex(1): PooledUnsafeDirectByteBuf(ridx: 0, widx: 1, cap: 2048) - IndexOutOfBoundsException (... < TeltonikaProtocolDecoder:711 < *:689 < ExtendedObjectDecoder:73 < ... < StandardLoggingHandler:62 < ... < NetworkMessageHandler:34 < ...)
2025-01-21 16:17:00  INFO: [U439381d7] id: 357073290746516, time: 2025-01-21 16:16:57, lat: 32.24781, lon: -8.52020, speed: 28.6, course: 0.0
2025-01-21 16:17:01  INFO: [U439381d7: teltonika < 105.66.128.247] ff
2025-01-21 16:17:01  WARN: [U439381d7] error - readerIndex(0) + length(2) exceeds writerIndex(1): PooledUnsafeDirectByteBuf(ridx: 0, widx: 1, cap: 2048) - IndexOutOfBoundsException (... < TeltonikaProtocolDecoder:711 < *:689 < ExtendedObjectDecoder:73 < ... < StandardLoggingHandler:62 < ... < NetworkMessageHandler:34 < ...)
2025-01-21 16:17:02  INFO: [U439381d7: teltonika < 105.74.3.71] 0066cafe018b000f3335343031373131303639383539308e0100000194897039c8010000000000000000000000000000000048000a000400ef01001504004502000101000400b5000000b6000000422fc000430f47000200f10000ebf20048ffffff440000000001
2025-01-21 16:17:02  INFO: [U439381d7: teltonika > 105.74.3.71] 00050000018b01
2025-01-21 16:17:02  INFO: Event id: 354017110698590, time: 2025-01-21 16:17:02, type: deviceOnline, notifications: 0
2025-01-21 16:17:02  INFO: [U439381d7] id: 354017110698590, time: 2025-01-21 16:17:01, lat: 0.00000, lon: 0.00000, course: 0.0
2025-01-21 16:17:02  INFO: [U439381d7: teltonika < 105.66.134.23] ff
2025-01-21 16:17:02  WARN: [U439381d7] error - readerIndex(0) + length(2) exceeds writerIndex(1): PooledUnsafeDirectByteBuf(ridx: 0, widx: 1, cap: 2048) - IndexOutOfBoundsException (... < TeltonikaProtocolDecoder:711 < *:689 < ExtendedObjectDecoder:73 < ... < StandardLoggingHandler:62 < ... < NetworkMessageHandler:34 < ...)
2025-01-21 16:17:03  INFO: [U439381d7: teltonika < 105.71.16.193] 0077cafe0197000f3335323632353639353432353130368e0100000194897031f800fb7b2e1a13d5d47a00fc01650d00130000000f000700ef0100f00100150500c80000450100010100b300000600b5000500b600030042361400430fce0044000000090000000200f10000ebf20010136e7b300000000001
2025-01-21 16:17:03  INFO: [U439381d7: teltonika > 105.71.16.193] 00050000019701
2025-01-21 16:17:03  INFO: [U439381d7] id: 352625695425106, time: 2025-01-21 16:16:59, lat: 33.27807, lon: -7.58134, speed: 10.3, course: 357.0
2025-01-21 16:17:03  INFO: [U439381d7: teltonika < 105.66.128.69] 00b3cafe011e000f3335323632353639353739333239378e0100000194897012b800fafd04511338e77f018201610e001d0000001c000b00ef0100f00100150400c80000450100010100b40000715200511c005200005964000900b5000c00b6000700426af600430f500044000000540000005505f1005a00000073033400080010066d83a40053000643f3005706757ed80064000035910066000000000067000000000069009814f3006b000146340000000001
Anton Tananaev12 days ago

Looks like it's failing on this:

2025-01-21 16:17:02  INFO: [U439381d7: teltonika < 105.66.134.23] ff

It seems like a keepalive message that you shouldn't need for UDP protocol. Disable it on the device side.

Simo12 days ago

Thanks, Anton. I will try to make changes on the device. I suspect that it is keeping the connection alive and will get back to you.

But I want to know, if I find the error, how can I determine which device caused it from the logs? I know I need to decode the hex to find the device's IMEI, but since many devices send hex codes, I'm not sure how to figure out which one is related to the error.

Anton Tananaev12 days ago

You can try to determine it by the IP address.

Simo12 days ago

Thank you so much for your help, I really appreciate it!

hypoclone12 days ago

@simo are you running traccar on Linux, if yes I can send you the command you can use to pull all logs related to a particular device id

Simo12 days ago

Yes, indeed! Thank you so much, @hypoclone

Track-trace12 days ago

@hypoclone

Can you post the command here. Interested to know howto extract all data from the log from ubuntu server

Turbovix11 days ago

@Rastrear-rastrear you can use grep for this:

grep 512365478 /opt/traccar/logs/tracker-server.log

Anton Tananaev11 days ago

How is this going to help? And what would you grep for?

hypoclone11 days ago

Hello @simo, @Track-trace, sorry for the late reply was night time here yesterday and had a long work day today.
The below is what i use to get all logs related to a particular device.
for word in $(cat tracker-server.log | grep XXXXXXXXXXXXXXX | cut -c 29-37 | uniq); do cat tracker-server.log | grep $word; done
Where XXXXXXXXXXXXXXX is the device id

Track-trace11 days ago

@hypoclone, for some reason it did not work for me your suggestion but a modified command did:

grep 13878000 tracker-server.log | cut -c 29-37 | sort | uniq | while read word; do grep "$word" tracker-server.log; done 

And this writes it to a file called output.txt

grep 13878000 tracker-server.log | cut -c 29-37 | sort | uniq | while read word; do grep "$word" tracker-server.log; done > output.txt

Thanks for the tip, its helpfull.