Dealing with packet fragmentation

First of all, I would like to thanks Mr. Tananaev for development and share of Traccar software.

I will try to be succinct in my doubts, but primarily involves the implementation of a new protocol used in Maxtrack's AVL devices like MXT162. The question that I'm facing now involves the reconstructions of fragmented packages send by telephony company through the breaking of the original data packet sent by AVL device. According the new protocol used by Maxtrack, AVL devices like MXT162 can send more than 4kB each time that sent some data to server. So, the local telephony company, break each large data package sent by the AVL device in small data chunk -- typically each data chunk has 900 ~ 1400B.

In my point of view I cannot deal with that because each package carry just a [HEADER] + [SIZE] + [CRC] + [MT] + [PF] + [DATA]. The header has a fix stream of bytes: 0xAA55AA55 (4 bytes), indicating the beginning of a package (here, "message"). After, the size (2 bytes) of data package; from the first CRC byte to the last byte in data. The CRC has 2 bytes and is calculated from the first MT byte to the last byte of data. The MT indicate the type of messages and has 2 bytes of length. The PF indicate the packet format and has 2 bytes of length. The data has information reported by the devices. The data has variable length according with the message type (MT). There are any mention that the preamble (HEADER) is unique and in case when the telephony company breaking the packages in chunk's, In my point of view, it's too hard reconstruct the messages because I do not have any information that after receive a first chunk containing data from the HEADER to the some parts of DATA, but subsequent chunk of DATA it's hard recognize which initial chunk they belongs. This situation occur when I had a considerable number of AVL devices connected in one server port. At any moment one or more devices can be send different messages, and I dont know if telephony operator send the messages chunks in order and what chunk fragment belongs a previous received by the server. How to deal with that?

Above, I send a original message sent by the AVL device and after, the same message sent by the telephony operator breaking in some packet's:

55AA55AAAD0AEBE20100000008DE8FE7880610AEC999A20722B20108A436104018DEA69FE5052218080010011D1D1923F42511C0D0E528F90630FFFF0338E9012A171003180028E80730E80738E80740004800500058006800320A0802120208021A0208023A3F0A1D080010001803201C28EE1B3000380148EDEF0150B1FF0158EC86026800121808011000201430013A0C08D405100518EF0F20F1A40148021A040800100042210A0C08001000180020002898E116120B080010D60F180020E4B0091A04080010007A0208002AB20108A336104018EEA49FE5052218080010011D1D1923F42511C0D0E528F90630FFFF0338E9012A171003180028E80730E80738E80740004800500058006800320A0802120208021A0208023A3F0A1D080010001803202528A31C3000380148EEF30150C1FD0158E886026800121808011000206530013A0C08D405100518EF0F20F1A40148021A040800100042210A0C08001000180020002898E116120B080010D60F180020E4B0091A04080010007A0208002AB20108A236104018FEA29FE5052218080010011D1D1923F42511C0D0E528F90630FFFF0338E9012A171003180028E80730E80738E80740004800500058006800320A0802120208021A0208023A3F0A1D080010001803202A28C01C3000380148ECF00150D1FB0158E486026800121808011000201330013A0C08D405100518EF0F20F1A40148021A040800100042210A0C08001000180020002898E116120B080010D60F180020E4B0091A04080010007A0208002AB20108A1361040188EA19FE5052218080010011D1D1923F42511C0D0E528F90630FFFF0338E9012A171003180028E80730E80738E80740004800500058006800320A0802120208021A0208023A3F0A1D080010001803202D28CE1C3000380148ABF10150E1F90158E086026800121808011000201430013A0C08D405100518EF0F20F1A40148021A040800100042210A0C08001000180020002898E116120B080010D60F180020E4B0091A04080010007A0208002AB20108A0361040189E9F9FE5052218080010011D1D1923F42511C0D0E528F90630FFFF0338E9012A171003180028E80730E80738E80740004800500058006800320A0802120208021A0208023A3F0A1D080010001803202E28D41C3000380148BFF50150F1F70158DC86026800121808011000206430013A0C08D405100518EF0F20F1A40148021A040800100042210A0C08001000180020002898E116120B080010D60F180020E4B0091A04080010007A0208002AB201089F36104018AE9D9FE5052218080010011D1D1923F42511C0D0E528F90630FFFF0338E9012A171003180028E80730E80738E80740004800500058006800320A0802120208021A0208023A3F0A1D080010001803202F28D91C3000380148AAF2015081F60158D886026800121808011000206430013A0C08D405100518EF0F20F1A40148021A040800100042210A0C08001000180020002898E116120B080010D60F180020E4B0091A04080010007A0208002AB201089E36104018BE9B9FE5052218080010011D1D1923F42511C0D0E528F90630FFFF0338E9012A171003180028E80730E80738E80740004800500058006800320A0802120208021A0208023A3F0A1D080010001803202F28DE1C3000380148F6F2015091F40158D486026800121808011000201330013A0C08D405100518EF0F20F1A40148021A040800100042210A0C08001000180020002898E116120B080010D60F180020E4B0091A04080010007A0208002AB201089D36104018CE999FE5052218080010011D1D1923F42511C0D0E528F90630FFFF0338E9012A171003180028E80730E80738E80740004800500058006800320A0802120208021A0208023A3F0A1D080010001803203028E11C3000380148DDF60150A1F20158D086026800121808011000201430013A0C08D405100518EF0F20F1A40148021A040800100042210A0C08001000180020002898E116120B080010D60F180020E4B0091A04080010007A0208002AB201089C36104018DE979FE5052218080010011D1D1923F42511C0D0E528F90630FFFF0338E9012A171003180028E80730E80738E80740004800500058006800320A0802120208021A0208023A3F0A1D080010001803203128E81C3000380148DBF30150B1F00158CC86026800121808011000201330013A0C08D405100518EF0F20F1A40148021A040800100042210A0C08001000180020002898E116120B080010D60F180020E4B0091A04080010007A0208002AB201089B36104018ED959FE5052218080010011D1D1923F42511C0D0E528F90630FFFF0338E9012A171003180028E80730E80738E80740004800500058006800320A0802120208021A0208023A3F0A1D080010001803203328F11C30003801489AF40150C1EE0158C886026800121808011000201430013A0C08D405100518EF0F20F1A40148021A040800100042210A0C08001000180020002898E116120B080010D60F180020E4B0091A04080010007A0208002AB201089A36104018FD939FE5052218080010011D1D1923F42511C0D0E528F90630FFFF0338E9012A171003180028E80730E80738E80740004800500058006800320A0802120208021A0208023A3F0A1D080010001803203428F91C3000380148C1F80150D1EC0158C486026800121808011000206530013A0C08D405100518EF0F20F1A40148021A040800100042210A0C08001000180020002898E116120B080010D60F180020E4B0091A04080010007A0208002AB2010899361040188D929FE5052218080010011D1D1923F42511C0D0E528F90630FFFF0338E9012A171003180028E80730E80738E80740004800500058006800320A0802120208021A0208023A3F0A1D080010001803203528FF1C300038014886F50150E1EA0158C086026800121808011000201330013A0C08D405100518EF0F20F1A40148021A040800100042210A0C08001000180020002898E116120B080010D60F180020E4B0091A04080010007A0208002AB2010898361040189D909FE5052218080010011D1D1923F42511C0D0E528F90630FFFF0338E9012A171003180028E80730E80738E80740004800500058006800320A0802120208021A0208023A3F0A1D080010001803203628841D3000380148B2F50150F1E80158BC86026800121808011000206430013A0C08D405100518EF0F20F1A40148021A040800100042210A0C08001000180020002898E116120B080010D60F180020E4B0091A04080010007A0208002AB201089736104018AD8E9FE5052218080010011D1D1923F42511C0D0E528F90630FFFF0338E9012A171003180028E80730E80738E80740004800500058006800320A0802120208021A0208023A3F0A1D080010001803203728891D3000380148D9F9015081E70158B886026800121808011000206530013A0C08D405100518EF0F20F1A40148021A040800100042210A0C08001000180020002898E116120B080010D60F180020E4B0091A04080010007A0208002AB201089636104018BD8C9FE5052218080010011D1D1923F42511C0D0E528F90630FFFF0338E9012A171003180028E80730E80738E80740004800500058006800320A0802120208021A0208023A3F0A1D0800100018032038288F1D3000380148BDF6015091E50158B486026800121808011000201430013A0C08D405100518EF0F20F1A40148021A040800100042210A0C08001000180020002898E116120B080010D60F180020E4B0091A04080010007A020800

And the chunks sent by the telephony operator:
First chunk:

55AA55AAAD0AEBE20100000008DE8FE7880610AEC999A20722B20108A436104018DEA69FE5052218080010011D1D1923F42511C0D0E528F90630FFFF0338E9012A171003180028E80730E80738E80740004800500058006800320A0802120208021A0208023A3F0A1D080010001803201C28EE1B3000380148EDEF0150B1FF0158EC86026800121808011000201430013A0C08D405100518EF0F20F1A40148021A040800100042210A0C08001000180020002898E116120B080010D60F180020E4B0091A04080010007A0208002AB20108A336104018EEA49FE5052218080010011D1D1923F42511C0D0E528F90630FFFF0338E9012A171003180028E80730E80738E80740004800500058006800320A0802120208021A0208023A3F0A1D080010001803202528A31C3000380148EEF30150C1FD0158E886026800121808011000206530013A0C08D405100518EF0F20F1A40148021A040800100042210A0C08001000180020002898E116120B080010D60F180020E4B0091A04080010007A0208002AB20108A236104018FEA29FE5052218080010011D1D1923F42511C0D0E528F90630FFFF0338E9012A171003180028E80730E80738E80740004800500058006800320A0802120208021A0208023A3F0A1D080010001803202A28C01C3000380148ECF00150D1FB0158E486026800121808011000201330013A0C08D405100518EF0F20F1A40148021A040800100042210A0C08001000180020002898E116120B080010D60F180020E4B0091A04080010007A0208002AB20108A1361040188EA19FE5052218080010011D1D1923F42511C0D0E528F90630FFFF0338E9012A171003180028E80730E80738E80740004800500058006800320A0802120208021A0208023A3F0A1D080010001803202D28CE1C3000380148ABF10150E1F90158E086026800121808011000201430013A0C08D405100518EF0F20F1A40148021A040800100042210A0C08001000180020002898E116120B080010D60F180020E4B0091A04080010007A0208002AB20108A0361040189E9F9FE5052218080010011D1D1923F42511C0D0E528F90630FFFF0338E9012A171003180028E80730E80738E80740004800500058006800320A0802120208021A0208023A3F0A1D080010001803202E28D41C3000380148BFF50150F1F70158DC86026800121808011000206430013A0C08D405100518EF0F20F1A40148021A040800100042210A0C08001000180020002898E116120B080010D60F180020E4B0091A04080010007A0208002AB201089F36104018AE9D9FE5052218080010011D1D1923F42511C0D0E528F90630FFFF0338E9012A171003180028E80730E80738E80740004800500058006800320A0802120208021A0208023A3F0A1D080010001803202F28D91C3000380148AAF2015081F60158D886026800121808011000206430013A0C08D405100518EF0F20F1A40148021A040800100042210A0C08001000180020002898E116120B080010D60F180020E4B0091A04080010007A0208002AB201089E36104018BE9B9FE5052218080010011D1D1923F42511C0D0E528F90630FFFF0338E9012A171003180028E80730E80738E80740004800500058006800320A0802120208021A0208023A3F0A1D080010001803202F28DE1C3000

Second chunk:

380148F6F2015091F40158D486026800121808011000201330013A0C08D405100518EF0F20F1A40148021A040800100042210A0C08001000180020002898E116120B080010D60F180020E4B0091A04080010007A0208002AB201089D36104018CE999FE5052218080010011D1D1923F42511C0D0E528F90630FFFF0338E9012A171003180028E80730E80738E80740004800500058006800320A0802120208021A0208023A3F0A1D080010001803203028E11C3000380148DDF60150A1F20158D086026800121808011000201430013A0C08D405100518EF0F20F1A40148021A040800100042210A0C08001000180020002898E116120B080010D60F180020E4B0091A04080010007A0208002AB201089C36104018DE979FE5052218080010011D1D1923F42511C0D0E528F90630FFFF0338E9012A171003180028E80730E80738E80740004800500058006800320A0802120208021A0208023A3F0A1D080010001803203128E81C3000380148DBF30150B1F00158CC86026800121808011000201330013A0C08D405100518EF0F20F1A40148021A040800100042210A0C08001000180020002898E116120B080010D60F180020E4B0091A04080010007A0208002AB201089B36104018ED959FE5052218080010011D1D1923F42511C0D0E528F90630FFFF0338E9012A171003180028E80730E80738E80740004800500058006800320A0802120208021A0208023A3F0A1D080010001803203328F11C30003801489AF40150C1EE0158C886026800121808011000201430013A0C08D405100518EF0F20F1A40148021A040800100042210A0C08001000180020002898E116120B080010D60F180020E4B0091A04080010007A0208002AB201089A36104018FD939FE5052218080010011D1D1923F42511C0D0E528F90630FFFF0338E9012A171003180028E80730E80738E80740004800500058006800320A0802120208021A0208023A3F0A1D080010001803203428F91C3000380148C1F80150D1EC0158C486026800121808011000206530013A0C08D405100518EF0F20F1A40148021A040800100042210A0C08001000180020002898E116120B080010D60F180020E4B0091A04080010007A0208002AB2010899361040188D929FE5052218080010011D1D1923F42511C0D0E528F90630FFFF0338E9012A171003180028E80730E80738E80740004800500058006800320A0802120208021A0208023A3F0A1D080010001803203528FF1C300038014886F50150E1EA0158C086026800121808011000201330013A0C08D405100518EF0F20F1A40148021A040800100042210A0C08001000180020002898E116120B080010D60F180020E4B0091A04080010007A0208002AB2010898361040189D909FE5052218080010011D1D1923F42511C0D0E528F90630FFFF0338E9012A171003180028E80730E80738E80740004800500058006800320A0802120208021A0208023A3F0A1D080010001803203628841D3000380148B2F50150F1E80158BC86026800121808011000206430013A0C08D405100518EF0F20F1A40148021A040800100042210A0C08001000180020002898E116120B080010D60F180020E4B0091A04080010007A0208002AB201089736104018AD8E9FE5052218080010011D1D1923F42511C0D0E528

Third chunk:

F90630FFFF0338E9012A171003180028E80730E80738E80740004800500058006800320A0802120208021A0208023A3F0A1D080010001803203728891D3000380148D9F9015081E70158B886026800121808011000206530013A0C08D405100518EF0F20F1A40148021A040800100042210A0C08001000180020002898E116120B080010D60F180020E4B0091A04080010007A0208002AB201089636104018BD8C9FE5052218080010011D1D1923F42511C0D0E528F90630FFFF0338E9012A171003180028E80730E80738E80740004800500058006800320A0802120208021A0208023A3F0A1D0800100018032038288F1D3000380148BDF6015091E50158B486026800121808011000201430013A0C08D405100518EF0F20F1A40148021A040800100042210A0C08001000180020002898E116120B080010D60F180020E4B0091A04080010007A020800

You have the length field. Why is it hard to reconstruct full frame?

I'm also not sure how your question is related to Traccar server. We handle this case for virtually every supported protocol.

Thanks a lot Tananaev,

I suppose that question is pertinent the server, because I plan expand the supported devices. I will search in the current software version a similar approach described here.

Thanks for your support advice and patience!

This is standard UDP fragmentation handling. If it's TCP, its even easier, as the OS will put them back together into one data chunk for you. You just need to read and wait for the full input to come to you.

Hello friends, I also plan to add support for MXT-160 and MXT-162 trackers, you could share the protocol with me so I can help with the development