How to handle buffered data

Malkit Singh8 years ago

Hi @Anton,
Another doubt and I think this could help many others also.

I am using device which stores/buffers data in it if no network (GPRS) connection available.
And as soon as it receives the network back, it starts sending stored data and current (where device is moving currently) but as a result of this, I am getting wrong summary and total distance travelled report.
Root and event reports seems to be ok.
Any workaround to handle such cases?

Anton Tananaev8 years ago

I can't think of any simple workaround. We need to go through position history and re-calculate distance for everything in this case.

Malkit Singh8 years ago

And what could be the reason for negative distance traveled report? I have set my server and device timezone to IST ( indian standard time) do you think changing this back to UTC can help?

Anton Tananaev8 years ago

The only reason for negative values can be that your device reports incorrect odometer, or it doesn't report odometer in every message.

Malkit Singh8 years ago

ok,
Can you explain me meaning of status in this data stored in attribute field of positions table?
{"ignition":false,"status":4294966271,"ip":"101.62.203.220","distance":2.1,"totalDistance":115935.96}
And all calculations depends on whats stored in this field?

Anton Tananaev8 years ago

For status description you should look at the protocol documentation:

https://www.traccar.org/protocols/

Yes, all calculations depend on the data in the attributes field.

Malkit Singh8 years ago

@Anton I think this can solve this.
Let's say you had GPRS working till 4:00 PM and then you got disconnected for 1 hour and in attributes field, you had distance traveled 50 KM.
And when you start receiving packet at 5:00 PM then calculate distance from last lati longi ( from 4:00 PM) till new lati longi which you receive at 5:00 PM and ignore all packets which have timestamp less than 5:00 PM.
I think this can help in some way. But this also makes way to cheat in tracking ( person driving vehicle can remove sim from device and then insert back after driving to some unknown locations).
But if we can have this kind of exceptional configuration in config file that would be helpful in such cases.
What you think?

Anton Tananaev8 years ago

It's exactly how Traccar works, but how would you calculate distance for missing period?

abyss8 years ago

@Malkit Singh

There is a few ways how devices send archive data:

  1. FIFO, as soon as it get connected, it sends archive from oldest to newest.
  2. LIFO, as soon as it get connected, it sends archive from newest to oldest.
  3. Without order, just as it want. Current and archive are mixed, positions in archive packet has one order, but packets order is other.

If you need accurate distance calculation, you need to use first type of device or device that calculate distance internally.

There is now good solution to handle 2 and 3 type of devices.
May be only force distance recalculation by command from administrator before generating month report or something can help.

Malkit Singh8 years ago

if we give some option to user to skip this missing distance calculation then that could be helpful in such cases.
Actually, this is what happened in my case, I started receiving my buffered data and that corrupted my actual distance report. If I had this option to disable buffered data then this situation could have avoided.

Anton Tananaev8 years ago

I don't see how disabling buffered data would solve the problem.

Malkit Singh8 years ago

@Anton Tananaev
Here is my solution to this problem. I have written my own service in NodeJs that take device id, fromdatetime and todatetime. ( I am using MySql database )
And inside this service, I am fetching all records for given device between fromdatetime and todatetime in ascending order of devicetime ( here this devicetime order is important).
And then by iterating over each record in this list of records, I am calculating distance between two points. and saving them to some global variable.
So, I will be using traccar api's to fetch events, route reports etc. But to calculate distance I will be depending upon this temporary patch of mine.
I hope you do something about this in 3.8 version or upcoming versions.

Anton Tananaev8 years ago

My concern is performance implications of iterating through all positions. Also, at which point would you do it?

Contributions are welcome if you want to implement this functionality.

Malkit Singh8 years ago

I would love to contribute, have cloned source code and trying to understand the structure and process of building using IntelliJ.
As far as performance is concerned, it's really not possible to calculate distance on each packet receive event. We need to figure out some way to run few things in the background like CRON jobs or something. And to limit this iteration we can always take advantage of "from" and "to" date ranges.
Is there any existing functionality which runs in the background?

Anton Tananaev8 years ago

There is a task that clears old data. Check "clearPositionsHistory" method.