Extreme CPU load
That is my initial thought, but if it was that way, CPU would be at max all the time Traccar is operating. That is not true. Some time ago I figured out that CPU is overloaded only when Traccar is close to run out of memory (we gave it 6G of heap). While Traccar has some memory left, the load would be nominal.
Have you tried disabling all computed attributes and see if it's even related to computed attributes?
No. Please standby.
Purged computed attributes. RAM and CPU usage is nominal and is not climbing.
So must be some issue with the computed attributes.
Yes. What would be the proper way to go?
Don't really have any ideas. I still suspect that the expressions you're using might be too complicates, so it slowly accumulates more and more memory. Do you see any accumulating delay in data processing?
Only during CPU overload. Data processing rate looks the same during "normal" period, however I can't provide any specific charts or data to rely on about that.
Would you please explain why would it accumulate memory? I thought Traccar processes the incoming data and stores it into database, then just frees the memory when it is done. Shouldn't the garbage collector sort this out anyway?
Maybe it's a JEXL bug, not ours?
Would you please explain why would it accumulate memory?
My thinking is that if new data is coming in faster than it is processed, it will keep taking more and more memory. If you checked and there are no delays, then it's probably not it.
Maybe it's a JEXL bug, not ours?
Could be. You can try older version. Revert this commit:
https://github.com/traccar/traccar/commit/f4f1b05f8d5211476ae073d8a3c2dbd10cbcbe8a
Or maybe we're just calling their API incorrectly.
Update: out of curiosity, I waited until some memory will build up, then cut the data flow from the devices. If Traccar indeed wasn't keeping up with the amount of data it received, then after this the memory should be freeing up, but it doesn't. I guess we're blaming JEXL after all.
I'll try an older version later.
then after this the memory should be freeing up, but it doesn't
Not necessarily. Java usually doesn't release memory. It has its own internal memory management, so it could look from outside that it uses a lot, but internally most of it is free. You have to check CPU usage instead.
Checked this using VisualVM. Memory is not freed inside heap.
Have you tried downgrading to see if it's a problem with JEXL version?
Not yet. Kind of busy on the job these days. I'll check during my day off tomorrow.
It probably means that the issues is completely different. Maybe the expression is too complicated and it just takes too much CPU?