Server CPU Usage spiked to 100% always

Hello all,

First of all, thank you, Anton, for the awesome work you've been doing.

I've been using Traccar for almost 3 years now. I have around 200 devices sending their location to the server every 10 seconds. The server has been running smoothly until the last few days. All of a sudden, my server's CPU usage jumped up to 100% from an average of 10% to 30%. I'm using Teltonika and Ruptela devices. Nothing has changed on the server side to trigger this spike in CPU usage. I added a couple more devices, and that's all. I have followed the MySQL optimization methods mentioned in the documentation.

Server Specs: 8 vCPUs, 16GB RAM, CentOS 7
Database: Local MySQL

Thinks I've tried so far with no luck:

  • Restarted Traccar service, MySQL service
  • Restarted all server
  • Upgraded Traccar version from 5.6 to 5.9 (Issue started when it was 5.6)

htop Screenshot: https://photos.app.goo.gl/cVrT33i2aDXtHpmZ7
traccar.conf: https://photos.app.goo.gl/ehxCv7i991fTBPXE9

I would greatly appreciate any assistance in figuring this out.

Anton Tananaeva year ago

Why do you have so many instances of Traccar?

Thank you Anton for the reply,

All those processes spin up as soon as I start the traccar.service, somehow. Isn't it supposed to be like that? I wonder if there are any ways to limit the number of processes. I have around 20 processes running just for Traccar, all of which start automatically with the traccar service.

Below is traccar.service:

[Unit]
Description=traccar
After=network.target

[Service]
Type=simple
WorkingDirectory=/opt/traccar
ExecStart=/opt/traccar/jre/bin/java -Xmx5G -jar tracker-server.jar conf/traccar.xml
SyslogIdentifier=traccar
SuccessExitStatus=143
WatchdogSec=600
Restart=on-failure
RestartSec=10

[Install]
WantedBy=multi-user.target

Added -Xmx5G as suggested on the official documentation.

Anton Tananaeva year ago

Are those processes or threads?

Track-tracea year ago

Isn't database.positionsHistoryDays hanging your server ?

Anton Tananaeva year ago

That parameter doesn't exist on the mentioned versions.

Thanks for the responses.

@Anton
Probably thread, i mean process as it has pid
@Track-trace
I found database.positionsHistoryDays in this forum, I've put it there since the beginning. Will remove it and restart to see if it makes any difference.

Anton Tananaeva year ago

If those are separate processes, something is seriously wrong. There should be only one.

Track-tracea year ago

Those are Threads, he was using HTOP when he posted the image of the processes. When @Nizamuddin Mohamed will just use TOP it will only show one Java process. My traccar server shows 46 threads in HTOP for Java traccar.

Those are threads, In htop, threads would show as processes. I removed database.positionsHistoryDays from traccar.xml and restarted but no luck. The CPU bumps to 100% as soon as the the Traccar service is running.

ps aux | grep traccar : https://photos.app.goo.gl/qf2acJJ896YVguQX8
htop combined threads: https://photos.app.goo.gl/9eu6QpvQPQKk1kBg7

Anton Tananaeva year ago

Do you have any computed attributes? Do how have geofences?

No computed attributes, but i have around 15 geofences.

Anton Tananaeva year ago

If your geofences are complicated (many points), that might be the cause.

Thanks again for the reply. Removed all the Geofences, restarted the MySQL service, and restarted the Traccar service. However, the issue still remains. I also tried removing the position history older than 45 days, but it didn't make any difference.

Anton Tananaeva year ago

As I mentioned previously, database.positionsHistoryDays doesn't exist, so it doesn't affect anything.

OK, more questions:

  1. Do you use any external services, like geocoding and geolocation? If yes, try disabling them.
  2. Is your database local or remote?