Mysql Throws error after working fine for weeks, PLEASE HELP its on LIVE PRODUCTION SERVER

karthik balu7 years ago

Traccar fails to start, it was working fine for more than a 3 weeks but suddenly after server restart its not connecting to mysql server and throws below error

I was able to connect to the same database, same credentials, JDBC connector using a simple java code
Traccar version 3.14 running on AWS EC2 AMI, mysql 5.7.11

Please help

Below is the attached traccar wrapper log

WARNING|wrapper|Service traccar|17-10-26 04:22:01|YAJSW: yajsw-stable-12.09
WARNING|wrapper|Service traccar|17-10-26 04:22:01|OS   : Linux/4.9.51-10.52.amzn1.x86_64/amd64
WARNING|wrapper|Service traccar|17-10-26 04:22:01|JVM  : Oracle Corporation/1.8.0_151//usr/java/jdk1.8.0_151/jre/64
INFO|wrapper|Service traccar|17-10-26 04:22:01|exec:"java" "-classpath" "/opt/traccar/wrapperApp.jar:/opt/traccar/tracker-server.jar:/opt/traccar/lib/log4j-1.2.17.jar:/opt/traccar/lib/joda-time-2.9.9.jar:
/opt/traccar/lib/h2-1.4.196.jar:/opt/traccar/lib/mysql-connector-java-5.1.44.jar:/opt/traccar/lib/postgresql-42.1.4.jre7.jar:/opt/traccar/lib/mssql-jdbc-6.2.1.jre7.jar:/opt/traccar/lib/HikariCP-java7-2.4.
12.jar:/opt/traccar/lib/slf4j-api-1.7.21.jar:/opt/traccar/lib/netty-3.10.6.Final.jar:/opt/traccar/lib/async-http-client-1.9.40.jar:/opt/traccar/lib/slf4j-simple-1.7.25.jar:/opt/traccar/lib/javax.json-1.0.
4.jar:/opt/traccar/lib/jetty-server-9.2.22.v20170606.jar:/opt/traccar/lib/javax.servlet-api-3.1.0.jar:/opt/traccar/lib/jetty-http-9.2.22.v20170606.jar:/opt/traccar/lib/jetty-io-9.2.22.v20170606.jar:/opt/t
raccar/lib/jetty-servlet-9.2.22.v20170606.jar:/opt/traccar/lib/jetty-security-9.2.22.v20170606.jar:/opt/traccar/lib/jetty-webapp-9.2.22.v20170606.jar:/opt/traccar/lib/jetty-xml-9.2.22.v20170606.jar:/opt/t
raccar/lib/jetty-jndi-9.2.22.v20170606.jar:/opt/traccar/lib/jetty-util-9.2.22.v20170606.jar:/opt/traccar/lib/jetty-proxy-9.2.22.v20170606.jar:/opt/traccar/lib/jetty-client-9.2.22.v20170606.jar:/opt/tracca
r/lib/websocket-server-9.2.22.v20170606.jar:/opt/traccar/lib/websocket-common-9.2.22.v20170606.jar:/opt/traccar/lib/websocket-api-9.2.22.v20170606.jar:/opt/traccar/lib/websocket-client-9.2.22.v20170606.ja
r:/opt/traccar/lib/websocket-servlet-9.2.22.v20170606.jar:/opt/traccar/lib/jersey-container-servlet-2.25.1.jar:/opt/traccar/lib/jersey-container-servlet-core-2.25.1.jar:/opt/traccar/lib/javax.inject-2.5.0
-b32.jar:/opt/traccar/lib/jersey-common-2.25.1.jar:/opt/traccar/lib/javax.annotation-api-1.2.jar:/opt/traccar/lib/jersey-guava-2.25.1.jar:/opt/traccar/lib/hk2-api-2.5.0-b32.jar:/opt/traccar/lib/hk2-utils-
2.5.0-b32.jar:/opt/traccar/lib/aopalliance-repackaged-2.5.0-b32.jar:/opt/traccar/lib/hk2-locator-2.5.0-b32.jar:/opt/traccar/lib/javassist-3.20.0-GA.jar:/opt/traccar/lib/osgi-resource-locator-1.0.1.jar:/op
t/traccar/lib/jersey-server-2.25.1.jar:/opt/traccar/lib/jersey-client-2.25.1.jar:/opt/traccar/lib/jersey-media-jaxb-2.25.1.jar:/opt/traccar/lib/validation-api-1.1.0.Final.jar:/opt/traccar/lib/javax.ws.rs-
api-2.0.1.jar:/opt/traccar/lib/jersey-media-json-jackson-2.25.1.jar:/opt/traccar/lib/jersey-entity-filtering-2.25.1.jar:/opt/traccar/lib/jackson-jaxrs-base-2.8.4.jar:/opt/traccar/lib/jackson-core-2.8.4.ja
r:/opt/traccar/lib/jackson-databind-2.8.4.jar:/opt/traccar/lib/jackson-jaxrs-json-provider-2.8.4.jar:/opt/traccar/lib/jackson-module-jaxb-annotations-2.8.4.jar:/opt/traccar/lib/jackson-annotations-2.8.4.j
ar:/opt/traccar/lib/liquibase-core-3.5.3.jar:/opt/traccar/lib/snakeyaml-1.17.jar:/opt/traccar/lib/mail-1.4.7.jar:/opt/traccar/lib/activation-1.1.jar:/opt/traccar/lib/jxls-2.4.2.jar:/opt/traccar/lib/common
s-jexl-2.1.1.jar:/opt/traccar/lib/commons-beanutils-1.9.2.jar:/opt/traccar/lib/jcl-over-slf4j-1.7.12.jar:/opt/traccar/lib/logback-core-1.1.3.jar:/opt/traccar/lib/jxls-poi-1.0.13.jar:/opt/traccar/lib/poi-3
.16.jar:/opt/traccar/lib/poi-ooxml-3.16.jar:/opt/traccar/lib/poi-ooxml-schemas-3.16.jar:/opt/traccar/lib/xmlbeans-2.6.0.jar:/opt/traccar/lib/stax-api-1.0.1.jar:/opt/traccar/lib/curvesapi-1.04.jar:/opt/tra
ccar/lib/velocity-1.7.jar:/opt/traccar/lib/commons-collections-3.2.1.jar:/opt/traccar/lib/commons-lang-2.4.jar:/opt/traccar/lib/velocity-tools-2.0.jar:/opt/traccar/lib/commons-digester-1.8.jar:/opt/tracca
r/lib/commons-chain-1.1.jar:/opt/traccar/lib/commons-logging-1.1.jar:/opt/traccar/lib/servlet-api-2.3.jar:/opt/traccar/lib/commons-validator-1.3.1.jar:/opt/traccar/lib/dom4j-1.1.jar:/opt/traccar/lib/oro-2
.0.8.jar:/opt/traccar/lib/sslext-1.2-0.jar:/opt/traccar/lib/struts-core-1.3.8.jar:/opt/traccar/lib/antlr-2.7.2.jar:/opt/traccar/lib/struts-taglib-1.3.8.jar:/opt/traccar/lib/struts-tiles-1.3.8.jar:/opt/tra
ccar/lib/commons-collections4-4.1.jar:/opt/traccar/lib/ical4j-2.0.4.jar:/opt/traccar/lib/commons-codec-1.9.jar:/opt/traccar/lib/commons-lang3-3.3.2.jar:/opt/traccar/lib/threetenbp-1.3.3.jar:/opt/traccar/l
ib/ch-smpp-5.0.9.jar:/opt/traccar/lib/ch-commons-util-6.0.2.jar:/opt/traccar/lib/ch-commons-charset-3.0.2.jar" "-Dfile.encoding=UTF-8" "-Xrs" "-Dwrapper.service=true" "-Dwrapper.console.visible=false" "-D
wrapper.port=15003" "-Dwrapper.key=-6934459343783858464" "-Dwrapper.teeName=-6934459343783858464$1508991721076" "-Dwrapper.tmp.path=/opt/traccar/tmp" "-Djna_tmpdir=/opt/traccar/tmp" "-Dwrapper.config=/opt
/traccar/conf/wrapper.conf" "-Dwrapper.visible=false" "-Dwrapper.pidfile=/var/run/wrapper.traccar.pid" "org.rzo.yajsw.app.WrapperJVMMain" 
INFO|wrapper|Service traccar|17-10-26 04:22:01|posix_spawn pid 3177
INFO|wrapper|Service traccar|17-10-26 04:22:01|started process 3177
INFO|wrapper|Service traccar|17-10-26 04:22:01|started process with pid 3177
INFO|3177/0|Service traccar|17-10-26 04:22:02|[INFO] StandardFileSystemManager - Using "/tmp/vfs_cache" as temporary files store.
INFO|3177/0|Service traccar|17-10-26 04:22:03|[main] INFO com.zaxxer.hikari.HikariDataSource - HikariPool-1 - Starting...
INFO|3177/0|Service traccar|17-10-26 04:22:03|Thu Oct 26 04:22:03 UTC 2017 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ 
and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to '
false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
INFO|3177/0|Service traccar|17-10-26 04:22:04|[main] INFO com.zaxxer.hikari.HikariDataSource - HikariPool-1 - Start completed.
INFO|3177/0|Service traccar|17-10-26 04:22:04|Thu Oct 26 04:22:04 UTC 2017 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ 
and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to '
false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
INFO|3177/0|Service traccar|17-10-26 04:22:04|Thu Oct 26 04:22:04 UTC 2017 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ 
and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to '
false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
INFO|3177/0|Service traccar|17-10-26 04:22:04|Thu Oct 26 04:22:04 UTC 2017 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ 
and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to '
false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
INFO|3177/0|Service traccar|17-10-26 04:22:04|Thu Oct 26 04:22:04 UTC 2017 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ 
and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to '
false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
INFO|3177/0|Service traccar|17-10-26 04:22:04|Thu Oct 26 04:22:04 UTC 2017 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ 
and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to '
false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
INFO|3177/0|Service traccar|17-10-26 04:22:04|Thu Oct 26 04:22:04 UTC 2017 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ 
and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to '
false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
INFO|3177/0|Service traccar|17-10-26 04:22:04|Thu Oct 26 04:22:04 UTC 2017 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ 
and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to '
false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
INFO|3177/0|Service traccar|17-10-26 04:22:04|Thu Oct 26 04:22:04 UTC 2017 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ 
and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to '
false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
INFO|3177/0|Service traccar|17-10-26 04:22:04|Thu Oct 26 04:22:04 UTC 2017 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ 
and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to '
false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
INFO|3177/0|Service traccar|17-10-26 04:22:05|Thu Oct 26 04:22:05 UTC 2017 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ 
and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to '
false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
INFO|3177/0|Service traccar|17-10-26 04:22:05|INFO 10/26/17 4:22 AM: liquibase: Clearing database change log checksums
INFO|3177/0|Service traccar|17-10-26 04:22:05|INFO 10/26/17 4:22 AM: liquibase: Waiting for changelog lock....
INFO|3177/0|Service traccar|17-10-26 04:22:15|java.lang.reflect.InvocationTargetException
INFO|3177/0|Service traccar|17-10-26 04:22:15|	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
INFO|3177/0|Service traccar|17-10-26 04:22:15|	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
INFO|3177/0|Service traccar|17-10-26 04:22:15|	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
INFO|3177/0|Service traccar|17-10-26 04:22:15|	at java.lang.reflect.Method.invoke(Method.java:498)
INFO|3177/0|Service traccar|17-10-26 04:22:15|	at org.rzo.yajsw.app.WrapperJVMMain.executeMain(WrapperJVMMain.java:60)
INFO|3177/0|Service traccar|17-10-26 04:22:15|	at org.rzo.yajsw.app.WrapperJVMMain.main(WrapperJVMMain.java:43)
INFO|3177/0|Service traccar|17-10-26 04:22:15|Caused by: liquibase.exception.LockException: liquibase.exception.DatabaseException: liquibase.exception.DatabaseException: com.mysql.jdbc.exceptions.jdbc4.My
SQLNonTransientConnectionException: Communications link failure during rollback(). Transaction resolution unknown.
INFO|3177/0|Service traccar|17-10-26 04:22:15|	at liquibase.lockservice.StandardLockService.acquireLock(StandardLockService.java:242)
INFO|3177/0|Service traccar|17-10-26 04:22:15|	at liquibase.lockservice.StandardLockService.waitForLock(StandardLockService.java:170)
INFO|3177/0|Service traccar|17-10-26 04:22:15|	at liquibase.Liquibase.clearCheckSums(Liquibase.java:1335)
INFO|3177/0|Service traccar|17-10-26 04:22:15|	at org.traccar.database.DataManager.initDatabaseSchema(DataManager.java:292)
INFO|3177/0|Service traccar|17-10-26 04:22:15|	at org.traccar.database.DataManager.<init>(DataManager.java:81)
INFO|3177/0|Service traccar|17-10-26 04:22:15|	at org.traccar.Context.init(Context.java:281)
INFO|3177/0|Service traccar|17-10-26 04:22:15|	at org.traccar.Main.main(Main.java:35)
INFO|3177/0|Service traccar|17-10-26 04:22:15|	... 6 more
INFO|3177/0|Service traccar|17-10-26 04:22:15|Caused by: liquibase.exception.DatabaseException: liquibase.exception.DatabaseException: com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException:
 Communications link failure during rollback(). Transaction resolution unknown.
INFO|3177/0|Service traccar|17-10-26 04:22:15|	at liquibase.database.AbstractJdbcDatabase.rollback(AbstractJdbcDatabase.java:1168)
INFO|3177/0|Service traccar|17-10-26 04:22:15|	at liquibase.lockservice.StandardLockService.acquireLock(StandardLockService.java:205)
INFO|3177/0|Service traccar|17-10-26 04:22:15|	... 12 more
INFO|3177/0|Service traccar|17-10-26 04:22:15|Caused by: liquibase.exception.DatabaseException: com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: Communications link failure during rol
lback(). Transaction resolution unknown.
INFO|3177/0|Service traccar|17-10-26 04:22:15|	at liquibase.database.jvm.JdbcConnection.rollback(JdbcConnection.java:340)
INFO|3177/0|Service traccar|17-10-26 04:22:15|	at liquibase.database.AbstractJdbcDatabase.rollback(AbstractJdbcDatabase.java:1166)
INFO|3177/0|Service traccar|17-10-26 04:22:15|	... 13 more
INFO|3177/0|Service traccar|17-10-26 04:22:15|Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: Communications link failure during rollback(). Transaction resolution unknown
.
INFO|3177/0|Service traccar|17-10-26 04:22:15|	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
INFO|3177/0|Service traccar|17-10-26 04:22:15|	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
INFO|3177/0|Service traccar|17-10-26 04:22:15|	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
INFO|3177/0|Service traccar|17-10-26 04:22:15|	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
INFO|3177/0|Service traccar|17-10-26 04:22:15|	at com.mysql.jdbc.Util.handleNewInstance(Util.java:425)
INFO|3177/0|Service traccar|17-10-26 04:22:15|	at com.mysql.jdbc.Util.getInstance(Util.java:408)
INFO|3177/0|Service traccar|17-10-26 04:22:15|	at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:918)
INFO|3177/0|Service traccar|17-10-26 04:22:15|	at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:897)
INFO|3177/0|Service traccar|17-10-26 04:22:15|	at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:886)
INFO|3177/0|Service traccar|17-10-26 04:22:15|	at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:860)
INFO|3177/0|Service traccar|17-10-26 04:22:15|	at com.mysql.jdbc.ConnectionImpl.rollback(ConnectionImpl.java:4584)
INFO|3177/0|Service traccar|17-10-26 04:22:15|	at liquibase.database.jvm.JdbcConnection.rollback(JdbcConnection.java:337)
INFO|3177/0|Service traccar|17-10-26 04:22:15|	... 14 more
INFO|wrapper|Service traccar|17-10-26 04:22:15|waitpid 3177 59136
INFO|wrapper|Service traccar|17-10-26 04:22:15|exit code posix process: 59136 application: 0
Anton Tananaev7 years ago

Looks like there is some database connection problem.

karthik balu7 years ago

I was able to connnect using JDBC simple java connect code without any issue

Also traccar logs filled up all disk space , so i disabled the logs in traccar.xml, after that looks like the server stopped working
so i disabled logs

<entry key='web.debug'>false</entry>
<entry key='logger.enable'>false</entry>
<entry key='logger.level'>none</entry>

Is it possible to see more logs to understand better

Also found this in my traccar startup log

INFO: Starting traccar ...
INFO: lib not found: /opt/traccar/lib/extended/glazedlists/glazedlists-1.8.0_java15.jar
INFO: lib not found: /opt/traccar/lib/extended/glazedlists/commons-beanutils-1.8.2.jar
INFO: lib not found: /opt/traccar/lib/extended/glazedlists/commons-beanutils-1.8.2.jar
INFO: lib not found: /opt/traccar/lib/extended/glazedlists/glazedlists-1.8.0_java15.jar

Anton Tananaev7 years ago

Maybe your database got locked. You need to manually unlock it in this case.

karthik balu7 years ago

Oh god! Thanks man, I fixed it

this helped
https://www.traccar.org/forums/topic/liquibase-waiting-for-changelog-lock/

But wondering if this could be handled in the code to such crash scenarios

David Stanley3 years ago

HILLBILLY HACK SOLUTION as far as for MSSQL Server Restart Server Traccar server-start fails.
I found an inductive solution for MSSQL restart fails of the traccar service. However crude it may be, it has resolved 100% of my server instances with a simple sequencing technique. I found I had to DELAY starting the traccar service until AFTER ALL MSSQL services were fully started, so that when Traccar services went after the database lock, there was a database lock to 'find'. Common sense, really, can't get a lock if database is not up yet. :) In about 1 in 3 restarts, Traccar got to the 'up and ready' state, 'faster' than SQL did, even when setting traccar to delayed start. Logs indicated the database lock problem, but in closer investigation, it was just because the SQL was not fully up and running and in a ready state yet. After reinstalling SQL several times, and also getting the same error in brand new VM instances, I began to realize there was nothing inherently wrong with all those test SQLs. It was in loading sequencing and state completion. Traccar was finishing faster than SQL, randomly. That made it harder to run to ground, but once timing was identified, solving the sequencing problem, I have not had a traccar failure to start after restart failure in over 2 dozen servers since, with MSSQL servers.

Figures that MSSQL takes it own sweet time loading. :) And, sometimes, but not every time. Microsoft...whatya gonna do.

No set of settings in the services console would completely eliminate the random event of traccar trying to get a lock on a database that was not 'there' yet'. Once Traccar's 'premature lock-ulation happened' happened, many times, no combination retry console settings would bring it online without human intervention...the kiss of death with 2 dozen servers and climbing. I needed a bulletproof, never fail solution on restarting a server,
on every server, every restart, even MS-hijacking-for-unplanned windows-update-restarts(those being the worst).

SOLUTION: I scripted a VERY simple services command in Windows Task Scheduler. It checks for a running instance of the traccar service,
5 minutes after my nightly server restart. IF the traccar service is not already running, it fires it up. My SQL server instances are fully up and running by 5 minutes after server restart, then, so, if traccar's normal start has failed for any reason, this commanding task backup task gives it a do-over, and it can then acquire the needed lock. If traccar is already running, the task completes with success without refiring a news instance. if traccar is NOT running, it fires it up. Never had a single hang since that time, on dozens of instances.

Windows Task Scheduler : commanding action: [ sc.exe ] parameter: [ start traccar ] trigger: timer-based (I set mine 5 minutes after restart)
set the task to NOT fire up IF an instance is already running via the console interface
set task to repeat a number of times if unsuccessful (I set mine to 59 retries, never needs it but to be safe)

This careful set of settings and the commanding syntax has completely resolved the issue of traccar services starting a bit faster than MSSQL and hunting for a database lock that is not quite 'there' yet. A hillbilly, non-scientific hack, for sure, but it has saved me dozens of alerts in the middle of the night since perfecting this little technique.

You will want to open a command window as an administrator, run sc.exe queryex to verify that your traccar service name is traccar etc, but this has done the trick for me, without question, on dozens of servers without fail.

hope it is helpful to other mssql users.