Size of Database
When you have 100K devices... that means, you will have 300 000 - 400 000 USD incoming taxes every month! Where is the problem to buying and put few terabytes hdd ? ;)
But..., why you need packages every 5 seconds?
Make some optimization on your devices, let them to send information only when have some new states and when passed some meters in moving state. Otherwise you will have a problems and with reporting queries.
Thank you @Kaloyan Kanev,
we are using traccar of our SaaS platform and on this our users(manufacture, ev rental company) will add their device and use it, so 100k is not our devices and we have a metric called idle time and for that we have to store every device data, even if it is not running.
Also we are based in India so tax related stuff is different.
Since you have told me about hdd, we will provide our user to get their data even they are very old and can get reports like route and trips, I think on hdd it will be a bottleneck
Ok, let's go back to your questin "Size of Database"
The size of database depending of two major factors.
1: Time period to sending package to the server. Now you have this interval 5 seconds, if you make it to 10 seconds, you will have double times smaller size of your DB. My opinion is, not need every 5 seconds info from the car when she is in parking state!
2: Quantity of parameters which devices sending to the server. There, depending of the gps model that you using.
Thanks @kaloyan kanev for you amazing insight, I just want to know one thing is that 1.1 gb per device per year is possible or we are doing something wrong
Very possible with a high frequency of reporting.
The basic problem is that 5 second reporting period.
Your vehicles will not travel very far in five seconds. Less distance than the accuracy range of the GPS.
I suggest that you make it every 30 seconds or even every 5 minutes.
@hairydog for our use case 5 sec reporting is kind a important. Now this is off topic but I want to know that in this case how would you provision your server.
what i am thinking is an several AWS EC2 with mysql RDS. I know that for best performance local db is preferred but if we use db locally then ec2 storage will be a bottleneck for us.
Let me know what you think
Hello everyone,
our device send data at every 5 sec and due to this, our db is filling up very fast.
Based on our calculation one device will take 1.1gb of data per year and since we are fleet management company we are planning to use 100k vehicle within a the end of next years. i.e (100k gb data per year)
I want to know that on what factor apart from frequency does db size depends, is it on protocol or something else?
Also how would you tackle this type of work load as we can't delete old data and have to persists every device data.