Server Error Data Loss

jaimzj9 years ago

Dear Anton,

I faced one Issue today, I had made a change to the position table, where I added an additional column (type int) with value null.

What Happened for 2 hours after that went unnoticed by me, and I lost all my vehicles gps data between these 2 hours.

The log shows db error for each time it tried to insert into position table. and then probably discarded the location data. the error was regarding 'the new column not having a default value set'.

I think it would be a good idea to.

  1. Set some exception cases where in-case of error and position entry being unable to update, the data is stored still in a certain file which can be later.
  2. Imported into db. once the issue is rectified. (thus avoiding losing of data).
Anton Tananaev9 years ago

Where do you want to store the data if there is a problem with database?

jaimzj9 years ago

The only thing, I could think of was in any case that the database fails or returns error.

Any data accepted by server, could be stored in lets say a text file or with comma seperated values each row.

This might come handy, until the administrator notices and fixes the problem after which he can import the data into their database.

In my case I am using mysql.

Anton Tananaev9 years ago

In your particular case it's your own fault. You should always check that everything works after making any changes.

In case of connection problems, database connection pooling library will automatically wait and re-connect. So the data would be safe if database comes back up eventually.

I can't think of any other use case where another layer of buffering is needed.

jaimzj9 years ago

I agree It was my fault, However I still strongly feel limiting it to database connection problem only wouldn't be as helpful.

Anton Tananaev9 years ago

I can consider it if you provide some real use cases for this feature.

jaimzj9 years ago

Honestly I cant think of any use real good use case.

But, yes If not creating a buffer, for a certain unspecific (user created issue/problem).

May I recommend an alternate thought of mine, which might come handy and be useful for any and all such issues.

That is in-case that the server app is for any reason not functioning as expected.

  1. Either the server process stops and alerts the admin to take necessary action.
  2. Or the server process stops acknowledging Messages from client until the problem is rectified.

Primarily the idea is to avoid-data loss, use case as well is to prevent data loss, both due to person made error or even just something unexpected happening.

Again this is not a use-case just my feedback :) ...

I did realise from this mistake of mine. and I have now set-up a staging server, and whatever changes I carry out, first I am now carrying it out on this before moving to Live Production

Anton Tananaev9 years ago

I agree that it's good to have such functionality, but there is so many more higher priority features that I don't think I would be able to implement this one any time soon.

jaimzj9 years ago

I understand :)