Adding documents to database while space on disk drive runs out
Hi, I just had the problem that the disk space of the drive where a BaseX database instance was stored ran out while adding a new document to the database via the server interface. The result was a half-finished document in the database resulting in ArrayIndexOutOfBounds-exceptions when querying the database and strange behavior of the GUI (views are not shown and even disabled). Would it be possible to check for sufficient disk space before writing to the database files? Moreover, it would be nice if the failed transaction of adding the document while the space ran out could be rolled back somehow to get the database back into a consistent state. Thank you for your support! Best regards, Thomas
Dear Thomas, I’m sorry that your data got corrupted. I agree that such a test would be a desirable addition, I’m afraid it’s often difficult to guess how much space will be occupied by a specific update operation. We could generally disable update operations when the remaining space falls below a given limit, but once again such a limit would be strongly dependent on the used system and use cases. Rollbacks are on our long-term todo list (along with, or even without MVCC), similar to incremental backups, but if memory gets low, they may as well lead to unexpected behavior. All the best, Christian _________________________________________
I just had the problem that the disk space of the drive where a BaseX database instance was stored ran out while adding a new document to the database via the server interface. The result was a half-finished document in the database resulting in ArrayIndexOutOfBounds-exceptions when querying the database and strange behavior of the GUI (views are not shown and even disabled). Would it be possible to check for sufficient disk space before writing to the database files? Moreover, it would be nice if the failed transaction of adding the document while the space ran out could be rolled back somehow to get the database back into a consistent state.
Thank you for your support!
Best regards, Thomas _______________________________________________ BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
Dear Christian, thank you for your response. Actually, I could recover most of the data by exporting everything from the corrupted database and re-importing into a new database instance (the partial document was of course lost). I think the best solution would be to add a parameter to the global configuration which defines a memory limit that is checked for before performing any write operations. The default value of this parameter could be 0 (= no limit) and that way the current behavior is not changed, but the user could specify his/her own limit based on his/her system and use cases. Best regards, Thomas
-----Original Message----- From: Christian Grün [mailto:christian.gruen@gmail.com] Sent: Thursday, November 22, 2012 7:08 PM To: Thomas Kaltofen; Thomas Kaltofen Cc: basex-talk@mailman.uni-konstanz.de Subject: Re: [basex-talk] Adding documents to database while space on disk drive runs out
Dear Thomas,
I'm sorry that your data got corrupted. I agree that such a test would be a desirable addition, I'm afraid it's often difficult to guess how much space will be occupied by a specific update operation. We could generally disable update operations when the remaining space falls below a given limit, but once again such a limit would be strongly dependent on the used system and use cases.
Rollbacks are on our long-term todo list (along with, or even without MVCC), similar to incremental backups, but if memory gets low, they may as well lead to unexpected behavior.
All the best, Christian _________________________________________
I just had the problem that the disk space of the drive where a BaseX database instance was stored ran out while adding a new document to the database via the server interface. The result was a half-finished document in the database resulting in ArrayIndexOutOfBounds-exceptions when querying the database and strange behavior of the GUI (views are not shown and even disabled). Would it be possible to check for sufficient disk space before writing to the database files? Moreover, it would be nice if the failed transaction of adding the document while the space ran out could be rolled back somehow to get the database back into a consistent state.
Thank you for your support!
Best regards, Thomas _______________________________________________ BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
participants (2)
-
Christian Grün -
Thomas Kaltofen