So we think we got the data back. The people that created it will check in the next few days.

The reason for these problems almost certain an out of diskspace situation. I don't know if this is something that BaseX could be better protected against in the future.
It would certainly be hard to incorporate in automatet tests.

I think this was just luck and maybe it wont work for anyone who tries that for their data.

Here is what I did:
* I got the source code for BaseX 9 branch and had a look what hapens at org.basex.data.DiskData.write(DiskData.java:144).
* This pointed me to the inf.basex file. inf.basex ins only 319 bytes.
* I compared that to a (way to old) backup of that same database where inf.basex is 63569 bytes.
* So I replaced that too small inf.basex file with the old version.
* I tried to open the db in basex running the IDE debugger.
  I could do a "xquery ." until it stopped with a Util.notExpected Data Access out of bounds exception.
* I disabled that check in TabelDiskAccess.java and used "xquery ." on the open database again which worked.
* I did an export to some directory and the result had only one obvieous defect: a missing attribute name on the root tag.

So maybe there is a way to improve the redundancy of the database structure files like this inf.basex? Or have a tool that can recreate it from other files?

We activated an external backup process today so hopefully this will never happen again.

Best regards

Am 12.05.2025 um 14:19 schrieb Christian Grün:
Hi Omar,

I am sorry to hear that. BaseX provides some basic recovery mechanisms, but it does not cover all cases that may happen (and of course it is always too late to give the advise to do regular backups when a failure occurs…).

If OPTIMIZE ALL fails, a simple OPTIMIZE works sometimes, but I guess not in this case?

In general it's a good sign if INSPECT reports no errors. I could have a look at the database this week (for free, as I cannot promise anything).

Best,
Christian


Omar Siam <Omar.Siam@oeaw.ac.at> schrieb am Mo., 12. Mai 2025, 14:04:
Hi!

We had some serious problems with our storage last week and
unfortunately that killed one of our important BaseX databases.

Is there any way to restore the data?

Does BaseX GmbH provide paid data rescue services?

The symptoms are:

There is data in that database:

Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
-a---          12.05.2025    13:16       25214377 atv.basex
-a---          12.05.2025    13:16         451580 atvl.basex
-a---          12.05.2025    13:16         106065 atvr.basex
-a---          12.05.2025    13:16            131 ftxx.basex
-a---          12.05.2025    13:16         449615 ftxy.basex
-a---          12.05.2025    13:16         697581 ftxz.basex
-a---          12.05.2025    13:59            319 inf.basex
-a---          12.05.2025    13:16             39 pth.basex
-a---          12.05.2025    13:16             28 swl.basex
-a---          12.05.2025    13:16       29798400 tbl.basex
-a---          12.05.2025    13:16          23650 tbli.basex
-a---          12.05.2025    13:16         693232 tokl.basex
-a---          12.05.2025    13:16         105495 tokr.basex
-a---          12.05.2025    13:16       53480811 txt.basex
-a---          12.05.2025    13:16         669153 txtl.basex
-a---          12.05.2025    13:16         166050 txtr.basex
-a---          12.05.2025    13:59              0 upd.basex

Inspect reports 0 documents.

Checking main table (0 nodes):
- 0 invalid node kinds
- 0 invalid parent references
- 0 wrong parent/descendant relationships
No inconsistencies found.
'dc_shawi_eng_000' inspected in 0.63 ms.

Optimize all yields:

Improper use? Potential bug? Your feedback is welcome:
Contact: basex-talk@mailman.uni-konstanz.de
Version: BaseX 9.3.2
Java: Azul Systems, Inc., 21.0.4
OS: Windows 11, amd64
Stack Trace:
java.lang.NullPointerException: Cannot invoke
"org.basex.index.path.PathIndex.write(org.basex.io.out.DataOutput)"
because "this.paths" is null
         at org.basex.data.DiskData.write(DiskData.java:144)
         at org.basex.data.DiskData.close(DiskData.java:160)
         at org.basex.core.Datas.unpin(Datas.java:52)
         at org.basex.core.cmd.Close.close(Close.java:45)
         at org.basex.core.cmd.OptimizeAll.optimizeAll(OptimizeAll.java:126)
         at org.basex.core.cmd.OptimizeAll$1.run(OptimizeAll.java:42)
         at org.basex.core.cmd.ACreate.update(ACreate.java:90)
         at org.basex.core.cmd.OptimizeAll.run(OptimizeAll.java:38)
         at org.basex.core.Command.run(Command.java:257)
         at org.basex.core.Command.execute(Command.java:93)
         at org.basex.api.client.LocalSession.execute(LocalSession.java:132)
         at org.basex.api.client.Session.execute(Session.java:36)
         at org.basex.core.CLI.execute(CLI.java:92)
         at org.basex.core.CLI.execute(CLI.java:76)
         at org.basex.BaseX.console(BaseX.java:177)
         at org.basex.BaseX.<init>(BaseX.java:152)
         at org.basex.BaseX.main(BaseX.java:43)

Best regards


-- 
Mag. Ing. Omar Siam
Austrian Center for Digital Humanities and Cultural Heritage
Österreichische Akademie der Wissenschaften | Austrian Academy of Sciences
Stellvertretende Behindertenvertrauensperson | Deputy representative for disabled persons
Bäckerstraße 13, 1010 Wien, Österreich | Vienna, Austria
T: +43 1 51581-7295
omar.siam@oeaw.ac.at | www.oeaw.ac.at/acdh