Hi,
we have a rather strange and hard to track problem with corrupted
databases.
Our setup is:
- Docker container with a Tomcat that hosts BaseX with some
custom RESTXQ services
- BaseX 9.2.4
- Java 14
- Docker runs on a Linux VM
Workflow
- Create database with RESTXQ service call
- Import JSON document with RESTXQ service call
- call this multiple times.
After some Import calls, the import fails and the database is
corrupt from this point on.
We first thought that it has something to do with the content of
the document. But we found no pattern. Sometimes it works, but
sometime it does not.
There is no concurrency involved. There are no other clients that
read or write to the database.
We also tried to deactivate the UPDINDEX setting. But it had no
effect and we could reproduce the error with and without the
automatic index update.
The logs in case of errors look like this:
06:58:00.683 172.18.0.2:33728 admin REQUEST [PUT] /c42-core/api/v1/restxq/user/documents/c42-index/metadata%40document
06:58:00.700 172.18.0.2:33728 admin 500 Unexpected error: Improper use? Potential bug? Your feedback is welcome:
Contact: basex-talk@mailman.uni-konstanz.de Version: BaseX 9.2.4 Java: Oracle Corporation, 14.0.1 OS: Linux, amd64
Stack Trace: java.lang.ArrayIndexOutOfBoundsException: Index 4 out of bounds for length 1
at org.basex.io.random.TableDiskAccess.fpre(TableDiskAccess.java:507)
at org.basex.io.random.TableDiskAccess.cursor(TableDiskAccess.java:467)
at org.basex.io.random.TableDiskAccess.read1(TableDiskAccess.java:156)
at org.basex.data.Data.kind(Data.java:304)
at org.basex.query.up.DataUpdates.prepare(DataUpdates.java:133)
at org.basex.query.up.ContextModifier.prepare(ContextModifier.java:90)
at org.basex.query.up.Updates.prepare(Updates.java:168)
at org.basex.query.QueryContext.update(QueryContext.java:678)
at org.basex.query.QueryContext.iter(QueryContext.java:332)
at org.basex.http.restxq.RestXqResponse.serialize(RestXqResponse.java:73)
at org.basex.http.web.WebResponse.create(WebResponse.java:63)
at org.basex.http... 16.59 ms
Our service does not much. It just calls db:replace().
declare variable $documents:IMPORT_OPTS := map {'chop': fn:false(), 'stripns': fn:false(), 'intparse': fn:true()};
declare
%rest:PUT("{$xml}")
%rest:consumes("application/xml")
%rest:produces("application/json")
%rest:path("/user/documents/{$databaseId}/{$documentId}")
%updating
function documents:create($databaseId as xs:string, $documentId as xs:string, $xml as document-node())
{
if (db:exists($databaseId)) then (
update:output(response:empty(204, ())),
db:replace($databaseId, documents:decode($documentId), $xml, $documents:IMPORT_OPTS)
)
else (
update:output(response:json(errors:error('C42UDO002', map {'databaseId': $databaseId}), 404)), ()
)
};
I've attached the example input document.
One addition:
We could not reproduce this error running the Docker container on a Windows host.
Any feedback or hints to solve this are greatly appreciated.
Best regards
Johannes