I'm using server version 8.6.1 and writing a Java client to move lots of documents from a SQL database to a BaseX database.

The basic version works just fine. Tested up to 1M documents.

Now I'm trying to do exactly the same but with AUTOFLUSH disabled + manually FLUSH every N documents and it fails after just a few inserts.

Pseudo code:

(set up client session, create and open new database)

session.execute(new Set(MainOptions.AUTOFLUSH, false));

int i = 0;

while(doc = nextDocs()) {

  session.add(doc.getKey(), doc.getInputStream());

  if(i++ % 10 == 0) { /* value reduced to 10 for testing purposes */

    session.execute(new Flush());

  }

}

(tear down session)

Remember tha

After the first FLUSH has been issued, the ADD  directive starts randomly throwing a org.basex.core.BaseXException (when using the BaseXClient implementation it was its inner ok() method that started returning false).

I tracked the root cause in the server: the SAXParser complaining about the documents not being well-formed (the exact same documents are all imported fine with AUTOFLUSH on). In the server standard logs I can see why: parts of those documents are suddenly missing (usually the beginning + command part) at parse time!

That is the symptom, now for the cause I'm also observing that the ADD and FLUSH directive in the logs are out of order, exactly like what would happen in the FLUSH commands were not properly queued.

I was expecting something like :

ADD

ADD

ADD

(N times)

FLUSH

ADD

ADD

etc.

Instead I'm observing 

ADD

FLUSH

ADD

FLUSH

ADD

FLUSH

(etc)

then 

ADD

ADD

ADD

(lots of them)

ERROR

So for now I'm keeping autoflush on :) but is there anything obvious I could try there?

Thanks,

Colin