So this happened again overnight but I have more information now.
1) I had switched off UPDINDEX on some of the databases and one of these was affected so this is not an UPDINDEX issue 2) I found that there was a hard crash of the JVM - which would account for why many databases were left locked and one had the folder with the numbers after 3) My Upstart script restarts the server quite happily which disguises 2) if you’re not looking closely!
The hard crash was due to the JVM not being able to allocate memory. What I’m not sure is why my script to OPTIMIZE all the databases seems to run quite happily manually but is failing when run by cron overnight.
I will continue monitoring.
Regards, James
On 8 Jan 2016, at 19:11, James Ball basex-talk@jamesball.co.uk wrote:
Christian,
I can’t reliably reproduce the behaviour at all at the moment but I’m losing a fair number of databases to the problem. I first thought it was related to running out of disk space mid-operation but I don’t think that’s the cause. I just wanted to check that I wasn’t doing anything I shouldn’t before I try to find a reproducible error.
UPDINDEX is really useful for my use case but seems to leave me with database in GB range until I OPTIMIZE them. I shall continue monitoring.
Many thanks, James
On 8 Jan 2016, at 19:00, Christian Grün christian.gruen@gmail.com wrote:
So another question - these seem to come about from running OPTIMIZE ALL on databases with UPDINDEX TRUE. Is there any reason that OPTIMIZE ALL can’t be run on such databases?
It shouldn’t cause any problems. Here is a little command scripts that demonstrates that it should work alright:
<commands> <set option='updindex'>true</set> <create-db name='db'/> <add path='a.xml'><a/></add> <add path='b.xml'><b/></add> <optimize-all/> <info-db/> <close/> </commands>
Can you also reproduce the behavior with smaller data sets?
Thanks, Christian