17 May
2018
17 May
'18
11:16 p.m.
correction: ofcourse executing the command: SET AUTOFLUSH false On Thu, May 17, 2018 at 11:15 PM, halit tiryaki <halit.tiryaki@gmail.com> wrote: > > How did you invoke flush? Did you use the client/server architecture? > I'm using the C# client. > Simply executing the commands on the session: > - SET AUTOFLUSH true > - FLUSH (after ~200 docs of insert) > > > > The stack trace seems to indicate that you were deleting data. Is it > part of your import step? > yes, outdated documents are dropped in the import process > > Thanks > Halit > > On Thu, May 17, 2018 at 10:28 PM, Christian Grün < > christian.gruen@gmail.com> wrote: > >> Hi Halit, >> >> >> Some more questions: >> >> > The Flush() invoke in longer periods during the process might have >> been corrupted the state in race conditions. >> >> How did you invoke flush? Did you use the client/server architecture? >> >> Restored the db from a backup and restarted the import with >>> AutoFlush:true, this time I got: >>> >> >> The stack trace seems to indicate that you were deleting data. Is it part >> of your import step? >> >> Best, >> Christian >> >> >> >>> >>> >>> Improper use? Potential bug? Your feedback is welcome: >>> Contact: basex-talk@mailman.uni-konstanz.de >>> Version: BaseX 9.0.2 beta >>> Java: Oracle Corporation, 1.8.0_151 >>> OS: Linux, amd64 >>> Stack Trace: >>> java.lang.RuntimeException: Key does not exist: >>> '86295d851a084fb2bde4181e80760099' >>> at org.basex.util.Util.notExpected(Util.java:61) >>> at org.basex.index.value.UpdatableDiskValues.delete(UpdatableDi >>> skValues.java:99) >>> at org.basex.data.Data.indexDelete(Data.java:1060) >>> at org.basex.data.Data.delete(Data.java:663) >>> at org.basex.query.up.atomic.Delete.apply(Delete.java:39) >>> at org.basex.query.up.atomic.AtomicUpdateCache.applyUpdates(Ato >>> micUpdateCache.java:296) >>> at org.basex.query.up.atomic.AtomicUpdateCache.execute(AtomicUp >>> dateCache.java:280) >>> at org.basex.query.up.DataUpdates.apply(DataUpdates.java:168) >>> at org.basex.query.up.ContextModifier.apply(ContextModifier.java:120) >>> at org.basex.query.up.Updates.apply(Updates.java:157) >>> at org.basex.query.QueryContext.iter(QueryContext.java:341) >>> at org.basex.query.QueryProcessor.iter(QueryProcessor.java:90) >>> at org.basex.core.cmd.AQuery.query(AQuery.java:92) >>> at org.basex.core.cmd.XQuery.run(XQuery.java:22) >>> at org.basex.core.Command.run(Command.java:257) >>> at org.basex.core.Command.execute(Command.java:93) >>> at org.basex.server.ClientListener.run(ClientListener.java:140) >>> >>> On Mon, May 14, 2018 at 4:10 PM, Christian Grün < >>> christian.gruen@gmail.com> wrote: >>> >>>> Hi Halit, >>>> >>>> Thanks for reporting the stack trace. The only thing that I can derive >>>> from your error message is that something has gone wrong with your >>>> database in an earlier state. Unfortunately, it’s difficult to guess >>>> what might have been the reason. Feel free to give us an update if you >>>> should manage to get the issue reproduced and reproducible for us. >>>> >>>> Best, >>>> Christian >>>> >>>> >>>> >>>> On Wed, May 9, 2018 at 4:28 PM, halit tiryaki <halit.tiryaki@gmail.com> >>>> wrote: >>>> > I'm using the C# driver. but still, it doesn't look like a high level >>>> > problem. neither a query problem. thats why I didn't post some code >>>> > snippets. To make sure, I just tried to execute the INSPECT command >>>> in the >>>> > CLI. same exception. Also the db:optimize fails with the same >>>> exception, >>>> > although with a different stacktrace, which I will post below. >>>> > >>>> > Altough the race condition point of two processes interfering is a >>>> good >>>> > point, I'm currently having trouble to do anything useful with a >>>> session >>>> > from which a race condition would matter. If, then it must have >>>> happened >>>> > while the import was running along with the other, non-importing >>>> process. >>>> > >>>> > As I mentioned in the first post, while the import process was still >>>> > ongoing, the other process already started to have the exception at a >>>> very >>>> > late progress of the import. After the import was done, I was still >>>> able to >>>> > operate on the db as normal, but only with the session in the process >>>> which >>>> > did the long-run import. >>>> > >>>> > To restore the second process its session state to normal, I >>>> restarted the >>>> > webapplication, which closes both sessions. After that, I'm no longer >>>> to >>>> > operate on the db. Not even directly from the CLI. >>>> > >>>> > Seems it is a sort of a persisted data state which the low level >>>> basex code >>>> > can't handle when trying to reread it into to the memory. >>>> > >>>> > I 've already googled around for the same exception in BaseX, seems >>>> it was >>>> > an issue in some older versions too. >>>> > >>>> > >>>> > here the stacktrace when running db:optimize: >>>> > >>>> > Improper use? Potential bug? Your feedback is welcome: >>>> > Contact: basex-talk@mailman.uni-konstanz.de >>>> > Version: BaseX 9.0.2 beta >>>> > Java: Oracle Corporation, 1.8.0_151 >>>> > OS: Linux, amd64 >>>> > Stack Trace: >>>> > java.lang.ArrayIndexOutOfBoundsException: 52 >>>> > at org.basex.util.hash.TokenSet.key(TokenSet.java:128) >>>> > at org.basex.data.Data.name(Data.java:388) >>>> > at org.basex.io.serial.Serializer.node(Serializer.java:414) >>>> > at org.basex.io.serial.Serializer.node(Serializer.java:158) >>>> > at org.basex.io.serial.Serializer.node(Serializer.java:345) >>>> > at org.basex.io.serial.Serializer.node(Serializer.java:158) >>>> > at org.basex.io.serial.Serializer.serialize(Serializer.java:109) >>>> > at org.basex.core.cmd.OptimizeAll$DBParser.parse(OptimizeAll. >>>> java:200) >>>> > at org.basex.build.Builder.parse(Builder.java:77) >>>> > at org.basex.build.DiskBuilder.build(DiskBuilder.java:77) >>>> > at org.basex.core.cmd.OptimizeAll.optimizeAll(OptimizeAll.java:122) >>>> > at org.basex.query.up.primitives.db.DBOptimize.apply(DBOptimize >>>> .java:124) >>>> > at org.basex.query.up.DataUpdates.apply(DataUpdates.java:175) >>>> > at org.basex.query.up.ContextModifier.apply(ContextModifier.java:120) >>>> > at org.basex.query.up.Updates.apply(Updates.java:157) >>>> > at org.basex.query.QueryContext.iter(QueryContext.java:341) >>>> > at org.basex.query.QueryProcessor.iter(QueryProcessor.java:90) >>>> > at org.basex.core.cmd.AQuery.query(AQuery.java:92) >>>> > at org.basex.core.cmd.XQuery.run(XQuery.java:22) >>>> > at org.basex.core.Command.run(Command.java:257) >>>> > at org.basex.core.Command.execute(Command.java:93) >>>> > at org.basex.server.ClientListener.run(ClientListener.java:140) >>>> > >>>> > >>>> > On Wed, May 9, 2018 at 11:45 AM, Alexander Holupirek < >>>> alex@holupirek.de> >>>> > wrote: >>>> >> >>>> >> Hi, >>>> >> >>>> >> (please also respond to the list. Others might have suggestions as >>>> well >>>> >> ;-) >>>> >> >>>> >> it still would be great if you could provide even more details. For >>>> >> instance, what programming language do you use?Ideally include a >>>> Short, Self >>>> >> Contained, Correct (Compilable), Example (SSCCE) [1] in order to let >>>> people >>>> >> reproduce the behaviour. >>>> >> >>>> >> From a high level perspective, are you sure the sessions do not >>>> interfere? >>>> >> Do you additionally work with BaseXGUI on the database, while your >>>> program >>>> >> is importing? >>>> >> >>>> >> Cheers, >>>> >> Alex >>>> >> >>>> >> >>>> >> [1] http://sscce.org/ >>>> >> >>>> >> > On 9. May 2018, at 09:26, halit tiryaki <halit.tiryaki@gmail.com> >>>> wrote: >>>> >> > >>>> >> > hello, thanks for the fast response. >>>> >> > >>>> >> > It happened in the process of a large Import. There were two >>>> processes >>>> >> > with each a respective Session to BaseX. One process with the Open >>>> session >>>> >> > which did the Import was doing fine. The Second process however >>>> Starter >>>> >> > having the mentioned exception. After stopping Both processes and >>>> restarting >>>> >> > the app, no basex Session is now able to operate normal. >>>> >> > >>>> >> > The ArrayIndexOutOfBoundsException already comes in executing the >>>> >> > command INSPECT. >>>> >> > >>>> >> > The db-Info returns following: >>>> >> > >>>> >> > >>>> >> > info db: >>>> >> > Database Properties >>>> >> > NAME: XXX >>>> >> > SIZE: 5877 kB >>>> >> > NODES: 124051 >>>> >> > DOCUMENTS: 5497 >>>> >> > BINARIES: 0 >>>> >> > TIMESTAMP: 2018-05-09T05:20:44.000Z >>>> >> > UPTODATE: false >>>> >> > >>>> >> > Resource Properties >>>> >> > INPUTPATH: >>>> >> > INPUTSIZE: 0 b >>>> >> > INPUTDATE: 2018-05-09T04:32:10.539Z >>>> >> > >>>> >> > Indexes >>>> >> > TEXTINDEX: true >>>> >> > ATTRINDEX: true >>>> >> > TOKENINDEX: false >>>> >> > FTINDEX: false >>>> >> > TEXTINCLUDE: /item_User/node()/User:Email >>>> >> > ATTRINCLUDE: /item_UserImage/node()/UserImage:Id >>>> >> > TOKENINCLUDE: >>>> >> > FTINCLUDE: /item_XXX/node()/XXX:[german] >>>> >> > LANGUAGE: English >>>> >> > STEMMING: false >>>> >> > CASESENS: false >>>> >> > DIACRITICS: false >>>> >> > STOPWORDS: >>>> >> > UPDINDEX: true >>>> >> > AUTOOPTIMIZE: false >>>> >> > MAXCATS: 100 >>>> >> > MAXLEN: 96 >>>> >> > SPLITSIZE: 0 >>>> >> >>>> >> > Alexander Holupirek <alex@holupirek.de> schrieb am Mi., 9. Mai >>>> 2018 >>>> >> > 08:44: >>>> >> > Hi, >>>> >> > >>>> >> > could you please provide some more details? >>>> >> > The best would be a reproducible example. >>>> >> > >>>> >> > Thanks, >>>> >> > Alex >>>> >> > >>>> >> > >>>> >> > > On 9. May 2018, at 06:50, halit tiryaki <halit.tiryaki@gmail.com >>>> > >>>> >> > > wrote: >>>> >> > > >>>> >> > > Hello, >>>> >> > > >>>> >> > > got a serious problem here. any ideas? thanks >>>> >> > > >>>> >> > > Improper use? Potential bug? Your feedback is welcome: >>>> >> > > Contact: basex-talk@mailman.uni-konstanz.de >>>> >> > > Version: BaseX 9.0.2 beta >>>> >> > > Java: Oracle Corporation, 1.8.0_151 >>>> >> > > OS: Linux, amd64 >>>> >> > > Stack Trace: >>>> >> > > java.lang.ArrayIndexOutOfBoundsException: 4288 >>>> >> > > at org.basex.io.random.TableDiskAccess.read1(TableDiskAccess. >>>> java:151) >>>> >> > > at org.basex.data.Data.kind(Data.java:294) >>>> >> > > at org.basex.query.value.node.DBNode$4.next(DBNode.java:332) >>>> >> > > at org.basex.query.value.node.DBNode$4.next(DBNode.java:323) >>>> >> > > at org.basex.query.expr.path.IterStep$1.next(IterStep.java:38) >>>> >> > > at org.basex.query.expr.path.IterStep$1.next(IterStep.java:32) >>>> >> > > at org.basex.query.QueryContext.next(QueryContext.java:392) >>>> >> > > at org.basex.query.expr.path.IterPath$1.next(IterPath.java:50) >>>> >> > > at org.basex.query.expr.path.IterPath$1.next(IterPath.java:34) >>>> >> > > at org.basex.query.expr.ParseExpr.item(ParseExpr.java:58) >>>> >> > > at org.basex.query.expr.ParseExpr.atomItem(ParseExpr.java:84) >>>> >> > > at org.basex.query.func.fn.FnConcat.item(FnConcat.java:20) >>>> >> > > at org.basex.query.expr.ParseExpr.value(ParseExpr.java:71) >>>> >> > > at org.basex.query.expr.gflwor.Let$LetEval.next(Let.java:177) >>>> >> > > at org.basex.query.expr.gflwor.GFLWOR.value(GFLWOR.java:72) >>>> >> > > at org.basex.query.expr.path.CachedPath.nodeIter(CachedPath. >>>> java:36) >>>> >> > > at org.basex.query.expr.path.AxisPath.iter(AxisPath.java:69) >>>> >> > > at org.basex.query.expr.gflwor.For$1.next(For.java:107) >>>> >> > > at org.basex.query.expr.gflwor.OrderBy$1.sort(OrderBy.java:73) >>>> >> > > at org.basex.query.expr.gflwor.OrderBy$1.next(OrderBy.java:54) >>>> >> > > at org.basex.query.expr.gflwor.GFLWOR.value(GFLWOR.java:72) >>>> >> > > at org.basex.query.expr.gflwor.Let$LetEval.next(Let.java:177) >>>> >> > > at org.basex.query.expr.gflwor.GFLWOR$1.next(GFLWOR.java:87) >>>> >> > > at org.basex.query.QueryContext.next(QueryContext.java:392) >>>> >> > > at org.basex.query.scope.MainModule$1.next(MainModule.java:122) >>>> >> > > at org.basex.core.cmd.AQuery.query(AQuery.java:94) >>>> >> > > at org.basex.core.cmd.XQuery.run(XQuery.java:22) >>>> >> > > at org.basex.core.Command.run(Command.java:257) >>>> >> > > at org.basex.core.Command.execute(Command.java:93) >>>> >> > > at org.basex.server.ClientListener.run(ClientListener.java:140) >>>> >> > >>>> >> >>>> > >>>> >>> >>> >