RuntimeException: Key should not exist
Hey there, I use basex in production for a couple of years now without problems. However on Saturday I got the error messages down below [1]. My storage architecture is that documents are saved throughout the day in a single database. For the next day a new database is created. When I got these errors and created a new database documents could be saved again. I think it has something to do with the index structures. The error messages were all similar but with different values after "key should not exist". Deployment method: docker BaseX 9.3.1 [Standalone] Try 'help' to get more information.
info
General Information: Version: 9.3.1 Used Memory: 10 MB Global options: AUTHMETHOD: Basic CACHETIMEOUT: 3600 DBPATH: /srv/basex/data DEBUG: false FAIRLOCK: false GZIP: false HOST: localhost HTTPLOCAL: false IGNORECERT: false IGNOREHOSTNAME: false KEEPALIVE: 600 LANG: English LANGKEYS: false LOG: true LOGMSGMAXLEN: 1000 LOGPATH: .logs NONPROXYHOSTS: PARALLEL: 8 PARSERESTXQ: 3 PASSWORD: PORT: 1984 PROXYHOST: PROXYPORT: 0 REPOPATH: /srv/basex/repo RESTPATH: RESTXQERRORS: true RESTXQPATH: SERVERHOST: SERVERPORT: 1984 STOPPORT: 8985 TIMEOUT: 30 USER: WEBPATH: /srv/basex/webapp Local options ADDARCHIVES: true ADDCACHE: false ADDRAW: false ARCHIVENAME: false ATTRINCLUDE: ATTRINDEX: true AUTOFLUSH: true AUTOOPTIMIZE: false BINDINGS: CASESENS: false CATFILE: CHECKSTRINGS: true CHOP: true COMPPLAN: true COPYNODE: true CREATEFILTER: *.xml CREATEONLY: false CSVPARSER: DEFAULTDB: false DIACRITICS: false DTD: false ENFORCEINDEX: false EXPORTER: FORCECREATE: false FTINCLUDE: FTINDEX: false FULLPLAN: false HTMLPARSER: INLINELIMIT: 100 INTPARSE: false JSONPARSER: LANGUAGE: en LSERROR: 0 MAINMEM: false MAXCATS: 100 MAXLEN: 96 MAXSTAT: 30 MIXUPDATES: false PARSER: xml QUERYINFO: false RUNQUERY: true RUNS: 1 SERIALIZE: true SERIALIZER: SKIPCORRUPT: false SPLITSIZE: 0 STEMMING: false STOPWORDS: STRIPNS: false TAILCALLS: 256 TEXTINCLUDE: TEXTINDEX: true TEXTPARSER: TOKENINCLUDE: TOKENINDEX: false UPDINDEX: false WITHDB: true WRITEBACK: false XINCLUDE: true XMLPLAN: false [1] Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk@mailman.uni-konstanz.de Version: BaseX 9.3.1 Java: IcedTea, 1.8.0_212 OS: Linux, amd64 Stack Trace: java.lang.RuntimeException: Key should not exist: 'urn:diasid:fty:kopal:0200507050000000000047' at org.basex.util.Util.notExpected(Util.java:61) at org.basex.index.value.UpdatableDiskValues.add(UpdatableDiskValues.java:77) at org.basex.data.Data.indexAdd(Data.java:1090) at org.basex.data.Data.insert(Data.java:828) at org.basex.query.up.atomic.Insert.apply(Insert.java:44) at org.basex.query.up.atomic.AtomicUpdateCache.applyUpdates(AtomicUpdateCache.java:294) at org.basex.query.up.atomic.AtomicUpdateCache.execute(AtomicUpdateCache.java:278) at org.basex.core.cmd.Replace.replace(Replace.java:106) at org.basex.core.cmd.Replace.access$000(Replace.java:20) at org.basex.core.cmd.Replace$1.run(Replace.java:58) at org.basex.core.cmd.ACreate.update(ACreate.java:90) at org.basex.core.cmd.Replace.run(Replace.java:55) at org.basex.core.Command.run(Command.java:257) at org.basex.http.rest.RESTCmd.run(RESTCmd.java:105) at org.basex.http.rest.RESTCmd.run(RESTCmd.java:93) at org.basex.http.rest.RESTExec.run0(RESTExec.java:31) at org.basex.http.rest.RESTCmd.run(RESTCmd.java:70) at org.basex.core.Command.run(Command.java:257) at org.basex.core.Command.execute(Command.java:93) at org.basex.core.Command.execute(Command.java:116) at org.basex.http.rest.RESTServlet.run(RESTServlet.java:31) at org.basex.http.BaseXServlet.service(BaseXServlet.java:65) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at org.eclipse.jetty.servlet.ServletHolder$NotAsyncServlet.service(ServletHolder.java:1391) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:760) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:547) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:590) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1607) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1297) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:485) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1577) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1212) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:500) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:547) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:270) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103) at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) at java.lang.Thread.run(Thread.java:748) Regards, Björn
Hi Björn, Thanks for the stack trace. You are right, it’s the updatable index structure that seemed to get corrupt; the actual database should not have been affected. I haven’t observed this before; and I bet it’s very hard to get this reproduced… If it happens again, please give us a note; we could then possibly have a look at the index files. Best, Christian On Mon, Apr 27, 2020 at 9:03 AM Braunschweig, Björn <bjoern.braunschweig@gwdg.de> wrote:
Hey there,
I use basex in production for a couple of years now without problems. However on Saturday I got the error messages down below [1].
My storage architecture is that documents are saved throughout the day in a single database. For the next day a new database is created. When I got these errors and created a new database documents could be saved again. I think it has something to do with the index structures. The error messages were all similar but with different values after „key should not exist“.
Deployment method: docker
BaseX 9.3.1 [Standalone]
Try 'help' to get more information.
info
General Information:
Version: 9.3.1
Used Memory: 10 MB
Global options:
AUTHMETHOD: Basic
CACHETIMEOUT: 3600
DBPATH: /srv/basex/data
DEBUG: false
FAIRLOCK: false
GZIP: false
HOST: localhost
HTTPLOCAL: false
IGNORECERT: false
IGNOREHOSTNAME: false
KEEPALIVE: 600
LANG: English
LANGKEYS: false
LOG: true
LOGMSGMAXLEN: 1000
LOGPATH: .logs
NONPROXYHOSTS:
PARALLEL: 8
PARSERESTXQ: 3
PASSWORD:
PORT: 1984
PROXYHOST:
PROXYPORT: 0
REPOPATH: /srv/basex/repo
RESTPATH:
RESTXQERRORS: true
RESTXQPATH:
SERVERHOST:
SERVERPORT: 1984
STOPPORT: 8985
TIMEOUT: 30
USER:
WEBPATH: /srv/basex/webapp
Local options
ADDARCHIVES: true
ADDCACHE: false
ADDRAW: false
ARCHIVENAME: false
ATTRINCLUDE:
ATTRINDEX: true
AUTOFLUSH: true
AUTOOPTIMIZE: false
BINDINGS:
CASESENS: false
CATFILE:
CHECKSTRINGS: true
CHOP: true
COMPPLAN: true
COPYNODE: true
CREATEFILTER: *.xml
CREATEONLY: false
CSVPARSER:
DEFAULTDB: false
DIACRITICS: false
DTD: false
ENFORCEINDEX: false
EXPORTER:
FORCECREATE: false
FTINCLUDE:
FTINDEX: false
FULLPLAN: false
HTMLPARSER:
INLINELIMIT: 100
INTPARSE: false
JSONPARSER:
LANGUAGE: en
LSERROR: 0
MAINMEM: false
MAXCATS: 100
MAXLEN: 96
MAXSTAT: 30
MIXUPDATES: false
PARSER: xml
QUERYINFO: false
RUNQUERY: true
RUNS: 1
SERIALIZE: true
SERIALIZER:
SKIPCORRUPT: false
SPLITSIZE: 0
STEMMING: false
STOPWORDS:
STRIPNS: false
TAILCALLS: 256
TEXTINCLUDE:
TEXTINDEX: true
TEXTPARSER:
TOKENINCLUDE:
TOKENINDEX: false
UPDINDEX: false
WITHDB: true
WRITEBACK: false
XINCLUDE: true
XMLPLAN: false
[1] Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk@mailman.uni-konstanz.de Version: BaseX 9.3.1 Java: IcedTea, 1.8.0_212 OS: Linux, amd64 Stack Trace: java.lang.RuntimeException: Key should not exist: 'urn:diasid:fty:kopal:0200507050000000000047' at org.basex.util.Util.notExpected(Util.java:61) at org.basex.index.value.UpdatableDiskValues.add(UpdatableDiskValues.java:77) at org.basex.data.Data.indexAdd(Data.java:1090) at org.basex.data.Data.insert(Data.java:828) at org.basex.query.up.atomic.Insert.apply(Insert.java:44) at org.basex.query.up.atomic.AtomicUpdateCache.applyUpdates(AtomicUpdateCache.java:294) at org.basex.query.up.atomic.AtomicUpdateCache.execute(AtomicUpdateCache.java:278) at org.basex.core.cmd.Replace.replace(Replace.java:106) at org.basex.core.cmd.Replace.access$000(Replace.java:20) at org.basex.core.cmd.Replace$1.run(Replace.java:58) at org.basex.core.cmd.ACreate.update(ACreate.java:90) at org.basex.core.cmd.Replace.run(Replace.java:55) at org.basex.core.Command.run(Command.java:257) at org.basex.http.rest.RESTCmd.run(RESTCmd.java:105) at org.basex.http.rest.RESTCmd.run(RESTCmd.java:93) at org.basex.http.rest.RESTExec.run0(RESTExec.java:31) at org.basex.http.rest.RESTCmd.run(RESTCmd.java:70) at org.basex.core.Command.run(Command.java:257) at org.basex.core.Command.execute(Command.java:93) at org.basex.core.Command.execute(Command.java:116) at org.basex.http.rest.RESTServlet.run(RESTServlet.java:31) at org.basex.http.BaseXServlet.service(BaseXServlet.java:65) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at org.eclipse.jetty.servlet.ServletHolder$NotAsyncServlet.service(ServletHolder.java:1391) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:760) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:547) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:590) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1607) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1297) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:485) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1577) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1212) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:500) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:547) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:270) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103) at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) at java.lang.Thread.run(Thread.java:748)
Regards,
Björn
Dear Christian, Further investigation revealed that a server crash was involved. I do feel relieved that my db should not have been affected. Do you recommend to somehow drop and recreate the indices? Thanks, Björn -----Ursprüngliche Nachricht----- Von: Christian Grün [mailto:christian.gruen@gmail.com] Gesendet: Dienstag, 28. April 2020 11:24 An: Braunschweig, Björn <bjoern.braunschweig@gwdg.de> Cc: basex-talk@mailman.uni-konstanz.de Betreff: Re: [basex-talk] RuntimeException: Key should not exist Hi Björn, Thanks for the stack trace. You are right, it’s the updatable index structure that seemed to get corrupt; the actual database should not have been affected. I haven’t observed this before; and I bet it’s very hard to get this reproduced… If it happens again, please give us a note; we could then possibly have a look at the index files. Best, Christian On Mon, Apr 27, 2020 at 9:03 AM Braunschweig, Björn <bjoern.braunschweig@gwdg.de> wrote:
Hey there,
I use basex in production for a couple of years now without problems. However on Saturday I got the error messages down below [1].
My storage architecture is that documents are saved throughout the day in a single database. For the next day a new database is created. When I got these errors and created a new database documents could be saved again. I think it has something to do with the index structures. The error messages were all similar but with different values after „key should not exist“.
Deployment method: docker
BaseX 9.3.1 [Standalone]
Try 'help' to get more information.
info
General Information:
Version: 9.3.1
Used Memory: 10 MB
Global options:
AUTHMETHOD: Basic
CACHETIMEOUT: 3600
DBPATH: /srv/basex/data
DEBUG: false
FAIRLOCK: false
GZIP: false
HOST: localhost
HTTPLOCAL: false
IGNORECERT: false
IGNOREHOSTNAME: false
KEEPALIVE: 600
LANG: English
LANGKEYS: false
LOG: true
LOGMSGMAXLEN: 1000
LOGPATH: .logs
NONPROXYHOSTS:
PARALLEL: 8
PARSERESTXQ: 3
PASSWORD:
PORT: 1984
PROXYHOST:
PROXYPORT: 0
REPOPATH: /srv/basex/repo
RESTPATH:
RESTXQERRORS: true
RESTXQPATH:
SERVERHOST:
SERVERPORT: 1984
STOPPORT: 8985
TIMEOUT: 30
USER:
WEBPATH: /srv/basex/webapp
Local options
ADDARCHIVES: true
ADDCACHE: false
ADDRAW: false
ARCHIVENAME: false
ATTRINCLUDE:
ATTRINDEX: true
AUTOFLUSH: true
AUTOOPTIMIZE: false
BINDINGS:
CASESENS: false
CATFILE:
CHECKSTRINGS: true
CHOP: true
COMPPLAN: true
COPYNODE: true
CREATEFILTER: *.xml
CREATEONLY: false
CSVPARSER:
DEFAULTDB: false
DIACRITICS: false
DTD: false
ENFORCEINDEX: false
EXPORTER:
FORCECREATE: false
FTINCLUDE:
FTINDEX: false
FULLPLAN: false
HTMLPARSER:
INLINELIMIT: 100
INTPARSE: false
JSONPARSER:
LANGUAGE: en
LSERROR: 0
MAINMEM: false
MAXCATS: 100
MAXLEN: 96
MAXSTAT: 30
MIXUPDATES: false
PARSER: xml
QUERYINFO: false
RUNQUERY: true
RUNS: 1
SERIALIZE: true
SERIALIZER:
SKIPCORRUPT: false
SPLITSIZE: 0
STEMMING: false
STOPWORDS:
STRIPNS: false
TAILCALLS: 256
TEXTINCLUDE:
TEXTINDEX: true
TEXTPARSER:
TOKENINCLUDE:
TOKENINDEX: false
UPDINDEX: false
WITHDB: true
WRITEBACK: false
XINCLUDE: true
XMLPLAN: false
[1] Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk@mailman.uni-konstanz.de Version: BaseX 9.3.1 Java: IcedTea, 1.8.0_212 OS: Linux, amd64 Stack Trace: java.lang.RuntimeException: Key should not exist: 'urn:diasid:fty:kopal:0200507050000000000047' at org.basex.util.Util.notExpected(Util.java:61) at org.basex.index.value.UpdatableDiskValues.add(UpdatableDiskValues.java :77) at org.basex.data.Data.indexAdd(Data.java:1090) at org.basex.data.Data.insert(Data.java:828) at org.basex.query.up.atomic.Insert.apply(Insert.java:44) at org.basex.query.up.atomic.AtomicUpdateCache.applyUpdates(AtomicUpdateC ache.java:294) at org.basex.query.up.atomic.AtomicUpdateCache.execute(AtomicUpdateCache. java:278) at org.basex.core.cmd.Replace.replace(Replace.java:106) at org.basex.core.cmd.Replace.access$000(Replace.java:20) at org.basex.core.cmd.Replace$1.run(Replace.java:58) at org.basex.core.cmd.ACreate.update(ACreate.java:90) at org.basex.core.cmd.Replace.run(Replace.java:55) at org.basex.core.Command.run(Command.java:257) at org.basex.http.rest.RESTCmd.run(RESTCmd.java:105) at org.basex.http.rest.RESTCmd.run(RESTCmd.java:93) at org.basex.http.rest.RESTExec.run0(RESTExec.java:31) at org.basex.http.rest.RESTCmd.run(RESTCmd.java:70) at org.basex.core.Command.run(Command.java:257) at org.basex.core.Command.execute(Command.java:93) at org.basex.core.Command.execute(Command.java:116) at org.basex.http.rest.RESTServlet.run(RESTServlet.java:31) at org.basex.http.BaseXServlet.service(BaseXServlet.java:65) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at org.eclipse.jetty.servlet.ServletHolder$NotAsyncServlet.service(Servle tHolder.java:1391) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:760) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java: 547) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.ja va:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java :590) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper. java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandle r.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandle r.java:1607) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandle r.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandle r.java:1297) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler .java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:4 85) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler .java:1577) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler .java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler .java:1212) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.ja va:141) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper. java:127) at org.eclipse.jetty.server.Server.handle(Server.java:500) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java: 383) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:547) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java :270) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(Abstrac tConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103) at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool .java:806) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThread Pool.java:938) at java.lang.Thread.run(Thread.java:748)
Regards,
Björn
participants (2)
-
Braunschweig, Björn -
Christian Grün