Ah, good point. I’m running BaseX in Tomcat so I will have to dig into the config to see where Xmx flag is set. And what the memory relationship is between Tomcat and BaseX
Robert
From: Christian Grün <christian.gruen@gmail.com>
Date: Monday, August 19, 2024 at 9:50 AM
To: Chavez, Robert <r.chavez@northeastern.edu>
Cc: basex-talk@mailman.uni-konstanz.de <basex-talk@mailman.uni-konstanz.de>
Subject: Re: [basex-talk] Basex java.lang.ArrayIndexOutOfBoundsException
Hi Robert,
In fact out of memory errors are always critical in Java. Maybe you will need (well, would have needed) to adjust the memory that is used by the JVM
with the Xmx flag [1]. What value is currently assigned?
Best,
Christian
On Mon, Aug 19, 2024 at 3:37 PM Chavez, Robert <r.chavez@northeastern.edu>
wrote:
Thank you for the suggestion Christian.
In the logs everything seems to be working just fine until this one call – which returns the error “Out of Main Memory”. However, looking at the systems logs themselves, the system reports ample memory at that time. I’m not exactly sure when BaseX reported that error at that time.
After that initial “Out of Main Memory” error, all requests (PUT requests, simple GUI requests) fail with either:
java.lang.ArrayIndexOutOfBoundsException
-or
java.lang.NegativeArraySizeException
The only thing I can think of is that he system was reaching maximum harddisk capacity and maybe ran out of wiggle room at that point.
3:59:40.282 10.112.72.182:36826 basex-rest REQUEST [PUT] /basex/rest/psc/jqa/jqadiaries-v31-1819-11-p194.xml
13:59:43.889 10.112.72.182:36826 basex-rest 500 Out of Main Memory. 3607.46 ms
13:59:45.358 10.112.72.182:36828 basex-rest REQUEST [PUT] /basex/rest/psc/jqa/jqadiaries-v31-1819-09-p165.xml
13:59:45.746 10.112.72.182:36828 basex-rest 500 Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk@mailman.uni-konstanz.de Version: BaseX 10.4 Java: Red Hat, Inc., 11.0.16 OS: Linux, amd64 Stack Trace: java.lang.RuntimeException: Free slot exceeds file offset: 49357094248 + 110461 > 49357197312 at org.basex.util.Uti14:08:00.444 10.112.72.182:37070 basex-rest 500 Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk@mailman.uni-konstanz.de Version: BaseX 10.4 Java: Red Hat, Inc., 11.0.16 OS: Linux, amd64 Stack Trace: java.lang.ArrayIndexOutOfBoundsException 9.96 ms
14:08:05.549 10.112.72.182:37074 basex-rest REQUEST [PUT] /basex/rest/psc/jqa/jqadiaries-v51-1803-08-p001.xml
14:08:05.556 10.112.72.182:37074 basex-rest 500 Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk@mailman.uni-konstanz.de Version: BaseX 10.4 Java: Red Hat, Inc., 11.0.16 OS: Linux, amd64 Stack Trace: java.lang.NegativeArraySizeException: -1236248751 at org.basex.io.random.DataAccess.readBytes(DataAccess.java:196) at org.basex.io.random.DataAccess.readToken(DataAccess.java:175) at org.basex.io.random.DataAccess.readToken(DataAccess.java:166) at org.basex.data.DiskData.txt(DiskData.java:306) at org.basex.data.DiskData.text(DiskData.java:273) at org.basex.index.value.DiskValues.key(DiskValues.java:430) at org.basex.index.value.DiskValues.indexEntry(DiskValues.java:330) at org.basex.index.value.DiskValues.get(DiskValues.java:206) at org.basex.index.value.UpdatableDiskValues.delete(UpdatableDiskValues.java:98) at org.basex.data.Data.indexDelete(Data.java:1083) at org.basex.data.Data.delete(Data.java:683) at org.basex.query.up.atomic.Replace.apply(Replace.java:57) at org.basex.query.up.atomic.AtomicUpdateCache.ap... 7.06 ms
14:08:10.590 10.112.72.182:370
From: Christian Grün <christian.gruen@gmail.com>
Date: Friday, August 9, 2024 at 5:06 AM
To: Chavez, Robert <r.chavez@northeastern.edu>
Cc: basex-talk@mailman.uni-konstanz.de <basex-talk@mailman.uni-konstanz.de>
Subject: Re: [basex-talk] Basex java.lang.ArrayIndexOutOfBoundsExceptionHi Robert,
The size in the screenshot indicates that your input may have exceeded some limits for a single database instance [1]. Usually, an insert operation that would cause an overflow is rejected. Based on your database logs, can you trace back what was the last seemingly successful insert, and the first operation that raised the error message?
With regard to the database optimization, does it make a difference if you choose "Full optimization"?
Best,
Christian
On Wed, Aug 7, 2024 at 11:07 PM Chavez, Robert <r.chavez@northeastern.edu> wrote:
Greetings,
We are suddenly seeing the following error message when using the REST API and also when using to Web GUI when trying to list the contents of a database by clicking on the database name and also when clicking on “optimize” (screen shot of gui attached).
Does anyone have experience with hw to handle this?
Version: BaseX 10.4
Java: Amazon.com Inc., 17.0.12
OS: Linux, amd64
Thank you,
Robert
Unexpected error: Improper use? Potential bug? Your feedback is welcome:
Contact: basex-talk@mailman.uni-konstanz.de
Version: BaseX 10.4
Java: Amazon.com Inc., 17.0.12
OS: Linux, amd64
Stack Trace:
java.lang.ArrayIndexOutOfBoundsException: Index 2147483647 out of bounds for length 17322
at org.basex.io.random.TableDiskAccess.fpre(TableDiskAccess.java:514)
at org.basex.io.random.TableDiskAccess.cursor(TableDiskAccess.java:474)
at org.basex.io.random.TableDiskAccess.read5(TableDiskAccess.java:180)
at org.basex.data.Data.textRef(Data.java:467)
at org.basex.data.DiskData.text(DiskData.java:272)
at org.basex.query.func.db.DbListDetails$2.get(DbListDetails.java:96)
at org.basex.query.func.db.DbListDetails$2.get(DbListDetails.java:1)
at org.basex.query.func.fn.FnSubsequence.value(FnSubsequence.java:108)
at org.basex.query.expr.IterMap.value(IterMap.java:106)
at org.basex.query.expr.gflwor.Let$LetEval.next(Let.java:146)
at org.basex.query.expr.gflwor.GFLWOR$1.next(GFLWOR.java:78)
at org.basex.query.QueryContext.next(QueryContext.java:375)
at org.basex.query.expr.constr.Constr.add(Constr.java:73)
at org.basex.query.expr.constr.CElem.item(CElem.java:149)
at org.basex.query.expr.constr.CElem.item(CElem.java:1)
at org.basex.query.expr.ParseExpr.value(ParseExpr.java:51)
at org.basex.query.expr.ParseExpr.iter(ParseExpr.java:46)
at org.basex.query.expr.constr.Constr.add(Constr.java:72)
at org.basex.query.expr.constr.CElem.item(CElem.java:149)
at org.basex.query.expr.constr.CElem.item(CElem.java:1)
at org.basex.query.expr.ParseExpr.value(ParseExpr.java:51)
at org.basex.query.expr.ParseExpr.iter(ParseExpr.java:46)
at org.basex.query.expr.List$1.iter(List.java:213)
at org.basex.query.expr.List$1.next(List.java:181)
at org.basex.query.QueryContext.next(QueryContext.java:375)
at org.basex.query.expr.constr.Constr.add(Constr.java:73)
at org.basex.query.expr.constr.CElem.item(CElem.java:149)
at org.basex.query.expr.constr.CElem.item(CElem.java:1)
at org.basex.query.expr.ParseExpr.value(ParseExpr.java:51)
at org.basex.query.func.StaticFuncCall.evalArgs(StaticFuncCall.java:147)
at org.basex.query.func.FuncCall.value(FuncCall.java:53)
at org.basex.query.expr.ParseExpr.iter(ParseExpr.java:46)
at org.basex.query.expr.gflwor.GFLWOR$1.next(GFLWOR.java:82)
at org.basex.query.scope.MainModule$1.next(MainModule.java:55)
at org.basex.http.restxq.RestXqResponse.serialize(RestXqResponse.java:79)
at org.basex.http.web.WebResponse.create(WebResponse.java:58)
at org.basex.http.restxq.RestXqServlet.run(RestXqServlet.java:72)
at org.basex.http.BaseXServlet.service(BaseXServlet.java:69)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:733)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:541)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:690)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:373)
at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:868)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1589)
at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.base/java.lang.Thread.run(Thread.java:840)