Hi,
we experience some problems after the upgrade to version 12 with the fulltext search.
We are using xquery:fork-join() to run the fulltext search on multiple databases at the same time (see attachment). This worked fine with version 11 but has strange effects after the upgrade to version 12.
With BaseX 11 the log output is like this:
09:03:51.054 [0:0:0:0:0:0:0:1]:55079 admin REQUEST [POST] /dba/query?file=ftsearch.xq 09:03:51.203 SERVER admin TRACE local:search-db(ALL) 100.4 ms 09:03:51.204 SERVER admin TRACE local:search-db(ALL) 1721 09:03:51.243 SERVER admin TRACE local:matches() 38.84 ms 09:03:51.270 SERVER admin TRACE local:search-db(MD) 166.69 ms 09:03:51.270 SERVER admin TRACE local:search-db(MD) 1883 09:03:51.282 SERVER admin TRACE local:matches() 11.9 ms ... 09:03:52.284 SERVER admin TRACE local:search-db(XP) 307.84 ms 09:03:52.284 SERVER admin TRACE local:search-db(XP) 74128 09:03:52.525 SERVER admin TRACE local:matches() 241.64 ms 09:03:52.796 SERVER admin TRACE local:matches() 581.56 ms 09:03:52.796 SERVER admin TRACE local:search-distinct() search "test" 1710.79 ms 09:03:53.200 [0:0:0:0:0:0:0:1]:55079 admin 200 2160.7 ms
After about 2 seconds the search is finished.
With BaseX 12 two things might happen.
ConcurrentModificationException
This happens almost immediatelly.
500 Unexpected error: Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk@mailman.uni-konstanz.de Version: BaseX 12.0 Java: Eclipse Adoptium, 21 OS: Windows 10, amd64 Stack Trace: java.util.ConcurrentModificationException at java.base/java.util.ArrayList$Itr.checkForComodification(ArrayList.java:1095) at java.base/java.util.ArrayList$Itr.next(ArrayList.java:1049) at org.basex.query.QueryResources.close(QueryResources.java:94) at org.basex.query.QueryContext.close(QueryContext.java:543) at org.basex.http.restxq.RestXqResponse.finish(RestXqResponse.java:152) at org.basex.http.web.WebResponse.create(WebResponse.java:63) at org.basex.http.restxq.RestXqServlet.run(RestXqServlet.java:70) at org.basex.http.BaseXServlet.service(BaseXServlet.java:68) at jakarta.servlet.http.HttpServlet.service(HttpServlet.java:587) at org.eclipse.jetty.ee9.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1397) at org.eclipse.jetty.ee9.servlet.ServletHolder.handle(ServletHolder.java:765) at org.eclipse.jetty.ee9.servlet.ServletHandler.doHandle(ServletHandler.java:528) at org.eclipse.jetty.ee9.nested.ScopedHandler.handle(ScopedHandler.java:127) at org.eclipse.jetty.ee9.security.SecurityHandler.handle(SecurityHandler.java:603) at org.eclipse.jetty.ee9.nested.HandlerWrapper.handle(HandlerWrapper.java:124) at org.eclipse.jetty.ee9.nested.ScopedHandler.nextHandle(ScopedHandler.java:197) at org.eclipse.jetty.ee9.nested.SessionHandler.doHandle(SessionHandler.java:612) at org.eclipse.jetty.ee9.nested.ScopedHandler.nextHandle(ScopedHandler.java:195) at org.eclipse.jetty.ee9.nested.ContextHandler.doHandle(ContextHandler.java:1045) at org.eclipse.jetty.ee9.nested.ScopedHandler.nextScope(ScopedHandler.java:164) at org.eclipse.jetty.ee9.servlet.ServletHandler.doScope(ServletHandler.java:483) at org.eclipse.jetty.ee9.nested.ScopedHandler.nextScope(ScopedHandler.java:162) at org.eclipse.jetty.ee9.nested.SessionHandler.doScope(SessionHandler.java:589) at org.eclipse.jetty.ee9.nested.ScopedHandler.nextScope(ScopedHandler.java:162) at org.eclipse.jetty.ee9.nested.ContextHandler.doScope(ContextHandler.java:966) at org.eclipse.jetty.ee9.nested.ScopedHandler.handle(ScopedHandler.java:125) at org.eclipse.jetty.ee9.nested.ContextHandler.handle(ContextHandler.java:1719) at org.eclipse.jetty.ee9.nested.HttpChannel$RequestDispatchable.dispatch(HttpChannel.java:1565) at org.eclipse.jetty.ee9.nested.HttpChannel.dispatch(HttpChannel.java:724) at org.eclipse.jetty.ee9.nested.HttpChannel.handle(HttpChannel.java:512) at org.eclipse.jetty.ee9.nested.ContextHandler$CoreContextHandler$CoreToNestedHandler.handle(ContextHandler.java:3029) at org.eclipse.jetty.server.handler.ContextHandler.handle(ContextHandler.java:1071) at org.eclipse.jetty.server.Server.handle(Server.java:182) at org.eclipse.jetty.server.internal.HttpChannelState$HandlerInvoker.run(HttpChannelState.java:678) at org.eclipse.jetty.server.internal.HttpConnection.onFillable(HttpConnection.java:416) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:322) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:99) at org.eclipse.jetty.io.SelectableChannelEndPoint$1.run(SelectableChannelEndPoint.java:53) at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.runTask(AdaptiveExecutionStrategy.java:480) at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.consumeTask(AdaptiveExecutionStrategy.java:443) at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:293) at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.run(AdaptiveExecutionStrategy.java:201) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:311) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:981) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.doRunJob(QueuedThreadPool.java:1211) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1166) at java.base/java.lang.Thread.run(Thread.java:1583) 189.44 ms
Full text search never finishes
09:00:12.422 [0:0:0:0:0:0:0:1]:55041 admin REQUEST [POST] /dba/query?file=ftsearch.xq 09:00:12.633 SERVER admin TRACE local:search-db(ALL) 115.46 ms 09:00:12.635 SERVER admin TRACE local:search-db(ALL) 1721 09:00:12.672 SERVER admin TRACE local:matches() 37.09 ms 09:00:12.711 SERVER admin TRACE local:search-db(MD) 194.24 ms 09:00:12.711 SERVER admin TRACE local:search-db(MD) 1883 09:00:12.725 SERVER admin TRACE local:matches() 13.66 ms 09:00:12.851 SERVER admin TRACE local:search-db(LS) 334.48 ms 09:00:12.851 SERVER admin TRACE local:search-db(LS) 169524 09:00:12.923 SERVER admin TRACE local:search-db(CT) 406.58 ms 09:00:12.923 SERVER admin TRACE local:search-db(CT) 80086 ... 09:00:13.483 SERVER admin TRACE local:search-db(US) 47012 09:00:13.553 SERVER admin TRACE local:search-db(POC) 828.06 ms 09:00:13.554 SERVER admin TRACE local:search-db(POC) 10147 09:00:13.693 SERVER admin TRACE local:search-db(MR) 1020.89 ms 09:00:13.693 SERVER admin TRACE local:search-db(MR) 223155
The process "hangs" and never finishes and CPU is running on full load. The current database where it hangs is always a different one.
Does this have anything to do with the Jetty update? It seems to be related to the multithreading.
Best Regards Johannes