September 2019 - BaseX-Talk - mailman.uni-konstanz.de

Options for ft:search
by Sebastian Zimmer 15 Mar '21

15 Mar '21

Hi, I'd like to request a feature concerning the ft:search method: An `lserror` option that works exactly like the global option `LSERROR`. It would be nice if we could set the maximum Levenshtein distance specifically for each fuzzy search without having to adjust the global option. Another question on ft:search: Is there a reason why it doesn't have a `case` option just like ft:contains has one? Best regards, Sebastian

2 3

Feature Request: Save untitled, open scripts when exiting GUI
by Andreas Mixich 30 Jul '20

30 Jul '20

SublimeText has this nifty feature, that it saves all open tabs, even if the have no representation on the filesystem, so one can continue exactly, where one has left off. I find myself often to have several, unnamed, XQueries open, to test this concept, to compose that function, etc. It would be nice, if one could simply shut off and have them reopened next time without naming a file and saving each item in question. -- Minden jót, all the best, Alles Gute, Andreas Mixich

2 6

9.2.1 not found in maven central
by andreas.hengsbach＠nexoma.de 07 May '20

07 May '20

Hi, is there a reason, that I can't find the current Version (9.2.1) in maven central? Thanks and best regards, Andreas

3 3

"Out of Memory" when inserting data from one DB to another
by BIRKNER Michael 30 Oct '19

30 Oct '19

Hi to all, I get an "Out of Memory" error (using the BaseX GUI on Ubuntu Linux) when I try to insert quite a lot of data into a BaseX database. The use case: I have a database (size is about 2600 MB, 13718400 nodes) with information in <record> elements that should be added to <record> elements in another database. The <record>s have a 1 to 1 connection identified by an ID that is available in both databases. An example (simplified) of the DB with the information I want to add to the other DB: <collection> <record> <id>1</id> <data>Some data</data> <data>More data</data> <data>More data</data> ... </record> <record> <id>2</id> <data>Some data</data> <data>More data</data> <data>More data</data> ... </record> <record> <id>3</id> <data>Some data</data> <data>More data</data> <data>More data</data> ... </record> ... many many more <record>s </collection> Here an example (simplified) of the DB to which the above <data> elements should be added: <collection> <record> <id>1</id> <mainData>Main data</mainData> <mainData>More main data</mainData> <mainData>More main data</mainData> ...  </record> <record> <id>2</id> <mainData>Main data</mainData> <mainData>More main data</mainData> <mainData>More main data</mainData> ...  </record> <record> <id>3</id> <mainData>Main data</mainData> <mainData>More main data</mainData> <mainData>More main data</mainData> ...  </record> ... many many more <record>s </collection> This is the XQuery I use to insert the given <data> elements from the <record> in one database to the corresponding <record> in the other database. It results in an "Out of Memory" error: let $infoRecs := db:open('db-with-data')/collection/record let $mainRecs := db:open('db-to-insert-data')/collection/record for $infoRec in $infoRecs let $id := data($infoRec/id) let $mainRec := $mainRecs[id=$id] let $dataToInsert := $infoRec/*[not(name()='id')] return insert node ($dataToInsert) into $mainRec I assume that the error is a result of the large amount of data that is processed. My question is if a strategy exists to work with such an amount of data without getting an "Out of Memory" error? Thanks very much to everyone in advance for any hint and advice. If you need more information about DB setup or options just let me know. Best regrads, Michael Dieses Mail ist ausschließlich für die Verwendung durch die/den darin genannten AdressatInnen bestimmt und kann vertrauliche bzw rechtlich geschützte Informationen enthalten, deren Verwendung ohne Genehmigung durch den/ die AbsenderIn rechtswidrig sein kann. Falls Sie dieses Mail irrtümlich erhalten haben, informieren Sie uns bitte und löschen Sie die Nachricht. UID: ATU 16209706 I https://wien.arbeiterkammer.at/datenschutz

2 2

basex OOM on 30GB database upon running /dba/db-optimize/
by first name last name 06 Oct '19

06 Oct '19

Hi, Let's say there's a 30GB dataset [3] containing most threads/posts from [1]. After importing all of it, when I try to run /dba/db-optimize/ on it (which must have some corresponding command) I get the OOM error in the stacktrace attached. I am using -Xmx2g so BaseX is limited to 2GB of memory (the machine I'm running this on doesn't have a lot of memory). I was looking at [2] for some estimates of peak memory usage for this "db-optimize" operation, but couldn't find any. Actually it would be nice to know peak memory usage because.. of course, for any database (including BaseX) a common operation is to do server sizing, to know what kind of server would be needed. In this case, it seems like 2GB memory is enough to import 340k documents, weighing in at 30GB total, but it's not enough to run "dba-optimize". Is there any info about peak memory usage on [2] ? And are there guidelines for large-scale collection imports like I'm trying to do? Thanks, Stefan [1] https://www.linuxquestions.org/ [2] http://docs.basex.org/wiki/Statistics [3] https://drive.google.com/open?id=1lTEGA4JqlhVf1JsMQbloNGC-tfNkeQt2

3 13

Re: [basex-talk] Connection timeout
by Marie Starck 01 Oct '19

01 Oct '19

I am using an ssl certificate and have a keystore on my production Basex instance. Are there extra steps needed to make an HTTPS request from a server to a basex server when those are in place? On Thu, Sep 26, 2019 at 6:14 PM Marie Starck <marie.compsci14(a)gmail.com> wrote: > Hey Christian, > > This error message is from doing it with the nodejs basex client. I get > the same connection time out when doing an http or https request on 8984 > and 8986 (non-ssl and ssl respectively). > > Thank you, > > Marie > > On Thu, Sep 26, 2019 at 5:54 PM Christian Grün <christian.gruen(a)gmail.com> > wrote: > >> Hi Marie, >> >> SOCKET ERROR: { Error: connect ETIMEDOUT 159.89.126.125:1984 >>> >> >> By default, the REST endpoints are available on port 8984. >> >> Maybe this helps, >> Christian >> >> >>

2 1

HTTPServer + gzip compression
by Stefan Koch 01 Oct '19

01 Oct '19

Hi, I'm struggling to get gzip compression working. I'm using the REST module. http://docs.basex.org/wiki/REST Version: BaseX 9.2.2 Tried to add the handler in jetty.xml as descriped here: https://www.eclipse.org/jetty/documentation/current/gzip-filter.html But it didn't work. Tried gzip filter via web.xml - but it is deprecated since jetty 9.3. Gzip handler is the correct way to do it. Searched the mailling list, similar problem was reported here: https://mailman.uni-konstanz.de/pipermail/basex-talk/2019-February/014160.h… Unfortunately no solution :( Not much experience with embedded jetty - but adding gzip compression is straight forward in tomcat or apache - no clue why it doesn't work.Shouldn't be that hard - and it's kinda a default feature. Did anybody get it to work? Any tipps? Thanks Stefan Koch B&IT Solutions Dipl.-Kfm. Rudolf Markus Petri Lietzenburger Str. 77 10719 Berlin Tel: 0049 30 8867 6099 Fax: 0049 30 8867 6159 Mail: koch(a)buit-solutions.com Web: www.buit-solutions.com

2 1

Database corrupted on updating with RestXQ
by Marco Lettere 26 Sep '19

26 Sep '19

Hi all, starting with 9.2.1 we experience a strange error with a RESTXQ API of ours that we have been using for years. The typical pattern is lookup a document update it and store it back. We have done this milions of time and also all the tests work neatly. But when using it from inside the web application, at around the tenth operation we get the DB corrupted with the stack trace [1]. This seems to happen only when the database is nearly empty and, for sure, it doesn't appear on 8.x series. It feels like some concurrency flaw is introduced somewhere but it's hard to say because the API is eight years in age and it never changed significantly. I know it's hard without an SSCE... but while working to insulate this byzantine behaviour (I've been trying for days now) maybe someone in the list has a clue or is able to interpret the stack trace? Thanks and have a nice weekend, Marco. [1] Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk(a)mailman.uni-konstanz.de Version: BaseX 9.3 beta Java: Ubuntu, 13 OS: Linux, amd64 Stack Trace: java.lang.ArrayIndexOutOfBoundsException: Index 1 out of bounds for length 1 at org.basex.io.random.TableDiskAccess.fpre(TableDiskAccess.java:507) at org.basex.io.random.TableDiskAccess.cursor(TableDiskAccess.java:468) at org.basex.io.random.TableDiskAccess.read1(TableDiskAccess.java:156) at org.basex.data.Data.kind(Data.java:304) at org.basex.query.value.node.DBNode.<init>(DBNode.java:50) at org.basex.query.value.seq.DBNodeSeq.get(DBNodeSeq.java:139) at org.basex.query.value.seq.DBNodeSeq.get(DBNodeSeq.java:164) at org.basex.query.func.db.DbOpen.value(DbOpen.java:19) at org.basex.query.func.StandardFunc.optimize(StandardFunc.java:82) at org.basex.query.expr.Arr.inline(Arr.java:69) at org.basex.query.expr.path.Path.inline(Path.java:962) at org.basex.query.expr.gflwor.ForLet.inline(ForLet.java:66) at org.basex.query.expr.gflwor.GFLWOR.inline(GFLWOR.java:804) at org.basex.query.expr.gflwor.GFLWOR.inlineLets(GFLWOR.java:339) at org.basex.query.expr.gflwor.GFLWOR.optimize(GFLWOR.java:106) at org.basex.query.expr.gflwor.GFLWOR.inline(GFLWOR.java:785) at org.basex.query.expr.Try.inline(Try.java:106) at org.basex.query.expr.gflwor.GFLWOR.inline(GFLWOR.java:816) at org.basex.query.expr.gflwor.GFLWOR.inlineLets(GFLWOR.java:339) at org.basex.query.expr.gflwor.GFLWOR.optimize(GFLWOR.java:106) at org.basex.query.func.StaticFunc.inline(StaticFunc.java:294) at org.basex.query.func.StaticFuncCall.compile(StaticFuncCall.java:67) at org.basex.query.scope.MainModule.comp(MainModule.java:81) at org.basex.query.QueryCompiler.compile(QueryCompiler.java:114) at org.basex.query.QueryCompiler.compile(QueryCompiler.java:105) at org.basex.query.QueryContext.compile(QueryContext.java:312) at org.basex.query.QueryContext.iter(QueryContext.java:331) at org.basex.http.restxq.RestXqResponse.serialize(RestXqResponse.java:73) at org.basex.http.web.WebResponse.create(WebResponse.java:63) at org.basex.http.restxq.RestXqServlet.run(RestXqServlet.java:53) at org.basex.http.BaseXServlet.service(BaseXServlet.java:65) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:865) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:542) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1700) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1345) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1667) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1247) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.Server.handle(Server.java:505) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:370) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:267) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103) at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:698) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:804) at java.base/java.lang.Thread.run(Thread.java:835)

2 11

Connection timeout
by Marie Starck 26 Sep '19

26 Sep '19

Been debugging this for the past couple of days and I am a bit stumped. I am doing a simple get REST request to my BaseX server from a node server and I keep getting a connection timeout error. I also tried the basex node client and I get the same error message. All of this works fine on localhost and what's even weirder is that I can access the BaseX rest endpoint through my browser just fine. Any ideas? I have SSL installed on my BaseX server. Could that be the issue? SOCKET ERROR: { Error: connect ETIMEDOUT 159.89.126.125:1984 at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1173:14) errno: 'ETIMEDOUT', code: 'ETIMEDOUT', syscall: 'connect', address: '159.89.126.125', port: 1984 } Thank you for any help you can give me, Marie

2 1

readable encryption result
by Andre Bovy 23 Sep '19

23 Sep '19

Dear all, I try to make the result of the encryption readable by converting the result either in Base64 or in Hex. Yet I am not able to convert it back to be able to decrypt. Could you help me ? André Bovy let $encrypted := crypto:encrypt('message', 'symmetric','keykeyke','DES’) let $base64 := string(convert:string-to-base64($encrypted)) let $encrypted := convert:binary-to-string(xs:base64Binary($base64)) (: generates invalid characters :) return crypto:decrypt($encrypted, 'symmetric','keykeyke','DES')

2 2