- BaseX-Talk - mailman.uni-konstanz.de

Fuzzy full-text query
by Tim Thompson 06 Jul '21

06 Jul '21

Hello, I am trying to match a local dataset against entries in a full-text index. I ran into the following case and was confused about the fuzzy querying performance: <x>Baxter, Richard, 1615-1691</x>[text() contains text "Baxter, Richard, 615-1691" using fuzzy] The BaseX documentation says that the default fuzzy search will preserve a minimum of 1 error. However, the query above returns no result. If I add the "errors" parameter with a value of 1, it does work: <x>Baxter, Richard, 1615-1691</x>[text() contains text "Baxter, Richard, 615-1691" using fuzzy 1 errors] Is this expected behavior? Thanks in advance, Tim -- Tim A. Thompson Librarian for Applied Metadata Research Yale University Library

2 1

automatically running an unknown number of queries on a variable content set
by Graydon Saunders 06 Jul '21

06 Jul '21

Hello! So the idea is to -- via some automatic process, so "the command line" -- load a content set into BaseX and then run some arbitrary list of queries on that content set, returning an aggregated report. It's important that it's easy to add a query; we expect to be adding queries as we find new ways for the content to be inappropriate. (We're guaranteed it's valid; we are not guaranteed it's sensible, in the "and just how do you expect to publish forty seven levels of nested lists?" sense.) My first thought is to have a directory of individual queries and have the main query walk through them with xquery:eval(), but there might be a better way. Anyone got any suggestions? Thanks! Graydon

3 2

Increase automatic logout time
by Josselin Morvan 02 Jul '21

02 Jul '21

Hi everyone, We are using BaseX for the development of our web application and we are having issues with automatic logouts. We have developed forms with Xforms that allow us to process historical documents and we use Basex to manage our data directly in XML. Recently we put our application online to be able to share our data and work with the whole team more easily. For the management of the users and the permissions, we relied on the BaseX doc (https://docs.basex.org/wiki/Permissions <https://docs.basex.org/wiki/Permissions>). Globally everything works fine, except that we encounter a kind of automatic logout of the users maybe about 1 hour after their login, maybe even less, it’s hard to say… In fact, our forms are quite big and it often takes more than an hour to finish processing a single historical resource, and when the user save his work, if he is logged out, he loses all what he’ve done. Would you know a way to increase this logout time? On our local BaseX instance, we tried changing the database options, including Keepalive (https://docs.basex.org/wiki/Options#KEEPALIVE) We changed the value 600 to 60, just to make sure it was this option that needed to be changed, but it didn't seem to have any effect… For the moment we set up an automatic backup system, but this solution introduces other issues and is therefore not ideal. Thank you in advance for your help!! Take good care, Josselin.

3 2

Copy/modify, Update, or file:append - help appreciated
by Bridger Dyson-Smith 02 Jul '21

02 Jul '21

Hi all - I'm trying to keep track of some HTTP POST results in a file, something akin to the following: file:write-text( "/tmp/bridger-test.csv", "book" || "," || "title" || "," || "sip" || out:nl() ), for $i in 1 to 20 (: some http:send-request stuff :) return( prof:sleep(250), if ($request[1]//@status/data() = 201) then file:append( "/tmp/bridger-test.csv", "book: " $i || "," || "title: " $i || "," || "" ) else "something went wrong with the request; the response was: " || $request[1]//@status/data() I have something that kinda works, but I'd like it to be better XQuery, avoid the two step create-file-then-append, and leverage more of built-in awesomeness in the language. I'm sure that I need to have a better understanding of the Update syntax, and/or %updating annotation, but would someone be willing to give some hints and/or suggestions for a better way to handle situations like this? Thanks in advance for your time and trouble. Best, Bridger

2 1

Command Files - string syntax
by Jonathan Robie 29 Jun '21

29 Jun '21

I decided to try creating some command files to simplify administration: https://docs.basex.org/wiki/Commands#String_Syntax I was surprised that this did not work: % cat foo.bsx CREATE DB test ADD TO embedded.xml <root>embedded</root> # run query XQUERY <hits>{ count(//text()) }</hits> CLOSE % basex foo.bsx Stopped at /.../foo.bsx, 1/9: [XPST0003] Unexpected end of query: 'DB test...'. This works fine: % basex <foo.bsx BaseX 9.1 [Standalone] Try 'help' to get more information. > CREATE DB test Database 'test' created in 196.04 ms. > ADD TO embedded.xml <root>embedded</root> Resource(s) added in 101.36 ms. > # run query > XQUERY <hits>{ count(//text()) }</hits> <hits>1</hits> Query executed in 393.09 ms. > CLOSE Database 'test' was closed. What am I missing? Jonathan

4 7

Sed-like CLI tool
by Iwan Briquemont 23 Jun '21

23 Jun '21

Hi, BaseX works very well to do some quick queries or updates over files (rather than using it as a DB). I usually use the GUI for that but I think it could be quicker to have a simple CLI tool for the terminal like sed to do things like: Have a fancy XPath tool: (-e is -q in basex) echo "<root><ele>1</ele><ele>2</ele></root>" | basexsed -e 'root/ele' 1 2 Work over multiple documents quickly (somewhat equivalent to "for $path in $args let $doc := doc($path) return xquery:eval($expr, map { '': $doc}))) basexsed -e 'some/xquery' xmldocuments/*.xml # Output the result Allow multiple documents on stdin too, so you can chain expressions: echo "<doc1/><doc2/><doc1/>' | basexsed -e 'doc1' | basexsed -e 'doc1/x' Do updating expressions, by default outputting the changes: basexsed -e 'delete node //n' xmldocuments/*.xml # Outputs the docs without the n nodes (-i is -u in basex) basexsed -i -e 'delete node //n' xmldocuments/*.xml # Changes the files in place Automatically handle other formats like json, csv, html: basexsed -e 'map:keys(.)' jsondocuments/*.json # Outputs as json as well AFAIK everything is already there in the basex command, so the main change would be to support multiple files/documents in a streaming fashion, and then a few smaller things like using stdin as the context- item if no file is given and allowing the same arguments as in standard tools like sed. To make it nicer you could also add syntax highlighting to the console, some shortcuts for longer XQuery parameters like serialization indentation. This would provide an alternative to tools like Xidel ( https://www.videlibri.de/xidel.html ) or jq ( https://stedolan.github.io/jq/ ). Is this something you would be interested in implementing? Or have some people already done something similar? Regards, Iwan

3 4

Re: [basex-talk] Package for system integration
by Christian Grün 23 Jun '21

23 Jun '21

Hi Karlo, > Where does basex set the location of the configuration? If you point me > to the file, I might try changing it. By default, BaseX creates and updates the configuration files in the project’s "Home Directory". This directory is determined at startup time as described in the Wiki [1]. If you download and start the JAR file on a completely fresh system, the home directory will be the same as the user home directory. Hope this helps, Christian [1] https://docs.basex.org/wiki/Configuration#Home_Directory On Tue, Jun 22, 2021 at 6:37 AM <karlo(a)fastmail.com> wrote: > > Liam R. E. Quin writes: > > The things that are binary-only, if they have a compatible licence, > > could either be included or added in a separate package. > > Indeed, when registering it with a package repository, we would replace > those binary libaries with dependent packages from the repository. > > Liam R. E. Quin writes: > > Yes, it should look in $HOME for a conf file. > > Even better, ${XDG_CONFIG_HOME:-${HOME}/.config} > > Where does basex set the location of the configuration? If you point me > to the file, I might try changing it.

1 0

Re: [basex-talk] Package for system integration
by Christian Grün 22 Jun '21

22 Jun '21

Dear Kralj, That’s a great contribution; I really appreciate your efforts. A little while ago, Bridger asked users on the list to share their experiences installing and running BaseX. It would be great if those of you who are using Linux and UNIX-based systems could have a look at Kralj’s package and tell us about your observations! Thanks all, best regards, Christian On Sun, Jun 20, 2021 at 8:36 AM <karlo(a)fastmail.com> wrote: > > Dear Christian, > > With your help I have prepared a new version with a similar user > interface but that also builds BaseX. The released binaries run > perfectly of course, so the role of the source build package is > to demonstrate that the binaries are built from a free source. > > Because the package follows certain conventions, we can quickly prepare > BaseX for many different operating systems and maintain BaseX packages > for several package repositories. > > I explain the method in more detail below. For those who are just > joining the discussion, I include some background before continuing > the discussion about this package. > > I have the honor to be Your Majesty's humble and obedient servant. > > Kralj Karlo > > > > INTRODUCTION > > BaseX has a bespoke procedure initiating builds. If the build could > instead be initiated through execution of "make" in a directory > with the conventional inputs, it would be fast to port to the many > different operating systems whose packaging systems follow this > convention. > > I propose a procedure to create BaseX source packages. The resulting > BaseX source packages contain a Makefile with instructions for building > and installing from source. > > I create the source package as a directory containing the basex source code, > the basex-dist source code, a version-specific Makefile, and other files > to support the build. > > I distribute the source package, and someone else can build it. A build > of the source package produces a tree that can be installed by unpacking > to the specified PREFIX, /usr/local by default. > > To build it within the framework of a package manager, you would > configure the package manager to retrieve the source package and, > possibly, set a different PREFIX, set the dependencies, and copy > other package metadata (for example, homepage and description). > The package manager will likely already have tooling around the "make" > and "make install" commands, so you should not really need to understand > the build process. > > > > PROCEDURE > > Copy the program for creating the source package. > https://klarinet.kos.ovh/fossil/basex-make/tarball/basex-make.tar.gz > > You can alternatively use fossil, for example: > > fossil clone https://klarinet.kos.ovh/fossil/basex-make ../basex-make.fossil > fossil open ../basex-make.fossil > > I run "make update prune all tgz" from the basex-make directory > to do the following. > > * Download the latest sources and copy them to ./basex-src > * Generate the file ./basex-src/Makefile for building the sources. > * Archive the directory basex-src as basex-src.tar.gz > > Now I can send the basex-src.tar.gz to someone else. Here it is. > https://klarinet.kos.ovh/fossil/basex-make/uv/basex-src.tar.gz > > Then you can extract basex-src.tar.gz and run "make && make install" > to build and install. That does the following. > > * Build BaseX by the conventional process. > * Unpack the zip file created by the conventional BaseX build. > * Replace the conventional executibles. The new executables > handle the PREFIX and are written in sh rather than bash. > > > > BUGS > > * With this installation method BaseX looks for the configuration file > $PREFIX/lib/BaseX$VERSION/.basex and creates it if it does not exist. > It would be good to change that to look for $HOME/.basex instead. > * Certain vendor libraries are still distributed as binaries > in basex/basex-core/lib. I plan to remove these so we can truly call > it a "source" package. > * basex-src.tar.gz should instead be named BaseX$VERSION.tar.gz, and > it should contain a directory named BaseX$VERSION instead of basex-src. > > NOT BUGS > > * The minor complexity of this process is unfortunate. It would be nice > to modify release.pl so there are fewer layers, but this could break other > things that depend on the present build process. > > APPLICATION > > I propose that a tarball like that created by the present procedure be > distributed on the BaseX download webpage (https://basex.org/download/) > and that it be addressed as the "Source Package" with support for all > Unix-like operating systems. I believe it will in fact work on Windows > with Cygwin, but I don't care to support that. > > Once we have begin releasing source packages, we proceed to add BaseX > to various package repositories. I think we would start with Debian apt, > because we have already done one of those, and OpenBSD ports, because > that's what I use.

3 2

ft:normalize: potential bug?
by Alexander Witzigmann 21 Jun '21

21 Jun '21

hi, if you call ft:normalize using a string containing "*" ft:normalize("a*",map { 'stemming': 'true', 'language':'de' }) this results in Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk(a)mailman.uni-konstanz.de Version: BaseX 9.5.2 Java: Oracle Corporation, 13.0.2 OS: Windows 10, amd64 Stack Trace: java.lang.ArrayIndexOutOfBoundsException: Index -1 out of bounds for length 8 at org.basex.util.TokenBuilder.get(TokenBuilder.java:191) at org.basex.util.ft.GermanStemmer.resub(GermanStemmer.java:150) at org.basex.util.ft.GermanStemmer.stem(GermanStemmer.java:42) at org.basex.util.ft.Stemmer.nextToken(Stemmer.java:99) at org.basex.util.ft.FTLexer.nextToken(FTLexer.java:156) at org.basex.query.func.ft.FtTokenize.tokens(FtTokenize.java:49) at org.basex.query.func.ft.FtNormalize.item(FtNormalize.java:23) .... anything I missed? Alex

2 2

Error message, can this be fixed?
by Roland Haarbrink 21 Jun '21

21 Jun '21

Hi, Getting this message: org.basex.core.BaseXException: Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk(a)mailman.uni-konstanz.de Version: BaseX 7.9 Java: Oracle Corporation, 1.8.0_161 OS: Linux, amd64 Stack Trace: java.lang.ArrayIndexOutOfBoundsException When using this validator: https://validatie.geostandaarden.nl/etf-webapp/testruns/start-direct Can this be fixed? Thanks URL: http://localhost:8080/etf-webapp/testruns/start-direct Exception id: EXID-06e97066-fe8f-43cd-adc5-df641b843f75 de.interactive_instruments.etf.dal.basex.dao.BasexDataSource.create(BasexDat aSource.java:397) de.interactive_instruments.etf.dal.basex.dao.BasexDataSource.create(BasexDat aSource.java:381) de.interactive_instruments.etf.dal.basex.dao.BasexTestObjectDao.doCreate(Bas exTestObjectDao.java:134) de.interactive_instruments.etf.dal.dao.AbstractTestObjectDao.create(Abstract TestObjectDao.java:89) de.interactive_instruments.etf.webapp.controller.TestObjectController.addFil eTestData(TestObjectController.java:366) de.interactive_instruments.etf.webapp.controller.TestRunController.startDire ct(TestRunController.java:238) sun.reflect.GeneratedMethodAccessor245.invoke(Unknown Source) sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl .java:43) java.lang.reflect.Method.invoke(Method.java:498) org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(Invoc ableHandlerMethod.java:221) org.springframework.web.method.support.InvocableHandlerMethod.invokeForReque st(InvocableHandlerMethod.java:136) org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandle rMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:110) org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerA dapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:817) org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerA dapter.handleInternal(RequestMappingHandlerAdapter.java:731) org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.hand le(AbstractHandlerMethodAdapter.java:85) org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServl et.java:959) org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServle t.java:893) org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkSer vlet.java:968) org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.jav a:870) javax.servlet.http.HttpServlet.service(HttpServlet.java:648) org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.ja va:844) javax.servlet.http.HttpServlet.service(HttpServlet.java:729) org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Application FilterChain.java:292) org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterCh ain.java:207) org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Application FilterChain.java:240) org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterCh ain.java:207) org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.ja va:212) org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.ja va:106) org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase .java:502) org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:141 ) org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79) org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogVa lve.java:616) org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java :88) org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:522) org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Proce ssor.java:1095) org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(Abstrac tProtocol.java:672) org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.jav a:1500) org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java: 1456) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:11 49) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:6 24) org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.ja va:61) java.lang.Thread.run(Thread.java:748)

2 1