BaseX-Talk

basex-talk@mailman.uni-konstanz.de

12 participants
5242 discussions

Sed-like CLI tool
by Iwan Briquemont 23 Jun '21

23 Jun '21

Hi, BaseX works very well to do some quick queries or updates over files (rather than using it as a DB). I usually use the GUI for that but I think it could be quicker to have a simple CLI tool for the terminal like sed to do things like: Have a fancy XPath tool: (-e is -q in basex) echo "<root><ele>1</ele><ele>2</ele></root>" | basexsed -e 'root/ele' 1 2 Work over multiple documents quickly (somewhat equivalent to "for $path in $args let $doc := doc($path) return xquery:eval($expr, map { '': $doc}))) basexsed -e 'some/xquery' xmldocuments/*.xml # Output the result Allow multiple documents on stdin too, so you can chain expressions: echo "<doc1/><doc2/><doc1/>' | basexsed -e 'doc1' | basexsed -e 'doc1/x' Do updating expressions, by default outputting the changes: basexsed -e 'delete node //n' xmldocuments/*.xml # Outputs the docs without the n nodes (-i is -u in basex) basexsed -i -e 'delete node //n' xmldocuments/*.xml # Changes the files in place Automatically handle other formats like json, csv, html: basexsed -e 'map:keys(.)' jsondocuments/*.json # Outputs as json as well AFAIK everything is already there in the basex command, so the main change would be to support multiple files/documents in a streaming fashion, and then a few smaller things like using stdin as the context- item if no file is given and allowing the same arguments as in standard tools like sed. To make it nicer you could also add syntax highlighting to the console, some shortcuts for longer XQuery parameters like serialization indentation. This would provide an alternative to tools like Xidel ( https://www.videlibri.de/xidel.html ) or jq ( https://stedolan.github.io/jq/ ). Is this something you would be interested in implementing? Or have some people already done something similar? Regards, Iwan

3 4

Re: [basex-talk] Package for system integration
by Christian Grün 23 Jun '21

23 Jun '21

Hi Karlo, > Where does basex set the location of the configuration? If you point me > to the file, I might try changing it. By default, BaseX creates and updates the configuration files in the project’s "Home Directory". This directory is determined at startup time as described in the Wiki [1]. If you download and start the JAR file on a completely fresh system, the home directory will be the same as the user home directory. Hope this helps, Christian [1] https://docs.basex.org/wiki/Configuration#Home_Directory On Tue, Jun 22, 2021 at 6:37 AM <karlo(a)fastmail.com> wrote: > > Liam R. E. Quin writes: > > The things that are binary-only, if they have a compatible licence, > > could either be included or added in a separate package. > > Indeed, when registering it with a package repository, we would replace > those binary libaries with dependent packages from the repository. > > Liam R. E. Quin writes: > > Yes, it should look in $HOME for a conf file. > > Even better, ${XDG_CONFIG_HOME:-${HOME}/.config} > > Where does basex set the location of the configuration? If you point me > to the file, I might try changing it.

1 0

Re: [basex-talk] Package for system integration
by Christian Grün 22 Jun '21

22 Jun '21

Dear Kralj, That’s a great contribution; I really appreciate your efforts. A little while ago, Bridger asked users on the list to share their experiences installing and running BaseX. It would be great if those of you who are using Linux and UNIX-based systems could have a look at Kralj’s package and tell us about your observations! Thanks all, best regards, Christian On Sun, Jun 20, 2021 at 8:36 AM <karlo(a)fastmail.com> wrote: > > Dear Christian, > > With your help I have prepared a new version with a similar user > interface but that also builds BaseX. The released binaries run > perfectly of course, so the role of the source build package is > to demonstrate that the binaries are built from a free source. > > Because the package follows certain conventions, we can quickly prepare > BaseX for many different operating systems and maintain BaseX packages > for several package repositories. > > I explain the method in more detail below. For those who are just > joining the discussion, I include some background before continuing > the discussion about this package. > > I have the honor to be Your Majesty's humble and obedient servant. > > Kralj Karlo > > > > INTRODUCTION > > BaseX has a bespoke procedure initiating builds. If the build could > instead be initiated through execution of "make" in a directory > with the conventional inputs, it would be fast to port to the many > different operating systems whose packaging systems follow this > convention. > > I propose a procedure to create BaseX source packages. The resulting > BaseX source packages contain a Makefile with instructions for building > and installing from source. > > I create the source package as a directory containing the basex source code, > the basex-dist source code, a version-specific Makefile, and other files > to support the build. > > I distribute the source package, and someone else can build it. A build > of the source package produces a tree that can be installed by unpacking > to the specified PREFIX, /usr/local by default. > > To build it within the framework of a package manager, you would > configure the package manager to retrieve the source package and, > possibly, set a different PREFIX, set the dependencies, and copy > other package metadata (for example, homepage and description). > The package manager will likely already have tooling around the "make" > and "make install" commands, so you should not really need to understand > the build process. > > > > PROCEDURE > > Copy the program for creating the source package. > https://klarinet.kos.ovh/fossil/basex-make/tarball/basex-make.tar.gz > > You can alternatively use fossil, for example: > > fossil clone https://klarinet.kos.ovh/fossil/basex-make ../basex-make.fossil > fossil open ../basex-make.fossil > > I run "make update prune all tgz" from the basex-make directory > to do the following. > > * Download the latest sources and copy them to ./basex-src > * Generate the file ./basex-src/Makefile for building the sources. > * Archive the directory basex-src as basex-src.tar.gz > > Now I can send the basex-src.tar.gz to someone else. Here it is. > https://klarinet.kos.ovh/fossil/basex-make/uv/basex-src.tar.gz > > Then you can extract basex-src.tar.gz and run "make && make install" > to build and install. That does the following. > > * Build BaseX by the conventional process. > * Unpack the zip file created by the conventional BaseX build. > * Replace the conventional executibles. The new executables > handle the PREFIX and are written in sh rather than bash. > > > > BUGS > > * With this installation method BaseX looks for the configuration file > $PREFIX/lib/BaseX$VERSION/.basex and creates it if it does not exist. > It would be good to change that to look for $HOME/.basex instead. > * Certain vendor libraries are still distributed as binaries > in basex/basex-core/lib. I plan to remove these so we can truly call > it a "source" package. > * basex-src.tar.gz should instead be named BaseX$VERSION.tar.gz, and > it should contain a directory named BaseX$VERSION instead of basex-src. > > NOT BUGS > > * The minor complexity of this process is unfortunate. It would be nice > to modify release.pl so there are fewer layers, but this could break other > things that depend on the present build process. > > APPLICATION > > I propose that a tarball like that created by the present procedure be > distributed on the BaseX download webpage (https://basex.org/download/) > and that it be addressed as the "Source Package" with support for all > Unix-like operating systems. I believe it will in fact work on Windows > with Cygwin, but I don't care to support that. > > Once we have begin releasing source packages, we proceed to add BaseX > to various package repositories. I think we would start with Debian apt, > because we have already done one of those, and OpenBSD ports, because > that's what I use.

3 2

ft:normalize: potential bug?
by Alexander Witzigmann 21 Jun '21

21 Jun '21

hi, if you call ft:normalize using a string containing "*" ft:normalize("a*",map { 'stemming': 'true', 'language':'de' }) this results in Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk(a)mailman.uni-konstanz.de Version: BaseX 9.5.2 Java: Oracle Corporation, 13.0.2 OS: Windows 10, amd64 Stack Trace: java.lang.ArrayIndexOutOfBoundsException: Index -1 out of bounds for length 8 at org.basex.util.TokenBuilder.get(TokenBuilder.java:191) at org.basex.util.ft.GermanStemmer.resub(GermanStemmer.java:150) at org.basex.util.ft.GermanStemmer.stem(GermanStemmer.java:42) at org.basex.util.ft.Stemmer.nextToken(Stemmer.java:99) at org.basex.util.ft.FTLexer.nextToken(FTLexer.java:156) at org.basex.query.func.ft.FtTokenize.tokens(FtTokenize.java:49) at org.basex.query.func.ft.FtNormalize.item(FtNormalize.java:23) .... anything I missed? Alex

2 2

Error message, can this be fixed?
by Roland Haarbrink 21 Jun '21

21 Jun '21

Hi, Getting this message: org.basex.core.BaseXException: Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk(a)mailman.uni-konstanz.de Version: BaseX 7.9 Java: Oracle Corporation, 1.8.0_161 OS: Linux, amd64 Stack Trace: java.lang.ArrayIndexOutOfBoundsException When using this validator: https://validatie.geostandaarden.nl/etf-webapp/testruns/start-direct Can this be fixed? Thanks URL: http://localhost:8080/etf-webapp/testruns/start-direct Exception id: EXID-06e97066-fe8f-43cd-adc5-df641b843f75 de.interactive_instruments.etf.dal.basex.dao.BasexDataSource.create(BasexDat aSource.java:397) de.interactive_instruments.etf.dal.basex.dao.BasexDataSource.create(BasexDat aSource.java:381) de.interactive_instruments.etf.dal.basex.dao.BasexTestObjectDao.doCreate(Bas exTestObjectDao.java:134) de.interactive_instruments.etf.dal.dao.AbstractTestObjectDao.create(Abstract TestObjectDao.java:89) de.interactive_instruments.etf.webapp.controller.TestObjectController.addFil eTestData(TestObjectController.java:366) de.interactive_instruments.etf.webapp.controller.TestRunController.startDire ct(TestRunController.java:238) sun.reflect.GeneratedMethodAccessor245.invoke(Unknown Source) sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl .java:43) java.lang.reflect.Method.invoke(Method.java:498) org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(Invoc ableHandlerMethod.java:221) org.springframework.web.method.support.InvocableHandlerMethod.invokeForReque st(InvocableHandlerMethod.java:136) org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandle rMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:110) org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerA dapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:817) org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerA dapter.handleInternal(RequestMappingHandlerAdapter.java:731) org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.hand le(AbstractHandlerMethodAdapter.java:85) org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServl et.java:959) org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServle t.java:893) org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkSer vlet.java:968) org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.jav a:870) javax.servlet.http.HttpServlet.service(HttpServlet.java:648) org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.ja va:844) javax.servlet.http.HttpServlet.service(HttpServlet.java:729) org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Application FilterChain.java:292) org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterCh ain.java:207) org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Application FilterChain.java:240) org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterCh ain.java:207) org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.ja va:212) org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.ja va:106) org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase .java:502) org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:141 ) org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79) org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogVa lve.java:616) org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java :88) org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:522) org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Proce ssor.java:1095) org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(Abstrac tProtocol.java:672) org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.jav a:1500) org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java: 1456) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:11 49) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:6 24) org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.ja va:61) java.lang.Thread.run(Thread.java:748)

2 1

Package for system integration
by karlo＠fastmail.com 20 Jun '21

20 Jun '21

I have packaged basex in a way that I believe will be easier for integration with package managers. https://klarinet.kos.ovh/fossil/basex-root-install/uv/basex-9.4.3.tar.gz >From the root of that archive, you can build and install like this. make make install # as root The above will install to the prefix /usr/local. You can set a different prefix. Here is an example. make clean make PREFIX=/usr make PREFIX=/usr install # as root A secondary benefit of this package is the removal of the bash dependency. Could a package like this be distributed officially? If that is agreeable, I am happy to try incorporating this into the present build system. Here is a utility for preparing the package. It depends on sh and other Unix-like utilities. https://klarinet.kos.ovh/fossil/basex-root-install/dir?ci=tip With distinguished salutations, Kralj Karlo

3 6

getting request's headers inside a Web Application
by dudzikp＠yahoo.com 16 Jun '21

16 Jun '21

Dear all, I do not see an option to obtain the headers from the HTTP request from inside webapp function. Just like in the example: declare %rest:GET %rest:path('tests-html') ???%rest:headers('User-Agent')??? %output:method('html') function page-main:start-empty( ???$user-agent as xs:string??? ) as element(html) { ... } Is this possible? BR

3 3

http:sent-request with username, password and auth-method results in improper use msg
by Rob Stapper 15 Jun '21

15 Jun '21

Hi Cristian, When I add the username, password,auth-method to a http-request [1] I get an improper use message from BaseX[2]. The message suggest a bug. I hope this helps solving it. Kind regards, Rob [1] http:send-request( <http:request method='post' username='admin' password='admin' auth-method='Basic' > <http:body media-type='text/plain'/> </http:request> , 'http://LAPTOP-F17D4BIM:8984/test' , "test" ) [2] Error: Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk(a)mailman.uni-konstanz.de Version: BaseX 9.5.3 beta Java: Oracle Corporation, 1.8.0_271 OS: Windows 10, amd64 Stack Trace: java.lang.IllegalStateException: Already connected at sun.net.www.protocol.http.HttpURLConnection.setRequestProperty(Unknown Source) at org.basex.util.http.HttpClient.setContentType(HttpClient.java:258) at org.basex.util.http.HttpClient.sendRequest(HttpClient.java:82) at org.basex.query.func.http.HttpSendRequest.value(HttpSendRequest.java:34) at org.basex.query.expr.ParseExpr.iter(ParseExpr.java:46) at org.basex.query.scope.MainModule.iter(MainModule.java:97) at org.basex.query.QueryContext.iter(QueryContext.java:326) at org.basex.query.QueryContext.cache(QueryContext.java:607) at org.basex.query.QueryProcessor.cache(QueryProcessor.java:113) at org.basex.core.cmd.AQuery.query(AQuery.java:99) at org.basex.core.cmd.XQuery.run(XQuery.java:22) at org.basex.core.Command.run(Command.java:257) at org.basex.core.Command.execute(Command.java:93) at org.basex.gui.GUI.exec(GUI.java:416) at org.basex.gui.GUI.lambda$4(GUI.java:359) at java.lang.Thread.run(Unknown Source) Sent from Mail for Windows 10 -- This email has been checked for viruses by Avast antivirus software. https://www.avast.com/antivirus

2 2

differencing the string value of documents
by Graydon Saunders 12 Jun '21

12 Jun '21

Hello -- So I have XML documents A and B. For this purpose, the markup (which is known to be different) is not relevant; the thing I need to do is prove that all of the text in the string value of A is present in B in the same order in which it exists in A. Document B can have additional text and still be correct, so it is not sufficient to check if string(A) = string(B). (Ideally, B's extra text can be extracted and then compared against the sources of extra text in the transformation between A and B so as to confirm that it's at least all plausible new text.) I've been poking at this for a while and am getting that "reinventing the wheel by gnawing on anvils" feeling. Does a facility to do this exist in BaseX? I've had a look at the documentation and I observe some mention of diff.xqm but that appears to be a node-based diff. thanks! Graydon

5 9

Permissions
by SW-Service 12 Jun '21

12 Jun '21

Hello, should perm force authentication and only the user "admin" would be authorised? Best regards Frank module namespace page = 'http://basex.org/test'; declare %rest:path("basex-rest/test/check-rest-perm") %perm:allow("admin") %rest:GET %output:method("xml") function page:test-check-rest-perm() as element(response) { let $response := element response { 'Response' } return $response }; <response>Response</response>

2 1

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

BaseX-Talk