- BaseX-Talk - mailman.uni-konstanz.de

Is it possible to limit a BaseX server maximum memory consumption?
by Jacob Borisov 12 Apr '17

12 Apr '17

Hello. Is it possible to hardly limit a maximum memory consumption of a BaseX server instance process (something like a cache_size option in MongoDB) avoiding getting Java heap space errors simultaneously? Or, in the current implementation, it's required for the process to be able to allocate memory enough for storing a whole data set of the currently accessed database? Thanks.

2 3

RESTXQ and URI-Parameters
by Peter Hinkelmanns 12 Apr '17

12 Apr '17

Dear all, I noticed, that the implementation of RestXQ in BaseX behaves differently compared to the implemation in eXist. If you have a simple API like declare %rest:path("/test/{$params1}/{$params2}") %rest:GET function test($params1 as xs:string*, $params2 as xs:string*) { <response> <p1>{$params1}</p1> <p2>{$params2}</p2> </response> }; And you call this Uri "/test/a/b;c=2" BaseX returns <response> <p1>a</p1> <p2>b</p2> </response> versus the response of eXist <response> <p1>a</p1> <p2>b;c=2</p2> </response> --> It seems to be not possible to pass Matrix URIs in BaseX over RestXQ right now. eXist would accept as well a call like "test/a;e;f=5/b;c=2". BaseX passes out at this point, because nothing after the first ; is evaluated and thus no matching RestXQ API can be found. My questions: -Is this a bug? -Is there a feature missing in the current implementation of RestXQ in BaseX? -Or is there an easy configuration for BaseX available, which will allow working with Matrix-Uri-Parameters and RestXQ? Thanks and bye Peter

2 4

systemd basex.service ubuntu 16.04
by Stephen White 10 Apr '17

10 Apr '17

Does anyone have a working combination for launch basexhttp as a service using systemd? If not, what is the suggested method to auto launch on reboot. Cheers, Steve

2 1

Re: [basex-talk] Basex with JSmooth
by Maximilian Gärber 10 Apr '17

10 Apr '17

Hi, you might need to check that all needed libs are on your classpath, the .bat does this with this line: set CP=%MAIN%/target/classes;%MAIN%/lib/*;%MAIN%/../basex-core/lib/* Replace %MAIN% with wathever JSmooth gives you as the project directory variable in such a way that e.g. %PROJECT%/target/classes will match your new setup. NOTE: * Please always post to the list as well, others might add valuable feedback anytime * If you include more infos, e.g. error messages, more people might be able to help Br, Max 2017-04-10 9:34 GMT+02:00 Tushar Deshmukh <tdlwala(a)gmail.com>: > Namaste Max, > > PFA a screen shot for making .exe file. I m not able to create it. > > I am not able to instantiate BaseXHTTP class in java also (after importing > org.basex.*;). Please what is going wrong. > > For normal user .exe file is more appropriate than yajsw. > > Mangal Bhavna Sahit, > Tushar > > On Mon, Apr 10, 2017 at 12:41 PM, Maximilian Gärber <mgaerber(a)arcor.de> > wrote: >> >> Hi, >> >> it is org.basex.BaseXHTTP >> >> just have a look at the start script: >> >> https://github.com/BaseXdb/basex/blob/0f970efc419bc067bd03a66438177eb7f884d… >> >> Br, >> Max >> >> 2017-04-10 4:49 GMT+02:00 Tushar Deshmukh <tdlwala(a)gmail.com>: >> > Namaste, >> > >> > I want to start Basex HTTP server using Jsmooth >> > (I want to make .exe file from BasexHTTP.jar.) >> > I am not able to do so because which class is to be instantiate. Any >> > link/Help is appreciated. >> > >> > Mangal Bhavna Sahit, >> > Tushar > >

1 0

Basex with JSmooth
by Tushar Deshmukh 10 Apr '17

10 Apr '17

Namaste, I want to start Basex HTTP server using Jsmooth (I want to make .exe file from BasexHTTP.jar.) I am not able to do so because which class is to be instantiate. Any link/Help is appreciated. Mangal Bhavna Sahit, Tushar

2 1

server and/or web application up times
by Fabrice ETANCHAUD 07 Apr '17

07 Apr '17

Dear all, Is there a mean to obtain server and/or web application up times ? Best regards, Fabrice

2 2

BaseX 8.6.3
by Christian Grün 06 Apr '17

06 Apr '17

Hi all, We have released a new patch version of BaseX. some minor bugs have been fixed [1], performance of iterative query evaluation has been improved, and a new fetch:xml-binary function is now available, which allows you to convert binary data (which may have an XML encoding declaration embedded) to XML [2]. Have fun, Christian [1] https://github.com/BaseXdb/basex/commits/master [2] http://docs.basex.org/wiki/Fetch_Module#fetch:xml-binary

1 0

Re: [basex-talk] Basex Docker Container: How to Control Options
by Christian Grün 05 Apr '17

05 Apr '17

Hi Eliot, Did you try to specify the option in the local section of the .basex file (at the bottom, after all other options)? Best, Christian Am 02.04.2017 10:52 schrieb "Eliot Kimber" <ekimber(a)contrext.com>: I’ve worked around the configuration file problem by setting the options on the Java command line by setting BASEX_JVM in my Dockerfile: ENV BASEX_JVM="-Dorg.basex.CHOP=false -Dorg.basex.CATFILE=/opt/dita-ot/DITA-OT/catalog-dita.xml -Dorg.basex.DTD=true -Dorg.basex.SKIPCORRUPT=true" That seems to work. I guess I could also set this environment variable in my docker-compose.yml file, although except for CATFILE, these are invariant for this application. Cheers, E. -- Eliot Kimber http://contrext.com On 4/1/17, 10:44 PM, "Eliot Kimber" <basex-talk-bounces@mailman. uni-konstanz.de on behalf of ekimber(a)contrext.com> wrote: Perhaps a more useful question is: is there a workaround for this? That is, I need to have the DTD, CATFILE, CHOP, and SKIPCORRUPT options set when I load files into a database, which I’m doing through the remote API. Is there either a separate options file that I should be using or a way to specify these options at document load time? Cheers, E. -- Eliot Kimber http://contrext.com On 4/1/17, 10:23 PM, "Eliot Kimber" <basex-talk-bounces@mailman. uni-konstanz.de on behalf of ekimber(a)contrext.com> wrote: I dug into the code and I think this is a code bug but I don’t understand the overall flow well enough to know where the bug is, but I suspect it’s a simple change. If I debug the BasexGUI class and have a ~/.basex file that includes CATFILE=foo it will fail with an unknown option error because the option class being used is StaticOptions and StaticOptions does not define CATFILE as an option. The CATFILE option, along with a few others, is defined on MainOptions: private static final Option<?>[] INHERIT = { CHOP, INTPARSE, STRIPNS, DTD, XINCLUDE, CATFILE }; Looking at BaseXHTTP it uses HTTPContext which only uses StaticOptions, not MainOptions, which means it can never allow the INHERIT options. So something must have changed from 8.1 ish to now to change how options are handled since these options worked before I moved to the latest BaseX container and versions. But there definitely seems to be an oversight in the initial handling of the options file in the BaseXHTTP server (and in the BaseXGUI as well since I get the same behavior there as well). I would make a pull request but I don’t presume to know what policies or general practices underlie the current options mechanism deisgn. Cheers, Eliot -- Eliot Kimber http://contrext.com On 3/26/17, 10:20 PM, "Eliot Kimber" <basex-talk-bounces@mailman. uni-konstanz.de on behalf of ekimber(a)contrext.com> wrote: I found my original mail thread from January 2016: https://mailman.uni-konstanz.de/pipermail/basex-talk/2016- January/010171.html Basically I had to append my options to the existing .basex file rather than simply provide a file. However, that isn’t an option here because there is no existing /srv/.basex file (the place where the file needs to be). As an experiment I tried doing this: RUN touch /srv/.basex && \ echo "DTD=true" >> .basex && \ echo "CATFILE=/opt/dita-ot/DITA-OT/catalog-dita.xml" >> .basex && \ echo "SKIPCORRUPT=true" >> .basex && \ echo "CHOP=false" >> .basex && \ echo "DBPATH=/srv/BaseXData" >> .basex && \ echo "REPOPATH=/srv/BaseXRepo" >> .basex && \ echo "WEBPATH=/srv/BaseXWeb" >> .basex && \ chown basex /srv/.basex RUN echo "/srv/.basex:\n===========" && cat /srv/.basex && echo "=======" And I get this from the build command: /srv/.basex: =========== DTD=true CATFILE=/opt/dita-ot/DITA-OT/catalog-dita.xml SKIPCORRUPT=true CHOP=false DBPATH=/srv/BaseXData REPOPATH=/srv/BaseXRepo WEBPATH=/srv/BaseXWeb ======= When I start up this container I get this: /srv/.basex: Unknown option 'DTD'. /srv/.basex: Unknown option 'CATFILE'. /srv/.basex: Unknown option 'SKIPCORRUPT'. /srv/.basex: Unknown option 'CHOP'. /srv/.basex: writing new configuration file. Note that it’s only complaining about DTD, CATFILE, SKIPCORRUPT, and CHOP, but not DBPATH, REPOPATH, and WEBPATH. I tried reordering things, no difference in result (but the messages reflect the order change). So it must be something about how these particular options are processed rather than an issue with the .basex file, at least as far as I can see. Cheers, Eliot -- Eliot Kimber http://contrext.com On 3/26/17, 3:29 PM, "Eliot Kimber" <basex-talk-bounces@mailman. uni-konstanz.de on behalf of ekimber(a)contrext.com> wrote: Discovered that if I use USER root in the Dockerfile I have to then do USER basex before the end so that the container will be running as basex, otherwise it does not start up correctly. I also have to chown the /srv/.basex file to basex otherwise it can’t overwrite it on startup. Made some progress but something is still not working. I now have a Dockerfile based on basex/dba that results in a running system with the both the dba and my custom app runnable. However, if I then add this: COPY basex/dot_basex /srv/.basex As before, on startup I get this: docker run --name linkmgr-test -p 8984:8984 link-manager-test /srv/.basex: Unknown option 'CATFILE'. /srv/.basex: Unknown option 'DTD'. /srv/.basex: Unknown option 'SKIPCORRUPT'. /srv/.basex: Unknown option 'CHOP'. /srv/.basex: writing new configuration file. Creating /srv/BaseXWeb/WEB-INF/web.xml Creating /srv/BaseXWeb/WEB-INF/jetty.xml I think this Unknown option this is actually an issue with the file itself but I don’t recall what the exact cause and/or solution was. Cheers, E. -- Eliot Kimber http://contrext.com On 3/26/17, 1:40 PM, "Eliot Kimber" < basex-talk-bounces(a)mailman.uni-konstanz.de on behalf of ekimber(a)contrext.com> wrote: I’m trying to set up a Docker container with my own Web app using the latest official containers as a base and following the instructions in the latest BaseX docs. As part of this setup I need to add several additional options. I also want to include the dba app. Following the instructions in the docs I’m putting my additional options in /srv/.basex and these take effect. However, if I use the basex/dba image as a base, the other options, such as the custom REPOPATH and WEBPATH settings do not get used. It’s not clear to me from looking at the various Dockerfiles how the dba container sets the options and how I can then add to them without overwriting them. Of course I can duplicate the options in my Dockerfile but it seems like I should be able to add my options additively but I’m not seeing how to do it, so I feel I’m missing something. In my Dockerfile I have: COPY basex/dot_basex /srv/.basex If I don’t include this line then the container otherwise works in that my Web app is there but it requires these additional options in order to function properly. If I include it then the effective options reflect only my changes and not those required for the custom Web apps to work. Thanks, Eliot -- Eliot Kimber http://contrext.com

3 5

Re: [basex-talk] Basex Docker Container: How to Control Options
by Eliot Kimber 02 Apr '17

02 Apr '17

Discovered that if I use USER root in the Dockerfile I have to then do USER basex before the end so that the container will be running as basex, otherwise it does not start up correctly. I also have to chown the /srv/.basex file to basex otherwise it can’t overwrite it on startup. Made some progress but something is still not working. I now have a Dockerfile based on basex/dba that results in a running system with the both the dba and my custom app runnable. However, if I then add this: COPY basex/dot_basex /srv/.basex As before, on startup I get this: docker run --name linkmgr-test -p 8984:8984 link-manager-test /srv/.basex: Unknown option 'CATFILE'. /srv/.basex: Unknown option 'DTD'. /srv/.basex: Unknown option 'SKIPCORRUPT'. /srv/.basex: Unknown option 'CHOP'. /srv/.basex: writing new configuration file. Creating /srv/BaseXWeb/WEB-INF/web.xml Creating /srv/BaseXWeb/WEB-INF/jetty.xml I think this Unknown option this is actually an issue with the file itself but I don’t recall what the exact cause and/or solution was. Cheers, E. -- Eliot Kimber http://contrext.com On 3/26/17, 1:40 PM, "Eliot Kimber" <basex-talk-bounces(a)mailman.uni-konstanz.de on behalf of ekimber(a)contrext.com> wrote: I’m trying to set up a Docker container with my own Web app using the latest official containers as a base and following the instructions in the latest BaseX docs. As part of this setup I need to add several additional options. I also want to include the dba app. Following the instructions in the docs I’m putting my additional options in /srv/.basex and these take effect. However, if I use the basex/dba image as a base, the other options, such as the custom REPOPATH and WEBPATH settings do not get used. It’s not clear to me from looking at the various Dockerfiles how the dba container sets the options and how I can then add to them without overwriting them. Of course I can duplicate the options in my Dockerfile but it seems like I should be able to add my options additively but I’m not seeing how to do it, so I feel I’m missing something. In my Dockerfile I have: COPY basex/dot_basex /srv/.basex If I don’t include this line then the container otherwise works in that my Web app is there but it requires these additional options in order to function properly. If I include it then the effective options reflect only my changes and not those required for the custom Web apps to work. Thanks, Eliot -- Eliot Kimber http://contrext.com

1 4

Deadlock with concurrent write and reads in BaseX
by Simon Chatelain 31 Mar '17

31 Mar '17

Hello all, I am facing an issue with my application using BaseX 8.6.2. Basically what I have is one thread (Add thread) receiving data from a socket as XML fragment and storing it in BaseX. On the other hand, my application provides a REST interface to retrieve data from the DB. When retrieving data what happen is, I first execute a simple Xquery in a thread (Primary Read), and for each hit returned by this first query, I execute another query in another thread (Secondary read) to retrieve additional data. The problem is that I end up in a situation where all three threads are stopped waiting for something. The Add thread is park as it cannot acquire a write lock, while the Primary read is running, as it should be. But the Secondary read is also parked, thus blocking the Primary read, and so we end up in a kind of deadlock. The PARALLEL option in .basex is set to 4, and the FAIRLOCK option is set to true. Some technical details: In the primary read I am using session.setOutputStream(pos); session.execute(new XQuery(flwor)); where session is a LocalSession and pos is a PipedOutputStream In Secondary read, I am reading the other side of the piped stream, and for each line read from the stream, I execute another query (well not exactly in the real case scenario, but this simplification seems good enough to exhibit the problem, and is shown in the attached BaseXDeadlock.java example). You can also see the stacks of all three threads in BaseXDeadlockStacks.txt In this example, if I don’t start the Add thread, then Primary and Secondary read are running as expected. On the other hand, if I change my implementation of Primary and Secondary read, and use a blocking queue to transfer data from Primary to Secondary read like shown below: Query query = session.query(flwor) while (query.more()) { queue.put(query.next()); } And then taking from the queue in Secondary read, then it works. This implementation is shown in BaseXNoDeadlock.java. Using the latter is good enough for me, as it does the job, even if there is a small performance penalty compared to the first implementation, in my real case scenario. So my question is: is my first implementation fundamentally broken or is there something wrong on BaseX side? Best regards and many thanks to all the BaseX team for this efficient and nice to use tool. Simon Chatelain

2 1