- BaseX-Talk - mailman.uni-konstanz.de

Re: [basex-talk] Linux lessons
by Dirk Kirsten 19 Jan '15

19 Jan '15

Hi Wendell, On 01/10/2015 10:31 PM, Wendell Piez wrote: > Hi Dirk, > > I am still looking at this - I hope you'll be patient with another > couple of questions. If you don't have the time to instruct me please > let me know and accept my apologies. :-) Sure, no problem. We are happy to help, but please always cc the mailing list, so others can benefit from this questions as well. I am quite sure you are not the only one wondering about stuff like this. > > All the projects are local, and they typically rely on other > applications in addition to BaseX (they tend to be XML/XSLT > applications). So for convenience I want to run a single copy of BaseX > with a single data directory, with different databases for the > different projects. > > It's fine if multiple servers run the different projects on multiple > ports, but it's also fine if they all live together on one (8984 is > fine). So there could be a single startup script for all the projects > (as long as the code supporting each project can be distributed), or > each project could have its own server. > > I guess that means running something like the basexhttp shell script > to run the server from a different directory, setting it up with its > own webapp and thus all the advantages (and encapsulation) you cite? > > How do I go about doing this? I have tried copying and modifying the > basexhttp script, but I can't get the classpaths correct. (It seems to > be designed to depend on its location in the file system relative to > BaseX.) I am afraid I am missing a simple way to start up BaseX to run > restXQ, except keep the latter code in a project directory. -- whether > with a completely different webapp, or sharing the "space" among > several projects - as long as their restXQ files can be found. I would also advise against copying the basexhttp script. It is not really a clean solution, involves lots of manually copying if we change the script in the next version (and for the 8.0 snapshots this did happen). Also, copying the binary itself by principle sounds like not the best solution. > > So -- I can give the project its own webapp directory, no problem. I > can put a jetty.xml file in there. But how do I start BaseX (installed > in /home/wendell/basex) to run from this webapp? (If my script calls > ~/bin/basexhttp, BaseX starts up with its own webapp and .basex > configuration etc.) So if I understood your setup correctly, all BaseX settings in your different projects are the same except the RESTXQ path? So one option could be to set the RESTXQ path manually before starting basexhttp. Using the latest 8.0 snapshot, this is done using the global variable BASEX_JVM. So executing export BASEX_JVM=-Dorg.basex.RESTXQPATH=/my/restxq/directory && basexhttp should use the path provided on the command line. If you want to startup all your projects this way you could write a little script which does this for you. If you call baexhttp from a subdirectory (e.g. /home/wendell/superawesomeproject), it shouldn't startup the standard BaseX webapp, but instead look for a web.xml in /home/wendell/superawesomeproject/src/main/webapp/WEB-INF/. I am not sure right now how to change this path or if this is even possible using basexhttp. Another option - which I personally prefer, also because it avoids creating your own scripts and provides easy configuration options - is using Maven. This way, a simple mvn jetty:run inside your project directory should be enough. The pom.xml configures the Maven execution. For example, the jetty maven plugin configuration looks something like this: <plugin> <groupId>org.mortbay.jetty</groupId> <artifactId>jetty-maven-plugin</artifactId> <version>${jettyVersion}</version> <configuration> <webAppConfig> <contextPath>/</contextPath> <descriptor>WEB-INF/${webXml}</descriptor> <tempDirectory>.tmp</tempDirectory> </webAppConfig> <webAppSourceDirectory>${basedir}/${webapp}/public</webAppSourceDirectory> <jettyXml>WEB-INF/jetty.xml</jettyXml> <stopKey>keyToStopJetty</stopKey> <stopPort>${stopPort}</stopPort> </configuration> </plugin> Notice, that in this case the web.xml is actually in a variable. This way, you could startup different web applications using the command mvn jetty:run -Dwebapp=project1-web.xml and so on. So you can use different web.xml (where you can set the RESTXQ path, see http://docs.basex.org/wiki/Web_Application#Configuration for details) for your different projects. Of yourse, you could also use complete separate pom.xml using mvn -f myotherpom.xml jetty:run I hope this helps. Cheers, Dirk > > Thanks again for the hints -- > > Cheers, Wendell > > > > > > On Sat, Jan 10, 2015 at 3:31 PM, Wendell Piez <wapiez(a)wendellpiez.com> wrote: > > Dirk, thanks! > > > > Cheers, Wendell > > > > > > On Thu, Jan 8, 2015 at 4:05 AM, Dirk Kirsten <dk(a)basex.org> wrote: > >> Hello Wendell, > >> > >> your last guess is correct: BaseX can only handle a single RESTXQ directory. So solutions using a colon a separator will not work, simply because we do not support it. It does not depend on the OS (although I personally think you did a wise thing in trying out GNU/Linux ;). > >> > >> The usual approach would be to have a separate .basex in your webapp directory. So if you start basex from this directory it will use this values instead your default ones. More likely you might want to store the BaseX specific values in your jetty.xml (if using Jetty), so they are used for the application container. This has several advantages: > >> - Your web application can not access some default functionality, which shouldn't really be available. > >> - You can use different BaseX version for different projects by including BaseX within the webapp itself > >> - You can also define different configurations for your different webapps (e.g. all your projects run on a different port, so you can run them simultanously) > >> > >> Cheers, > >> Dirk > >> > >> On 01/07/2015 11:32 PM, Wendell Piez wrote: > >>> Hi again, > >>> > >>> Correction: of course, under the "composite" path, neither path works. > >>> > >>> Perhaps BaseX can only have a single RestXQ directory at a time? > >>> > >>> Thanks again for your patience with me. :-) > >>> > >>> Cheers, Wendell > >>> > >>> On Wed, Jan 7, 2015 at 5:29 PM, Wendell Piez <wapiez(a)wendellpiez.com> wrote: > >>>> Dear BaseX people, > >>>> > >>>> I am a hopeless newbie when it comes to Linux. Seriously, my ignorance > >>>> is astonishing. I am now running Linux Mint. It's a huge amount of fun > >>>> but I'm left guessing about many simply things. > >>>> > >>>> I am running the current BaseX 7.9. I am using it to provide a > >>>> pipeline architecture calling lots of XSLT2, producing HTML and SVG. > >>>> It works well for this. > >>>> > >>>> To my RESTXQPATH I wish to append to - but not override (if possible) - a path. > >>>> > >>>> By editing my .basex configuration I have managed to reset it, but not > >>>> simply to append to it. > >>>> > >>>> That is, > >>>> > >>>> RESTXQPATH = /home/wendell/basex/webapp:/home/wendell/XQuery-project/restxq > >>>> > >>>> With this, the server skips the basex/webapp directory. If I remove > >>>> the second segment and restart, it goes live again. > >>>> > >>>> What is the simplest way for me to set up my BaseX so that I can > >>>> maintain my project xqm files (including a project's RestXQ), in the > >>>> project, but keep BaseX otherwise (including its data directories) > >>>> separate? Over the longer term I can see several such directories > >>>> where I'd like BaseX to look. > >>>> > >>>> On another operating system I used to start up using different shell > >>>> scripts; is that the best way here (and if so does my shell script > >>>> have to call Java or can I use the scripts in the basex/bin and pass > >>>> in a value somehow)? > >>>> > >>>> In general, what recommendations can you offer for no-fuss > >>>> configuration of RestXQ for separate projects? > >>>> > >>>> Thanks! Wendell > >>>> > >>>> > >>>> -- > >>>> Wendell Piez | http://www.wendellpiez.com > >>>> XML | XSLT | electronic publishing > >>>> Eat Your Vegetables > >>>> _____oo_________o_o___ooooo____ooooooo_^ > >>> > >>> > >>> > >> > >> -- > >> Dirk Kirsten, BaseX GmbH, http://basexgmbh.de > >> |-- Firmensitz: Blarerstrasse 56, 78462 Konstanz > >> |-- Registergericht Freiburg, HRB: 708285, Geschäftsführer: > >> | Dr. Christian Grün, Dr. Alexander Holupirek, Michael Seiferle > >> `-- Phone: 0049 7531 28 28 676, Fax: 0049 7531 20 05 22 > >> > > > > > > > > -- > > Wendell Piez | http://www.wendellpiez.com > > XML | XSLT | electronic publishing > > Eat Your Vegetables > > _____oo_________o_o___ooooo____ooooooo_^ > > > -- Dirk Kirsten, BaseX GmbH, http://basexgmbh.de |-- Firmensitz: Blarerstrasse 56, 78462 Konstanz |-- Registergericht Freiburg, HRB: 708285, Geschäftsführer: | Dr. Christian Grün, Dr. Alexander Holupirek, Michael Seiferle `-- Phone: 0049 7531 28 28 676, Fax: 0049 7531 20 05 22

2 4

module import / base URI declaration
by Hans-Juergen Rennau 18 Jan '15

18 Jan '15

Dear BaseX team, I think that the resolving of module import hints does not work properly if the importing module has a relative base URI declaration. In the example below, module1 and module2 both are located in a directory "tt". As module 1 has a base URI declaration setting the base URI to "..", its import of module2 should require the following import hint: "tt/module2.xq", rather than "module2.xq". However, this produces an error; the error is not thrown when the import hint is erroneously set to "module" - which is, of course, also wrong behaviour. Compare: http://www.w3.org/TR/xquery-30/#id-base-uri-decl Cheers,Hans-Jürgen PScall: basex -b uri=dummy.xml main.xq=>[XQST0059] Could not retrieve module 'C:/projects/infospace/pcol/tmp/tt/module2.xq'. main.xq:=======import module namespace m="http://www.ttools.org/xquery-functions" at "tmp/module1.xq"; declare variable $uri external; m:foo($uri) module1.xq:=========module namespace m="http://www.ttools.org/xquery-functions"; import module namespace i="http://www.ttools.org/xquery-functions" at "tt/module2.xq"; declare base-uri ".."; declare function m:foo($uri as xs:string) {i:bar($uri)}; module2.xq:==========module namespace m="http://www.ttools.org/xquery-functions"; declare base-uri ".."; declare function m:bar($uri as xs:string) {doc($uri)};

2 3

try catch mechanisms and inlining
by jean-marc Mercier 17 Jan '15

17 Jan '15

Hello all, I wrote some basic debugging / profiling tools for xquery code, using try{} catch *{} mechanisms. However, profiling tools are not reliable. The suspected reason is that inlining is not activated in try catch expression. If confirmed, is it a desired behavior ? Here is the basic test : declare function local:test($i){fn:for-each(1 to $i, function($a){2*$a})}; declare function local:fun-profile($fun,$args) {try{local:call($fun,$args)} catch *{()}}; declare function local:fun-profile2($fun,$args) {local:call($fun,$args)}; With default options (ie inlining activated) prof:time(count(local:fun-profile2(local:test#1, 10000000))) Output : 683.12 ms prof:time(count(local:fun-profile(local:test#1, 10000000))) Output : 2465.47 ms With inlining desactivated (ie INLINELIMIT=0) prof:time(count(local:fun-profile2(local:test#1, 10000000))) Output : 2480.32 ms

2 6

Pruned, optimized DB shows same document count, db size
by Hondros, Constantine (ELS-AMS) 16 Jan '15

16 Jan '15

Hello all, I've pruned about half the documents of a +-5GB database using XQuery delete. I then optimised the database using db:optimize. The database metadata still shows the original number of documents, and the overall db filesize remains roughly the same. I was sort of hoping this pruning would improve performance, but if there's any difference it's negligable. Am I missing something obvious? Thanks in advance, Constantine Hondros ________________________________ Elsevier B.V. Registered Office: Radarweg 29, 1043 NX Amsterdam, The Netherlands, Registration No. 33156677, Registered in The Netherlands.

3 7

fn:nilled() not working
by ankit kumar 15 Jan '15

15 Jan '15

I am trying to run fn:nilled() function in base-x, but every time a node with xsi:nil="true" is passed to the function it return false. Please help me out.

2 1

RESTXQ example of query parameters
by C. M. Sperberg-McQueen 15 Jan '15

15 Jan '15

On the documentation page on RESTXQ [1], the following example is given in the section headed 'Query parameters': declare %rest:path("/params") %rest:query-param("id", "{$id}") %rest:query-param("add", "{$add}", 42, 43, 44) function page:params($value as xs:string?, $answer as xs:integer+) { <result id="{ $id }" sum="{ sum($add) }"/> }; Is it really the case that %rest:query-param("id", "{$id}") %rest:query-param("add", "{$add}", 42, 43, 44) will cause variables named $id and $add to be in scope in the function, even though there are no arguments with either of these two names? (And what values will $value and $answer have?) Or is there a typo in the example? If there is a typo in the example, it would be nice to fix it; if there is no typo in the example, it would be nice to explain to the user what is going on. I can't find any mention of rest:query-param in the draft RESTXQ spec [2], and where I do find mentions of rest:query-param, I have not yet found any explicit statement about what it does, in formal terms. I have been guessing, based on the other examples in [1] and on the statement there that "The following annotations can be used to bind request values to function arguments", that an annotation of the form rest:query-param(X, "{$Y}", Z) binds the value of the query parameter X in the URI to $Y, where $Y is the name of a parameter of the function being annotated. I apologize for not simply tying the example to see if it works. Michael [1] http://docs.basex.org/wiki/RESTXQ [2] http://exquery.github.com/exquery/exquery-restxq-specification/restxq-1.0-s… -- **************************************************************** * C. M. Sperberg-McQueen, Black Mesa Technologies LLC * http://www.blackmesatech.com * http://cmsmcq.com/mib * http://balisage.net ****************************************************************

2 1

RESTXQ and access blocking
by Lars Johnsen 15 Jan '15

15 Jan '15

Hi all I was wondering how to block general access to BaseX when using RESTXQ. Our javascript/jquery web-application communicates with BaseX using commands like: $('#myobject').load('objects') where the term 'objects' is defined as a path in a .xqm-file. declare %rest:path("/objects") However, databases are exposed using the URL "/rest" which seems built into the rest-module. For example, in the javascript/jquery console (f.ex. in Chrome ), a div could be filled up with content outside of the application by typing things like: $('div').load('rest/my_database') and general queries could be made using the rest-interface http://docs.basex.org/wiki/REST. Is there a way to prevent this, while at the same time using BaseX as web-server (one way is to use BaseX only as a backend database)? Or how to limit the URLs permitted? Best Lars

3 4

Optimization for procedure call ?
by jean-marc Mercier 14 Jan '15

14 Jan '15

Hello, I am facing some performances issues, probably due to unexpected query reevaluation. Might it be due to the optimizer ? Here is a code to test declare function local:test($i){ let $data := fn:for-each(1 to 10000000, function($a){2*$a} ) (: let $data := xquery:eval("fn:for-each(1 to 10000000, function($a){2*$a} )") :) return (1 to $i) ! local:test2($data,.) }; declare function local:test2($data, $dummy) { count($data) }; (1 to 10) ! prof:time(local:test(.)) Output : 683.76 ms 1137.95 ms 1727.68 ms 2151.06 ms 2694.84 ms 3189.57 ms 3725.1 ms 4277.51 ms 4815.74 ms 5417.79 ms Note that using xquery:eval (toggling the comment inside local:test). seems to perform correctly. However I can't use this trick as a workaround. (1 to 10) ! prof:time(local:test(.)) Ouput : 2472.71 ms 825.61 ms 3013.14 ms 986.81 ms 2982.97 ms 778.24 ms 1195.33 ms 3280.7 ms 976.96 ms 913.69 ms

2 6

O'Reilly: eXist. A NoSQL Document Database and Application Platform
by Christian Grün 13 Jan '15

13 Jan '15

Hi all, Our friends from eXist (or, more specifically, Adam Retter and Erik Siegel) have just published an O'Reilly book on the eXist database. It is available both on paper and as epub [1]. Have fun, Christian [1] http://shop.oreilly.com/product/0636920026525.do

1 0

User login
by Andy Bunce 13 Jan '15

13 Jan '15

Hi, Just investigating user handling, I am using the latest snapsnap. I start basexhttp with -U guest -P guest and/or I set org.basex.user and org.basex.password in web.xml Sometimes I have created a user ( guest with password guest) and permission none via dba. Sometimes I have deleted the guest user. In all cases I appear to be able to run restxq queries and in those queries read from databases. Am I misunderstanding something here? /Andy

2 5