- BaseX-Talk - mailman.uni-konstanz.de

Performance
by Marco Lettere 21 Mar '14

21 Mar '14

Hello to everyone, while trying to trace the performance I could expect from my RestXQ app I created a DB with something like 300/400k documents. For the test the documents are rather homogenoeus and stored plainly in the DB root collection. I had two major objectives. 1) understand the behaviour of the query when run on a significant (even if not huge) number of documents. I've noticed that something like collection('db')/a[@att = 'xxx' and b/text() = 'zzz'] on my laptop (Intel I7, 8 GB Ram and SSD) takes some 5 to 7 seconds. Is this reasonable? I was expecting slightly better performance but I must admit that the documents are rather similar thus maybe the indexing is not very selective. What I noticed and this is feeling strange is that the time is often spent in rather similar proportion between query compilation and query evaluation and in some runs (1 every 5) most of the time is spent in compilation and only few msec in evaluation. But the query is rather simple so I assume that time spent in opening large collections (or accessing the indexes) is computed in the compilation phase. Is it like that? Is there a way to restructure my query to be somehow more selective reducing the time of accessing the collection/index? 2) to measure the difference between the same query when run directly and when run through xquery:eval. Here I noticed that the difference between plain and xquery:eval() execution is negligible when the time for the evaluation increases. So I assume that the xquery:eval is using the indexing system even if this is not reported in the info view of the GUI. Is it like that? Thank you all. Regards, Marco.

2 1

Namespaces ...
by Marco Lettere 21 Mar '14

21 Mar '14

Hello all, I've a few questions related to namespaces. I've noticed that on one of the info panels of the the GUI all the namespaces of the documents stored in a DB are listed. Is there a way to get this information from inside an XQuery? My problem is that I have to perform queries that are dynamically generated on a heterogeneuos set of documents. By now I have to store somewhere an association between one input parameter of the query and a corresponding set of namespaces to be used. This is then transformed into a list of "declare namespace ..." statements. I thought I could pregenerate a declare statement for every namespace in the DB by default thus avoiding to maintain the association. Would there be a significant impact on query or query compilation performance? Thanks, Marco.

3 2

restriction on number of namespaces has apparently been lifted – thank you
by Imsieke, Gerrit, le-tex 21 Mar '14

21 Mar '14

Today I noticed that I could actually build an index of all XSLT, XProc, Relax NG and Schematron files on my hard disk (3316 files). I couldn’t do that 2 years ago because the maximum number of distinct namespaces in a DB was limited to 256 or so. Thanks, BaseX team, for lifting this restriction! This has already proved really useful: I knew that I wrote an XProc step that conditionally invoked a step whose local name I remembered. The simple XPath expression collection('home')//*:declare-step[*:choose//*:paths] helped me identify the two relevant files. Since we do a lot of development in XML-syntax languages, an XML database is really really good for structured searches on these files. I bet you XQuery devs still use grep to query your code ;) Gerrit

1 1

mailing list archives 403?
by Jesse Clark 20 Mar '14

20 Mar '14

Is anybody else getting a 403 Forbidden error trying to access the mailing list archives? https://mailman.uni-konstanz.de/pipermail/basex-talk/ -Jesse

2 2

Bug (?) - trailing whitespace in text nodes
by Hans-Juergen Rennau 20 Mar '14

20 Mar '14

Dear BaseX team, I think I observed a bug concerning trailing whitespace in text nodes. Please consider this input document: <para>xxx <emphasis role="italic">abc</emphasis> yyy.</para> [Note the blanks between xxx and <emphasis>.] The result of the following "null-transformation" ======================= declare function local:edit($n as node()) as node()? { typeswitch($n) case document-node() return document {for $c in $n/node() return local:edit($c)} case element() return element {node-name($n)} {for $ac in $n/(@*, node()) return local:edit($ac)} default return $n }; local:edit(.) ======================= should be identical, but what I get is this: <para>xxx<emphasis role="italic">abc</emphasis>yyy.</para> The blanks after "xxx" are gone! When transforming mixed content like docbook, this can have awkware consequences. Kind regards, Hans-Jürgen

3 5

Question about BaseX JAVA API Usage ...
by Christopher Lyons 20 Mar '14

20 Mar '14

I've just starting using BaseX (V.7.8.1) and have a very quick question that I hope somebody can help me with. I've reviewed the BaseX docs and tried looking for an answer to this on the Wiki but have not been successful. I've installed BaseX (on Win7 Pro/64-bit) and have been using the GUI interface (non-server) and it has been working great. I just started exploring the JAVA API and am having a few basic problems. I'm trying to use the RunQueries.java example file (currently just need to do some local queries from a JAVA app). Just an FYI, the links from BaseX doc site to the GitHub example repository are all broken. The examples are on GitHub but it looks like they were recently moved/updated and the links must have changed and haven't been updated on the BaseX site. I'm an experienced programmer but am just getting up-to-speed with JAVA, so my question may be more related to that than a specific BaseX problem but hopefully someone can point me in the right direction. I use Eclipse and when I create a test class with the RunQueries.java code, Eclipse is not able to resolve the BaseX package imports at the start of the class. Namely, the following code: import org.basex.core.*; import org.basex.core.cmd.*; import org.basex.data.*; import org.basex.io.serial.*; import org.basex.query.*; import org.basex.query.iter.*; import org.basex.query.value.item.*; Should the general install of BaseX put these packages into their proper places? Or is there something else I have to do to get these packages properly installed and able to be resolved? Appreciate any help. Thanks, Chris Lyons

2 1

Re: [basex-talk] BaseX-Talk Digest, Vol 51, Issue 20
by John Best 19 Mar '14

19 Mar '14

Dear Christian, Good to see these features... What about MIXED Content searching ? :( On Wed, Mar 19, 2014 at 4:30 PM, <basex-talk-request(a)mailman.uni-konstanz.de > wrote: > Send BaseX-Talk mailing list submissions to > basex-talk(a)mailman.uni-konstanz.de > > To subscribe or unsubscribe via the World Wide Web, visit > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk > or, via email, send a message with subject or body 'help' to > basex-talk-request(a)mailman.uni-konstanz.de > > You can reach the person managing the list at > basex-talk-owner(a)mailman.uni-konstanz.de > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of BaseX-Talk digest..." > > > Today's Topics: > > 1. BaseX: Upcoming Features (Christian Gr?n) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Tue, 18 Mar 2014 13:03:26 +0100 > From: Christian Gr?n <christian.gruen(a)gmail.com> > To: BaseX <basex-talk(a)mailman.uni-konstanz.de>, BaseX > <basex-announce(a)mailman.uni-konstanz.de> > Subject: [basex-talk] BaseX: Upcoming Features > Message-ID: > <CAP94bnOKazkmPDE4qbw+MZ9OR= > ceCh+4vM9E3tcgOrBMFcKyBg(a)mail.gmail.com> > Content-Type: text/plain; charset=ISO-8859-1 > > Dear BaseX aficionados, > > An upcoming version of our software (7.9 or 8.0) will make it much > easier to perform updates and return items at the same time [1]. It > will also contain an xquery:update function and will bring xquery:eval > and xquery:evaluate together again [2] Another exciting and > sophisticated feature we are working on is query precompilation [3], > which will particularly speed up large XQuery applications and allow > for more request-specific optimizations. > > Next, we are working on a Database Administration interface based on > RESTXQ, and we will also drop out the confusing client/server dialog > from the GUI. > > Feature 1 and 2 are already available via a feature branch [4]. If you > are working with the git sources, we are excited to get your feedback! > > Christian > > [1] https://github.com/BaseXdb/basex/issues/873 > [2] https://github.com/BaseXdb/basex/issues/878 > [3] https://github.com/BaseXdb/basex/issues/874 > [4] https://github.com/BaseXdb/basex/tree/issue873 > > > ------------------------------ > > _______________________________________________ > BaseX-Talk mailing list > BaseX-Talk(a)mailman.uni-konstanz.de > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk > > > End of BaseX-Talk Digest, Vol 51, Issue 20 > ****************************************** > -- Have a nice day JBest

3 2

Error in webdav.xqm
by France Baril 19 Mar '14

19 Mar '14

I'm using BaseX with Oxygen, accessing content through a webdav folder. Since the upgrade to 7.8.1, I'm getting these errors in the terminal Window (see end of message). Interestingly, I have no xquery folder under /Users/archie and a search return no file named webdav.xqm on the computer. I haven't seen differences in the behaviour with the upgrade, just these messages. Regards, France Last login: Fri Feb 28 11:37:18 on ttys000 NoName:~ archie$ /Applications/basex/bin/basexhttp ; exit; BaseX 7.8.1 [Server] Server was started (port: 1984) HTTP Server was started (port: 8984) Stopped at /Users/archie/xquery/webdav.xqm, 1/246: [XPTY0004] Cannot treat xs:string as xs:integer: "300". com.bradmcevoy.http.exceptions.BadRequestException Stopped at /Users/archie/xquery/webdav.xqm, 1/246: [XPTY0004] Cannot treat xs:string as xs:integer: "300". com.bradmcevoy.http.exceptions.BadRequestException Stopped at /Users/archie/xquery/webdav.xqm, 1/246: [XPTY0004] Cannot treat xs:string as xs:integer: "300". com.bradmcevoy.http.exceptions.BadRequestException Stopped at /Users/archie/xquery/webdav.xqm, 1/246: [XPTY0004] Cannot treat xs:string as xs:integer: "300". com.bradmcevoy.http.exceptions.BadRequestException Stopped at /Users/archie/xquery/webdav.xqm, 1/246: [XPTY0004] Cannot treat xs:string as xs:integer: "300". com.bradmcevoy.http.exceptions.BadRequestException Stopped at /Users/archie/xquery/webdav.xqm, 1/246: [XPTY0004] Cannot treat xs:string as xs:integer: "300". com.bradmcevoy.http.exceptions.BadRequestException Stopped at /Users/archie/xquery/webdav.xqm, 1/246: [XPTY0004] Cannot treat xs:string as xs:integer: "300". com.bradmcevoy.http.exceptions.BadRequestException Stopped at /Users/archie/xquery/webdav.xqm, 1/246: [XPTY0004] Cannot treat xs:string as xs:integer: "300". com.bradmcevoy.http.exceptions.BadRequestException Stopped at /Users/archie/xquery/webdav.xqm, 1/246: [XPTY0004] Cannot treat xs:string as xs:integer: "300". com.bradmcevoy.http.exceptions.BadRequestException Stopped at /Users/archie/xquery/webdav.xqm, 1/246: [XPTY0004] Cannot treat xs:string as xs:integer: "300". com.bradmcevoy.http.exceptions.BadRequestException Stopped at /Users/archie/xquery/webdav.xqm, 1/246: [XPTY0004] Cannot treat xs:string as xs:integer: "300". com.bradmcevoy.http.exceptions.BadRequestException Stopped at /Users/archie/xquery/webdav.xqm, 1/246: [XPTY0004] Cannot treat xs:string as xs:integer: "300". com.bradmcevoy.http.exceptions.BadRequestException Stopped at /Users/archie/xquery/webdav.xqm, 1/246: [XPTY0004] Cannot treat xs:string as xs:integer: "300". com.bradmcevoy.http.exceptions.BadRequestException Stopped at /Users/archie/xquery/webdav.xqm, 1/246: [XPTY0004] Cannot treat xs:string as xs:integer: "300". com.bradmcevoy.http.exceptions.BadRequestException Stopped at /Users/archie/xquery/webdav.xqm, 1/246: [XPTY0004] Cannot treat xs:string as xs:integer: "300". com.bradmcevoy.http.exceptions.BadRequestException Stopped at /Users/archie/xquery/webdav.xqm, 1/246: [XPTY0004] Cannot treat xs:string as xs:integer: "300". com.bradmcevoy.http.exceptions.BadRequestException Stopped at /Users/archie/xquery/webdav.xqm, 1/246: [XPTY0004] Cannot treat xs:string as xs:integer: "300". com.bradmcevoy.http.exceptions.BadRequestException Stopped at /Users/archie/xquery/webdav.xqm, 1/246: [XPTY0004] Cannot treat xs:string as xs:integer: "300". com.bradmcevoy.http.exceptions.BadRequestException Stopped at /Users/archie/xquery/webdav.xqm, 1/246: [XPTY0004] Cannot treat xs:string as xs:integer: "300". com.bradmcevoy.http.exceptions.BadRequestException Stopped at /Users/archie/xquery/webdav.xqm, 1/246: [XPTY0004] Cannot treat xs:string as xs:integer: "300". com.bradmcevoy.http.exceptions.BadRequestException -- France Baril Architecte documentaire / Documentation architect france.baril(a)architextus.com (514) 572-0341

2 2

db:replace throwing error
by Jesse Clark 18 Mar '14

18 Mar '14

Hi All, The module docs for db:replace (http://docs.basex.org/wiki/Database_Module#db:replace) states that: "Replaces a document, specified by $path, in the database $db with the content of $input, or adds it as a new document." However, when I try to run a query using db:replace: return db:replace($database, $jsonXML, $targetProfilePath) I get the following error regardless of whether there is a document in the db at that path or not: [FODC0002] Resource "/targetProfiles/5321cff2e4b0372d556893f4" does not exist. db:add() seems to be working fine but I would prefer not to add multiple documents at the same path. I suppose I can work around this by first using db:delete() to remove any documents at the path and then using db:add() but I would appreciate any insight into why db:replace() isn't working as expected. Thanks, -Jesse

2 2

using evaluate on untrusted code
by Jesse Clark 18 Mar '14

18 Mar '14

Hello all, I have a project where I need to take 3rd party XQuery expressions and evaluate them in the context of a single xml document. The expressions are untrusted and I want to limit them to read-only and only allow them to access a single xml document being passed in as an external variable. I read in the XQuery Module docs (http://docs.basex.org/wiki/XQuery_Module) that with the evaluate() command the evaluated query is prevented from opening new database. I believe I could run the evaluate() call from a session connected as a read only user to prevent any update commands. Also, since I only want the evaluated expressions to be able to access a document declared as external I could, I believe, evaluate() from a session connected to an empty database as well. However, would the evaluated expressions still be able to execute functions from other BaseX modules like, for example, proc:system()? I know that in general using eval on untrusted code is a huge risk. Is there anyway to create a secure 'sandbox' for evaluating untrusted XQuery expressions? Any advice would be greatly appreciated. Thanks, -Jesse

2 3