March 2014 - BaseX-Talk - mailman.uni-konstanz.de

multi-threaded XQueries
by Ying Shanshan 23 Mar '14

23 Mar '14

Hi All, I created one database of 1GB in size, and have a batch of *read-only*queries to be performed. Each query is quite simple, so there is not much room to improve. All the queries are executed through *XQJ api.* If these queries are sequentially executed, it costs 5~8 seconds per query. To speed it up, I tried to executed it in a multi-thread way. But turns out to be much slower than the sequential one. This is what I wrote for each thread: public void run() { try { XQConnection conn = ds.getConnection(); XQSequence rs = conn.prepareExpression(input) .executeQuery(); //process the query result conn.close(); } } catch (Throwable e) { e.printStackTrace(); } } Anyone has the idea on how to improve the batch of queries. Thanks, -- Regards Shanshan School of Computing National University of Singapore

2 1

Confusing error using GROUP BY
by Charles Duffy 23 Mar '14

23 Mar '14

Howdy -- I've run into a somewhat confusing error message using GROUP BY in BaseX 7.8.1, while trying to answer a question on StackOverflow ( http://stackoverflow.com/questions/22585865/grouping-with-counts-in-xquery) Against the database at http://www.ibiblio.org/xml/examples/shakespeare/hamlet.xml, I'm running the following query: for $line in //SPEECH/LINE let $speaker := $line/../SPEAKER/text()[1] group by $speaker return <speaker name="{$speaker}" count="{count($line)}"/> ...and receiving the error: [XPTY0004] Single item expected, (db:open-pre("hamlet",748), db:open-pre("hamlet",750)) found. I've gone over the GROUP BY docs several times, and not seeing my error in the above. Could someone enlighten?

2 1

Re: [basex-talk] pearl pigments
by tony-zhao＠vip.188.com 22 Mar '14

22 Mar '14

1 0

Traling empty nodes in csv:serialize
by Lars Johnsen 22 Mar '14

22 Mar '14

Hi all I have a question about csv:serialize and blank nodes. Trailing blanks are indicated with a separator, but those in the middle are. csv:serialize( <rows> <row> <f>1</f> <s>2</s> <t>3</t></row> <row> <f></f> <s></s> <t>3</t></row> <row> <f>1</f> <s></s> <t>3</t></row> <row> <f>1</f> <s></s> <t></t></row> <row> <f>1</f> <s>2</s> <t></t></row> </rows>) Here is the output (BaseX 7.8.1) , where line 4 and 5 are missing trailing separators: 1,2,3 ,,3 1,,3 1 1,2 Could this be fixed, or is it a feature? All the best Lars G Johnsen National Library of Norway

3 3

Re: [basex-talk] Bug (?) - trailing whitespace in text nodes
by Hans-Juergen Rennau 21 Mar '14

21 Mar '14

Mm, the documentation says: "Chops all leading and trailing whitespaces from text nodes while building a database, and discards empty text nodes. By default, this option is set to true, as it often reduces the database size by up to 50%. It can also be turned off on command line via -w." The text states clearly that chopping affects only text nodes stored into a database. At any rate - the problem remains, whether or not I use option -w, and whether or not I use prolog option db:chop. (Side remark: it would be a serious issue if the prolog option were required, as this would imply that standard conformant behaviour could only be achieved by making the code unportable.) Kind regards, Hans-Juergen -------------------------------------------- Dirk Kirsten <dk(a)basex.org> schrieb am Do, 20.3.2014: Betreff: Re: [basex-talk] Bug (?) - trailing whitespace in text nodes An: "Hans-Juergen Rennau" <hrennau(a)yahoo.de>, "basex-talk(a)mailman.uni-konstanz.de" <basex-talk(a)mailman.uni-konstanz.de> Datum: Donnerstag, 20. März, 2014 18:28 Uhr Dear Hans-Jürgen, I am not quite sure it is intended that the CHOP option is applied to text nodes. At least the wording in the documentation ("while building a database") does not indicate it, while I think it actually does make sense. Christian will have to answer whether this works as is intended behavior. However, you can set the chop option to false within your XQuery by declaring it declare option db:chop "false"; and this should also affect reading in files from the file system. At least this works for me. Cheers, Dirk On 20/03/14 17:47, Hans-Juergen Rennau wrote: > My understanding is that it only affects database documents, and I used file input. > > Nevertheless, I also tried option -w, but always got the same result. > > Kind regards, > Hans-Jürgen > > > > > > Hans-Juergen Rennau <hrennau(a)yahoo.de> schrieb am 17:45 Donnerstag, 20.März 2014: > > Dear Dirk, thank you. > > But this is strange - I ran the query using a file as input - not a database document. > > I tried two versions: > BaseX 7.8.2 beta f505185 [Standalone] > BaseX 8.0 beta 606f18b [Standalone] > > OS is Windows 7. > > I always get this result: > <para>xxx<emphasis role="italic">abc</emphasis>yyy.</para> > > Using a different XQuery processor, I get this result, as expected: > <para>xxx <emphasis role="italic">abc</emphasis> yyy.</para> > > Kind regards, > Hans-Jürgen > > > > > > Dirk Kirsten <dk(a)basex.org> schrieb am 17:10 Donnerstag, 20.März 2014: > > Dear Hans-Jürgen, > > When running local:edit() on an in-memory node I get the expected and > correct result, including whitespaces. > > I guess that you run this command on a database node and the XML > documents were parsed with CHOP being true (which is the default), this > would explain the behavior and would be as expected. If you do not want > this you might consider setting the CHOP option (see > https://docs.basex.org/wiki/Options#CHOP) to false. > > Cheers, > Dirk > > > On 20/03/14 16:57, Hans-Juergen Rennau wrote: >> Dear BaseX team, >> >> I think I observed a bug concerning trailing whitespace in text nodes. >> >> Please consider this input document: >> >> <para>xxx <emphasis role="italic">abc</emphasis> yyy.</para> >> >> >> [Note the blanks between xxx and <emphasis>.] >> >> The result of the following "null-transformation" >> >> ======================= >> >> declare function > local:edit($n as node()) >> as node()? { >> typeswitch($n) >> case document-node() return >> document {for $c in $n/node() return local:edit($c)} >> case element() return >> element {node-name($n)} >> {for $ac in $n/(@*, node()) return local:edit($ac)} >> default return $n >> }; >> local:edit(.) >> ======================= >> >> should be identical, but what I get is this: >> >> <para>xxx<emphasis role="italic">abc</emphasis>yyy.</para> >> >> The > blanks after "xxx" are gone! >> >> >> When transforming mixed content like docbook, this can have awkware consequences. >> >> Kind regards, >> Hans-Jürgen >> >> >> >> _______________________________________________ >> BaseX-Talk mailing list >> BaseX-Talk(a)mailman.uni-konstanz.de >> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >> > -- Dirk Kirsten, BaseX GmbH, http://basex.org |-- Firmensitz: Blarerstrasse 56, 78462 Konstanz |-- Registergericht Freiburg, HRB: 708285, Geschäftsführer: | Dr. Christian Grün, Dr. Alexander Holupirek, Michael Seiferle `-- Phone: 0049 7531 28 28 676, Fax: 0049 7531 20 05 22

2 4

feature's requests and not publicly released features
by Fabrice Etanchaud 21 Mar '14

21 Mar '14

Dear all, First of all, BaseX is just fantastic ! Could you please tell me if you may release the following features in near future ? 1. User defined indexes Let's consider you have to index multi lingual documents. Currently, one have to partition each document in mono-lingual documents, and add them in separate collections because there can be only one fulltext index per collection, and because all text() nodes will be indexed. Are others BaseX users interested with user defined indexes (attribute, text, and fulltext) ? Wouldn't user defined indexes dramatically improve performance (during requests, and optimization) ? 2. Break current collection limitation on the maximum number of nodes. 3. Possibility to store millions of small documents, and do efficient 'db:replaces' or 'db:delete'. Last, How does BaseX team provide non-open-source features ? Best regards, Fabrice Etanchaud Questel/Orbit

2 1

Performance
by Marco Lettere 21 Mar '14

21 Mar '14

Hello to everyone, while trying to trace the performance I could expect from my RestXQ app I created a DB with something like 300/400k documents. For the test the documents are rather homogenoeus and stored plainly in the DB root collection. I had two major objectives. 1) understand the behaviour of the query when run on a significant (even if not huge) number of documents. I've noticed that something like collection('db')/a[@att = 'xxx' and b/text() = 'zzz'] on my laptop (Intel I7, 8 GB Ram and SSD) takes some 5 to 7 seconds. Is this reasonable? I was expecting slightly better performance but I must admit that the documents are rather similar thus maybe the indexing is not very selective. What I noticed and this is feeling strange is that the time is often spent in rather similar proportion between query compilation and query evaluation and in some runs (1 every 5) most of the time is spent in compilation and only few msec in evaluation. But the query is rather simple so I assume that time spent in opening large collections (or accessing the indexes) is computed in the compilation phase. Is it like that? Is there a way to restructure my query to be somehow more selective reducing the time of accessing the collection/index? 2) to measure the difference between the same query when run directly and when run through xquery:eval. Here I noticed that the difference between plain and xquery:eval() execution is negligible when the time for the evaluation increases. So I assume that the xquery:eval is using the indexing system even if this is not reported in the info view of the GUI. Is it like that? Thank you all. Regards, Marco.

2 1

Namespaces ...
by Marco Lettere 21 Mar '14

21 Mar '14

Hello all, I've a few questions related to namespaces. I've noticed that on one of the info panels of the the GUI all the namespaces of the documents stored in a DB are listed. Is there a way to get this information from inside an XQuery? My problem is that I have to perform queries that are dynamically generated on a heterogeneuos set of documents. By now I have to store somewhere an association between one input parameter of the query and a corresponding set of namespaces to be used. This is then transformed into a list of "declare namespace ..." statements. I thought I could pregenerate a declare statement for every namespace in the DB by default thus avoiding to maintain the association. Would there be a significant impact on query or query compilation performance? Thanks, Marco.

3 2

restriction on number of namespaces has apparently been lifted – thank you
by Imsieke, Gerrit, le-tex 21 Mar '14

21 Mar '14

Today I noticed that I could actually build an index of all XSLT, XProc, Relax NG and Schematron files on my hard disk (3316 files). I couldn’t do that 2 years ago because the maximum number of distinct namespaces in a DB was limited to 256 or so. Thanks, BaseX team, for lifting this restriction! This has already proved really useful: I knew that I wrote an XProc step that conditionally invoked a step whose local name I remembered. The simple XPath expression collection('home')//*:declare-step[*:choose//*:paths] helped me identify the two relevant files. Since we do a lot of development in XML-syntax languages, an XML database is really really good for structured searches on these files. I bet you XQuery devs still use grep to query your code ;) Gerrit

1 1

mailing list archives 403?
by Jesse Clark 20 Mar '14

20 Mar '14

Is anybody else getting a 403 Forbidden error trying to access the mailing list archives? https://mailman.uni-konstanz.de/pipermail/basex-talk/ -Jesse

2 2