BaseX-Talk August 2015

basex-talk@mailman.uni-konstanz.de

35 participants
51 discussions

Start a nNew thread

BaseX Package and Jetty
by Eckey, Florian 05 Aug '15

05 Aug '15

3 5

Destination of result-document from XSLT module
by Lizzi, Vincent 05 Aug '15

05 Aug '15

I'm trying to use the XSLT Module in BaseX 8.2.3 with Saxon 9.6 to run an XSLT that produces several output documents using xsl:result-document. I'm having trouble setting the location of the output documents. I want to have xsl:result-document create the output documents in a temporary folder because the documents need to be zipped together. According to Saxon's documentation, the a relative path in the href attribute of xsl:result-document will be resolved using either the path of the Destination, or the current directory. The XSLT Module does not appear have a way to provide a path for a destination document. What I'm seeing is that the result documents are created in BaseX's home directory. The XSLT works as expected when run using Saxon from the command line, where it's possible to set a destination path. Is there a way to specify a Base Output URI to the XSLT Module? Or, would it be possible to specify a file URI output location to a method like xslt:transform? One possible workaround is to provide an absolute path as a parameter to the XSLT, and use that parameter in the xsl:result-document href location. Here is a self-contained example code: declare function local:example($in, $xsl, $zipPath) { let $tempDir := file:create-temp-dir('test', 'example') let $x := xslt:transform-text($in, $xsl) return let $zip := archive:create-from($tempDir) return ( file:write-binary($zipPath, $zip) (: , file:delete($tempDir, true()) :) ) }; let $xsl := <xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:xs="http://www.w3.org/2001/XMLSchema" version="2.0"> <xsl:template match="/"> <xsl:result-document href="doc1.xml"> <test>this is a test 1 <xsl:apply-templates/></test> </xsl:result-document> <xsl:result-document href="doc2.xml"> <test>this is a test 2 <xsl:apply-templates/></test> </xsl:result-document> </xsl:template> </xsl:stylesheet> let $doc := <test>this is input</test> let $zipPath := 'report.zip' return local:example($doc, $xsl, $zipPath) The expected output is a zip file report.zip that contains doc1.xml and doc2.xml. However, what I'm seeing is that report.zip is created as an empty zip file and doc1.xml and doc2.xml are placed in BaseX's home directory. Thanks, Vincent

5 7

Performance with joins?
by Tim Thompson 04 Aug '15

04 Aug '15

Hello, I have a database containing two resources/documents: they both represent the same set of library catalog records (7728 "records" in each), but they each contain different data that I want to join. The first resource looks like this: <marc:collection xmlns:marc="http://www.loc.gov/MARC21/slim"> <marc:record> <marc:leader>01225cam a2200373Mi 4500</marc:leader> <marc:controlfield tag="001">5323084</marc:controlfield> <marc:datafield ind1="1" ind2="4" tag="245"> <marc:subfield code="a">Els teleclubs a les illes Balears :</marc:subfield> </marc:datafield> </marc:record> <marc:record> <marc:leader>01225cam a2200373Mi 4500</marc:leader> <marc:controlfield tag="001">5323084</marc:controlfield> <marc:datafield ind1="1" ind2="4" tag="245"> <marc:subfield code="a">Els teleclubs a les illes Balears :</marc:subfield> </marc:datafield> </marc:record> <marc:record> <marc:leader>00818cam a2200241Mi 4500</marc:leader> <marc:controlfield tag="001">6310976</marc:controlfield> <marc:datafield ind1="0" ind2="0" tag="245"> <marc:subfield code="a">Diccionari manual de sinònims i antònims de la llengua catalana /</marc:subfield> </marc:datafield> </marc:record> </marc:collection> The second one looks like this: <root> <row> <LANGUAGE>cat</LANGUAGE> <ITEM_ID>5912416</ITEM_ID> <BIB_ID>5323084</BIB_ID> <VENDOR_CODE>MXBKSMX</VENDOR_CODE> </row> <row> <LANGUAGE>cat</LANGUAGE> <ITEM_ID>5912416</ITEM_ID> <BIB_ID>5323084</BIB_ID> <VENDOR_CODE>PUVILL</VENDOR_CODE> </row> <row> <LANGUAGE>cat</LANGUAGE> <ITEM_ID>5935043</ITEM_ID> <BIB_ID>6310976</BIB_ID> <VENDOR_CODE>PUVILL</VENDOR_CODE> </row> </root> I have a simple query that joins the two using the value of the marc:controlfield[@tag = '001'] from resource 1 and the BIB_ID from resource 2. The query: declare namespace marc="http://www.loc.gov/MARC21/slim"; for $m in collection( "latin_hold_20150730" )/marc:collection/marc:record, $r in collection( "latin_hold_20150730" )/root/row[BIB_ID = $m/marc:controlfield[@tag = '001']] group by $key := $r/ITEM_ID return <test n="{ $key }">{ $m/marc:datafield[@tag = '245']/marc:subfield[@code = 'a']/string() }</test> When I run this using Saxon (substituting fn:doc for fn:collection), it only takes a second to execute. In BaseX 8.2.3 (using the GUI), however, it takes a very long time to execute (around 6 minutes!). The BaseX database has both attribute and text indexes enabled. Any idea what is causing it to take so long? Here is the BaseX Query Info: Total Time: 383756.3 ms Compiling: - pre-evaluating collection("latin_hold_20150730") - pre-evaluating collection("latin_hold_20150730") Query: declare namespace marc="http://www.loc.gov/MARC21/slim"; for $m in collection( "latin_hold_20150730" )/marc:collection/marc:record, $r in collection( "latin_hold_20150730" )/root/row[BIB_ID = $m/marc:controlfield[@tag = '001']] group by $key := $r/ITEM_ID return <test n="{ $key }">{ $m/marc:datafield[@tag = '245']/marc:subfield[@code = 'a']/string() }</test> Optimized Query: for $m_0 in (db:open-pre("latin_hold_20150730",0), ...)/marc:collection/marc:record for $r_1 in (db:open-pre("latin_hold_20150730",0), ...)/root/row[(BIB_ID = $m_0/marc:controlfield[(@tag = "001")])] let (: post-group :) $m_4 := $m_0 group by $key_2 := $r_1/ITEM_ID return element test { (attribute n { ($key_2) }, $m_4/marc:datafield[(@tag = "245")]/marc:subfield[(@code = "a")]/string()) } Result: - Hit(s): 7587 Items - Updated: 0 Items - Printed: 505 KB - Read Locking: local [latin_hold_20150730] - Write Locking: none Timing: - Parsing: 0.43 ms - Compiling: 1.96 ms - Evaluating: 383737.5 ms - Printing: 16.41 ms - Total Time: 383756.3 ms Query plan: <QueryPlan compiled="true"> <GFLWOR> <For> <Var name="$m" id="0"/> <IterPath> <DBNodeSeq size="2"> <DBNode name="latin_hold_20150730" pre="0"/> <DBNode name="latin_hold_20150730" pre="365692"/> </DBNodeSeq> <IterStep axis="child" test="marc:collection"/> <IterStep axis="child" test="marc:record"/> </IterPath> </For> <For> <Var name="$r" id="1"/> <IterPath> <DBNodeSeq size="2"> <DBNode name="latin_hold_20150730" pre="0"/> <DBNode name="latin_hold_20150730" pre="365692"/> </DBNodeSeq> <IterStep axis="child" test="root"/> <IterStep axis="child" test="row"> <CmpG op="="> <CachedPath> <IterStep axis="child" test="BIB_ID"/> </CachedPath> <IterPath> <VarRef> <Var name="$m" id="0"/> </VarRef> <IterStep axis="child" test="marc:controlfield"> <CmpG op="="> <CachedPath> <IterStep axis="attribute" test="tag"/> </CachedPath> <Str value="001" type="xs:string"/> </CmpG> </IterStep> </IterPath> </CmpG> </IterStep> </IterPath> </For> <GroupBy> <Spec> <Var name="$key" id="2"/> <IterPath> <VarRef> <Var name="$r" id="1"/> </VarRef> <IterStep axis="child" test="ITEM_ID"/> </IterPath> </Spec> </GroupBy> <CElem> <QNm value="test" type="xs:QName"/> <CAttr> <QNm value="n" type="xs:QName"/> <VarRef> <Var name="$key" id="2"/> </VarRef> </CAttr> <MixedPath> <VarRef> <Var name="$m" id="4"/> </VarRef> <IterStep axis="child" test="marc:datafield"> <CmpG op="="> <CachedPath> <IterStep axis="attribute" test="tag"/> </CachedPath> <Str value="245" type="xs:string"/> </CmpG> </IterStep> <IterStep axis="child" test="marc:subfield"> <CmpG op="="> <CachedPath> <IterStep axis="attribute" test="code"/> </CachedPath> <Str value="a" type="xs:string"/> </CmpG> </IterStep> <FnString name="string([item])"/> </MixedPath> </CElem> </GFLWOR> </QueryPlan> Thanks in advance! Tim -- Tim A. Thompson Metadata Librarian (Spanish/Portuguese Specialty) Princeton University Library

2 4

Round tripping JSON
by Marc van Grootel 04 Aug '15

04 Aug '15

Hi, I am working on some REST API code and I'm struggling with POSTing of JSON. What I want to achieve is to have the function receive an XDM map (from JSON using fn:parse-json). I also want to output from such a map (in effect roundtripping some JSON). The test POST body I'm using is: {"foo": "bar"} When reading the docs I'm led to believe that this should achieve it: declare %rest:POST("{$body}") %rest:path("/json") %rest:consumes("application/json;lax=yes;format=map") %rest:produces("application/json;format=map") function api:json($body) { $body }; No matter what I do I always get this as response: <json type="object"> <foo>bar</foo> </json> Somehow I cannot force the function to receive a map(*) as $body argument. When I wrap $body in a trace() I see that it gets XML and not a map(*). What am I missing? Or do I really have to go JSON > XML > map {....}? -- --Marc

3 18

File permissions on .basex files
by Amanda Galtman 03 Aug '15

03 Aug '15

Hi, It seems that the commands and functions that operate on the .basex files, such as ADD and db:add(...), make the .basex files writable by the creator but not by anyone else. On the other hand, executing a query requires write permission for at least some of the .basex files, even if the query does not attempt to update the database. As a result, if I create a database and want anyone other than me to be able to use it (via the GUI, for example), I have to change permissions on the .basex files after creating the database. Apparently, I have to change the permissions again if I later update the database and rebuild the indexes. Do folks on this list have a slick way to keep the permissions as open as needed? I tried searching the list archive but didn't find anything that looked relevent. Would it be a reasonable enhancement request to make this a bit smoother? For example, an option could let you specify that the .basex files being created or modified should be writable by all. Alternatively or in addition, perhaps it is possible to stop requiring write permission for non-updating queries; I'm not familiar with the reason why write permission is needed. Thanks, Amanda

2 4

Slow query
by Menashè Eliezer 03 Aug '15

03 Aug '15

Hello, I wonder if the attached query can be optimised. I'm attaching all relevant information. Basex 7.9, Debian, powerful server. This is just an example. The queries will be built based on a compilation of a search form. So reordering the conditions for having smaller subset right from the begging isn't relevant. Any help would be appreciated. 40 seconds are not acceptable. -- With kind regards, Menashè

2 45

Applying indexes
by Menashè Eliezer 03 Aug '15

03 Aug '15

Hello, 1. In which cases the "applying index" phrase should I see in the query plan? I have an old query (the one I've sent here at 06/22/2015) where only for one condition I've seen "applying index", and even this one doesn't appear anymore using a newer (8.2.2) version of BaseX and a larger database. I do see the right properties of the db: Indexes Up-to-date: true TEXTINDEX: true ATTRINDEX: true 2. Only once a value which is mentioned in the query is found in the index, in case of "=" condition? 3. Always, in case of "<" or "<" condition, assuming the specific text/attribute has been indexed? 4. Everything I see as a reply to index:facets is indexed, right? -- With kind regards, Menashè

2 4

No debug info
by Menashè Eliezer 03 Aug '15

03 Aug '15

Hello, I don't know why I don't see anymore query log events like "admin OK QUERY" in the .logs folder. Not even when I use the -d flag. BaseX 8.2.2 -- With kind regards, Menashè

2 2

compiling and building basex-core with maven
by Rob Stapper 03 Aug '15

03 Aug '15

Hi Christian, I'm trying to compile and build the basex-core. All is going quit well except that one testrun is failing which results in not being able to build the basex-core. I started the process from within the basex-core-directory with the command: mvn -U clean install. After a zillion test almost as last the "org.basex.server.LockingTest" fails with the message: java.lang.AssertionError: Parallel execution expected at org.junit.Assert.fail<Assert.java:88> at org.junit.Assert.assertTrue<Assert.java:41> at org.junit.Assert.LockingTest.testQueries<LockingTest.java:125> at org.junit.Assert.LockingTest. LockingTest.<LockingTest.java:146> I'm running on a simple windows-system with windows vista home edition. No apache-serve running. Can you give me a hint to get the basex-core build? TIA. Rob PS. maven on Basex-test also give problems but basex-api and basex-examples works fine. --- Dit e-mailbericht is gecontroleerd op virussen met Avast antivirussoftware. https://www.avast.com/antivirus

3 3

Returning REST Response From Updating Function
by Eliot Kimber 02 Aug '15

02 Aug '15

I'm trying to understand how to use db:output() in the context of a REST function that does a bunch of stuff that updates and then wants to return a result. I have my updating functions using db:output() to return XML elements that are log entries, which I have up to now then formatted as HTML for return by the Web app. However, there doesn't appear to be a way to get the stuff returned by db:output before it gets returned to the ultimate caller. Have I missed something? My REST-handling function is declared as %updating: declare %updating %rest:path("/repo/{$repo}/{$branch}/updateLinkManagementIndexes") Which I understand to be a requirement if the function itself calls any updating functions. I tried e.g.: let $result := f:myUpdatingFunction() return db:output(f:formatLogItems($result)) But that results in the "no updating functions" message on the variable assignment. Is there a way to do what I want? Thanks, Eliot ---- Eliot Kimber, Owner Contrext, LLC http://contrext.com

2 4

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

BaseX-Talk August 2015