- BaseX-Talk - mailman.uni-konstanz.de

Subject: Re: BaseX request from Saxon XSL transformation
by ski n 06 Jan '16

06 Jan '16

Hi Christophe, It's possible to query Basex directly from XSLT. However this can be cumbersome because you need a lot of escapes to correctly execute the query. If you only need to query a relative small XML you can avoid this by getting by getting the complete XML and use a normal xpath in the xslt (in stead of xquery): 1) First create a function in a separate XSLT: <?xml version="1.0"?> <xsl:stylesheet version="2.0" exclude-result-prefixes="xs basex saxon" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:functx="http://www.functx.com" xmlns:org="http://www.example.org" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:saxon="http://saxon.sf.net/" xmlns:basex="java:org.basex.server.ClientSession"> <xsl:function name="org:getBasexDoc"> <xsl:param name="database"/> <xsl:variable name="session" select="basex:new('localhost', 1984, 'admin','admin')"/> <xsl:variable name="query"> <xsl:text>XQUERY for $doc in db:open('</xsl:text><xsl:value-of select="$database"/><xsl:text>') return $doc</xsl:text> </xsl:variable> <xsl:variable name="result"> <xsl:value-of select="basex:execute($session,string($query))"/> </xsl:variable> <xsl:copy-of select="saxon:parse($result)"/> </xsl:function> </xsl:stylesheet> 2) Include the stylesheet in you main stylesheet: xsl:include href="basex.xsl"/> 3) Set the XML document in a variable <xsl:variable name="dbname"> <xsl:copy-of select="drs:getBasexDoc('dbname')"/> </xsl:variable> 4) Query the document with xpath <xsl:value-of select="$dbname//xpath"/> See also: https://mailman.uni-konstanz.de/pipermail/basex-talk/2013-March/004776.html Raymond

1 0

Error Java ArrayIndexOutOfBoundsException
by kleist 06 Jan '16

06 Jan '16

Hello again, I got the following error: Error: Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk(a)mailman.uni-konstanz.de Version: BaseX 8.3.1 Java: Oracle Corporation, 1.8.0_45 OS: Mac OS X, x86_64 Stack Trace: java.lang.ArrayIndexOutOfBoundsException after running: xquery version "3.0"; declare namespace tei = "http://www.tei-c.org/ns/1.0"; db:create('kleist_index', for $item in db:open('kleist-data') return $item update delete node .//(tei:note|tei:rdg|tei:lb|tei:del|tei:reg|tei:sic), db:open('kleist-data')/db:path(.)) This error only appears on my second computer (both MacBooks). The one, which works, runs java version "1.8.0_31" Java(TM) SE Runtime Environment (build 1.8.0_31-b13) Java HotSpot(TM) 64-Bit Server VM (build 25.31-b07, mixed mode) the other, where my xq is not working java version "1.8.0_45" (Java 8-Update 66). Any ideas, what's the problem? Best, Günter

2 1

XQuery Update: remove-elements-not-contents
by kleist 06 Jan '16

06 Jan '16

Hi Christian, hello all, to write a new index to get better results out of mostly mixed-content elements, I have to delete some elements: db:create('index', for $item in db:open('data') return $item update delete node .//(note|sic), db:open('data')/db:path(.)) After that I have to remove some elements, but not the content, like the FunctX-function functx:remove-elements-not-contents does (http://www.xqueryfunctions.com/xq/functx_remove-elements-not-contents.html) to get <p>This is only an illustrative example, not a real text.</p> from <p>This is only an illustrative <hi>example</hi>, not a <q>real</q> text.</p> Is there any way, to do it with the help of BaseX XQuery Update? Best regards, Günter

2 1

XQery Performance-Problem when using variables
by kleist 05 Jan '16

05 Jan '16

Dear members, dear Christian, when using a for-loop inside of a for-loop I run into serious performance-problems, when I'm using variables to return results. For example this code-line takes 36780ms (!!!) for evaluation: return <li><a>{$title}</a>{$quote}</li> When I substitute the variable $time with the defining code, it takes only 711ms for evaluation (!!): <li><a> { if ($ele/ancestor::tei:TEI//tei:titleStmt/tei:title[@type="short"]) then $ele/ancestor::tei:TEI//tei:titleStmt/tei:title[@type="short"]/string() else if ($ele/ancestor::tei:TEI//tei:titleStmt/tei:title[@type="article"]) then $ele/ancestor::tei:TEI//tei:titleStmt/tei:title[@type="article"]/string() else if ($ele/ancestor::tei:TEI//tei:titleStmt/tei:title[@type="main"]) then $ele/ancestor::tei:TEI//tei:titleStmt/tei:title[@type="main"]/string() else ()} </a>{$quote}</li> The only difference in the code is using a variable and it differs in performance by a factor of more than 50. In my real application (using more variables) it differs by factor more than 150. I tried to simulate the problem with a simple factbook-query. Here is the code: xquery version "3.0"; let $query := "Pa.*" for $city in doc('factbook')//city/name for $hits in ft:mark($city[.//text() contains text {$query} using wildcards]) let $country_of_city := $city/ancestor::country/name return (: fast version: Evaluating 12.54ms :) <hit><city>{$hits}</city><country>{$city/ancestor::country/name}</country></hit> (: slow version: Evaluating 35.78 :) (: <hit><city>{$hits}</city><country>{$country_of_city}</country></hit> :) Also here is a performance bottleneck (not as dramatically: factor 3). Is there any solution to the problem? Now I'm working in my code without variables in the result part of the xquery, but it makes realy ugly non maintainable code. Any help would be appreciated. - Günter

2 8

Guidance on Indexing
by Mansi Sheth 05 Jan '16

05 Jan '16

Hello, A very happy new year to all of you !!! I have some very basic questions with indexing. 1. Most of my xqueries are of below nature '/Archives/descendant::apiCalls[contains(@name,"com.sun")]/@name', where apiCalls could be 3-4 level under 'Archives'. Xqueries are accessed via REST Based on this, I used attribute indexing, after each update to DB. Am I correct ? Should I have been using fulltext indexing instead ? Why ? 2. I have 1000s of documents, spanning over 100 XML DB, with total space around 400 GB currently. Each query is taking roughly 30 mins, to run. Though expectable performance, but I know I can do better with indexing. Currently, when I looked at one of the DBs, > open bi_output_3 Database 'bi_output_3' was opened in 38.22 ms. > info db Database Properties Name: bi_output_3 Size: 3938 MB Nodes: 16193129 Documents: 35 Binaries: 0 Timestamp: 2016-01-03T13:40:40.000Z Resource Properties Timestamp: 2016-01-03T13:40:40.776Z Encoding: UTF-8 CHOP: true Indexes Up-to-date: false TEXTINDEX: false ATTRINDEX: false FTINDEX: false LANGUAGE: English STEMMING: false CASESENS: false DIACRITICS: false STOPWORDS: UPDINDEX: false AUTOOPTIMIZE: false MAXCATS: 100 MAXLEN: 96 When looked at its HDD footprint: ubuntu@<abc>/BaseXDB/bi_output_3$ ls -l total 4032992 -rw-rw-r-- 1 ubuntu ubuntu 2209449064 Jan 1 17:00 atv.basex -rw-rw-r-- 1 ubuntu ubuntu 4 Jan 1 16:35 atvl.basex -rw-rw-r-- 1 ubuntu ubuntu 0 Jan 1 16:35 atvr.basex -rw-rw-r-- 1 ubuntu ubuntu 6414 Jan 3 13:40 doc.basex -rw-rw-r-- 1 ubuntu ubuntu 6 Jan 1 17:00 ftxx.basex -rw-rw-r-- 1 ubuntu ubuntu 0 Jan 1 17:00 ftxy.basex -rw-rw-r-- 1 ubuntu ubuntu 0 Jan 1 17:00 ftxz.basex -rw-rw-r-- 1 ubuntu ubuntu 829 Jan 3 13:40 inf.basex -rw-rw-r-- 1 ubuntu ubuntu 28 Jan 1 17:00 swl.basex -rw-rw-r-- 1 ubuntu ubuntu 1916444672 Jan 3 13:40 tbl.basex -rw-rw-r-- 1 ubuntu ubuntu 3796037 Jan 3 13:40 tbli.basex -rw-rw-r-- 1 ubuntu ubuntu 45462 Jan 1 17:00 txt.basex -rw-rw-r-- 1 ubuntu ubuntu 4 Jan 1 16:35 txtl.basex -rw-rw-r-- 1 ubuntu ubuntu 0 Jan 1 16:35 txtr.basex ubuntu@<abc>/BaseXDB/bi_output_3$ pwd /veracode/msheth/BaseXDB/bi_output_3 ubuntu@<abc>/BaseXDB/bi_output_3$ My concern is, at each DB update, I am using attribute indexing, but info command on basex prompt tells me otherwise. Am I misreading something ? Is there a way to fix this once DB is created ? Its takes me 48 hours, to create DBs from scratch... :) Reading thru UPDINDEX and AUTOOPTIMIZE ALL commands, tells me to open each DB and run these commands. Is that my option ? Do we have a xquery script somewhere which I can use to do this ? Thanks, - Mansi

4 14

ft:search and map-function
by kleist 05 Jan '16

05 Jan '16

Hi all, in the documentation for the ft:search-function is the following example: ft:search("db", ("A", "B"), map { "mode": "all words", "distance": { "max": "5", "unit": "words" } }) It's throwing this error for line 3: "Invalid value, simple expression expected." I'm using BaseX Version 8.3.1. Did the notation change or what's going wrong? Best regards, Günter

2 1

GUI: Horizontal splitting needed
by Alex R. Loayza 05 Jan '16

05 Jan '16

Hello, I'd like to suggest an improvement to the GUI: the ability to split the window horizontally, as you can do in IDEA or Visual Studio. I'm finding myself all too frequently scrolling up and down between functions or variables. After a while, it wears you out. A horizontal split would make BaseX's already great GUI, even better. Hope you agree and we can find it in BaseX 8.4! Best regards, Alex

2 1

Problems with option XINCLUDE
by Hans-Juergen Rennau 04 Jan '16

04 Jan '16

Dear BaseX team, I face unexpected problem when trying to switch off xinclude processing. From the documentation I understand that what I have to do is set the option XINCLUDE to the value "false". But how? Attempt #1========= I added to the config file .basex the line: XINCLUDE = false Result: no effect, and these messages: C:/Program Files (x86)/BaseX/.basex: Unknown option 'XINCLUDE'. C:/Program Files (x86)/BaseX/.basex: writing new configuration file. Did I misunderstand the documentation? Attempt #2======== I inserted into the query this initial line:declare option db:XINCLUDE 'false'; Result: Program crash with this message:[FODC0002] "file:///C:/projects/otds/otds-config/h.xml" (Line 1): Vorzeitiges Dateiende. where the named file (h.xml) is the output (!) file specified either like this:basex foo.xq > h.xmlor like this:basex -o h.xml foo.xq Note. Using neither > ... nor -o ..., I do get the correct output, written to stdout. But I must capture it, and the capturing fails! Attempt #3======== Patching the basex.bat file, inserting option -D: java -Dorg.basex.XINCLUDE=false -cp "%CP%" %BASEX_JVM% org.basex.BaseX %* Result:Same as attempt #2 So I'm at a loss. I need to switch off xinclude because my input files use xinclude in combination with xpointer attributes, like this: <xi:include href="../../../globals/notifications.xml" xpointer="xpointer(/config/*)" parse="xml" /> Applying doc(...) to such a file, yields this result: [FODC0002] "file:///C:/projects/otds/otds-config/branches/path/to/bar.xml" (Line 11): "include" mit href "../../../globals/notifications.xml" nicht erfolgreich, und es wurde kein "fallback"-Element gefunden. Can you help me? Hans-Jürgen PS: Using version:BaseX 8.3 beta 474994c [Standalone]

2 2

AWS Deployment, .war-file writing
by kleist 04 Jan '16

04 Jan '16

Hi all, I want to deploy my baxex-App to Amazon Web Services (AWS). Therefore I need a .war file of my existing application, not only the war-file of basex. How can I do it? There is the excellent tutorial of Paul Swennenhuis about Deployment on AWS (Part II). But it's a little bit outdated. AWS is now using tomcat 8 and it is no more stored at /var/lib/tomcat... Does anybody know, where I can find the new root-directory? Thanks for any help, Günter

1 0

Ignore option for full-text-search
by kleist 04 Jan '16

04 Jan '16

Hi Christian, my long-time project (kleist-digital.de) is nearly only based on documents (TEI-xml), so I'm working mostly with mixed content. Are there any plans for future versions to implement the ignore option for full-text-search? It would help a lot. Best, Günter

2 2