BaseX-Talk January 2015

basex-talk@mailman.uni-konstanz.de

36 participants
62 discussions

Optimization for procedure call ?
by jean-marc Mercier 14 Jan '15

14 Jan '15

Hello, I am facing some performances issues, probably due to unexpected query reevaluation. Might it be due to the optimizer ? Here is a code to test declare function local:test($i){ let $data := fn:for-each(1 to 10000000, function($a){2*$a} ) (: let $data := xquery:eval("fn:for-each(1 to 10000000, function($a){2*$a} )") :) return (1 to $i) ! local:test2($data,.) }; declare function local:test2($data, $dummy) { count($data) }; (1 to 10) ! prof:time(local:test(.)) Output : 683.76 ms 1137.95 ms 1727.68 ms 2151.06 ms 2694.84 ms 3189.57 ms 3725.1 ms 4277.51 ms 4815.74 ms 5417.79 ms Note that using xquery:eval (toggling the comment inside local:test). seems to perform correctly. However I can't use this trick as a workaround. (1 to 10) ! prof:time(local:test(.)) Ouput : 2472.71 ms 825.61 ms 3013.14 ms 986.81 ms 2982.97 ms 778.24 ms 1195.33 ms 3280.7 ms 976.96 ms 913.69 ms

2 6

O'Reilly: eXist. A NoSQL Document Database and Application Platform
by Christian Grün 13 Jan '15

13 Jan '15

Hi all, Our friends from eXist (or, more specifically, Adam Retter and Erik Siegel) have just published an O'Reilly book on the eXist database. It is available both on paper and as epub [1]. Have fun, Christian [1] http://shop.oreilly.com/product/0636920026525.do

1 0

User login
by Andy Bunce 13 Jan '15

13 Jan '15

Hi, Just investigating user handling, I am using the latest snapsnap. I start basexhttp with -U guest -P guest and/or I set org.basex.user and org.basex.password in web.xml Sometimes I have created a user ( guest with password guest) and permission none via dba. Sometimes I have deleted the guest user. In all cases I appear to be able to run restxq queries and in those queries read from databases. Am I misunderstanding something here? /Andy

2 5

Get Path physical of item in database
by nguyen my 13 Jan '15

13 Jan '15

I want to get the physical path of an item in database by xquery . but i don't known how. the path as: Users/Emotive/Documents/Open Test Framework/OtxProject16/ZZZ/OTX or target path as in BaseX Gui: "/NewDefaultPackage1/" this is xquery code. declare default element namespace 'http://iso.org/OTX/1.0.0'; declare namespace xsi = 'http://www.w3.org/2001/XMLSchema-instance'; let $node := collection()/otx[name="something"] how to get the physical path of $node ? or other xquery. Please help me.

2 1

Add an XML String
by Achim Baumgartner 12 Jan '15

12 Jan '15

Hello Christian, I am adding an XML String to the database in a Java Programm via Add (String target, String input); Now my problem is input contains some special characters like German ( ß) It is explained a couple of times that serialization controls Output but what about the input? Where can I set the input character encoding? My ß always turns into a character mess inside the database kind regards Achim Gesendet über Yahoo Mail für Android

2 1

The JSON serialization controversy
by Florent Gallaire 12 Jan '15

12 Jan '15

Whether my database this XML file: <?xml version="1.0" encoding="UTF-8"?> <mots> <mot> <nom>iatrogène</nom> <racine>gène</racine> </mot> <mot> <nom>hippiatre</nom> <racine>hippo</racine> </mot> </mots> The only easy way to have JSON serialization with BaseX is to use the "format=jsonml" option which returns: ["mots", ["mot", ["nom", "iatrogène"], ["racine", "gène"]], ["mot", ["nom", "hippiatre"], ["racine", "hippo"]]] This output is not at all what I need, and think I'm not the only one in this case. A quick search on Google give us simple online tools which have an useful output: http://codebeautify.org/xmltojson { "mots": { "mot": [ { "nom": "iatrogène", "racine": "gène" }, { "nom": "hippiatre", "racine": "hippo" } ] } } http://www.utilities-online.info/xmltojson { "mots": { "mot": [ { "nom": "iatrogène", "racine": "gène" }, { "nom": "hippiatre", "racine": "hippo" } ] } } Could we have this "easyjson" option in BaseX ? I know that it has not at all the power of lossless conversion from JSON to XML and back, like the "direct" and "attributes" options, but who really cares ? I really think it's what most users need today to use BaseX with client side JSON oriented frameworks. Best regards Florent -- FLOSS Engineer & Lawyer

3 8

SQL memory error
by Erol Akarsu 12 Jan '15

12 Jan '15

Hi Christian, I have another issue with following code snippet. It works fine with small JDBC result rows returned from database. But if your sql query returns say 40 million rows, then you will get out of memory exception easily. I guess Basex first read whole 40 million rows into memory first, then it will allow user to use sql data. Is there a way for basex to read sql rows block by block (say 1000 by 1000) and write into a file? I appreciate your help. declare variable $sqlfile as xs:string external; declare variable $outputfile as xs:string external; let $lines := file:read-text-lines ($sqlfile) let $lines := for $line in $lines return fn:normalize-space($line) let $sqlStmt := fn:string-join($lines," ") let $addBegin := file:append-text($outputfile,"<RECORDS>","UTF-8") let $url := "jdbc:oracle:thin:" let $conn := sql:connect($url) let $res := sql:execute ($conn,$sqlStmt) let $list := for $rec in $res let $is := for $line in $rec/* return element {fn:data($line/@name)} {$line/text()} return file:append( $outputfile, <record>{$is}</record>) return ($list,file:append-text($outputfile,"</RECORDS>","UTF-8")) On Tue, Jan 6, 2015 at 4:22 AM, Christian Grün <christian.gruen(a)gmail.com> wrote: > Hi Erol, > > > I have several xml document index keeping products features and have one > > master index file of products.. Currently, we have about 25 million > products > > that have size about 25G xquery index size. > > One solution is to generate a BaseX command script, which contains > multiple subsequent update operations. It can e.g. be created as > follows: > > let $records := count(db:open('db')//product-node) > let $commands := > for tumbling window $w in (1 to $records) > start at $s when true() > only end at $e when $e - $s = 999999 > return 'for $product-node in //product-node' || > '[position() = ' || $s || ' to ' || $e || '] ' || > 'return insert node <a/> into $product-node' > return file:write-text-lines('script.bxs', $commands) > > ..and run via "basex script.bxs". > > Hope this helps, > Christian > > > > > > My xquery application is reading features data from auxiliary xml > document > > and enhancing master document. > > > > In order to do it, I am inserting data into master document with a > generic > > insert command: > > > > insert node <FEATURES_NODE> into <PRODUCT_NODE> > > > > But xquery update is executing update process after all insert requests > > collected. But with huge size of my index, the memory of machine (35G) is > > exhausted easily and can NOT finish process. > > > > Is there way to execute individual xquery update requests one by one so > that > > process can finish? > > > > Erol Akarsu > > > > >

2 8

Update: BaseX Server Down
by Alex R. Loayza 11 Jan '15

11 Jan '15

Hello again, Regarding my mail of a few hours ago, I'd like to tell you that it turns out that BaseX Server and Jetty are actually working, despite the misleading message "Can't communicate with the server". CORS requests are also working. It remains a mystery why the BaseX Server command line window keeps posting such message. Wrong error messages are very confusing, indeed (as I happened to find today). Cordially, Alex

2 1

BaseX Server down
by Alex R. Loayza 10 Jan '15

10 Jan '15

Hello, I tried toenable Jetty to accepts CORS requests by adding to web.xml, the followinglines: <filter> <filter-name>cross-origin</filter-name> <filter-class>org.eclipse.jetty.servlets.CrossOriginFilter</filter-class> <init-param> <param-name>allowedOrigins</param-name> <param-value></param-value> </init-param> <init-param> <param-name>allowedMethods</param-name> <param-value></param-value> </init-param> <init-param> <param-name>allowedHeaders</param-name> <param-value></param-value> </init-param></filter><filter-mapping> <filter-name>cross-origin</filter-name> <url-pattern>/*</url-pattern></filter-mapping> However,now when I try to start BaseX 7.9 Server, I only get a message “Can’t communicatewith the server”. The message persists even though I removed the added lines toweb.xml and restarted my PC (I’m on Win8). Could youplease give me some light on how can I get BaseX Server to work again? And, howcan I enable BaseX/Jetty Server for CORS requests? Thank youvery much for any help. Regards, Alex

1 0

Performance issue due to evaluation during function composition ?
by jean-marc Mercier 09 Jan '15

09 Jan '15

Hello (and happy new year 2015 !) I am facing some performance issues opening "big" databases due to an (unexpected for me) argument evaluation during function composition. Is this a normal behavior ? Cheers Here is a code description : declare variable $db := db:open("MyBigDataBase"); declare function local:elts($db){$db//element()}; declare function local:compose($f1 as function(*), $f2 as function(*)) as function(*){ function($a) { $f1($f2($a)) }}; declare function local:count_elt($db) {count(local:elts($db))}; prof:time(local:count_elt($db)), prof:time(count(local:elts($db))), prof:time(local:compose(count#1,local:elts#1) ($db)) ouput : 3.1 ms 0.02 ms 100047.53 ms 134666491 134666491 134666491

3 8

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

BaseX-Talk January 2015