January 2013 - BaseX-Talk - mailman.uni-konstanz.de

Database locking pre-release published
by Jens Erat 15 Jan '13

15 Jan '13

Dear BaseX community, we're glad to announce some pre-release-version of a much-requested and long-awaited feature: database locking. Database locking will enable higher parallelism, it will be possibly to perform updates to some database without blocking all other databases (actually even parallel updates to different databases). Please notice this is a pre-release developer version which is not yet fully tested and optimized. We will announce as soon as we regard it ready for production. It is included in most current [development release]. Have a look at our [wiki documentation] for more information on limitations and [how to enable]. During the next weeks, we will continue to include more optimizations and reduce the number of queries requiring global locks and add fixes if necessary. We will be glad receiving your feedback! As I'll be writing my thesis on these locking enhancements, I'd also be glad on some user stories and requirements (which could be considered in prioritizing efforts on optimizations, too). You can also mail me directly if you do not want to post to the list. Regards from Lake Constance, Germany, Jens Erat [development release]: http://files.basex.org/releases/latest/ [wiki documentation]: http://docs.basex.org/wiki/Transaction_Management [how to enable]: http://docs.basex.org/wiki/Transaction_Management#How_to_enable -- Jens Erat [phone]: tel:+49-151-56961126 [mail]: mailto:email@jenserat.de [jabber]: xmpp:jabber@jenserat.de [web]: http://www.jenserat.de PGP: 350E D9B6 9ADC 2DED F5F2 8549 CBC2 613C D745 722B

1 0

Determining whether a function exists
by Charles Duffy 15 Jan '13

15 Jan '13

Howdy -- I'm trying to determine whether sql:execute-prepared exists, to have a module that functions in both BaseX 7.3 and 7.5. However -- exists(function-lookup(xs:QName("sql:execute-prepared"), 2)) returns false even in BaseX 7.5, even though function-name(function-lookup(xs:QName("sql:execute-prepared"), 2)) properly returns "sql:execute-prepared". Am I going to have to use a try block to detect this case?

2 3

256 namespaces limit
by Imsieke, Gerrit, le-tex 15 Jan '13

15 Jan '13

Dear Team, Do you plan on increasing the 256 namespaces limit any time soon? I know that there is the STRIPNS option, but this does not fit my use case (which is to index all my XSLT and XProc files) too well. Gerrit

3 2

unzip
by Jason Harrop 14 Jan '13

14 Jan '13

Hello eXist and MarkLogic both have mechanisms for unzipping a resource when it is loaded into the database. In eXist, you can use a trigger and its unzip module. For MarkLogic, you can use its Office OpenXML Extract pipeline. See http://robotbeerbash.wordpress.com/2009/08/31/getting-started-with-open-xml… What's the easiest way to do this sort of thing in basex? thanks .. Jason

4 7

Re: [basex-talk] best way to partition large data sets among collections
by pw＠themail.co.uk 14 Jan '13

14 Jan '13

Hi Christian Thank you for directing me to the profiling module, I think that is just what I need Cheers Peter > > > >---- Original Message ---- >From: christian.gruen(a)gmail.com >To: pw(a)themail.co.uk >Subject: Re: [basex-talk] best way to partition large data sets among >collections >Date: Mon, 14 Jan 2013 19:13:12 +0100 > >>Hi Peter, >> >>> Do you have any information to guide me here; what sorts of XQuery >>> expressions should I match with large numbers of collections, and >>> which with small numbers of collections? >> >>hmm, there is no answer I get in mind that could give you general >>guidance here, as XQuery provides just too many possibilities for >>writing slow and fast queries. Its similar to the question how to >>write efficient Java, Perl of whatever.. To do get more information >on >>why your queries are not as fast as they ideally should be, you >could >>e.g... >> >> check in the InfoView/query info if the relevant index structures >are used >> use functions of the Profiling Module [1] to track down >bottlenecks >> use -Xmx get do low level profiling >> >>Next, you may pass us on snippets of your code that may, in your >>opinion, be subject to being optimized. >>Christian >> >>[1] http://docs.basex.org/wiki/Profiling_Module >> >> >>> If I am using data-rich XML there is a high ration of nodes to >>> content. What are the rules of thumb for this type of content? >>> >>> Are there any recommendations specifically for GML? >>> >>> Many thanks >>> >>> Peter >>> >>>> >>>> >>>> >>>>---- Original Message ---- >>>>From: christian.gruen(a)gmail.com >>>>To: pw(a)themail.co.uk >>>>Subject: Re: [basex-talk] best way to partition large data sets >among >>>>collections >>>>Date: Mon, 14 Jan 2013 13:05:00 +0100 >>>> >>>>>Hi Peter, >>>>> >>>>>thanks for the link. Theres no general answer for your question, >as >>>>>an application may both run flawlessly with a single or hundreds >of >>>>>databases, depending on how your XQuery expressions look like. If >>>>you >>>>>do regular updates, I suggest to split your data into fixed >>>>instances >>>>>that will never change, and use all indexes, and updating >instances >>>>>that may eventually be merged with the fixed instances if no more >>>>>changes are expected. >>>>> >>>>>Christian >>>>>___________________________ >>>>> >>>>>On Sun, Jan 13, 2013 at 1:09 AM, <pw(a)themail.co.uk> wrote: >>>>>> Hello List >>>>>> >>>>>> I am experimenting with statistical data ( >>>>>> http://www.semantechs.co.uk/ ) and found that organising 2.5Gb >of >>>>xml >>>>>> data into 12 unevenly sized collections ranging from 40 to >400Mb >>>>>> performs much more slowly than 36 collections each containing >>>>>> approximately 75Mb of data. >>>>>> >>>>>> What rules of thumb are there to guide me in designing the most >>>>>> performant database? >>>>>> >>>>>> Many thanks >>>>>> >>>>>> Peter >>>>>> >>>>>> _______________________________________________ >>>>>> BaseX-Talk mailing list >>>>>> BaseX-Talk(a)mailman.uni-konstanz.de >>>>>> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >>>>> >>> >>> >>

1 0

Re: [basex-talk] best way to partition large data sets among collections
by pw＠themail.co.uk 14 Jan '13

14 Jan '13

Hi Christian Do you have any information to guide me here; what sorts of XQuery expressions should I match with large numbers of collections, and which with small numbers of collections? If I am using data-rich XML there is a high ration of nodes to content. What are the rules of thumb for this type of content? Are there any recommendations specifically for GML? Many thanks Peter > > > >---- Original Message ---- >From: christian.gruen(a)gmail.com >To: pw(a)themail.co.uk >Subject: Re: [basex-talk] best way to partition large data sets among >collections >Date: Mon, 14 Jan 2013 13:05:00 +0100 > >>Hi Peter, >> >>thanks for the link. Theres no general answer for your question, as >>an application may both run flawlessly with a single or hundreds of >>databases, depending on how your XQuery expressions look like. If >you >>do regular updates, I suggest to split your data into fixed >instances >>that will never change, and use all indexes, and updating instances >>that may eventually be merged with the fixed instances if no more >>changes are expected. >> >>Christian >>___________________________ >> >>On Sun, Jan 13, 2013 at 1:09 AM, <pw(a)themail.co.uk> wrote: >>> Hello List >>> >>> I am experimenting with statistical data ( >>> http://www.semantechs.co.uk/ ) and found that organising 2.5Gb of >xml >>> data into 12 unevenly sized collections ranging from 40 to 400Mb >>> performs much more slowly than 36 collections each containing >>> approximately 75Mb of data. >>> >>> What rules of thumb are there to guide me in designing the most >>> performant database? >>> >>> Many thanks >>> >>> Peter >>> >>> _______________________________________________ >>> BaseX-Talk mailing list >>> BaseX-Talk(a)mailman.uni-konstanz.de >>> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >>

2 1

7.5.1 beta XMLDB api issues
by Jason Harrop 14 Jan '13

14 Jan '13

https://github.com/BaseXdb/basex-examples/blob/master/src/main/java/org/bas… is not working for me. I'm using source code from github (7.5.1 beta). Is it just me? Also, any update on the status of March 2011 regarding the questions: Is it possible to create child Collections within a database through the XMLDB API? And is it possible to insert XMLResources in a child Collection through the XMLDB API? thanks .. Jason XMLDBInsert returns: === XMLDBInsert === * Get collection. * Create new resource. Exception in thread "main" java.lang.NullPointerException at com.plutext.basex.samples.XMLDBInsert.main(XMLDBInsert.java:52) ============== [basex-talk] xmldb api and child collections Andreas Weiler andreas.weiler at uni-konstanz.de Sun Mar 27 14:10:31 CEST 2011 Previous message: [basex-talk] xmldb api and child collections Next message: [basex-talk] BaseX 6.6 Update - Soft-Files.com Messages sorted by: [ date ] [ thread ] [ subject ] [ author ] Sorry, it seems that this is not supported right now. We will have a look into this. In the meantime, you could use our own API for your scenario. Kind regards, Andreas Am 26.03.2011 um 17:59 schrieb frans at planet: > Is it possible to create child Collections within a database through the XMLDB API? And is it possible to insert XMLResources in a child Collection through the XMLDB API? > > Regards, > Frans > > _______________________________________________ > BaseX-Talk mailing list > BaseX-Talk at mailman.uni-konstanz.de > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk

2 1

best way to partition large data sets among collections
by pw＠themail.co.uk 14 Jan '13

14 Jan '13

Hello List I am experimenting with statistical data ( http://www.semantechs.co.uk/ ) and found that organising 2.5Gb of xml data into 12 unevenly sized collections ranging from 40 to 400Mb performs much more slowly than 36 collections each containing approximately 75Mb of data. What rules of thumb are there to guide me in designing the most performant database? Many thanks Peter

2 1

[BXSQ0002] calling sql:execute() in BaseX 7.5.1 snapshot
by Charles Duffy 14 Jan '13

14 Jan '13

Howdy -- I have the following code, which worked perfectly with BaseX 7.3: declare function sqlimport:load-sql-source-raw($config as element(config), $sql_source as element(sql-source)) { let $handle := sql:connect( concat($sql_source/@uri, "/", $sql_source/@schema/string()), $sql_source/@username, $sql_source/@password) let $all_column_names := sql:execute( sql:prepare( $handle, "SELECT COLUMN_NAME FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_SCHEMA=? AND TABLE_NAME=?"), <sql:parameters> <sql:parameter type='string'>{$sql_source/@schema/string()}</sql:parameter> <sql:parameter type='string'>{$sql_source/@table/string()}</sql:parameter> </sql:parameters> )/sql:column[@name="COLUMN_NAME"] let $column_names := $all_column_names/text()[not(.=$sql_source/ignore-column/text())] return sql:execute($handle, concat("SELECT ", string-join($column_names, ", "), " FROM ", $sql_source/@table/string())) }; With BaseX 7.5 and current snapshots of 7.5.1, however, it yields the following error: [BXSQ0002] No opened connection with id 1 ...positioned at the sql:execute() call. I would understand sql:connect() failing -- but having the connection appear to take place without errors, followed by a failure with the execution, is surprising. Guidance?

2 1

BaseX Users Meetup, XMLPrague
by Christian Grün 14 Jan '13

14 Jan '13

Dear all, we've now assigned two slots for our Friday's Users Meetup in Prague. The pre-conference schedule is online: http://www.xmlprague.cz/pre-conference-friday/ We are still interested in your presentations, live demos, etc.. Give us a note! Next, you can still register for the conference, which is really budget-friendly, compared to other events of that kind. See you soon, Christian BaseX Team 2013

1 0