- BaseX-Talk - mailman.uni-konstanz.de

xquery update impact on server?
by Pascal Heus 09 Nov '11

09 Nov '11

All: I'm running an long "xquery delete node" on one of my database and this pretty much seem to make my server unavailable for other operations. I expect this query to run for a few hours (affecting a few million nodes, database is around 800Mb). I'm running this on a Amazon EC2 small instance (1 CPU, 1.7Gb RAM) and the CPU has been stuck at 100%. If I attempt to query any database on the server (not only the one being updated), I basically don't get anything back. So do heavy xquery update requests basically lock down the server? Anyone else having similar experience? Feedback/suggestions? best *P

4 5

Bugreport
by Hillert, Martin, le-tex 08 Nov '11

08 Nov '11

Ziel der Query ist es, nach der Manipulation der DB selbige zu optimieren. Query: let $w := (//work)[position() = 17]//impression[1] return ( if ($w/@has_archive_data) then delete node $w/@has_archive_data else (), db:optimize("index") ) Error: Potential bug? Improper use? Your feedback is welcome: Contact: basex-talk(a)mailman.uni-konstanz.de Version: BaseX 7.0.1 Java: Sun Microsystems Inc., 1.6.0_07 OS: Linux, i386 Stack Trace: java.lang.ArrayIndexOutOfBoundsException Beste Grüße, Martin Hillert -- Martin Hillert Senior Analyst Enterprise Applications le-tex publishing services GmbH Weissenfelser Str. 84, 04229 Leipzig, Germany Phone +49 341 355356 130, Fax +49 341 355356 530 martin.hillert(a)le-tex.de, http://www.le-tex.de Registergericht / Commercial Register: Amtsgericht Leipzig Registernummer / Registration Number: HRB 24930 Geschäftsführer: Gerrit Imsieke, Svea Jelonek, Thomas Schmidt, Dr. Reinhard Vöckler

3 3

Re: [basex-talk] BaseX 7.0.1: Release early, release often..
by Ralf Jung 07 Nov '11

07 Nov '11

Hi, > Yep, that was not necessary. Here is a version, which I hope works > correctly: That's looking great, I just added a missing " " around the last $FILE. FILE="${BASH_SOURCE[0]}" while [ -h "$FILE" ] ; do SOURCE="$(readlink "$FILE")" FILE="$( cd -P "$(dirname "$FILE")" && \ cd -P "$(dirname "$SOURCE")" && pwd )/$(basename "$SOURCE")" done BASEX_DIR="$( cd -P "$(dirname "$FILE")" && pwd )" Of course, in the remainder of the script, $PWD has to be replaced by BASEX_DIR... or, to make things slightly more readable, BASEX_DIR could be the actual dir with the main jar, not the bin dir. While I was at it, I also fixed the "list all jars in lib/" part to work properly if the path basex lives in contains a ' ' (which would previously have been replaced by a ':'). You can find the result attached. It works here, but should be tested on some different setups :D Kind regards, Ralf

1 0

Re: [basex-talk] very common attribute values/custom indexing
by Christian Grün 07 Nov '11

07 Nov '11

Ross, > The DSI Utilities (http://dsiutils.dsi.unimi.it/) are a very> interesting set of classes. Thanks; I'll soon have a look at that package. Just some quick feedback, regarding the extension of index entries: we'd get even better results by extending the index entries with id's to the unique location paths.. e.g.: <a> <b>dig</b> <c>dig</c> <c>diz</c> </a> Path Index: 0: a/b 1: a/c Value Index: dig.0 -> /a/b/text() dig.1 -> /a/c/text()[.='bla'] diz.1 -> /a/c/text()[.='blu'] This way, we could completely give up path inversions, at least in a number of trivial cases. In both cases, I guess that most effort has to be put into the adaptation of the query optimizer. Volunteers are welcome.. ;) Christian___________________________ On Wed, Nov 2, 2011 at 11:29 PM, Ross Judson <rossjudson(a)gmail.com> wrote: > You guys are wizards at making the JVM run fast, without a doubt. I > haven't looked to see how namespaces are handled right now, within the > indexing scheme. In my particular case I do not have namespaces, but > clearly they need to be handled, generally. > > The DSI Utilities (http://dsiutils.dsi.unimi.it/) are a very > interesting set of classes. I have previously used these to construct > indexes of RDF-type triples. They are _extremely_ high performance, > able to process at average rates of hundreds of thousands of tuples > per second. > > The ImmutableExternalPrefixMap was very useful. I created compound > strings (subject-property-object) and stored them. I could then do a > very fast lookup on any prefix, like subject, or subject-property. The > prefix map would supply the applicable interval. By creating another > index compounding (object-property-subject), I could lookup all usages > of a given property value (object), restricting by property name, or > all the way to a specific subject. > > The ImmutableExternalPrefixMap uses an in-memory structure to compute > approximate intervals on any given query; it then reads disk to refine > and answer the query. The net result is that you generally have access > to the results of any given query in one sector read (more or less > like a hash structure), but the data is _ordered_, which means you can > perform range queries, at the cost of reading additional sectors as > necessary. > > Within BaseX you have already converted tokens to byte arrays. > ImmutableExternalPrefixMap does this as well. > > A useful experiment would be to add a new type of index to BaseX that > incorporates one of these prefix maps. > > Creating the compound index looks something like this (altering > ValueBuilder.build()). Replace: > > index.index(data.text(pre, text), pre); > > with: > > byte [] attrName = data.name(pre, Data.ATTR); > int parentElement = data.parent(pre, Data.ATTR); > byte [] elementName = data.name(parentElement, Data.ELEM); > TokenBuilder tb = new TokenBuilder(data.text(pre, text)); > tb.add(';'); > tb.add(attrName); > tb.add(';'); > tb.add(elementName); > index.index(tb.finish(), pre); > > Not sure if I have all of that right, but it shows the idea. When I > run with this the indexes on my data do expand a little, but not too > much (may a 10% growth?). Of course, it will be different for > everyone, depending on their content. > > By checking the database properties you can see that the index has > more keys in it. Next step would be to add a prefix-scanning mode to > the index iteration, in addition to equality scanning. In the example > above I use the ';' character to separate the composite token; (byte)0 > might be more appropriate.

2 1

Error: "xbug80.dtd" could not be parsed.
by Florent Georges 06 Nov '11

06 Nov '11

Hi, When using the BaseX-7.0.2pre-20111030 GUI, and try to create a new database, I fill in the form with the name and location, then press OK. Some files seem to be created, then after a few seconds I get the following error: "TestInputs/bug/bug80.xml" (Line 3): "xbug80.dtd" could not be parsed. I don't know what it means in this context. Any idea? Regards, -- Florent Georges http://fgeorges.org/ http://h2oconsulting.be/

2 7

BaseX 7.0.1: Release early, release often..
by Christian Grün 06 Nov '11

06 Nov '11

Hi all, we are glad to announce BaseX 7.0.1, which is basically the result of your initial feedback on 7.0. Those are the most important changes: DISTRIBUTIONS: - Windows installer was updated to support latest features - ZIP file was updated (initial config & directories added) - Short directory names are chosen if config file resides in app.dir. - Start scripts have been improved XQUERY: - much faster execution of count() when applied to opened databases SERVER (http://docs.basex.org/wiki/Startup_Options#BaseX_HTTP_Server) - Flag -c connects to an existing database server - Flag -s specifies a port for stopping the HTTP server (Jetty) - Flag -S starts the HTTP server as a service - running write operations will be completed before server is stopped API: - Ruby, Python, PHP, Java: clients updated Please check out the well-known links: - Homepage: http://basex.org/ - Documentation: http://docs.basex.org/ - Snapshots: http://files.basex.org/releases/latest/ - Sources: https://github.com/BaseXdb Enjoy the update! Christian BaseX Team

5 17

Simple Xquery '(/org)[1 to 5]' was failed in GUI
by 方立厅 05 Nov '11

05 Nov '11

Hi Michael, Thanks for your replying. So this BaseX does not compatible to XQuery 1.0 by this feature? "(/org)[$start to $end]" query excuted well in marklogic and bdbxml. And I need a solution to make my existing xquery modules could be exuted in BaseX without modifying the codes. Any suggestion? Thank you, Edward >> Dear Edward, >> >> the correct syntax is: >> (2 to 100)[position() = 5 to 20] > (sequence)[5] is only a shorthand for (sequence*)[position() eq 5] wher 'eq' compares two _single_ items for equality. > > With [positon() = 5 to 20] we perform a general comparison, such that the predicate evaluates to true if comparison of any of the items inside both sequences > evaluates to true. > Thus position() = (5,6,7,?,20) evaluates to true if position() is between 5 and 20, while > position() eq (5 to 20) throws an error, as eq is not allowed to compare single items with sequences. > > Hope this helped clear things up a little, s/o please correct me if my explanation was faulty :-) > > Thanks > Michael > Am 05.11.2011 um 03:54 schrieb ???: > >> Hi, >> I'm using BaseX first time. and using 7.0.1. >> about 5000 org.xml was loaded into the db in my test enviroment. >> >> But I got an error msg instead of the result i want when I was excuting an simple xquery '(/org)[1 to 5]' in GUI: >> Query: (/org)[1 to 5] >> Error: [FORG0006] (1 to 5) not allowed as condition type. >> >> The xquery '(/org)[position()> 2 and position()< 5]' excuted very well. >> >> Thanks, >> Edward >> >> _______________________________________________ >> BaseX-Talk mailing list >> BaseX-Talk(a)mailman.uni-konstanz.de >> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk

3 2

Simple Xquery '(/org)[1 to 5]' was failed in GUI
by 方立厅 05 Nov '11

05 Nov '11

Hi, I'm using BaseX first time. and using 7.0.1. about 5000 org.xml was loaded into the db in my test enviroment. But I got an error msg instead of the result i want when I was excuting an simple xquery '(/org)[1 to 5]' in GUI: Query: (/org)[1 to 5] Error: [FORG0006] (1 to 5) not allowed as condition type. The xquery '(/org)[position() > 2 and position() < 5]' excuted very well. Thanks, Edward

2 1

Re: [basex-talk] Importing a document using xquery statements?
by Lukas Kircher 04 Nov '11

04 Nov '11

So far there is no XQuery function that allows you to create or delete a database. I remember this being a conscious design decision, as creating/deleting a database requires CREATE permissions, but I might be wrong... Maybe one of my team mates can help me out here? Cheers, Lukas On Fri, Nov 4, 2011 at 10:22 AM, Erdal Karaca < erdal.karaca.de(a)googlemail.com> wrote: > Thanks for the quick reply! > Unfortunately, this function needs an existing database. I could not > find a function like db:create or the like... How would i create the > database, first? > > > 2011/11/4, Lukas Kircher <lukaskircher1(a)googlemail.com>: > > Hi Erdal, > > > > you could have a look at our BaseX-specific database functions - > > db:add() [1] might be what you are looking for. > > > > Hope I could help, regards, > > Lukas > > > > > > [1] http://docs.basex.org/wiki/Database_Functions#db:add > > > > > > > > On Fri, Nov 4, 2011 at 9:54 AM, Erdal Karaca < > erdal.karaca.de(a)googlemail.com > >> wrote: > > > >> I have set up eclipse XQDT to use basex as interpreter... Now, I need > >> to import a document into a database. Is that possible without using > >> the server commands in a command line? For example some basex specific > >> functions? > >> _______________________________________________ > >> BaseX-Talk mailing list > >> BaseX-Talk(a)mailman.uni-konstanz.de > >> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk > >> > > > > > -- > 1 + 1 = 10 >

1 0

Importing a document using xquery statements?
by Erdal Karaca 04 Nov '11

04 Nov '11

I have set up eclipse XQDT to use basex as interpreter... Now, I need to import a document into a database. Is that possible without using the server commands in a command line? For example some basex specific functions?

2 1