January 2013 - BaseX-Talk - mailman.uni-konstanz.de

XSD Validation
by Seenivasan Gunabalan 30 Jan '13

30 Jan '13

Hi all, I have two questions regarding XSD validation, 1. resolving paths while using imported XSDs 2. error reporting 1. resolving paths while using imported XSDs consider this example let $doc := <a /> let $schema := doc("../xsd/XDW.xsd") return validate:xsd($doc,$schema) this returns an error "[BXVA0001] Validation failed: src-resolve: Cannot resolve the name 'cda:NullFlavor' to a(n) 'type definition' component.". This is because the XDW.xsd schema imports two more XSD's with the following import statements that uses relative paths to refer to schemas that reside in the same folder as the root XDW.xsd. <xs:import namespace="http://docs.oasis-open.org/ns/bpel4people/ws-humantask/types/200803" schemaLocation="ws-humantask-types.xsd"/> <xs:import namespace="urn:hl7-org:v3" schemaLocation="HW_POCD_MT000040.xsd"/> The problem occurs with both file and http based uris. The only way I have to make validations through imported schemas to work is by using absolute paths in the import statements which is very annoying/limiting especially when handling multiple deployments. I just wanted to know whether this is the expected behaviour in which case it'll require us to take countermeasures at application deployment level. 2. Error reporting Basex reports the first error while validating with xsd schema, while java uses to report the collection of errors that do not depend on each other. This latter beaviour allos a user to correct multiple bugs simultaneously befor redoing the validation. I understand that the choice to return an exception on validation failure complicates this but couldn't it be possible to return a more complex description of the validation result containing multiple error marks? For information, we use Basex 7.5 on Ubuntu 12.04. Thanks in advance, Seenivasan

3 3

Fwd: java hosting provider with BaseX
by marco fregonese 29 Jan '13

29 Jan '13

---------- Forwarded message ---------- From: marco fregonese <marcofregonese(a)gmail.com> Date: 2013/1/29 Subject: Re: [basex-talk] java hosting provider with BaseX To: Alexander Holupirek <alexander.holupirek(a)uni-konstanz.de> Hi Alexander, I'm developing a java dynamic web application with Tomcat and BaseX. So far I could set up BaseX on JVMHost: everything is OK. On Openshift I set up Tomcat and BaseX. Tomcat is OK, but when I try to run basex/bin/basexserver from shell after ssh logging into openshift, the console says "Server is running or permission was denied." I checked with the command "ps -A" but it looks like BaseX server is not running, so I think permission was denied. Do you have any idea to solve this? Thanks, Marco. 2012/11/28 Alexander Holupirek <alexander.holupirek(a)uni-konstanz.de> > Hi Marco, > > i could setup BaseX on > > • http://www.jvmhost.com/ > • https://openshift.redhat.com/app/ > • http://jelastic.com/ > > All the best, > Alex > > On 28.11.2012, at 10:31, marco fregonese wrote: > > > Hi there, > > I was wondering if there is any java web hosting provider supporting > BaseX. > > I couldn't find any so far. > > Thanks. > > Bye. >

2 1

concurrency within xquery
by Seenivasan Gunabalan 29 Jan '13

29 Jan '13

Hi all, Is there a way to control concurrency from inside xqueries? Though its possible to achieve concurrency through REST (and RESTXQ?) interface, I am looking for ways to declaratively ask an xquery to run subqueries in a concurrent way. Thanks in advance Seenivasan

5 4

Fwd: BaseX Performance Test - hitting a performance wall
by Meredith Gregory 29 Jan '13

29 Jan '13

Dear Christian, We've been using your experimental locking to improve performance of BaseX in our application. It definitely helps, but right behind this bottleneck is file-io blocking. i'd like to get a deeper understanding of how your file-io is organized and what your caching scheme looks like. What sort of file-io caching mechanism is in place now? What are your plans? Is there anything we can do to help that along? Best wishes, --greg ---------- Forwarded message ---------- From: Jason Klassen <Jason.Klassen(a)protegra.com> Date: Tue, Jan 22, 2013 at 9:35 AM Subject: BaseX Performance Test - hitting a performance wall ... I’ve run the performance tests with 7.5.1 servers using DB Locking and with varying levels of parallel reads. *Unfortunately there is no performance gain*. While the socket.io blocking is gone the disk io blocking right after impacts performance just as much. Results are:**** ** ** previous (8 reads)**** 32 reads**** 64 reads**** 192 reads**** first**** last**** first**** last**** first**** Last**** first**** last**** UAT 10 by 1000**** 8.76**** 14.16**** 8.79**** 12.99**** 9.12**** 12.73**** 8.4**** 12.9**** ** ** Running with 192 reads and causing 10 by 10 by 1000, 10 users by 10 concurrent searches on 300ms delay each, by an affiliate with 1000. (above is 10 users by 1 concurrent search by an affiliate with 1000)**** ** ** I’ve attached some screenshots showing what the basex jar is doing when the tests run. It seems only ~1-2 threads can read from disk at any given time. The red sections are all 3 common issues, TableDiskAccess.read1, TableDiskAccess.read2, DataAccess.readToken, the dumps are attached as well. ** ** ** ** Looking at the source for basex TableDiskAccess<https://github.com/BaseXdb/basex/blob/master/src/main/java/org/basex/io/ran…> and DataAccess<https://github.com/BaseXdb/basex/blob/master/src/main/java/org/basex/io/ran…>there is no file system caching. Every test is doing a buffer read on the exact same database even though it does not change from a write. So instead of basex realizing this, it just blindly goes to the disk each time. **** ** ** We’ll run some tests on a vm backed by ssd to see if that can resolve the disk access*. If it can’t, and without any kind of caching logic, this looks to be as far as we can go with basex performance and I’m not sure it will be enough.* Even if caching logic existed for read, I’m still concerned that many writes with basex will block it the same way as it doesn’t appear to do any log or page caching as most relational dbs do now to limit disk io impacts. Sharding the dbs across 16 servers would have helped for socket.io but it will not solve physical disk io. **** ** ** Jason**** ** ** *From:* Sasa Ivetic *Sent:* January-18-13 3:05 PM *...* ** ** Hi everyone,**** ** ** I’ve spent a few hours today writing a simple BaseX test (mimics the BaseXPersistTest in KVDB) in order to analyze performance of different machines we have available to us. Results are interesting, and demonstrate that the test is likely heavily I/O bound on our Production server hosted by Rackforce. I’ve attached two quick graphs I made from the results. “BaseX Test Results Chart.png” is a graph of total runtime based on specified parameters, with Y Axis being measured in ms (lower is better). “BaseX Test Results Chart – Queries Per Second” is a graph of total queries per second (higher is better).**** ** ** PARALLEL parameter is a BaseX Server setting indicating how many read/write processes can run concurrently**** T parameter is the number of client threads used to create read/write requests**** QPT parameter is the number of queries each thread runs**** ** ** Amazon EC2 is the High I/O instance, with 16 cores, and 1TB worth of SSD-backed storage**** My laptop is a Core i7 (4 cores w/ HT) with a 7200rpm drive**** Beta is our own internal VM**** Prod is the Rackforce VM, in their high performance cloud and with the high performance SAN (which we’re pretty sure is SSD backed as well) **** ** ** For purposes of the above test, 5% of total queries were write queries. On average, each test executed just under 19k read queries and just over 1k write queries (for a total of 20k queries). The test is only a micro-benchmark, but I feel it reflects daily Protunity usage reasonably well.**** ** ** I will also try to spend some time testing raw I/O throughput (and more importantly) latency with third-party software to get a better grasp of the differences between the various servers.**** ** ** ** ** _________________**** Sasa Ivetic**** 204-488-5697**** ** ** -- L.G. Meredith Managing Partner Biosimilarity LLC 7329 39th Ave SW Seattle, WA 98136 +1 206.650.3740 http://biosimilarity.blogspot.com

2 1

text() vs string()
by Wendell Piez 29 Jan '13

29 Jan '13

Hi, This may be a question about XQuery Full Text, or only about common usage (or misusage?) of XPath; in either case I hope it's on topic. Please tell me if not. In BaseX [A]: let $test := <test> The apple never falls far from the tree. Apples and trees. Trees and  apples. <fruit>Apple</fruit> trees. </test> return $test/*[text() contains text ('apple' ftand 'tree') using stemming using language 'en'] This returns  Apples and trees. As an experienced XPath user, this is what I expect, assuming "contains text" allows a sequence of nodes as its first argument (and returns true if any of them satisfies the test). Only the second 'p' element has a child text node whose value contains both "apple" and "tree". Of course the problem in the others is the mixed content: in the first, an element node 'em' intervenes, while in the third, a comment intervenes, so both these cases contain text nodes with either "apple" or "tree", but not both. In the case of the fourth 'p', there is no text node child containing "apple" at all, only a grandchild. Assuming I want all four back, I can write either: [B] return $test/*[string() contains text ('apple' ftand 'tree') using stemming using language 'en'] or [C] return $test/*[. contains text ('apple' ftand 'tree') using stemming using language 'en'] In the case of [B], the string() function casts the element to a string, flattening its structure. [C] passes the element itself to the "contains text" operation, which happily has the same effect. I have several related questions about this: 1. Unless I learn better, I'm going to prefer [B] or [C], because in my world, mixed content is common; is there any reason (performance or otherwise) to prefer [A] in cases where I know it will be robust? Is there any reason to prefer [B] or prefer [C]? 2. I see examples like [A] offered frequently in the XQuery literature, of "text()" being used apparently to refer to an element's string (text) value not to its text node children. And I see this usage in running code. I can only imagine that those who write it are simply not aware that mixed content will complicate their queries like this; maybe they have just never thought about it, or they don't know what text() actually does. In any case, the error is pernicious, since nothing tells you the query you gave isn't the one you intended -- it even works, until the day it doesn't, and the cases where gives correct but unwanted results may be rare. But maybe I'm wrong and they just know something about XQuery, XQuery FT, or their tools, that I don't. What do the experts say? Cheers, Wendell -- Wendell Piez | http://www.wendellpiez.com XML | XSLT | electronic publishing Eat Your Vegetables _____oo_________o_o___ooooo____ooooooo_^

5 4

Scientific Numbers in Scatter Plot
by IVES Greg 25 Jan '13

25 Jan '13

Hi I find BaseX a very powerful tool. One thing I noticed however is that it misunderstands numbers in scientific notation in the scatter plot display. Please see the attached plot. The samples should form a rough diagonal line. However, look at the scale on the left and you can see the exponent field is 'dis-continuous'. It looks like the chart formatter treats it as a non-scientific number or possibly even lexicographic order. Best Regards Greg

2 3

Collection related functions [SEC=UNCLASSIFIED]
by Thamm, Russell 25 Jan '13

25 Jan '13

UNCLASSIFIED Hi, Does baseX provide any collection related functions? In particular, I am after equivalents to the eXist functions: xmldb:create-collection xmldb:get-child-collections xmldb:get-child-resources I have searched the module library documentation and can't seem to find anything. I can duplicate the action of the latter two using db:list but that appears to be rather inefficient. BaseX doesn't seem to like empty collections, so the first might be superfluous. Cheers IMPORTANT: This email remains the property of the Department of Defence and is subject to the jurisdiction of section 70 of the Crimes Act 1914. If you have received this email in error, you are requested to contact the sender and delete the email.

2 2

Basex performance
by Mark Dwyer 24 Jan '13

24 Jan '13

Hi, I was considering the use of BASEX for an XPATH based solution I have which uses XPATH to select data from moderately large messages. Say between 20 and 50 Mb. Are there any relative performance stats in using BASEX over SUN/ORACLE Java implementations of XPATH ? Mark A copy of the LCH.Clearnet e-mail disclaimer can be found at: www.lchclearnet.com/disclaimer/email LCH.Clearnet Limited, Registered Office: Aldgate House, 33 Aldgate High Street, London EC3N 1EA. Recognised as a Clearing House under the Financial Services & Markets Act 2000. Reg. in England No. 25932. LCH.Clearnet SA, Siège Social: 18 rue du Quatre Septembre, 75002 Paris, Chambre de Compensation conformément au Code Monétaire et Financier. LCH.Clearnet LLC, Principal Office: 17 State Street, New York, NY 10004. Registered with the U.S. Commodity Futures Trading Commission as a derivatives clearing organization under the U.S. Commodity Exchange Act.

2 1

Performance with fulltext searches
by Charles Duffy 24 Jan '13

24 Jan '13

Howdy, all -- If I run the following query against a large, full-text-indexed store (output from "svn log --xml" on a repository with several hundred thousand entries), I get results back in well under a second: count(/log/logentry[./paths/path/text() contains text "trunk"]) However, the following variant takes multiple minutes to execute -- actually, has never yet completed before I've lost patience and cancelled it: count(/log/logentry[ ./paths/path/text() contains text "trunk" and not( (./paths/path/text() contains text "tags") or (./paths/path/text() contains text "branches") ) ]) Is there a way to get reasonable(ish) performance for the more complex query?

3 2

HTTPS
by pw＠themail.co.uk 24 Jan '13

24 Jan '13

Dear List Since Win7, Win2008 etc don't by default allow establishing WebDAV connections with basic authentication on http connections, what is the best way to configure the BaseX httpserver for https? TIA Peter

2 1