- BaseX-Talk - mailman.uni-konstanz.de

Whitespace handling on ingest
by Wendell Piez 22 Feb '13

22 Feb '13

Hi, I see the 'CHOP' option, turned on by default, for trimming leading and trailing whitespace and eliminating empty text nodes. What about going further? Is there a good way to normalize whitespace entirely, collapsing any runs of tab-LF-space into single spaces in my data? I think I mentioned earlier the idea of specifying an XSLT transformation to filter data on ingest (for a similar requirement, namely removing all comments and PIs). That might be going too far but any hints you can give me (or pointers to docs) about functionality to address this sort of thing in general would be welcome. Thanks! Wendell -- Wendell Piez | http://www.wendellpiez.com XML | XSLT | electronic publishing Eat Your Vegetables _____oo_________o_o___ooooo____ooooooo_^

2 3

GUI enhancement to show only query results as text in a tree?
by Amanda Galtman 22 Feb '13

22 Feb '13

Hi, The BaseX GUI is very nice for interactively tweaking a query or exploring my data set. However, one thing I wish I could do with the GUI is see query results as text in a tree with expandable/collapsible nodes. An example of what I mean is in the right pane of this Qizx screen capture: http://www.axyana.com/qizxopen/_distrib/docs/manual/qizxstudio.html#d0e4472 The tree in the screen capture makes it easy to distinguish between adjacent result items, and the collapsibility of result items makes it easier to decide when you want to browse the results at a high level or dig into a specific result. By contrast, the BaseX GUI's text view shows all the text of the query results, but they run together. The folder visualization has expandable/collapsible nodes, but because it shows the query-result nodes in the context of the entire database, you might have to expand a lot of nodes to discover where the query-result nodes are. While it can be useful to see the query-result nodes in context, I often want to see only the results themselves; and if I decide I need more context, I tweak my query to return more context. Would it be possible in a future release of BaseX to get a text tree-view of just the query results? Better yet, is it already available but I don't know about it? Thanks, Amanda

2 2

Re: [basex-talk] BaseX xpath querying takes more time than SAX parser takes
by Christian Grün 22 Feb '13

22 Feb '13

…glad to hear it works. ___________________________ > Thanks Christian for your response, > > I have done a mistake in my code. I have executed different xpaths against > both causes the time difference. Now I have executed same xpath against both > and BaseX xpath evaluation is faster than SAX. > > Regards, > Elango. > > > On Fri, Feb 22, 2013 at 3:51 PM, Christian Grün <christian.gruen(a)gmail.com> > wrote: >> >> Hi Elango, >> >> to say anything on this, we probably need more information on how you >> are processing your data via SAX, and how you receive your result via >> session.query(expression).execute(). Do you have some self-contained >> code that allows us to get more insight into how you are testing? >> >> Best, >> Christian >> >> >> On Wed, Feb 20, 2013 at 11:56 AM, elangovan MuthuSwamy >> <elangovan.h(a)gmail.com> wrote: >> > Hi All, >> > >> > I am trying to use BaseX Client and server with large XML files(13 MB >> > size). >> > I have created a database for the XML and evaluated an xpath in that. >> > When I >> > executed an xpath it takes around 250 milliseconds to fetch the results. >> > In >> > case of SAX parser I could get the response for the same xpath within 5 >> > seconds. >> > >> > I used following BaseXClient code for this. >> > >> > BaseXClient session = new BaseXClient("localhost", 1984, "admin", >> > "admin"); >> > >> > // code to fetch the large XML file size. Read from the file. >> > String xmlString = ....... >> > >> > InputStream bais = new ByteArrayInputStream(xmlString.getBytes()); >> > session.create("database", bais); >> > >> > String expression = "/configuration/protocol[starts-with(name, >> > 'http')]/name"; >> > >> > session.query(expression).execute(); // this line takes 250 milliseconds >> > to >> > give the result. Where as same xpath can be executed within 5 >> > milliseconds >> > in SAX parser. >> > >> > Am I basically missing anything or something else. >> > >> > >> > Please advice me. >> > >> > Thanks in advance. >> > >> > -- >> > Regards, >> > Elango. > > > > > -- > Regards, > Elango.

1 0

Additional info on the invoke problem
by Arto Viitanen 22 Feb '13

22 Feb '13

I am not sure if this tells more of the problem (xquery:invoke does not open the collection), but: if the query does something with the collection plus predicates retrieved, the query is evaluated. So, in my example collection('suuryritys_com/suuryritys.com/data/ou')//*[@cat='production'] if I add some extra content that forces the evaluation, like <a>{collection('suuryritys_com/suuryritys.com/data/ou')//*[@cat='production']}</a>/* I.e. add the result to a -element and retrieve it, the xquery:invoke("file-name.xq") evaluates the query, even if the database (suuryritys_com) is closed! -- Arto Viitanen Microteam Oy Finland

2 1

BaseX 7.6, XQJ API 1.2.3 and XQuery 3.0 features - syntax error for simple map operator
by Markus Ackermann 22 Feb '13

22 Feb '13

Hello BaseX-community, first of all, I really have to say that I really enjoy using BaseX and that I am really impressed by its active development and its ongoing improvement. I am still a little unsure about the Query 3.0 support via the BaseX XQJ driver from net.xqj, though. According to the Changelog of the website ([1]), XQuery 3.0 via XQJ is now supported: BaseX 7.3, 18.06.2012 [...] - Charles Foster's latest XQJ Driver supports XQuery 3.0 and the Update and Full Text extensions: http://xqj.net/basex/ The README file of Charles Foster's driver also states that XQuery 3.0 is supported since driver version 1.1.0: Version 1.1.0: * Supports XQuery 3.0 + XQuery Update Facility + XQuery Full Text extensions. * Various bug fixes. However, take this minimal example using the simple map operator (query A): BaseXXQDataSource dataSource = new BaseXXQDataSource(); dataSource.setServerName("localhost"); dataSource.setPort(1984); XQConnection con = dataSource.getConnection("dict_wsa", "lidell"); con.createExpression().executeQuery( "(1 to 3) ! (. * 2))"); //query A con.createExpression().executeQuery( "let $m := map { 'one' := 1, 'two' := 2 }" + "return ($m('one')"); //query B Executing it yield a syntax exception: Exception in thread "main" javax.xml.xquery.XQException: XQJQS001 - Invalid XQuery syntax, syntax does not pass static validation. Root Cause: net.xqj.basex.bin.bn: Lexical error at line 1, column 11. Encountered: " " (32), after : "!" at XQ3Test.main(XQ3Test.java:21) at XQ3Test.main(XQ3Test.java:21) For query B I also receive a syntax exception: Exception in thread "main" javax.xml.xquery.XQException: XQJQS001 - Invalid XQuery syntax, syntax does not pass static validation. Root Cause: net.xqj.basex.bin.bj: Encountered " "{" "{ "" at line 1, column 15. Was expecting one of: "=" ... "order" ... "," ... Is there another option I have to set to circumvent the 'static validation' or did I overlook anything else? Regards and thanks in advance for any information or advice, Markus Ackermann PS: Information on the execution environment: JVM: 1.7.0_13 (Oracle Corporation 23.7-b01) OS: Linux 3.0.0-30-generic amd64 (Ubuntu 11.10) BaseX 7.6

2 1

Re: [basex-talk] BaseX-Talk Digest, Vol 38, Issue 36
by Arto Viitanen 22 Feb '13

22 Feb '13

22.2.2013 11:04, basex-talk-request(a)mailman.uni-konstanz.de wrote: > Message: 3 > Date: Fri, 22 Feb 2013 09:00:09 +0100 > From: Marco Lettere <marco.lettere(a)dedalus.eu> > To: basex-talk(a)mailman.uni-konstanz.de > Subject: Re: [basex-talk] Indirect evaluation with xquery eval or > invoke on closed collection > Message-ID: <51272589.3000600(a)dedalus.eu> > Content-Type: text/plain; charset="iso-8859-1"; Format="flowed" > > This looks like the same stacktrace reported here: > > https://mailman.uni-konstanz.de/pipermail/basex-talk/2013-February/004683.h… > > BTW, I got also a major corruption of the DB. Your DB remains coherent > when this happens? > M. > > No, but then the query was readonly. Another, but similar query by my colleagues also gave some random answer; where as my gave the stack trace. They used C# api, so it might be partially reason for the difference. -- Arto Viitanen

1 0

corruption when adding documents via the GUI in a almost full database
by Fabrice Etanchaud 22 Feb '13

22 Feb '13

Dear all at BaseX, Under the GUI (7.5.1 beta), When adding documents in a almost full database (Properties/Resources panel), An error is raised, and then the node count gets negative (Properties/General panel). Is there a way to safely add documents until the maximum node number is reached, Or do I have to add logic based on db:info()/databaseproperties/nodes of the input and destination data ? Best regards, Fabrice

2 4

Indirect evaluation with xquery eval or invoke on closed collection
by Arto Viitanen 22 Feb '13

22 Feb '13

Following error comes from BaseX version 7.5, but it seems to exist also on 7.6: We have a query that uses collection: ("suuryritys" stands for "big company"): collection('suuryritys_com/suuryritys.com/data/ou')//*[@cat='production'] When evaluated on basexgui (or via C# api), the query evaluates, i.e. the database is opened, document (/suuryritys.com/data/ou) is opened ,data fetched and database closed. But, when we try to evaluate the query using either xquery:eval or xquery:invoke xquery:eval("collection('suuryritys_com/suuryritys.com/data/ou')//*[@cat='production']") instead of result, we get error message: Error: Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk(a)mailman.uni-konstanz.de Version: BaseX 7.5 Java: Oracle Corporation, 1.7.0_02 OS: Linux, i386 Stack Trace: java.lang.ArrayIndexOutOfBoundsException: 57 org.basex.util.Compress.pull(Compress.java:139) org.basex.util.Compress.unpack(Compress.java:113) org.basex.data.DiskData.txt(DiskData.java:268) org.basex.data.DiskData.text(DiskData.java:235) org.basex.io.serial.Serializer.node(Serializer.java:389) org.basex.io.serial.Serializer.serialize(Serializer.java:99) org.basex.io.serial.Serializer.serialize(Serializer.java:84) org.basex.query.iter.ValueBuilder.serialize(ValueBuilder.java:80) org.basex.query.iter.ValueBuilder.serialize(ValueBuilder.java:75) org.basex.core.cmd.AQuery.query(AQuery.java:91) org.basex.core.cmd.XQuery.run(XQuery.java:22) org.basex.core.Command.run(Command.java:342) org.basex.core.Command.exec(Command.java:321) org.basex.core.Command.execute(Command.java:78) org.basex.gui.GUI.exec(GUI.java:389) org.basex.gui.GUI$6.run(GUI.java:343) Query: xquery:invoke("invoke-file.xq") Query plan: <QueryPlan> <FNXQuery name="invoke(string[,bindings])"> <Str value="invoke-file.xq" type="xs:string"/> </FNXQuery> </QueryPlan> Same error comes, when the query is on a text file, and it is evaluated with xquery:invoke. The funny part is, if the database (suuryritys_com) is opened by hand (on BaseXGui's command line with "open suuryritys_com"), the query returns the result. It also seems, that if I first open some other database, then close it, and then open the database suuryritys_com, I get the same error. -- Arto Viitanen Microteam Oy Finland

2 1

Re: [basex-talk] BaseX-Talk Digest, Vol 38, Issue 33
by elangovan MuthuSwamy 21 Feb '13

21 Feb '13

Hi All, There was a misspell in end of first para. I could get the results within 5 milliseconds using SAX parser misspelled as 5 seconds. Sorry for the inconvenience caused. Thanks, Elango. > Hi All, > > I am trying to use BaseX Client and server with large XML files(13 MB > size). I have created a database for the XML and evaluated an xpath in > that. When I executed an xpath it takes around 250 milliseconds to fetch > the results. In case of SAX parser I could get the response for the same > xpath within 5 seconds. > > I used following BaseXClient code for this. > > BaseXClient session = new BaseXClient("localhost", 1984, "admin", "admin"); > > // code to fetch the large XML file size. Read from the file. > String xmlString = ....... > > InputStream bais = new ByteArrayInputStream(xmlString.getBytes()); > session.create("database", bais); > > String expression = "/configuration/protocol[starts-with(name, > 'http')]/name"; > > *session.query(expression).execute();* // this line takes 250 milliseconds > to give the result. Where as same xpath can be executed within 5 > milliseconds in SAX parser. > > Am I basically missing anything or something else. > > > Please advice me. > > Thanks in advance. > > -- > Regards, > Elango. >

1 0

Performance problem
by Nuria Garcia 20 Feb '13

20 Feb '13

We are testing basex to use with a huge database, and we have a performance problem. At the moment we have two small databases with thousands of documents to test: · Authors, 2.000 documents, 2MBs aprox. · Books (3 per author) with 6.000 documents, 5MBs aprox. We need join and group results, and this takes a long time: Query: <result>{for $Author in (db:open('Authors')//businessEvent), $Book in (db:open('Books')//businessEvent) for $income in distinct-values($Author//author/country) where $Author//author/idauthor=$Book//book/idauthor group by $income return <line><label>{ $income }</label><data>count($Book )</data></line>}</result> Compiling: - rewriting where clause to predicate(s) - pre-evaluating db:open("Authors") - simplifying descendant-or-self step(s) - pre-evaluating db:open("Books") - simplifying descendant-or-self step(s) - simplifying descendant-or-self step(s) - swapping operands: descendant::*:book/*:idauthor = $Author/descendant::author/idauthor - simplifying descendant-or-self step(s) - simplifying descendant-or-self step(s) Result: element result { for $Author in (document-node { "1" }, document-node { "2" }, ...)/descendant::*:businessEvent for $Book in (document-node { "1-1" }, document-node { "1-2" }, ...)/descendant::*:businessEvent[descendant::*:book/*:idauthor = $Author/descendant::author/idauthor] for $income in fn:distinct-values($Author/descendant::author/country) group by $income := $income return element line { element label { $income }, element data { "count($Book )" } } } Timing: - Parsing: 1.01 ms - Compiling: 3.99 ms - Evaluating: 100725.38 ms - Printing: 0.37 ms - Total Time: 100730.77 ms We have no idea why it’s taking so long time to show the results. If we don’t group results it takes a long time too. Regards, Nuria

2 3