August 2010 - BaseX-Talk - mailman.uni-konstanz.de

DeepFS: using GUI; turning whitespace chop off
by pw＠themail.co.uk 22 Sep '10

22 Sep '10

How do I turn off whitespace chopping when using the GUI to set up a DeepFS mapping? TIA

3 2

general question on basex database/collection design
by Sandra Maria Silcot 02 Sep '10

02 Sep '10

Hi all, Basex continues to impress. I had been under the (false) impression that a single database should contain all the data/files relevant for one particular application. I have discovered that this kind of query, which crosses/joins data across databases, works: for $c in basex:db("chaingang")//person[@key="ccc86809"] let $links := basex:db("fasdb")//linkGrp[link=$c/@target] return ($c, $links, basex:db("fasdb")//person[@key=($links//link/text())]) which gives me (as desired): <person isVdl="yes" target="ai14506" type="ccc" key="ccc86809"> <sources> <ccc id="ccc86809" n="86809"> <isVDL>yes</isVDL> <aiRef>14506</aiRef> <drupalNodeId>86809</drupalNodeId> </ccc> </sources> </person> <linkGrp id="L016612" size="3" c31a="1" dlm="1" ai="1"> <link type="dlm">dlm18216038</link> <link type="ai">ai14506</link> <link type="c31a">c31a31070390</link> </linkGrp> <person key= ... the 3 person records specified in the linkGrp. Wow! I had no idea this kind of joining across basex databases was doable. Performance is great, about 64ms. It means I can think about refactoring my rather large single db with 30 documents into far smaller and manageable chunks which become updatable (i.e. the overhead of optimizing becomes tolerable). Does anyone have any comment about this, and the pros and cons? Perhaps I should be thinking of a database for each record type/major document instead of "one big database". I can see a downside in that my queries get locked into an implementation specific syntax, but I am so pleased with what Basex fulltext querying is giving me, and the general performance and clean design, that with it being open source, I'm happy to wear this risk. Thoughts anyone? Hoping this helps someone else discover what this marvellous piece of software is capable of. Thanks again to the developers -- bravo!! Cheers, Sandra

2 3

bug in vb.net sample code for new query
by Jörg Beisiegel 30 Aug '10

30 Aug '10

Hi, I have recently discovered baseX. First of all: This is an amazing project. Thank you very much. I have tested the UI and it works great with an incredible performance. Now I want to automize xqueries against a lager list in vb.net. I have downloaded (the most recent files) from "basex - Revision 2441: /trunk/api/etc/vb" The execute function works fine but the query function throws an error during New Query creation. There appears to be a session object parameter missing: in code: Dim query As Query = session.Query(input) '...calling the Query function.... no problem Public Function Query(ByVal q As String) As Query Return New Query(Me, q) End Function ' ...then the Query function calls the Query class ' ... and here "s As Session" is required but missing. Class Query Private session As Session Private id As String Private nextItem As String ' see readme.txt Public Sub New(s As Session, query As String) session = s session.stream.WriteByte(0) session.Send(query) id = session.Receive() If Not session.Ok() Then Throw New IOException(session.Receive()) End If End Sub The query results in the following error message in the console window: " instead of 1 commands received. 8814 ms." and in the VB.net direct window I get the message "Eine Ausnahme (erste Chance) des Typs "System.IO.IOException" ist in BaseX.exe aufgetreten." Could you please help? ________________________________ [http://www.translatissimo.de/files/translatissimo_logo.png]<http://www.translatissimo.de/> Jörg Beisiegel Geschäftsführer / Managing Director Localization Consulting Desk phone: +49 7275 98933-22 mailto:joerg.beisiegel@translatissimo.de translatissimo Übersetzung und Lokalisierung GbR Bismarckstr. 19 * D-76870 Kandel * Germany http://www.translatissimo.de<http://www.translatissimo.de/> *** HINWEIS *** NOTE *** Falls die Mail Anhänge enthält, bitte Empfang bestätigen. If this mail contains attachments please confirm receipt.

3 5

bug report in basex 6.2.3 fulltext
by Sandra Maria Silcot 30 Aug '10

30 Aug '10

Hi all, The following query works: > XQUERY //b4a[shipNameNorm/text() contains text 'mount.+' using wildcards]/shipNameNorm <shipNameNorm>mountstewartelphinstone</shipNameNorm> <shipNameNorm>mountstewartelphinstone</shipNameNorm> <shipNameNorm>mountstewartelphinstone</shipNameNorm> Query executed in 22.31 ms. But with a space in the same query, the server just hangs with java using up all cpu and after a long time I get an out of memory error: 21:24:01.510 [127.0.0.1:35052] XQUERY //b4a[shipNameNorm/text() contains text 'mount .+' using wildcards]/shipNameNorm Error: Out of Main Memory. The following hints might help you: - increase Java's heap size with the flag -Xmx<size> - choose the internal XML parser in the GUI or via 'set intparse on' - deactivate the text and attribute indexes 18607.03 ms To double check, I have recreated the db and reindexed (as it had been created under an earlier version). This time I don't get an out of memory error, but this: > XQUERY //b4a[shipNameNorm/text() contains text 'mount .+' using wildcards]/shipNameNorm [XQST0054] Circular variable definition? So this still appears to me to be a bug. The db is about 1.3GB, 26 documents. and Nodes: 48959283 with Indexes: Path Summary: ON Text Index: ON Attribute Index: ON Full-Text Index: ON (wildcards) Many thanks for any guidance. Cheers, Sandra

5 6

Grouping and sorting
by reference＠gmx.de 27 Aug '10

27 Aug '10

XQuery: let $type := <type> <item id="id1"><value>3</value></item> <item id="id2"><value>3</value></item> <item id="id3"><value>6</value></item> <item id="id4"><value>5</value></item> <item id="id5"><value>5</value></item> </type> for $i in $type/item let $v := $i/value group by $v order by $v return $v Result in BaseX: <value>3</value> <value>5</value> <value>5</value> As I understand XQuery the result should be as follows: <value>3</value> <value>5</value> <value>6</value> Where ist my mistake or is it a bug in BaseX?

2 2

Question regards "add path"; does it break existing indexes?
by Sandra Maria Silcot 25 Aug '10

25 Aug '10

Hi all, Using Basex v6.2.3 jar under linux, sun jdk1.6.0_18. I have a database named 'linkdb' with a few documents and a full set of indexes built. I would like to be able to add/delete docs and simply optimise the indexes without needing to rebuild them from scratch. But I am having problems doing so. Before adding a document, the "info" command reports: Database Properties Name: linkdb Size: 29338 KB Nodes: 1085033 Height: 6 Database Creation ... Documents: 3 Whitespace Chopping: ON Entity Parsing: OFF Indexes Path Summary: ON Text Index: ON Attribute Index: ON Full-Text Index: ON All is well. I then add another document and I can access the new document via xquery, but now, even after an "optimize", "info db" reports: Database Properties Name: linkdb Size: 29414 KB Nodes: 1088588 Height: 6 Database Creation ... Documents: 4 Whitespace Chopping: ON Entity Parsing: OFF Indexes Path Summary: ON Text Index: OFF Attribute Index: OFF Full-Text Index: OFF So what has just happened to the text, attribute and fulltext indexes? The only process which seems to work is to completely rebuild them, like so: LOGIN admin OK SET pathindex on OK 0.19 ms SET textindex on OK 0.13 ms SET attrindex on OK 0.14 ms SET ftindex on OK 0.13 ms SET wildcards on OK 0.14 ms SET casesens off OK 0.13 ms SET scoring 2 OK 0.13 ms OPEN linkdb OK 0.69 ms CREATE INDEX TEXT OK 424.54 ms CREATE INDEX PATH OK 90.74 ms CREATE INDEX ATTRIBUTE OK 344.46 ms CREATE INDEX FULLTEXT OK 996.65 ms OPTIMIZE OK 1784.94 ms CLOSE OK 0.18 ms And now: Indexes Path Summary: ON Text Index: ON Attribute Index: ON Full-Text Index: ON (wildcards) In the .basex file in the home dir of the user the basex server is run as, I attempt to set global options like so: TEXTINDEX = ON ATTRINDEX = ON FTINDEX = ON WILDCARDS = ON CASESENS = OFF SCORING = 2 but this seems to be always ignored. What am I doing wrong, or is this the intended behaviour? Where does the ".basex" file need to be to take effect as a set of defaults. The behaviour is repeatable irrespective of whether I use the java client or run the commands via a perl script. Many thanks for any advice, Cheers, Sandra.

2 1

defining external variables when using BaseX in server mode
by Godmar Back 25 Aug '10

25 Aug '10

Hi, is it possible to create bindings for external variables when using BaseX's server mode? For instance, we have a query that declares: declare variable $id as xs:string external; When using XQJ, 'id' would be bound as bindNode(). In our server-based implementation, we manually substitute all occurrences of $id with its value before running the query. Obviously, this is not robust: for instance, occurrences of $id in string literals would be substituted as well, even when they should not (e.g., "$id") Is there a way to leave $id declared as external, but assign a value for it when using BaseX's server mode? - Godmar

1 0

note on BaseXClient.java
by Godmar Back 23 Aug '10

23 Aug '10

Hi, The example file BaseXClient uses an unbuffered stream to write to the socket: out = socket.getOutputStream(); void send(final String s) throws IOException { for(final byte t : s.getBytes()) out.write(t); out.write(0); } This yields extremely bad performance, especially for large queries (every single call to write will trigger a separate system call to the OS, which in turn will trigger a separate TCP segment on the wire). On a typical Ethernet, this means that 100 minimum sized packets are transmitted rather than a single packet to transmit a 100-byte query. On a different note, why is BaseXClient not part of the "official" BaseX API, or at least provided as part of the distributed .jar files in an org.basex.* package? - Godmar

3 3

exception when interrupting BaseX server
by Godmar Back 23 Aug '10

23 Aug '10

When I interrupt BaseXServer with SIGINT, I occasionally see backtraces like this one. - Godmar Starting BaseXServer on port 1985 BaseX 6.2.3 [Server] Server was started. java.io.IOException: Stream closed at sun.nio.cs.StreamEncoder.ensureOpen(StreamEncoder.java:26) at sun.nio.cs.StreamEncoder.write(StreamEncoder.java:99) at sun.nio.cs.StreamEncoder.write(StreamEncoder.java:116) at java.io.OutputStreamWriter.write(OutputStreamWriter.java:203) at java.io.Writer.write(Writer.java:140) at org.basex.server.Log.write(Log.java:76) at org.basex.server.ServerProcess.run(ServerProcess.java:178) java.io.IOException: Stream closed at sun.nio.cs.StreamEncoder.ensureOpen(StreamEncoder.java:26) at sun.nio.cs.StreamEncoder.write(StreamEncoder.java:99) at sun.nio.cs.StreamEncoder.write(StreamEncoder.java:116) at java.io.OutputStreamWriter.write(OutputStreamWriter.java:203) at java.io.Writer.write(Writer.java:140) at org.basex.server.Log.write(Log.java:76) at org.basex.server.ServerProcess.run(ServerProcess.java:178)

2 2

Bulk Loading XML documents with Add(docPath).execute(CONTEXT)
by pw＠themail.co.uk 22 Aug '10

22 Aug '10

When adding documents from a directory I get things working using a few small documents and Add(docPath).execute(CONTEXT) but when I want to add hundreds of documents (average size 1Mb) I get a memory error. I am using Jython and have -J-Xms1200M and -J-Xmx1200m should I use an iterator and only add one document at a time? [this doesn't seem to me to be the way to use Add() ] TIA Peter

2 1