- BaseX-Talk - mailman.uni-konstanz.de

Beginner with PHP
by Hänzi Martina (s) 04 Sep '10

04 Sep '10

Hi I got some basic problems with the php-examples, available on the homepage. where do i have to save the php-Files? and what url do i type in to execute them? i suppose it begins with http://localhost:1984/... when i start the server and enter http://localhost:1984/BaseXData/BaseXClient.php (path where the file is saved) i only get the hourglass of the browser. i also tried with the jaxrx server on port 8984 using http://localhost:8984/basex/jax-rx but i got the same result - hourglass. both times i startet the server via the Client - Server Administrators. am i doing something wrong? Thanks for your feedback greets Martina

2 4

Out of Main Memory issues when opening a tiny database
by Sandra Maria Silcot 02 Sep '10

02 Sep '10

Hi all, This I feel sure is "pilot error"; I am messing something up in my perl script to create the db. Have a tiny collection of 6 documents 75kb when loaded. It seems to load ok but when I go to use it I am getting Out of Main Memory errors. The JVM is 4gb using -Xms and -Xmx options on the java command to start the basex server (on an 8gb VM). On reviewing the logs I can see that my client is not doing a CLOSE for the database -- yesterday's log showed 175 OPENs and only 31 closes -- OOPS!! Ok, so I start getting this: OPEN chaingang Error: Out of Main Memory. 11286.05 ms I can see in top the JVMs ram just being gobbled up rapidly until it runs out and this message appears. Yes, I know I need to fix my clients but is it possible that even when the clients session closes, somehow the database is being kept open, and eventually we run out of memory? But that doesn;t really fit with the "gobbling ram" I am seeing, unless the former situation triggers the latter. IF this is the issue, could the basex server be made to nicely close any databases open in a session when that session is closed or the pipe gets broken? # when trying to recover 2010-08-31.log:22:24:11.781 DROP DB chaingang Error: Database 'chaingang' is currently locked. Or of a db has become corrupted somehow, could it cause such message? Are there any integrity check commands on the horizon? I feel a bit nervous about these kinds of memory errors as I operate in a VSphere v4 (VMWARE) environment and my JVM monitoring skills are rudimentary -- the JVM does look like it has a nice big contiguous slab of ram to work with. Clearly something bad/silly I have done is triggering this behaviour: basex just "runs away" with ram. Any thoughts? Has the api interface changed -- my perl client dates back to late March? Sorry to be so vague. Any similarly vague clues most welcome. On a similar vien, my web interface/query controller is modperl2 under apache, which means the script sits resident in ram. I have it opening/closing a basex session on each request cycle (and am about to go check what I do when it closes!). Perhaps it would be far better to have the basex connection done in the apache initialisation stage (when apache2 launches the modperl2 modules) so the session just sits there until apache restarts. Then each request cycle should just open/close the database? I ask here as it could possibly be related to my grief above. If not my apologies for confusing the matter. I am using this Sun jdk: /opt/java/jdk1.6.0_18/bin/java. Kind of regretting not using openSolaris as my OS platform rather than Linux. I do so love its trace facilities and its a great Java platform. I'm using 64bit Linux: 2.6.27-17-server #1 SMP Fri Mar 12 02:48:52 UTC 2010 x86_64 GNU/Linux ; and BaseX 6.2.6 [Server] Best wishes all, Sandra

3 3

general question on basex database/collection design
by Sandra Maria Silcot 02 Sep '10

02 Sep '10

Hi all, Basex continues to impress. I had been under the (false) impression that a single database should contain all the data/files relevant for one particular application. I have discovered that this kind of query, which crosses/joins data across databases, works: for $c in basex:db("chaingang")//person[@key="ccc86809"] let $links := basex:db("fasdb")//linkGrp[link=$c/@target] return ($c, $links, basex:db("fasdb")//person[@key=($links//link/text())]) which gives me (as desired): <person isVdl="yes" target="ai14506" type="ccc" key="ccc86809"> <sources> <ccc id="ccc86809" n="86809"> <isVDL>yes</isVDL> <aiRef>14506</aiRef> <drupalNodeId>86809</drupalNodeId> </ccc> </sources> </person> <linkGrp id="L016612" size="3" c31a="1" dlm="1" ai="1"> <link type="dlm">dlm18216038</link> <link type="ai">ai14506</link> <link type="c31a">c31a31070390</link> </linkGrp> <person key= ... the 3 person records specified in the linkGrp. Wow! I had no idea this kind of joining across basex databases was doable. Performance is great, about 64ms. It means I can think about refactoring my rather large single db with 30 documents into far smaller and manageable chunks which become updatable (i.e. the overhead of optimizing becomes tolerable). Does anyone have any comment about this, and the pros and cons? Perhaps I should be thinking of a database for each record type/major document instead of "one big database". I can see a downside in that my queries get locked into an implementation specific syntax, but I am so pleased with what Basex fulltext querying is giving me, and the general performance and clean design, that with it being open source, I'm happy to wear this risk. Thoughts anyone? Hoping this helps someone else discover what this marvellous piece of software is capable of. Thanks again to the developers -- bravo!! Cheers, Sandra

2 3

bug in vb.net sample code for new query
by Jörg Beisiegel 30 Aug '10

30 Aug '10

Hi, I have recently discovered baseX. First of all: This is an amazing project. Thank you very much. I have tested the UI and it works great with an incredible performance. Now I want to automize xqueries against a lager list in vb.net. I have downloaded (the most recent files) from "basex - Revision 2441: /trunk/api/etc/vb" The execute function works fine but the query function throws an error during New Query creation. There appears to be a session object parameter missing: in code: Dim query As Query = session.Query(input) '...calling the Query function.... no problem Public Function Query(ByVal q As String) As Query Return New Query(Me, q) End Function ' ...then the Query function calls the Query class ' ... and here "s As Session" is required but missing. Class Query Private session As Session Private id As String Private nextItem As String ' see readme.txt Public Sub New(s As Session, query As String) session = s session.stream.WriteByte(0) session.Send(query) id = session.Receive() If Not session.Ok() Then Throw New IOException(session.Receive()) End If End Sub The query results in the following error message in the console window: " instead of 1 commands received. 8814 ms." and in the VB.net direct window I get the message "Eine Ausnahme (erste Chance) des Typs "System.IO.IOException" ist in BaseX.exe aufgetreten." Could you please help? ________________________________ [http://www.translatissimo.de/files/translatissimo_logo.png]<http://www.translatissimo.de/> Jörg Beisiegel Geschäftsführer / Managing Director Localization Consulting Desk phone: +49 7275 98933-22 mailto:joerg.beisiegel@translatissimo.de translatissimo Übersetzung und Lokalisierung GbR Bismarckstr. 19 * D-76870 Kandel * Germany http://www.translatissimo.de<http://www.translatissimo.de/> *** HINWEIS *** NOTE *** Falls die Mail Anhänge enthält, bitte Empfang bestätigen. If this mail contains attachments please confirm receipt.

3 5

bug report in basex 6.2.3 fulltext
by Sandra Maria Silcot 30 Aug '10

30 Aug '10

Hi all, The following query works: > XQUERY //b4a[shipNameNorm/text() contains text 'mount.+' using wildcards]/shipNameNorm <shipNameNorm>mountstewartelphinstone</shipNameNorm> <shipNameNorm>mountstewartelphinstone</shipNameNorm> <shipNameNorm>mountstewartelphinstone</shipNameNorm> Query executed in 22.31 ms. But with a space in the same query, the server just hangs with java using up all cpu and after a long time I get an out of memory error: 21:24:01.510 [127.0.0.1:35052] XQUERY //b4a[shipNameNorm/text() contains text 'mount .+' using wildcards]/shipNameNorm Error: Out of Main Memory. The following hints might help you: - increase Java's heap size with the flag -Xmx<size> - choose the internal XML parser in the GUI or via 'set intparse on' - deactivate the text and attribute indexes 18607.03 ms To double check, I have recreated the db and reindexed (as it had been created under an earlier version). This time I don't get an out of memory error, but this: > XQUERY //b4a[shipNameNorm/text() contains text 'mount .+' using wildcards]/shipNameNorm [XQST0054] Circular variable definition? So this still appears to me to be a bug. The db is about 1.3GB, 26 documents. and Nodes: 48959283 with Indexes: Path Summary: ON Text Index: ON Attribute Index: ON Full-Text Index: ON (wildcards) Many thanks for any guidance. Cheers, Sandra

5 6

Grouping and sorting
by reference＠gmx.de 27 Aug '10

27 Aug '10

XQuery: let $type := <type> <item id="id1"><value>3</value></item> <item id="id2"><value>3</value></item> <item id="id3"><value>6</value></item> <item id="id4"><value>5</value></item> <item id="id5"><value>5</value></item> </type> for $i in $type/item let $v := $i/value group by $v order by $v return $v Result in BaseX: <value>3</value> <value>5</value> <value>5</value> As I understand XQuery the result should be as follows: <value>3</value> <value>5</value> <value>6</value> Where ist my mistake or is it a bug in BaseX?

2 2

Question regards "add path"; does it break existing indexes?
by Sandra Maria Silcot 25 Aug '10

25 Aug '10

Hi all, Using Basex v6.2.3 jar under linux, sun jdk1.6.0_18. I have a database named 'linkdb' with a few documents and a full set of indexes built. I would like to be able to add/delete docs and simply optimise the indexes without needing to rebuild them from scratch. But I am having problems doing so. Before adding a document, the "info" command reports: Database Properties Name: linkdb Size: 29338 KB Nodes: 1085033 Height: 6 Database Creation ... Documents: 3 Whitespace Chopping: ON Entity Parsing: OFF Indexes Path Summary: ON Text Index: ON Attribute Index: ON Full-Text Index: ON All is well. I then add another document and I can access the new document via xquery, but now, even after an "optimize", "info db" reports: Database Properties Name: linkdb Size: 29414 KB Nodes: 1088588 Height: 6 Database Creation ... Documents: 4 Whitespace Chopping: ON Entity Parsing: OFF Indexes Path Summary: ON Text Index: OFF Attribute Index: OFF Full-Text Index: OFF So what has just happened to the text, attribute and fulltext indexes? The only process which seems to work is to completely rebuild them, like so: LOGIN admin OK SET pathindex on OK 0.19 ms SET textindex on OK 0.13 ms SET attrindex on OK 0.14 ms SET ftindex on OK 0.13 ms SET wildcards on OK 0.14 ms SET casesens off OK 0.13 ms SET scoring 2 OK 0.13 ms OPEN linkdb OK 0.69 ms CREATE INDEX TEXT OK 424.54 ms CREATE INDEX PATH OK 90.74 ms CREATE INDEX ATTRIBUTE OK 344.46 ms CREATE INDEX FULLTEXT OK 996.65 ms OPTIMIZE OK 1784.94 ms CLOSE OK 0.18 ms And now: Indexes Path Summary: ON Text Index: ON Attribute Index: ON Full-Text Index: ON (wildcards) In the .basex file in the home dir of the user the basex server is run as, I attempt to set global options like so: TEXTINDEX = ON ATTRINDEX = ON FTINDEX = ON WILDCARDS = ON CASESENS = OFF SCORING = 2 but this seems to be always ignored. What am I doing wrong, or is this the intended behaviour? Where does the ".basex" file need to be to take effect as a set of defaults. The behaviour is repeatable irrespective of whether I use the java client or run the commands via a perl script. Many thanks for any advice, Cheers, Sandra.

2 1

defining external variables when using BaseX in server mode
by Godmar Back 25 Aug '10

25 Aug '10

Hi, is it possible to create bindings for external variables when using BaseX's server mode? For instance, we have a query that declares: declare variable $id as xs:string external; When using XQJ, 'id' would be bound as bindNode(). In our server-based implementation, we manually substitute all occurrences of $id with its value before running the query. Obviously, this is not robust: for instance, occurrences of $id in string literals would be substituted as well, even when they should not (e.g., "$id") Is there a way to leave $id declared as external, but assign a value for it when using BaseX's server mode? - Godmar

1 0

note on BaseXClient.java
by Godmar Back 23 Aug '10

23 Aug '10

Hi, The example file BaseXClient uses an unbuffered stream to write to the socket: out = socket.getOutputStream(); void send(final String s) throws IOException { for(final byte t : s.getBytes()) out.write(t); out.write(0); } This yields extremely bad performance, especially for large queries (every single call to write will trigger a separate system call to the OS, which in turn will trigger a separate TCP segment on the wire). On a typical Ethernet, this means that 100 minimum sized packets are transmitted rather than a single packet to transmit a 100-byte query. On a different note, why is BaseXClient not part of the "official" BaseX API, or at least provided as part of the distributed .jar files in an org.basex.* package? - Godmar

3 3

exception when interrupting BaseX server
by Godmar Back 23 Aug '10

23 Aug '10

When I interrupt BaseXServer with SIGINT, I occasionally see backtraces like this one. - Godmar Starting BaseXServer on port 1985 BaseX 6.2.3 [Server] Server was started. java.io.IOException: Stream closed at sun.nio.cs.StreamEncoder.ensureOpen(StreamEncoder.java:26) at sun.nio.cs.StreamEncoder.write(StreamEncoder.java:99) at sun.nio.cs.StreamEncoder.write(StreamEncoder.java:116) at java.io.OutputStreamWriter.write(OutputStreamWriter.java:203) at java.io.Writer.write(Writer.java:140) at org.basex.server.Log.write(Log.java:76) at org.basex.server.ServerProcess.run(ServerProcess.java:178) java.io.IOException: Stream closed at sun.nio.cs.StreamEncoder.ensureOpen(StreamEncoder.java:26) at sun.nio.cs.StreamEncoder.write(StreamEncoder.java:99) at sun.nio.cs.StreamEncoder.write(StreamEncoder.java:116) at java.io.OutputStreamWriter.write(OutputStreamWriter.java:203) at java.io.Writer.write(Writer.java:140) at org.basex.server.Log.write(Log.java:76) at org.basex.server.ServerProcess.run(ServerProcess.java:178)

2 2