- BaseX-Talk - mailman.uni-konstanz.de

connection provblems with basex 7.6
by Andreas Wombacher 01 Aug '13

01 Aug '13

Hi, I am running a server in Java on a linux machine and want to connect to it from a windows machine (either GUI or java code). It is not working. I get as a response that the connection is refused. On the linux machine the server is running - I see the port being listened on. The firewall is also open on that port - I can telnet the port and get a response. Any idea? I use these settings in .basex # Client/Server Architecture HOST = 130.89.1.182 PORT = 19840 SERVERPORT = 1984 EVENTPORT = 1985 USER = XXXX PASSWORD = XXXX SERVERHOST = silo1.ewi.utwente.nl thanbks in advance. best regards Andreas

3 6

Out of memory bug
by Ivan Lagunov 31 Jul '13

31 Jul '13

Hello, I've got 'Out of memory' error and corrupted database again, this time it's for 6.3 version. Steps: 1. Started server, started terminal client, checked there are no text and attribute indexes. 2. Started my Java application that runs several tests on database: querying, adding, removing. 3. Then I forgot to reopen terminal client and used the one that I opened in Step 1. 4. Run optimize command for the collection and got error. 5. Cannot open the collection any more as I'm getting 'Out of memory'. It's clear that I shouldn't have used the terminal client that I opened before execution of my Java application. But sometimes it's hard to remember. And it's completely unacceptable that these actions result in the corrupted database. You really should catch ArrayIndexOutOfBoundsException everywhere in order to save database from being corrupted. Terminal client stacktrace: > optimize Possible bug? Feedback is welcome: basex-talk(a)mailman.uni-konstanz.de BaseX 6.3: java.lang.ArrayIndexOutOfBoundsException: -1 org.basex.data.PathSummary.add(PathSummary.java:68) org.basex.core.cmd.Optimize.stats(Optimize.java:68) org.basex.core.cmd.Optimize.run(Optimize.java:34) org.basex.core.Command.run(Command.java:236) org.basex.core.Command.exec(Command.java:218) org.basex.core.Command.execute(Command.java:66) org.basex.server.ServerProcess.run(ServerProcess.java:161) > open products Out of Main Memory. Best regards, Ivan

4 8

Re: [basex-talk] losing line breaks with xslt:transform(...)
by Simon Görke 31 Jul '13

31 Jul '13

Hi there, Setting CHOP=false in the .basex configuration solves my xslt issue but seems to have an ugly side-effect (bug?): Using basex commands in xml syntax is not working any more; if I call: basexclient.bat -c cmd.xml with the content of cmd.xml being (for example): <commands> <create-db name='xslt'/> </commands> it always returns " Syntax: <commands><...></commands>". Executing the same without CHOP=false in .basex works like a charm. Is there any chance this could be fixed? Best, Simon -----Ursprüngliche Nachricht----- Von: Christian Grün [mailto:christian.gruen@gmail.com] Gesendet: Mittwoch, 31. Juli 2013 12:39 An: Simon Görke Cc: BaseX Betreff: Re: [basex-talk] losing line breaks with xslt:transform(...) Hi Simon, good to hear you could narrow down the problem. > Is there a possibility to disable chopping globally ? Our databases > are either huge or contain text, but not both... the easiest way is to set CHOP=false in your .basex configuration [1]. As Gerrit indicates, the chopping flag caused frequent problems in the past. Due to legacy and storage issues, we are still hesitant to simply change the default, but we are working on it. Best, Christian [1] http://docs.basex.org/wiki/Options > -----Ursprüngliche Nachricht----- > Von: Christian Grün [mailto:christian.gruen@gmail.com] > Gesendet: Dienstag, 30. Juli 2013 17:26 > An: Simon Görke > Cc: BaseX > Betreff: Re: [basex-talk] losing line breaks with xslt:transform(...) > > Hi Simon, > > could you provide us with a minimized example that demonstrates the > (mis)behavior? > > Thanks in advance > Christian > ___________________________ > > 2013/7/30 Simon Görke <simon.goerke(a)ils.uni-stuttgart.de>: >> Christian, >> >> thanks for the hint, but no change for the better. I also tried with >> the basex-internal xslt-processor (instead of saxon) but resulted in >> the same problem. >> >> Any ideas welcome... >> >> Greets, >> Simon >> >> >> >> -----Ursprüngliche Nachricht----- >> Von: Christian Grün [mailto:christian.gruen@gmail.com] >> Gesendet: Freitag, 26. Juli 2013 14:56 >> An: Simon Görke >> Cc: BaseX >> Betreff: Re: [basex-talk] losing line breaks with xslt:transform(...) >> >> Hi Simon, >> >> not sure if that helps, but you can try to deactivate whitespace >> chopping in the prolog of your XQuery expression: >> >> declare option db:chop "no"; >> ... >> >> Best, >> Christian >> ___________________________ >> >> 2013/7/26 Simon Görke <simon.goerke(a)ils.uni-stuttgart.de>: >>> Hi, >>> >>> I have small issue regarding the xslt:transform function (or what >>> lies beneath). We are currently migrating a larger number of >>> existing stylesheets into a basex-environment (before they were used >>> with saxon on the filesystem). >>> >>> The problem is, that all line breaks which are generated explicitly >>> in the stylesheets (e.g. by using <xsl:text>
</xsl:text>) get lost. >>> Line feeds within a xsl:text element are correctly transformed. See >>> below for the xslt and example output. >>> >>> First we thought this might be linked to the whitespace chopping of >>> the database where the transformation result is stored, however I >>> get the same behavior when directly looking at the output of >>> xslt:transform(...) in the basex-gui. >>> Is there any options to influence this behavior? We are using saxon9 >>> within basex version 7.6. >>> >>> Thanks for your help >>> Simon >>> >>> >>> ------------------------------- >>> Excerpt from example stylesheet: >>> ... >>> <Document> >>> <xsl:text> >>> block1 line1 >>> block1 line2 >>> </xsl:text> >>> >>> <xsl:text>
</xsl:text> >>> <xsl:text>block2 line1</xsl:text> >>> <xsl:text>
</xsl:text> >>> <xsl:text>block2 line2</xsl:text> >>> >>> <xsl:text> >>> </xsl:text> >>> >>> <xsl:text>block3 line1</xsl:text> >>> >>> <xsl:text> >>> </xsl:text> >>> >>> <xsl:text>block3 line2</xsl:text> >>> >>> </Document> >>> .... >>> >>> ------------------------------------ >>> Result when running the stylesheet with saxon on the filesystem; All >>> line break variants actually result in new lines: >>> <Document> >>> block1 zeile1 >>> block1 zeile2 >>> block2 zeile1 >>> block2 zeile1 >>> block3 zeile1 >>> block3 zeile1 >>> </Document> >>> ---------------------------------- >>> >>> Result from calling xslt:transform in basex (saxon is used also >>> within basex); Only the line break within the xsl:text is visible in >>> the >>> output: >>> <Document> >>> block1 line1 >>> block1 line2block2 line1block2 line2block3 line1block3 line2 >>> </Document> >>> ---------------------------------- >>> >>> >>> >>> _______________________________________________ >>> BaseX-Talk mailing list >>> BaseX-Talk(a)mailman.uni-konstanz.de >>> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >> >

2 3

Xquery on streaming xml
by EV Narasimham 31 Jul '13

31 Jul '13

Hi, Can we use basex for processing streaming xml. If yes, how?. The idea being: - Create a ramdisk/ram drive having size of about 10GB. - load the file into ram drive. - Use basex to stream the xml file during xquery evaluation. Once data is extracted from the xml file, the xml file will be deleted. Since this activity is transient in nature, the creation of xml database can be avoided. Regards Nesh

2 1

Re: [basex-talk] losing line breaks with xslt:transform(...)
by Simon Görke 31 Jul '13

31 Jul '13

Hi Christian, I narrowed it down a little bit. Though turning it off in the xquery prologue as proposed did not help, the problem seems actually to be linked to the chopping option. What I did not mention in my first posting is that the xslts are also stored in a (second) basex database. What I found out is that the result of the transformation is ok (i.e. with all the line breaks required) if I *manually* create the database for the xslts in the basex-gui and *manually* set the chopping option to off in the creation window. Surprisingly, when I create the databases automatically using the following xml-snippet <commands> <set option='chop'>false</set> <create-db name='fromgme'/> <create-db name='export'/> <create-db name='xslt'/> </commands> All databases have chopping set to off according to the GUI, but still the line breaks are lost in the transformation ??? I also found this thread http://www.mail-archive.com/basex-talk@mailman.uni-konstanz.de/msg02257.html with a similar problem, however locally setting the chop value as proposed is no remedy for my problem. return (# db:chop "no" #) { xslt:transform($xml,$xslt) } Is there a possibility to disable chopping globally ? Our databases are either huge or contain text, but not both... Thanks in advance Simon -----Ursprüngliche Nachricht----- Von: Christian Grün [mailto:christian.gruen@gmail.com] Gesendet: Dienstag, 30. Juli 2013 17:26 An: Simon Görke Cc: BaseX Betreff: Re: [basex-talk] losing line breaks with xslt:transform(...) Hi Simon, could you provide us with a minimized example that demonstrates the (mis)behavior? Thanks in advance Christian ___________________________ 2013/7/30 Simon Görke <simon.goerke(a)ils.uni-stuttgart.de>: > Christian, > > thanks for the hint, but no change for the better. I also tried with > the basex-internal xslt-processor (instead of saxon) but resulted in > the same problem. > > Any ideas welcome... > > Greets, > Simon > > > > -----Ursprüngliche Nachricht----- > Von: Christian Grün [mailto:christian.gruen@gmail.com] > Gesendet: Freitag, 26. Juli 2013 14:56 > An: Simon Görke > Cc: BaseX > Betreff: Re: [basex-talk] losing line breaks with xslt:transform(...) > > Hi Simon, > > not sure if that helps, but you can try to deactivate whitespace > chopping in the prolog of your XQuery expression: > > declare option db:chop "no"; > ... > > Best, > Christian > ___________________________ > > 2013/7/26 Simon Görke <simon.goerke(a)ils.uni-stuttgart.de>: >> Hi, >> >> I have small issue regarding the xslt:transform function (or what >> lies beneath). We are currently migrating a larger number of existing >> stylesheets into a basex-environment (before they were used with >> saxon on the filesystem). >> >> The problem is, that all line breaks which are generated explicitly >> in the stylesheets (e.g. by using <xsl:text>
</xsl:text>) get lost. >> Line feeds within a xsl:text element are correctly transformed. See >> below for the xslt and example output. >> >> First we thought this might be linked to the whitespace chopping of >> the database where the transformation result is stored, however I get >> the same behavior when directly looking at the output of >> xslt:transform(...) in the basex-gui. >> Is there any options to influence this behavior? We are using saxon9 >> within basex version 7.6. >> >> Thanks for your help >> Simon >> >> >> ------------------------------- >> Excerpt from example stylesheet: >> ... >> <Document> >> <xsl:text> >> block1 line1 >> block1 line2 >> </xsl:text> >> >> <xsl:text>
</xsl:text> >> <xsl:text>block2 line1</xsl:text> >> <xsl:text>
</xsl:text> >> <xsl:text>block2 line2</xsl:text> >> >> <xsl:text> >> </xsl:text> >> >> <xsl:text>block3 line1</xsl:text> >> >> <xsl:text> >> </xsl:text> >> >> <xsl:text>block3 line2</xsl:text> >> >> </Document> >> .... >> >> ------------------------------------ >> Result when running the stylesheet with saxon on the filesystem; All >> line break variants actually result in new lines: >> <Document> >> block1 zeile1 >> block1 zeile2 >> block2 zeile1 >> block2 zeile1 >> block3 zeile1 >> block3 zeile1 >> </Document> >> ---------------------------------- >> >> Result from calling xslt:transform in basex (saxon is used also >> within basex); Only the line break within the xsl:text is visible in >> the >> output: >> <Document> >> block1 line1 >> block1 line2block2 line1block2 line2block3 line1block3 line2 >> </Document> >> ---------------------------------- >> >> >> >> _______________________________________________ >> BaseX-Talk mailing list >> BaseX-Talk(a)mailman.uni-konstanz.de >> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >

3 2

Re: [basex-talk] losing line breaks with xslt:transform(...)
by Simon Görke 30 Jul '13

30 Jul '13

Christian, thanks for the hint, but no change for the better. I also tried with the basex-internal xslt-processor (instead of saxon) but resulted in the same problem. Any ideas welcome... Greets, Simon -----Ursprüngliche Nachricht----- Von: Christian Grün [mailto:christian.gruen@gmail.com] Gesendet: Freitag, 26. Juli 2013 14:56 An: Simon Görke Cc: BaseX Betreff: Re: [basex-talk] losing line breaks with xslt:transform(...) Hi Simon, not sure if that helps, but you can try to deactivate whitespace chopping in the prolog of your XQuery expression: declare option db:chop "no"; ... Best, Christian ___________________________ 2013/7/26 Simon Görke <simon.goerke(a)ils.uni-stuttgart.de>: > Hi, > > I have small issue regarding the xslt:transform function (or what lies > beneath). We are currently migrating a larger number of existing > stylesheets into a basex-environment (before they were used with saxon > on the filesystem). > > The problem is, that all line breaks which are generated explicitly in > the stylesheets (e.g. by using <xsl:text>
</xsl:text>) get lost. > Line feeds within a xsl:text element are correctly transformed. See > below for the xslt and example output. > > First we thought this might be linked to the whitespace chopping of > the database where the transformation result is stored, however I get > the same behavior when directly looking at the output of > xslt:transform(...) in the basex-gui. > Is there any options to influence this behavior? We are using saxon9 > within basex version 7.6. > > Thanks for your help > Simon > > > ------------------------------- > Excerpt from example stylesheet: > ... > <Document> > <xsl:text> > block1 line1 > block1 line2 > </xsl:text> > > <xsl:text>
</xsl:text> > <xsl:text>block2 line1</xsl:text> > <xsl:text>
</xsl:text> > <xsl:text>block2 line2</xsl:text> > > <xsl:text> > </xsl:text> > > <xsl:text>block3 line1</xsl:text> > > <xsl:text> > </xsl:text> > > <xsl:text>block3 line2</xsl:text> > > </Document> > .... > > ------------------------------------ > Result when running the stylesheet with saxon on the filesystem; All > line break variants actually result in new lines: > <Document> > block1 zeile1 > block1 zeile2 > block2 zeile1 > block2 zeile1 > block3 zeile1 > block3 zeile1 > </Document> > ---------------------------------- > > Result from calling xslt:transform in basex (saxon is used also within > basex); Only the line break within the xsl:text is visible in the > output: > <Document> > block1 line1 > block1 line2block2 line1block2 line2block3 line1block3 line2 > </Document> > ---------------------------------- > > > > _______________________________________________ > BaseX-Talk mailing list > BaseX-Talk(a)mailman.uni-konstanz.de > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk

2 1

connection provblems with basex 7.6
by Andreas Wombacher 30 Jul '13

30 Jul '13

Hi, I am running a server in Java on a linux machine and want to connect to it from a windows machine (either GUI or java code). It is not working. I get as a response that the connection is refused. On the linux machine the server is running - I see the port being listened on. The firewall is also open on that port - I can telnet the port and get a response. Any idea? I use these settings in .basex # Client/Server Architecture HOST = 130.89.1.182 PORT = 19840 SERVERPORT = 1984 EVENTPORT = 1985 USER = XXXX PASSWORD = XXXX SERVERHOST = silo1.ewi.utwente.nl thanbks in advance. best regards Andreas

1 0

losing line breaks with xslt:transform(...)
by Simon Görke 26 Jul '13

26 Jul '13

Hi, I have small issue regarding the xslt:transform function (or what lies beneath). We are currently migrating a larger number of existing stylesheets into a basex-environment (before they were used with saxon on the filesystem). The problem is, that all line breaks which are generated explicitly in the stylesheets (e.g. by using <xsl:text>
</xsl:text>) get lost. Line feeds within a xsl:text element are correctly transformed. See below for the xslt and example output. First we thought this might be linked to the whitespace chopping of the database where the transformation result is stored, however I get the same behavior when directly looking at the output of xslt:transform(...) in the basex-gui. Is there any options to influence this behavior? We are using saxon9 within basex version 7.6. Thanks for your help Simon ------------------------------- Excerpt from example stylesheet: ... <Document> <xsl:text> block1 line1 block1 line2 </xsl:text> <xsl:text>
</xsl:text> <xsl:text>block2 line1</xsl:text> <xsl:text>
</xsl:text> <xsl:text>block2 line2</xsl:text> <xsl:text> </xsl:text> <xsl:text>block3 line1</xsl:text> <xsl:text> </xsl:text> <xsl:text>block3 line2</xsl:text> </Document> .... ------------------------------------ Result when running the stylesheet with saxon on the filesystem; All line break variants actually result in new lines: <Document> block1 zeile1 block1 zeile2 block2 zeile1 block2 zeile1 block3 zeile1 block3 zeile1 </Document> ---------------------------------- Result from calling xslt:transform in basex (saxon is used also within basex); Only the line break within the xsl:text is visible in the output: <Document> block1 line1 block1 line2block2 line1block2 line2block3 line1block3 line2 </Document> ----------------------------------

2 1

A Resilient BaseX deployment
by Andrew Miller 24 Jul '13

24 Jul '13

Hi Folks, We are looking at deploying BaseX as a resilient deployment. If the BaseX server fails, then a stand-by server is to take over. My problem is how to replicate the data from the active Master to the passive slave. I found a thread from July 2012 proposing replicating the data directory using rsync. However, I have concerns about this. Firstly, if BaseX is writing to the database at the time of failure then the database on the Master would be corrupted, wouldn't it? Secondly, rsync does not guarantee a consistent file set is transferred. If BaseX updates the database during the rsync transfer, or the server fails during the rsync transfer, the database on the Slave would be corrupt, wouldn't it? I am considering using a cron job to do a "CREATE BACKUP" on the database at regular intervals and push the resulting zip file to the slave. If the Master fails during this process, we can detect the failure (at best we get an incomplete zip file, which will not be valid). If this happens we can fall back to the previous backup. If the Master fails, we detect this (using monit), start BaseX on the slave and do a database restore. I think this should give us a resilient deployment. In the worse case scenario (master dying during a replication) we lose changes since the last back-up, but at least we get a working database. Does this sound like a sound strategy? When I do a database restore, does BaseX automatically cope if the latest backup is corrupt? -- Andrew Miller Crocodile RCS Ltd

2 2

Basex json module and namespaces
by Arto Viitanen 24 Jul '13

24 Jul '13

Is there any way to make json module (mainly json:serialize) handle namespaces? I tried following: let $str := '{"name":"Mike Mac","address":{"postnumber":20221,"city":"NYC","street":"23 west A 22"},"email":"mm(a)gmail.com"}' let $json := json:parse($str) return json:serialize($json) and I get the string, with linefeed etc. But I would like to add the $json element inside another element, which have default namespace. let $str := '{"name":"Mike Mac","address":{"postnumber":20221,"city":"NYC","street":"23 west A 22"},"email":"mm(a)gmail.com"}' let $json := json:parse($str) let $wrapper := copy $x := <wrapper xmlns="namespace-uri"/> modify (insert node $json into $x) return $x let $json2 := $wrapper/*[1] return json:serialize($json2) (That is, insert the $json inside the <wrapper> -element, get it out and serialize it.) Instead of the same result as in the first case, I get 'BXJS0001 JSON serialization: Element <json> has invalid attribute "xmlns". ' But since it is default namespace, there is no easy way to drop it. My question is: is there any way to let json:serialize handle or ignore namespaces; at least default namespaces? -- Arto Viitanen Microteam Oy Tampere, Finland

2 1