July 2013 - BaseX-Talk - mailman.uni-konstanz.de

JSON or JSONML output formatting
by Zachary Pride 16 Jul '14

16 Jul '14

Hello, I've been playing with BaseX lately and all-in-all am very impressed. Something I'm wrestling with however is the JSON output options. I've got various web applications that would really be simplified by receiving JSON straight away for the obvious scripting benefits. The JSON method has been giving me a lot of grief, stipulating rules I'm not sure how to follow. right now I have an element that "is typed as string and cannot be nested." I've been able to get JSONML to output but the JSON is this morass of nested Arrays... Attributes seem to get transformed into Objects but elements/tags and their values are pushed into Arrays for some rationale - which I suppose is on purpose. My question is how can I get Objects for simple key:value properties from <tag>value</tag> 's. Are there options I'm not utilizing? Would it be feasible to transform the XML output to some middle form first - setting it up to become Objects, before serializing it to JSON? Thanks! Zach Pride Associate Software Engineer Ultra Electronics, ProLogic 1000 Green River Drive, Ste. 201 Fairmont, WV 26554 zachary.pride(a)ultra-prologic.com Tel: +1 304 333 2680 ext.308 www.ultra-prologic.com This e-mail from Ultra Electronics, ProLogic and any attachments to it are confidential to the intended recipient and may also be privileged or controlled by U.S. export control laws and regulations. No technical data, information or other items provided by Ultra Electronics, ProLogic in connection with this email shall be shared or provided to any foreign persons, entities, or subsidiaries without the expressed written authorization of Ultra Electronic, ProLogic. If you have received it in error please notify the sender and delete it from your system. If you are not the intended recipient you must not copy it or use it for any purpose nor disclose or distribute its contents to any other person. All communications may be subject to interception or monitoring for operational and/or security purposes. Please rely on your own virus checking as the sender cannot accept any liability for any damage arising from any bug or virus infection. This e-mail from Ultra Electronics, ProLogic and any attachments to it are confidential to the intended recipient and may also be privileged or controlled by U.S. export control laws and regulations. No technical data, information or other items provided by Ultra Electronics, ProLogic in connection with this email shall be shared or provided to any foreign persons, entities, or subsidiaries without the expressed written authorization of Ultra Electronic, ProLogic. If you have received it in error please notify the sender and delete it from your system. If you are not the intended recipient you must not copy it or use it for any purpose nor disclose or distribute its contents to any other person. All communications may be subject to interception or monitoring for operational and/or security purposes. Please rely on your own virus checking as the sender cannot accept any liability for any damage arising from any bug or virus infection. Please consider the environment before printing this email.

4 4

Dynamic Module Loading
by Jean-philippe Magué 19 May '14

19 May '14

Hello, Exist-db has an util:import-module<http://exist-db.org/exist/apps/fundocs/view.html?uri=http://exist-db.org/xq…>function that allows to dynamically import an XQuery module at run time. I haven't found anything equivalent in BaseX. Is that right ? I have also tried to see how to implement such a function directly in Java, but as far as I understand the module parsing process, functions prefix are checked at compile time. That would compromise any attempt to write such a function. Am I right ? Best, Jean-Philippe -- Jean-Philippe Magué Maître de Conférences en Humanités Numériques Responsable de l'Atelier des Humanités Numériques : ahn.ens-lyon.fr Ecole Normale Supérieure de Lyon +33 (0)4 26 23 38 08 -- "Using computers will therefore lead us to a more profound and systematic knowledge of human expression; in principle, it can help us to be more humanistic than before." Busa, 1980.

6 16

get validate:xsd to work like validate:dtd
by France Baril 01 Aug '13

01 Aug '13

<xs:schema id="schema-images" xmlns:xs="http://www.w3.org/2001/XMLSchema"elementFormDefault ="qualified"> <xs:include schemaLocation="HowToContent_Common.xsd"/> ... </xs:element> </xs:schema> Does not pickup the definitions in the common .xsd files when running validate:xsd. It returns errors for items defined in the 'included' .xsd. Referencing a 'sub-DTD' does work with the DTD (validate:dtd) when the common element is included using a similar path. <!ENTITY % common PUBLIC "-//MOTOROLA//DTD HOW2 COMMON//EN" "HowToContent_Common.mod"> %common; How do I get validate:xsd to grab the sub-components? I'm currently using validate:xsd like this: *let* *$xsd* := *db:open*('AppResources')/*[*@id*=*$xsd-id*] *try* { *validate:xsd*(*$xml-file*, *$xsd*) } *catch* *** { <li *style*=*"color:red;"*><p>File '{* $xml-file*/*base-uri*()}' doesn't validate against the '{*$xsd*/* base-uri*()}' schema.</p><p>{'Error [' || *$err:code* || ']: ' || * $err:description*}</p></li> } My files validate correctly when I run validation in XML Oxygen, so I know the XSD itself is working. I tried using lower caps in the xs:include reference as I noticed that the .xsd files get renamed to lower caps on import (but not DTDs [because they were considered raw and unindexed?]). Anyhow, it didn't help. Regards, France -- France Baril Architecte documentaire / Documentation architect france.baril(a)architextus.com (514) 572-0341

3 10

connection provblems with basex 7.6
by Andreas Wombacher 01 Aug '13

01 Aug '13

Hi, I am running a server in Java on a linux machine and want to connect to it from a windows machine (either GUI or java code). It is not working. I get as a response that the connection is refused. On the linux machine the server is running - I see the port being listened on. The firewall is also open on that port - I can telnet the port and get a response. Any idea? I use these settings in .basex # Client/Server Architecture HOST = 130.89.1.182 PORT = 19840 SERVERPORT = 1984 EVENTPORT = 1985 USER = XXXX PASSWORD = XXXX SERVERHOST = silo1.ewi.utwente.nl thanbks in advance. best regards Andreas

3 6

Out of memory bug
by Ivan Lagunov 31 Jul '13

31 Jul '13

Hello, I've got 'Out of memory' error and corrupted database again, this time it's for 6.3 version. Steps: 1. Started server, started terminal client, checked there are no text and attribute indexes. 2. Started my Java application that runs several tests on database: querying, adding, removing. 3. Then I forgot to reopen terminal client and used the one that I opened in Step 1. 4. Run optimize command for the collection and got error. 5. Cannot open the collection any more as I'm getting 'Out of memory'. It's clear that I shouldn't have used the terminal client that I opened before execution of my Java application. But sometimes it's hard to remember. And it's completely unacceptable that these actions result in the corrupted database. You really should catch ArrayIndexOutOfBoundsException everywhere in order to save database from being corrupted. Terminal client stacktrace: > optimize Possible bug? Feedback is welcome: basex-talk(a)mailman.uni-konstanz.de BaseX 6.3: java.lang.ArrayIndexOutOfBoundsException: -1 org.basex.data.PathSummary.add(PathSummary.java:68) org.basex.core.cmd.Optimize.stats(Optimize.java:68) org.basex.core.cmd.Optimize.run(Optimize.java:34) org.basex.core.Command.run(Command.java:236) org.basex.core.Command.exec(Command.java:218) org.basex.core.Command.execute(Command.java:66) org.basex.server.ServerProcess.run(ServerProcess.java:161) > open products Out of Main Memory. Best regards, Ivan

4 8

Re: [basex-talk] losing line breaks with xslt:transform(...)
by Simon Görke 31 Jul '13

31 Jul '13

Hi there, Setting CHOP=false in the .basex configuration solves my xslt issue but seems to have an ugly side-effect (bug?): Using basex commands in xml syntax is not working any more; if I call: basexclient.bat -c cmd.xml with the content of cmd.xml being (for example): <commands> <create-db name='xslt'/> </commands> it always returns " Syntax: <commands><...></commands>". Executing the same without CHOP=false in .basex works like a charm. Is there any chance this could be fixed? Best, Simon -----Ursprüngliche Nachricht----- Von: Christian Grün [mailto:christian.gruen@gmail.com] Gesendet: Mittwoch, 31. Juli 2013 12:39 An: Simon Görke Cc: BaseX Betreff: Re: [basex-talk] losing line breaks with xslt:transform(...) Hi Simon, good to hear you could narrow down the problem. > Is there a possibility to disable chopping globally ? Our databases > are either huge or contain text, but not both... the easiest way is to set CHOP=false in your .basex configuration [1]. As Gerrit indicates, the chopping flag caused frequent problems in the past. Due to legacy and storage issues, we are still hesitant to simply change the default, but we are working on it. Best, Christian [1] http://docs.basex.org/wiki/Options > -----Ursprüngliche Nachricht----- > Von: Christian Grün [mailto:christian.gruen@gmail.com] > Gesendet: Dienstag, 30. Juli 2013 17:26 > An: Simon Görke > Cc: BaseX > Betreff: Re: [basex-talk] losing line breaks with xslt:transform(...) > > Hi Simon, > > could you provide us with a minimized example that demonstrates the > (mis)behavior? > > Thanks in advance > Christian > ___________________________ > > 2013/7/30 Simon Görke <simon.goerke(a)ils.uni-stuttgart.de>: >> Christian, >> >> thanks for the hint, but no change for the better. I also tried with >> the basex-internal xslt-processor (instead of saxon) but resulted in >> the same problem. >> >> Any ideas welcome... >> >> Greets, >> Simon >> >> >> >> -----Ursprüngliche Nachricht----- >> Von: Christian Grün [mailto:christian.gruen@gmail.com] >> Gesendet: Freitag, 26. Juli 2013 14:56 >> An: Simon Görke >> Cc: BaseX >> Betreff: Re: [basex-talk] losing line breaks with xslt:transform(...) >> >> Hi Simon, >> >> not sure if that helps, but you can try to deactivate whitespace >> chopping in the prolog of your XQuery expression: >> >> declare option db:chop "no"; >> ... >> >> Best, >> Christian >> ___________________________ >> >> 2013/7/26 Simon Görke <simon.goerke(a)ils.uni-stuttgart.de>: >>> Hi, >>> >>> I have small issue regarding the xslt:transform function (or what >>> lies beneath). We are currently migrating a larger number of >>> existing stylesheets into a basex-environment (before they were used >>> with saxon on the filesystem). >>> >>> The problem is, that all line breaks which are generated explicitly >>> in the stylesheets (e.g. by using <xsl:text>
</xsl:text>) get lost. >>> Line feeds within a xsl:text element are correctly transformed. See >>> below for the xslt and example output. >>> >>> First we thought this might be linked to the whitespace chopping of >>> the database where the transformation result is stored, however I >>> get the same behavior when directly looking at the output of >>> xslt:transform(...) in the basex-gui. >>> Is there any options to influence this behavior? We are using saxon9 >>> within basex version 7.6. >>> >>> Thanks for your help >>> Simon >>> >>> >>> ------------------------------- >>> Excerpt from example stylesheet: >>> ... >>> <Document> >>> <xsl:text> >>> block1 line1 >>> block1 line2 >>> </xsl:text> >>> >>> <xsl:text>
</xsl:text> >>> <xsl:text>block2 line1</xsl:text> >>> <xsl:text>
</xsl:text> >>> <xsl:text>block2 line2</xsl:text> >>> >>> <xsl:text> >>> </xsl:text> >>> >>> <xsl:text>block3 line1</xsl:text> >>> >>> <xsl:text> >>> </xsl:text> >>> >>> <xsl:text>block3 line2</xsl:text> >>> >>> </Document> >>> .... >>> >>> ------------------------------------ >>> Result when running the stylesheet with saxon on the filesystem; All >>> line break variants actually result in new lines: >>> <Document> >>> block1 zeile1 >>> block1 zeile2 >>> block2 zeile1 >>> block2 zeile1 >>> block3 zeile1 >>> block3 zeile1 >>> </Document> >>> ---------------------------------- >>> >>> Result from calling xslt:transform in basex (saxon is used also >>> within basex); Only the line break within the xsl:text is visible in >>> the >>> output: >>> <Document> >>> block1 line1 >>> block1 line2block2 line1block2 line2block3 line1block3 line2 >>> </Document> >>> ---------------------------------- >>> >>> >>> >>> _______________________________________________ >>> BaseX-Talk mailing list >>> BaseX-Talk(a)mailman.uni-konstanz.de >>> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >> >

2 3

Xquery on streaming xml
by EV Narasimham 31 Jul '13

31 Jul '13

Hi, Can we use basex for processing streaming xml. If yes, how?. The idea being: - Create a ramdisk/ram drive having size of about 10GB. - load the file into ram drive. - Use basex to stream the xml file during xquery evaluation. Once data is extracted from the xml file, the xml file will be deleted. Since this activity is transient in nature, the creation of xml database can be avoided. Regards Nesh

2 1

Re: [basex-talk] losing line breaks with xslt:transform(...)
by Simon Görke 31 Jul '13

31 Jul '13

Hi Christian, I narrowed it down a little bit. Though turning it off in the xquery prologue as proposed did not help, the problem seems actually to be linked to the chopping option. What I did not mention in my first posting is that the xslts are also stored in a (second) basex database. What I found out is that the result of the transformation is ok (i.e. with all the line breaks required) if I *manually* create the database for the xslts in the basex-gui and *manually* set the chopping option to off in the creation window. Surprisingly, when I create the databases automatically using the following xml-snippet <commands> <set option='chop'>false</set> <create-db name='fromgme'/> <create-db name='export'/> <create-db name='xslt'/> </commands> All databases have chopping set to off according to the GUI, but still the line breaks are lost in the transformation ??? I also found this thread http://www.mail-archive.com/basex-talk@mailman.uni-konstanz.de/msg02257.html with a similar problem, however locally setting the chop value as proposed is no remedy for my problem. return (# db:chop "no" #) { xslt:transform($xml,$xslt) } Is there a possibility to disable chopping globally ? Our databases are either huge or contain text, but not both... Thanks in advance Simon -----Ursprüngliche Nachricht----- Von: Christian Grün [mailto:christian.gruen@gmail.com] Gesendet: Dienstag, 30. Juli 2013 17:26 An: Simon Görke Cc: BaseX Betreff: Re: [basex-talk] losing line breaks with xslt:transform(...) Hi Simon, could you provide us with a minimized example that demonstrates the (mis)behavior? Thanks in advance Christian ___________________________ 2013/7/30 Simon Görke <simon.goerke(a)ils.uni-stuttgart.de>: > Christian, > > thanks for the hint, but no change for the better. I also tried with > the basex-internal xslt-processor (instead of saxon) but resulted in > the same problem. > > Any ideas welcome... > > Greets, > Simon > > > > -----Ursprüngliche Nachricht----- > Von: Christian Grün [mailto:christian.gruen@gmail.com] > Gesendet: Freitag, 26. Juli 2013 14:56 > An: Simon Görke > Cc: BaseX > Betreff: Re: [basex-talk] losing line breaks with xslt:transform(...) > > Hi Simon, > > not sure if that helps, but you can try to deactivate whitespace > chopping in the prolog of your XQuery expression: > > declare option db:chop "no"; > ... > > Best, > Christian > ___________________________ > > 2013/7/26 Simon Görke <simon.goerke(a)ils.uni-stuttgart.de>: >> Hi, >> >> I have small issue regarding the xslt:transform function (or what >> lies beneath). We are currently migrating a larger number of existing >> stylesheets into a basex-environment (before they were used with >> saxon on the filesystem). >> >> The problem is, that all line breaks which are generated explicitly >> in the stylesheets (e.g. by using <xsl:text>
</xsl:text>) get lost. >> Line feeds within a xsl:text element are correctly transformed. See >> below for the xslt and example output. >> >> First we thought this might be linked to the whitespace chopping of >> the database where the transformation result is stored, however I get >> the same behavior when directly looking at the output of >> xslt:transform(...) in the basex-gui. >> Is there any options to influence this behavior? We are using saxon9 >> within basex version 7.6. >> >> Thanks for your help >> Simon >> >> >> ------------------------------- >> Excerpt from example stylesheet: >> ... >> <Document> >> <xsl:text> >> block1 line1 >> block1 line2 >> </xsl:text> >> >> <xsl:text>
</xsl:text> >> <xsl:text>block2 line1</xsl:text> >> <xsl:text>
</xsl:text> >> <xsl:text>block2 line2</xsl:text> >> >> <xsl:text> >> </xsl:text> >> >> <xsl:text>block3 line1</xsl:text> >> >> <xsl:text> >> </xsl:text> >> >> <xsl:text>block3 line2</xsl:text> >> >> </Document> >> .... >> >> ------------------------------------ >> Result when running the stylesheet with saxon on the filesystem; All >> line break variants actually result in new lines: >> <Document> >> block1 zeile1 >> block1 zeile2 >> block2 zeile1 >> block2 zeile1 >> block3 zeile1 >> block3 zeile1 >> </Document> >> ---------------------------------- >> >> Result from calling xslt:transform in basex (saxon is used also >> within basex); Only the line break within the xsl:text is visible in >> the >> output: >> <Document> >> block1 line1 >> block1 line2block2 line1block2 line2block3 line1block3 line2 >> </Document> >> ---------------------------------- >> >> >> >> _______________________________________________ >> BaseX-Talk mailing list >> BaseX-Talk(a)mailman.uni-konstanz.de >> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >

3 2

Re: [basex-talk] losing line breaks with xslt:transform(...)
by Simon Görke 30 Jul '13

30 Jul '13

Christian, thanks for the hint, but no change for the better. I also tried with the basex-internal xslt-processor (instead of saxon) but resulted in the same problem. Any ideas welcome... Greets, Simon -----Ursprüngliche Nachricht----- Von: Christian Grün [mailto:christian.gruen@gmail.com] Gesendet: Freitag, 26. Juli 2013 14:56 An: Simon Görke Cc: BaseX Betreff: Re: [basex-talk] losing line breaks with xslt:transform(...) Hi Simon, not sure if that helps, but you can try to deactivate whitespace chopping in the prolog of your XQuery expression: declare option db:chop "no"; ... Best, Christian ___________________________ 2013/7/26 Simon Görke <simon.goerke(a)ils.uni-stuttgart.de>: > Hi, > > I have small issue regarding the xslt:transform function (or what lies > beneath). We are currently migrating a larger number of existing > stylesheets into a basex-environment (before they were used with saxon > on the filesystem). > > The problem is, that all line breaks which are generated explicitly in > the stylesheets (e.g. by using <xsl:text>
</xsl:text>) get lost. > Line feeds within a xsl:text element are correctly transformed. See > below for the xslt and example output. > > First we thought this might be linked to the whitespace chopping of > the database where the transformation result is stored, however I get > the same behavior when directly looking at the output of > xslt:transform(...) in the basex-gui. > Is there any options to influence this behavior? We are using saxon9 > within basex version 7.6. > > Thanks for your help > Simon > > > ------------------------------- > Excerpt from example stylesheet: > ... > <Document> > <xsl:text> > block1 line1 > block1 line2 > </xsl:text> > > <xsl:text>
</xsl:text> > <xsl:text>block2 line1</xsl:text> > <xsl:text>
</xsl:text> > <xsl:text>block2 line2</xsl:text> > > <xsl:text> > </xsl:text> > > <xsl:text>block3 line1</xsl:text> > > <xsl:text> > </xsl:text> > > <xsl:text>block3 line2</xsl:text> > > </Document> > .... > > ------------------------------------ > Result when running the stylesheet with saxon on the filesystem; All > line break variants actually result in new lines: > <Document> > block1 zeile1 > block1 zeile2 > block2 zeile1 > block2 zeile1 > block3 zeile1 > block3 zeile1 > </Document> > ---------------------------------- > > Result from calling xslt:transform in basex (saxon is used also within > basex); Only the line break within the xsl:text is visible in the > output: > <Document> > block1 line1 > block1 line2block2 line1block2 line2block3 line1block3 line2 > </Document> > ---------------------------------- > > > > _______________________________________________ > BaseX-Talk mailing list > BaseX-Talk(a)mailman.uni-konstanz.de > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk

2 1

connection provblems with basex 7.6
by Andreas Wombacher 30 Jul '13

30 Jul '13

Hi, I am running a server in Java on a linux machine and want to connect to it from a windows machine (either GUI or java code). It is not working. I get as a response that the connection is refused. On the linux machine the server is running - I see the port being listened on. The firewall is also open on that port - I can telnet the port and get a response. Any idea? I use these settings in .basex # Client/Server Architecture HOST = 130.89.1.182 PORT = 19840 SERVERPORT = 1984 EVENTPORT = 1985 USER = XXXX PASSWORD = XXXX SERVERHOST = silo1.ewi.utwente.nl thanbks in advance. best regards Andreas

1 0