August 2013 - BaseX-Talk - mailman.uni-konstanz.de

JSON or JSONML output formatting
by Zachary Pride 16 Jul '14

16 Jul '14

Hello, I've been playing with BaseX lately and all-in-all am very impressed. Something I'm wrestling with however is the JSON output options. I've got various web applications that would really be simplified by receiving JSON straight away for the obvious scripting benefits. The JSON method has been giving me a lot of grief, stipulating rules I'm not sure how to follow. right now I have an element that "is typed as string and cannot be nested." I've been able to get JSONML to output but the JSON is this morass of nested Arrays... Attributes seem to get transformed into Objects but elements/tags and their values are pushed into Arrays for some rationale - which I suppose is on purpose. My question is how can I get Objects for simple key:value properties from <tag>value</tag> 's. Are there options I'm not utilizing? Would it be feasible to transform the XML output to some middle form first - setting it up to become Objects, before serializing it to JSON? Thanks! Zach Pride Associate Software Engineer Ultra Electronics, ProLogic 1000 Green River Drive, Ste. 201 Fairmont, WV 26554 zachary.pride(a)ultra-prologic.com Tel: +1 304 333 2680 ext.308 www.ultra-prologic.com This e-mail from Ultra Electronics, ProLogic and any attachments to it are confidential to the intended recipient and may also be privileged or controlled by U.S. export control laws and regulations. No technical data, information or other items provided by Ultra Electronics, ProLogic in connection with this email shall be shared or provided to any foreign persons, entities, or subsidiaries without the expressed written authorization of Ultra Electronic, ProLogic. If you have received it in error please notify the sender and delete it from your system. If you are not the intended recipient you must not copy it or use it for any purpose nor disclose or distribute its contents to any other person. All communications may be subject to interception or monitoring for operational and/or security purposes. Please rely on your own virus checking as the sender cannot accept any liability for any damage arising from any bug or virus infection. This e-mail from Ultra Electronics, ProLogic and any attachments to it are confidential to the intended recipient and may also be privileged or controlled by U.S. export control laws and regulations. No technical data, information or other items provided by Ultra Electronics, ProLogic in connection with this email shall be shared or provided to any foreign persons, entities, or subsidiaries without the expressed written authorization of Ultra Electronic, ProLogic. If you have received it in error please notify the sender and delete it from your system. If you are not the intended recipient you must not copy it or use it for any purpose nor disclose or distribute its contents to any other person. All communications may be subject to interception or monitoring for operational and/or security purposes. Please rely on your own virus checking as the sender cannot accept any liability for any damage arising from any bug or virus infection. Please consider the environment before printing this email.

4 4

Dynamic Module Loading
by Jean-philippe Magué 19 May '14

19 May '14

Hello, Exist-db has an util:import-module<http://exist-db.org/exist/apps/fundocs/view.html?uri=http://exist-db.org/xq…>function that allows to dynamically import an XQuery module at run time. I haven't found anything equivalent in BaseX. Is that right ? I have also tried to see how to implement such a function directly in Java, but as far as I understand the module parsing process, functions prefix are checked at compile time. That would compromise any attempt to write such a function. Am I right ? Best, Jean-Philippe -- Jean-Philippe Magué Maître de Conférences en Humanités Numériques Responsable de l'Atelier des Humanités Numériques : ahn.ens-lyon.fr Ecole Normale Supérieure de Lyon +33 (0)4 26 23 38 08 -- "Using computers will therefore lead us to a more profound and systematic knowledge of human expression; in principle, it can help us to be more humanistic than before." Busa, 1980.

6 16

BaseX 7.7 error in working with EQNames
by Dimitre Novatchev 11 Sep '13

11 Sep '13

Hi, Given this XML document (in file example.xml): Freude, schöner Götterfunken, Tochter aus Elysium, Wir betreten feuertrunken, Himmlische, dein Heiligtum. This XQuery expression (in file TesQNametExpression.xquery, in the same directory): let $doc := doc('example.xml') return string-length($doc/Q{ http://example.com/one}p[1]/@Q{http://www.w3.org/XML/1998/namespace}lang) produces: 0 However the correct result (which I get with Saxon 9.5.01) is: 1 Here is the "Query Info" from the BaseX UI: Compiling: - pre-evaluating fn:doc("example.xml") - binding static variable $doc - removing unknown element/attribute Q{http://example.com/one}p - pre-evaluating fn:string-length(()) - removing variable $doc - simplifying flwor expression Query: let $doc := doc('example.xml') return string-length($doc/Q{ http://example.com/one}p[1]/@Q{http://www.w3.org/XML/1998/namespace}lang) Optimized Query: 0 Hope that this information would be useful for identifying and fixing the issue. -- Cheers, Dimitre Novatchev --------------------------------------- Truly great madness cannot be achieved without significant intelligence. --------------------------------------- To invent, you need a good imagination and a pile of junk ------------------------------------- Never fight an inanimate object ------------------------------------- To avoid situations in which you might make mistakes may be the biggest mistake of all ------------------------------------ Quality means doing it right when no one is looking. ------------------------------------- You've achieved success in your field when you don't know whether what you're doing is work or play ------------------------------------- Facts do not cease to exist because they are ignored. ------------------------------------- Typing monkeys will write all Shakespeare's works in 200yrs.Will they write all patents, too? :) ------------------------------------- I finally figured out the only reason to be alive is to enjoy it.

2 2

Too many opened files error
by Alexander Shpack 05 Sep '13

05 Sep '13

Hello! for $i in (1 to 1000) let $name := 'user_' || $i return ( db:create( $name ) ) Is it bug, or normal behavior? I would like to create little benchmark with multiple databases, multiple writtings and readings to it. -- s0rr0w

2 1

Re: [basex-talk] username after login with jetty's HashLoginModule AND parallel update queries
by Florian Eckey 29 Aug '13

29 Aug '13

Hi, that's the way I have realized the login before, but now I combine this with the http basic authentication, because otherwise you can download all files in the webapp directory without being logged in. I have realized it in that way, but now I can't get the username of the user who is logged in. Is there any way to read the "org.eclipse.jetty.security.UserIdentity" attribute in the session? I always get an error: HTTP ERROR 400 Problem accessing /restxq/getSession/org.eclipse.jetty.security.UserIdentity. Reason: Stopped at D:/Projects/******/basex77/basex/webapp/restxq/utilities.xqm, 35/18: [BXSE0002] Stored attribute cannot be retrieved: SessionAuthentication. Stack Trace: - D:/Projects/******/basex77/basex/webapp/restxq/utilities.xqm, 34/13 _____ Powered by Jetty:// The server stops at the line, where it tries to read the session attribute "org.eclipse.jetty.security.UserIdentity". Another question: Is there any way to realize parallel update queries in baseX? Cheers, Florian -----Ursprüngliche Nachricht----- Von: basex-talk-bounces(a)mailman.uni-konstanz.de [mailto:basex-talk-bounces@mailman.uni-konstanz.de] Im Auftrag von Dirk Kirsten Gesendet: Montag, 26. August 2013 11:44 An: basex-talk(a)mailman.uni-konstanz.de Betreff: Re: [basex-talk] username after login with jetty's HashLoginModule Hello Florian, you might want to ask this question on the jetty mailing list as this seems to be not really specific to BaseX. You most likely also will get a more useful response as we are no jetty experts per se. However, I also couldn't find any HashLoginModule in our default jetty.xml (and I did a quick google search and could find anything), so I am not really sure what you actually do. Might be useful to post the relevant part of your jetty.xml. Btw, in our RestXQ-based applications we mainly use application-defined authentication, i.e. we first check the proper login within each output function, so something like: if(not(_:logged-in())) then () else where _:logged-in() is simply declare function _:logged-in() as xs:boolean { boolean(_:id()) }; and we set the session variable during login. Cheers, Dirk On 08/26/2013 11:25 AM, Florian Eckey wrote: > Hey, > > I have a problem with the jetty authentication. I use the > HashLoginModule, which is configured in the jetty.xml. The problem is, > that I find no way to write the username in the session, so after > login I have no username to deal with. Has anyone solved the same or a > similar problem? Or are there any tips how to implement a secure > login, after that I can use the entered username in the session? > > regards > > Florian > > _______________________________________________ > BaseX-Talk mailing list > <mailto:BaseX-Talk@mailman.uni-konstanz.de> BaseX-Talk(a)mailman.uni-konstanz.de > <https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk > -- Dirk Kirsten, BaseX GmbH, <http://basex.org> http://basex.org |-- Firmensitz: Blarerstrasse 56, 78462 Konstanz |-- Registergericht Freiburg, HRB: 708285, Geschäftsführer: | Dr. Christian Grün, Dr. Alexander Holupirek, Michael Seiferle `-- Phone: 0049 7531 28 28 676, Fax: 0049 7531 20 05 22 _______________________________________________ BaseX-Talk mailing list <mailto:BaseX-Talk@mailman.uni-konstanz.de> BaseX-Talk(a)mailman.uni-konstanz.de <https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk

2 2

BaseX 7.7: The Happy-Birthday-Leo Edition
by Christian Grün 29 Aug '13

29 Aug '13

Hi everyone, we are excited to give you Version 7.7 of BaseX -- the Happy-Birthday-Leo Edition -- which provides a bunch of new and improved features: XQUERY - Support for XQuery 3.0 has been finalized http://docs.basex.org/wiki/XQuery_3.0 - the Unit Module allows standardized testing of XQuery applications http://docs.basex.org/wiki/Unit_Module - the Streaming Module speeds up operations on large files http://docs.basex.org/wiki/Streaming_Module - the Inspection Module provides reflection and documentation features http://docs.basex.org/wiki/Inspection_Module - we have added support for XQuery collations http://docs.basex.org/wiki/Full-Text#Collations - we have extended the Database, Archive, File and other Modules http://docs.basex.org/wiki/Module_Library WEB/HTTP SERVICES: - WebDAV has been enriched with locking features (sponsored feature!) http://docs.basex.org/wiki/WebDAV#Locking - RESTXQ has been improved and extended: http://docs.basex.org/wiki/RESTXQ CORE - improved database locking, including the use of custom lock strings http://docs.basex.org/wiki/Transaction_Management - database names have got more flexible http://docs.basex.org/wiki/Commands#Valid_Names - we have new options for simplifying the creation of large databases http://docs.basex.org/wiki/Options BaseX is one of the leading open source XML database and XQuery 3.0 processors, including full support for the W3C Update and Full Text extensions. An interactive GUI frontend gives you great insight into your XML documents, and the client-/server-architecture and web services make it a great framework for building powerful web applications. Please find more information at: http://basex.org Have fun with our new release! Looking forward to your feedback, Christian BaseX Team, 2013

3 2

Proposals. Custom query slot management, named instances and read-only databases
by Alexander Shpack 28 Aug '13

28 Aug '13

Hi everyone! Our current project, based on BaseX, was failed on launch, because we have any practice with optimization for NoSQL databases and take the bottleneck with write locks, but we take the unbelievable experience with BaseX. Today I know, how to get the maximum from this DB 1. Custom query slot management. It would be useful, if query command can take the number of taken slot. >From code side I know, that some type of requests is very simple, very fast, and DB can return rapidly the results back. And this type of requests may be sticked to some query slots permanently. From other side, heavy xQuery code may be managed automatically by BaseX 2. Named instances We need to start multiple BaseX servers to utilize more then one CPU core. It would be useful to set up the name for each instance by personal .basex config 3. Access mode Next proposal is simple. Allow to manage the type of access (read or write only, both) to DB for list of named instances. This proposal may help with the next architecture: many servers with one DBPATH. Server1 Server2 Server3 | | | -------------------------------- | | | | DB1 DB2 DB3 DB4 DB1 permissions Read: Server1, Server3 Write: Server2 DB2, DB3, DB4 permissions Read: Server1, Server2 Write: Server2 This may protect from DB corrupts. Is it interesting concept? -- s0rr0w

1 0

security issues
by Ertan Tike 27 Aug '13

27 Aug '13

Hi, We are providing an cloud based application development environment with using the basex database. We want to allow developers can write xquery commands to access database but some security issues has appeared. For example " file:list('c:/') " query returns the list of C:\ directory. Is it possible to exclude some of modules from basex engine or is there any other way to execute query in sandbox environment which allows only FLWOR expressions and basic modules like "Math". Scanning "file:" like prefixes may be a solution but maybe there is a better way to do it. Thank you for help. -- Ertan TİKE <http://www.6kare.com>

2 1

username after login with jetty's HashLoginModule
by Florian Eckey 26 Aug '13

26 Aug '13

Hey, I have a problem with the jetty authentication. I use the HashLoginModule, which is configured in the jetty.xml. The problem is, that I find no way to write the username in the session, so after login I have no username to deal with. Has anyone solved the same or a similar problem? Or are there any tips how to implement a secure login, after that I can use the entered username in the session? regards Florian

2 1

Re: [basex-talk] Documentation
by Thomas Daly 25 Aug '13

25 Aug '13

Thank you very much for your reply. I assume that in your example commands, mydoc.xml already existed as a file on your hard drive. In my case, the system looked for mydoc.xml rather than create it: > CREATE DB db1 Database 'db1' created in 14.41 ms. > ADD mydoc.xml Resource "/home/tcd/mydoc.xml" not found. Your 2nd ADD command worked, which I assume gives us an XML file empty apart from a root node <a/>: > ADD TO doc2.xml <a/> Path "doc2.xml" added in 2.72 ms. > INFO STORAGE PRE DIS SIZ ATS ID NS KIND CONTENT ------------------------------------------ 0 1 2 1 0 0 DOC doc2.xml 1 1 1 1 1 0 ELEM a Now I'd like to add a new node to the new document. I've tried two variants as follows: > XQUERY insert node <newnode/> into <a/> Query executed in 2.09 ms. > XQUERY insert node <newnode/> into doc('doc2.xml')/a Stopped at ., 1/33: [FODC0002] Resource '/home/tcd/doc2.xml' does not exist. Although the first XQUERY command appeared to work, if I export the XML file with the EXPORT command and examine it, unfortunately the new node is not present. The file just contains the <a/> node. Is there any variation of the EXPORT command that will simply echo the contents of the XML file to the terminal, rather than to a file? If not, that would be a useful addition. Best regards, Thomas At 11:00 25/08/2013, you wrote: >Send BaseX-Talk mailing list submissions to > basex-talk(a)mailman.uni-konstanz.de > >To subscribe or unsubscribe via the World Wide Web, visit > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >or, via email, send a message with subject or body 'help' to > basex-talk-request(a)mailman.uni-konstanz.de > >You can reach the person managing the list at > basex-talk-owner(a)mailman.uni-konstanz.de > >When replying, please edit your Subject line so it is more specific >than "Re: Contents of BaseX-Talk digest..." > > >Today's Topics: > > 1. Documentation (Thomas Daly) > 2. Re: Documentation (Dimitar Popov) > > >---------------------------------------------------------------------- > >Message: 1 >Date: Sat, 24 Aug 2013 16:25:17 +0100 >From: Thomas Daly <thomas.daly(a)live.com> >To: basex-talk(a)mailman.uni-konstanz.de >Subject: [basex-talk] Documentation >Message-ID: <BLU0-SMTP12145626E8EAB6750E955A2914F0(a)phx.gbl> >Content-Type: text/plain; charset="us-ascii"; format=flowed > >Hello, > >Thank you for this system, which looks promising. > >I would like to request better documentation to show how to do the >most basic things. I am trying to create a new database and add a >single new record to it. I don't want to start with a pre-written >XML document. So using the command line client tool: > > > CREATE DB users >Database 'users' created in 214.89 ms. > >Then I refer to the ADD command >http://docs.basex.org/wiki/Commands#ADD which suggests an XML string >can be added. So I try: > > > ADD <record>hello</record> >Name '' is invalid. > >All the examples of ADD involve adding existing XML files. A path to >the database is referred to, but having just created the database I >have no idea what the path is. > >Referring to tutorials and slides on page >http://docs.basex.org/wiki/Getting_Started, the link to XMLPrague >2013 is broken. BaseX adventures starts at an advanced level of >querying a host of XML files. W3Schools XQuery seems all about >querying existing databases, I can't find anything on how to add or >update a single record. > >I'm sure with enough trial and error I'll work out how to do it. My >point is that I feel it ought to be more obvious how to carry out the >most basic CRUD operations. > >Best regards, >Thomas > > > >------------------------------ > >Message: 2 >Date: Sat, 24 Aug 2013 23:31:36 +0200 >From: Dimitar Popov <dp(a)basex.org> >To: basex-talk(a)mailman.uni-konstanz.de >Subject: Re: [basex-talk] Documentation >Message-ID: <1377379896.5833.0.camel@darkrider> >Content-Type: text/plain; charset="UTF-8" > > >On Sat, 2013-08-24 at 16:25 +0100, Thomas Daly wrote: > > Hello, > > > > Thank you for this system, which looks promising. > > > > I would like to request better documentation to show how to do the > > most basic things. I am trying to create a new database and add a > > single new record to it. I don't want to start with a pre-written > > XML document. So using the command line client tool: > > > > > CREATE DB users > > Database 'users' created in 214.89 ms. > > > > Then I refer to the ADD command > > http://docs.basex.org/wiki/Commands#ADD which suggests an XML string > > can be added. So I try: > > > > > ADD <record>hello</record> > > Name '' is invalid. > > > > All the examples of ADD involve adding existing XML files. A path to > > the database is referred to, but having just created the database I > > have no idea what the path is. > >BaseX supports the notion of collections, i.e. one can store multiple >XML documents into a single database. Each non-empty database contains >at least one document. Each stored document must have a name. This is >the reason why when calling the ADD command with an XML fragment one >must specify the name of the new document. However, if a file is >specified as input, then the file name is implicitly used as a name of >the new document. > >Similar behavior has the CREATE DB command. If called with a file, then >a document with the name of the file will be created. If called with an >XML fragment, then the database name will be used as the name of the >first document. The following examples illustrate the two commands: > > > CREATE DB db1 >Database 'db1' created in 68.92 ms. > > > INFO STORAGE >PRE DIS SIZ ATS ID NS KIND CONTENT >----------------------------------------- > > > ADD mydoc.xml >Path "mydoc.xml" added in 15.36 ms. > > > INFO STORAGE >PRE DIS SIZ ATS ID NS KIND CONTENT >------------------------------------------- > 0 1 2 1 0 0 DOC mydoc.xml > 1 1 1 1 1 0 ELEM a > > > ADD TO doc2.xml <a/> >Path "doc2.xml" added in 4.5 ms. > > > INFO STORAGE >PRE DIS SIZ ATS ID NS KIND CONTENT >------------------------------------------- > 0 1 2 1 0 0 DOC mydoc.xml > 1 1 1 1 1 0 ELEM a > 2 3 2 1 2 0 DOC doc2.xml > 3 1 1 1 3 0 ELEM a > > > CREATE DB db2 mydoc.xml >Database 'db2' created in 12.57 ms. > > > INFO STORAGE >PRE DIS SIZ ATS ID NS KIND CONTENT >------------------------------------------- > 0 1 2 1 0 0 DOC mydoc.xml > 1 1 1 1 1 0 ELEM a > > > CREATE DB db3 <a/> >Database 'db3' created in 13.6 ms. > > > INFO STORAGE >PRE DIS SIZ ATS ID NS KIND CONTENT >----------------------------------------- > 0 1 2 1 0 0 DOC db3.xml > 1 1 1 1 1 0 ELEM a > > > > Referring to tutorials and slides on page > > http://docs.basex.org/wiki/Getting_Started, the link to XMLPrague > > 2013 is broken. > >Thanks, I fixed the link. Please, report such inaccuracies or feel free >to edit the wiki yourself. > > > BaseX adventures starts at an advanced level of querying a host > of XML files. W3Schools XQuery seems all about > > querying existing databases > >The BaseX documentation contains only the BaseX specific functionality - >the purpose is not to duplicate the W3C specification or the tutorials >at W3Schools. However, XQuery questions are readily answered on the >mailing list ;) > > > I can't find anything on how to add or update a single record. > >The XQuery Update Facility might be helpful to you [1]. > > > I'm sure with enough trial and error I'll work out how to do it. My > > point is that I feel it ought to be more obvious how to carry out the > > most basic CRUD operations. > >I couldn't agree more. On the other hand the BaseX team gives its best >to maintain the documentation and is always open to improvement >suggestions. So if you have any concrete ideas how the documentation can >be made more useful, they will be most welcome. > >Regards, >Dimitar > >[1] http://www.w3.org/TR/xquery-update-10/ > > > > >------------------------------ > >_______________________________________________ >BaseX-Talk mailing list >BaseX-Talk(a)mailman.uni-konstanz.de >https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk > > >End of BaseX-Talk Digest, Vol 44, Issue 24 >******************************************

2 1