June 2013 - BaseX-Talk - mailman.uni-konstanz.de

JSON or JSONML output formatting
by Zachary Pride 16 Jul '14

16 Jul '14

Hello, I've been playing with BaseX lately and all-in-all am very impressed. Something I'm wrestling with however is the JSON output options. I've got various web applications that would really be simplified by receiving JSON straight away for the obvious scripting benefits. The JSON method has been giving me a lot of grief, stipulating rules I'm not sure how to follow. right now I have an element that "is typed as string and cannot be nested." I've been able to get JSONML to output but the JSON is this morass of nested Arrays... Attributes seem to get transformed into Objects but elements/tags and their values are pushed into Arrays for some rationale - which I suppose is on purpose. My question is how can I get Objects for simple key:value properties from <tag>value</tag> 's. Are there options I'm not utilizing? Would it be feasible to transform the XML output to some middle form first - setting it up to become Objects, before serializing it to JSON? Thanks! Zach Pride Associate Software Engineer Ultra Electronics, ProLogic 1000 Green River Drive, Ste. 201 Fairmont, WV 26554 zachary.pride(a)ultra-prologic.com Tel: +1 304 333 2680 ext.308 www.ultra-prologic.com This e-mail from Ultra Electronics, ProLogic and any attachments to it are confidential to the intended recipient and may also be privileged or controlled by U.S. export control laws and regulations. No technical data, information or other items provided by Ultra Electronics, ProLogic in connection with this email shall be shared or provided to any foreign persons, entities, or subsidiaries without the expressed written authorization of Ultra Electronic, ProLogic. If you have received it in error please notify the sender and delete it from your system. If you are not the intended recipient you must not copy it or use it for any purpose nor disclose or distribute its contents to any other person. All communications may be subject to interception or monitoring for operational and/or security purposes. Please rely on your own virus checking as the sender cannot accept any liability for any damage arising from any bug or virus infection. This e-mail from Ultra Electronics, ProLogic and any attachments to it are confidential to the intended recipient and may also be privileged or controlled by U.S. export control laws and regulations. No technical data, information or other items provided by Ultra Electronics, ProLogic in connection with this email shall be shared or provided to any foreign persons, entities, or subsidiaries without the expressed written authorization of Ultra Electronic, ProLogic. If you have received it in error please notify the sender and delete it from your system. If you are not the intended recipient you must not copy it or use it for any purpose nor disclose or distribute its contents to any other person. All communications may be subject to interception or monitoring for operational and/or security purposes. Please rely on your own virus checking as the sender cannot accept any liability for any damage arising from any bug or virus infection. Please consider the environment before printing this email.

4 4

Dynamic Module Loading
by Jean-philippe Magué 19 May '14

19 May '14

Hello, Exist-db has an util:import-module<http://exist-db.org/exist/apps/fundocs/view.html?uri=http://exist-db.org/xq…>function that allows to dynamically import an XQuery module at run time. I haven't found anything equivalent in BaseX. Is that right ? I have also tried to see how to implement such a function directly in Java, but as far as I understand the module parsing process, functions prefix are checked at compile time. That would compromise any attempt to write such a function. Am I right ? Best, Jean-Philippe -- Jean-Philippe Magué Maître de Conférences en Humanités Numériques Responsable de l'Atelier des Humanités Numériques : ahn.ens-lyon.fr Ecole Normale Supérieure de Lyon +33 (0)4 26 23 38 08 -- "Using computers will therefore lead us to a more profound and systematic knowledge of human expression; in principle, it can help us to be more humanistic than before." Busa, 1980.

6 16

Out of memory bug
by Ivan Lagunov 31 Jul '13

31 Jul '13

Hello, I've got 'Out of memory' error and corrupted database again, this time it's for 6.3 version. Steps: 1. Started server, started terminal client, checked there are no text and attribute indexes. 2. Started my Java application that runs several tests on database: querying, adding, removing. 3. Then I forgot to reopen terminal client and used the one that I opened in Step 1. 4. Run optimize command for the collection and got error. 5. Cannot open the collection any more as I'm getting 'Out of memory'. It's clear that I shouldn't have used the terminal client that I opened before execution of my Java application. But sometimes it's hard to remember. And it's completely unacceptable that these actions result in the corrupted database. You really should catch ArrayIndexOutOfBoundsException everywhere in order to save database from being corrupted. Terminal client stacktrace: > optimize Possible bug? Feedback is welcome: basex-talk(a)mailman.uni-konstanz.de BaseX 6.3: java.lang.ArrayIndexOutOfBoundsException: -1 org.basex.data.PathSummary.add(PathSummary.java:68) org.basex.core.cmd.Optimize.stats(Optimize.java:68) org.basex.core.cmd.Optimize.run(Optimize.java:34) org.basex.core.Command.run(Command.java:236) org.basex.core.Command.exec(Command.java:218) org.basex.core.Command.execute(Command.java:66) org.basex.server.ServerProcess.run(ServerProcess.java:161) > open products Out of Main Memory. Best regards, Ivan

4 8

Circular module dependency
by Jean-philippe Magué 22 Jul '13

22 Jul '13

Hi, I'm having problem with modules importing each other. I have the two following restxq modules : moduleA.xqm ------------------ module namespace moduleA = 'http:/www.example.com/moduleA'; import module namespace moduleB = 'http:/www.example.com/moduleB' at 'moduleB.xqm'; declare function moduleA:f() { "" }; ------------------ moduleB.xqm ------------------ module namespace moduleB = 'http:/www.example.com/moduleB'; import module namespace moduleA = 'http:/www.example.com/moduleA' at 'moduleA.xqm'; ------------------ They fail to compile with the following error: Stopped at line 7, column 29 in /Users/jmague/work/basex/webapp/restxq/moduleA.xqm: [XQST0034] Duplicate declaration of function moduleA:f(...). Are cross imports just forbidden ? Yet, the XQuery specification state that (section 4.12.4 Cycles) : "Implementations must resolve cycles in the import graph, either at the level of module URIs or at the level of location URIs, and ensure that each module is imported only once." Best, Jean-Philippe -- Jean-Philippe Magué Maître de Conférences en Humanités Numériques Responsable de l'Atelier des Humanités Numériques : ahn.ens-lyon.fr Ecole Normale Supérieure de Lyon +33 (0)4 26 23 38 08 -- "Using computers will therefore lead us to a more profound and systematic knowledge of human expression; in principle, it can help us to be more humanistic than before." Busa, 1980.

3 4

Invitation: I AM LOOKING FOR ROMANCE @ Sun 30 Jun 2013 12:00 - 13:00 (sallymore02@gmail.com)
by Sally Moore 30 Jun '13

30 Jun '13

You have been invited to the following event. Title: I AM LOOKING FOR ROMANCE I AM LOOKING FOR ROMANCE My name is Sally Moore, I came acrose your profile when i was searching for a life partner I was inspired to send you a massage and ask you for friendship. As you know that having a prophesy of love over somebody there most be odds around the corner, but no matter the odds, love survive all odds. I am a girl who's love story is intresting. I am responsible, romantic and courageous. I belive, with good mind you have to admit my request Please recontact me with my e-mail address at......( sallymoore21(a)yahoo.com ) so I can send my picture and tell you more about myself. Yours Love Miss Sally When: Sun 30 Jun 2013 12:00 – 13:00 Ouagadougou Calendar: sallymore02(a)gmail.com Who: (Guest list has been hidden at organiser's request) Event details: https://www.google.com/calendar/event?action=VIEW&eid=aGhpZ21xZDRhYzYxdmQ3O… Invitation from Google Calendar: https://www.google.com/calendar/ You are receiving this courtesy email at the account basex-talk(a)mailman.uni-konstanz.de because you are an attendee of this event. To stop receiving future notifications for this event, decline this event. Alternatively, you can sign up for a Google account at https://www.google.com/calendar/ and control your notification settings for your entire calendar.

1 0

limiting time or space resources in calls to xquery:eval() ?
by C. M. Sperberg-McQueen 29 Jun '13

29 Jun '13

I'm contemplating the construction of an interface for advanced or dedicated users of a database, with a text box in which they type their queries as XQuery modules. (Non-advanced and non-dedicated users will make do with a variety of pre-defined queries; this interface is intended to provide an open-ended query interface for the few users who will need it.) One issue that arises is security: any use of an eval() function opens the door to code injection attacks. For now, I'm inclined to think that BaseX's rule that xquery:eval() does not accept updating expressions suffices to protect the database from harm. Parsing the query and declining potentially harmful queries is also a possibility, if I can persuade myself I can distinguish harmful from harmless queries. (Comments and advice on this topic welcome, even though it's not the focus of this question.) A second issue is resource usage. Imagine a hostile user who writes a query designed to tie up the server for a long time and consume a lot of memory. Or imagine a non-hostile but naive user who unintentionally concocts an extremely expensive query. Is there a way to put a call to eval() (or any call to the BaseX HTTP server) into some kind of box and specify limits on the amount of CPU time, the amount of clock time, and/or the amount of storage to be available for its evaluation? Michael -- **************************************************************** * C. M. Sperberg-McQueen, Black Mesa Technologies LLC * http://www.blackmesatech.com * http://cmsmcq.com/mib * http://balisage.net ****************************************************************

4 7

Comment filtrer le goudron de vos cigarettes ?
by Filtre AG 28 Jun '13

28 Jun '13

Petit, Léger, Lavable et Conserve le GOUDRON Si vous faites partie des fumeurs qui aimeraient diminuer les effets néfastes de la cigarette, ce mail est fait pour vous. Le Filtre AG est un porte-cigarette santé. Il permet de filtrer une très grande partie des goudrons qui sont présents dans vos cigarettes. Son utilisation est d'une simplicité déconcertante, il suffit simplement de fixer le filtre au bout de sa cigarette et de fumer. Dès la première bouffée, vous verrez apparaître le goudron dans le filtre, choc visuel assuré. Le Filtre AG est réutilisable à l'infini, il suffit de le nettoyer avec un mouchoir et vous pourrez le réutiliser et ça indéfiniment. Son prix dérisoire, vous permettra de préserver à minima vos poumons, votre gorge et votre dentition. Fini les odeurs sur les mains, fini les dents jaunes. Le Filtre AG a été testé au laboratoire national d'essai français. Soyez en sûr, le Filtre AG vous permettra de diminuer fortement les goudrons présents dans vos cigarettes tout en gardant la saveur originelle de vos cigarettes. Accéder à plus d'informations >> Le code promo filtronslegoudron vous permettra de bénéficier dès à présent de 40%* de réduction. *valable jusqu'au 15/07/2013 Enfin la solution pour limiter les effets néfastes de la cigarette Vidéo de démonstration *Filtre AG est une marque déposée auprès de l'institut nationale de la propriété industrielle. Pour ne plus recevoir de mail de notre part, veuillez cliquer STOPMAIL

1 0

indent parameter
by Hans-Juergen Rennau 28 Jun '13

28 Jun '13

Hello, it seems to me there is a couple of issues concerning the handling of indentation (serialization parameter "indent"). a) Using "indent=no" whitespace-only text nodes between element nodes are removed b) Using "indent=yes" the content of elements with xml:space="preserve" is indented, too I suppose both behaviors are in contraction to the spec ( http://www.w3.org/TR/xslt-xquery-serialization/#xml-indent ). As a particular consequence, (a) seems to preclude making an exact copy of a document (fragment) - all whitespace-only text nodes vanish. And as a consequence of (b) one has no possibility to locally protect the whitespace used in the source document, as required for example when reusing "programlisting" elements of docbook. Or did I overlook something? Kind regards, Hans-Juergen

2 3

Corrupt database
by Lars Iversen 28 Jun '13

28 Jun '13

Hi, We are using the REST API with BaseX 7.6. After doing approximately 100 PUT requests for creating documents in a database, we get the error listed below. So far I haven't been able to reproduce it, but I have the database if that would be any help. Any suggestion on how to un-corrupt the database? Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk(a)mailman.uni-konstanz.de Version: BaseX 7.6 Java: Oracle Corporation, 1.7.0_25 OS: Windows 7, x86 Stack Trace: java.lang.RuntimeException: Data Access out of bounds: - pre value: 0 - #used blocks: 0 - #total locks: 0 - access: -1 (0 > -1] org.basex.util.Util.notexpected(Util.java:53) org.basex.io.random.TableDiskAccess.cursor(TableDiskAccess.java:488) org.basex.io.random.TableDiskAccess.read1(TableDiskAccess.java:189) org.basex.data.Data.kind(Data.java:283) org.basex.query.value.node.DBNode.<init>(DBNode.java:49) org.basex.query.value.seq.DBNodeSeq.get(DBNodeSeq.java:108) org.basex.query.value.seq.DBNodeSeq.get(DBNodeSeq.java:96) org.basex.query.QueryResources.compile(QueryResources.java:62) org.basex.query.QueryContext.compile(QueryContext.java:200) org.basex.query.QueryProcessor.compile(QueryProcessor.java:66) org.basex.query.QueryProcessor.execute(QueryProcessor.java:95) org.basex.query.QueryProcessor.queryNodes(QueryProcessor.java:204) org.basex.core.cmd.AQuery.queryNodes(AQuery.java:185) org.basex.core.cmd.Cs.run(Cs.java:23) org.basex.core.Command.run(Command.java:342) org.basex.core.Command.exec(Command.java:321) org.basex.core.Command.execute(Command.java:78) org.basex.server.LocalSession.execute(LocalSession.java:122) org.basex.server.Session.execute(Session.java:37) org.basex.http.rest.RESTCode.open(RESTCode.java:43) org.basex.http.rest.RESTRetrieve.run(RESTRetrieve.java:41) org.basex.http.rest.RESTGet.run(RESTGet.java:67) org.basex.http.rest.RESTServlet.run(RESTServlet.java:14) org.basex.http.BaseXServlet.service(BaseXServlet.java:39) javax.servlet.http.HttpServlet.service(HttpServlet.java:848) org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:648) org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455) org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137) org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:559) org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:232) org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1072) org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:382) org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:194) org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1006) org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135) org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116) org.eclipse.jetty.server.Server.handle(Server.java:365) org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:485) org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:926) org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:988) org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:635) org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235) org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82) org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:627) org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:51) org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608) org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) java.lang.Thread.run(Unknown Source) Med venlig hilsen / Kind regards [Description: Systematic Logo]<http://www.systematic.com/> Lars Iversen Senior Systems Engineer Søren Frichs Vej 39, 8000 Aarhus C Denmark Mobile: +45 4196 5164 Lars.Iversen(a)systematic.com<mailto:Lars.Iversen@systematic.com> www.systematic.com<http://www.systematic.com>

2 1

One long running read query blocking other read xqueries execution
by Srinivasan Muthu 26 Jun '13

26 Jun '13

I use BaseX 7.7 build from couple days ago with GLOBALLOCK=false. I run using client-server mode. I have one long running read-only xQuery (say query1) that needs to be optimized but for now it takes about 3 minutes to run. While query1 is running all other read-only xQueries are queued up (from BaseX server logs) and wait for query1 completion. After query1 is done, all other xQueries get executed one after the other. Note query1 only accesses one document and all other xQueries run in parallel operate on a different document. Before I debugged this more, can you let me know if a read-only query can block other read-only query? and if so any hints at where I should look at? I am also in the process of debugging this more, so if I can get a sample code ready, I will share with you guys soon for making this easy. Thanks, Srini

2 2