BaseX-Talk March 2018

basex-talk@mailman.uni-konstanz.de

24 participants
20 discussions

SSL support for BaseX REST API
by Giavanna J Richards 21 Mar '18

21 Mar '18

I'm trying to determine how to enable SSL communications with the BaseX server, I have a java server which communicates with BaseX over its REST API for running xquery's. I see in the BaseX changelog that SSL support was added to version 7.5 in 2012 but I haven't been able to find any references to it in the documentation. I found some statements in the jetty.xml file which are commented out and would appear to enable SSL on port 8986. But I'm not at all familiar with jetty. Is there some documentation available for enabling this support? Thanks in advance! Giavanna Richards

5 7

Uncomprehensible (at least to me) behaviour with external bindings
by Marco Lettere 21 Mar '18

21 Mar '18

Hi all, trying to understand why we get the exception [2] running the code [1] in a Java app, I stumbled upon a strange behaviour of the GUI. I rewrote the query to: xquery:eval(" declare variable $_dep as xs:long* external; $_dep[2] * $_dep[2]", map{ "_dep" : (1,2,3)}) obtaining the correct result (4). Anyway if I assign (1,2,3) to a binding through the proper dialog of the UI I get the following exception at execution: Error: Stopped at /home/lettere/git/dasbox-frontend/dasbox-frontend-ui/webapp/ui/resources/js/monitoring/file, 2/12: [FORG0001] Cannot cast to xs:double: (1,2,3). Strangely enough this happens even if I leave the map overriding locally the values for the external parameters in the xquery:eval. The dialog wins? If I rewrite the query like this and remove the parameter values from the dialog: xquery:eval(" declare variable $_dep as xs:long* external := (1,2,3); $_dep[2] * $_dep[2]") I get yet another typecast exception: Error: Stopped at /home/lettere/git/dasbox-frontend/dasbox-frontend-ui/webapp/ui/resources/js/monitoring/file, 2/12: [XPTY0004] Cannot promote xs:integer+ to $_dep as xs:long*: (1, 2, 3). Is there a nice wording to explain all this different behaviour and might it be that the exception we get in the Java embedded version is somehow related to the same? Thanks a lot for helping, Marco. [1] declare variable $_history as xs:long* external; declare variable $_now as xs:dateTime external; declare variable $_dep as xs:long* external; declare variable $dep as xs:long external; $_dep[2] * $_dep[2] [2] Caused by: org.basex.query.QueryException: [FORG0001] Cannot cast to xs:double: [Ljava.lang.Object;@2f0539f0. at org.basex.query.QueryError.get(QueryError.java:1365) ~[BaseX.jar:8.6.7] at org.basex.query.QueryError.castError(QueryError.java:1571) ~[BaseX.jar:8.6.7] at org.basex.query.value.item.Dbl.parse(Dbl.java:180) ~[BaseX.jar:8.6.7] at org.basex.query.value.item.Item.dbl(Item.java:134) ~[BaseX.jar:8.6.7] at org.basex.query.value.type.AtomType.checkLong(AtomType.java:927) ~[BaseX.jar:8.6.7] at org.basex.query.value.type.AtomType$20.cast(AtomType.java:331) ~[BaseX.jar:8.6.7] at org.basex.query.value.type.AtomType$20.cast(AtomType.java:1) ~[BaseX.jar:8.6.7] at org.basex.query.QueryContext.cast(QueryContext.java:733) ~[BaseX.jar:8.6.7] at org.basex.query.QueryContext.bind(QueryContext.java:486) ~[BaseX.jar:8.6.7] at org.basex.query.QueryProcessor.bind(QueryProcessor.java:129) ~[BaseX.jar:8.6.7] at com.nw.basex.xquery.XQueryExecutor.bind(XQueryExecutor.java:89) ~[com.nw.dasbox-2018-03-15.jar:?] at com.nw.basex.xquery.XQueryExecutor.execute(XQueryExecutor.java:107) ~[com.nw.dasbox-2018-03-15.jar:?] at com.nw.dasbox.vdr.resolution.InlineXQueryResolutionEngine.evaluate(InlineXQueryResolutionEngine.java:91) ~[com.nw.dasbox-2018-03-15.jar:?] ... 10 more

2 2

Re: [basex-talk] TR: Marklogic XXE and XML Bomb prevention
by Bridger Dyson-Smith 14 Mar '18

14 Mar '18

Forwarding/replying to the list, since I'm officially Bad At Email. On Wed, Mar 14, 2018 at 11:56 AM, Bridger Dyson-Smith <bdysonsmith(a)gmail.com > wrote: > Hi Fabrice - > > On Wed, Mar 14, 2018 at 11:28 AM, Fabrice ETANCHAUD < > fetanchaud(a)pch.cerfrance.fr> wrote: > >> Hello, >> >> >> >> I found this MarkLogic post interesting, >> >> So I forward it to the BaseX users. >> >> I do not remember loading data I did not trust, but did somebody >> experience this kind of issue ? >> >> > I certainly haven't :) but clearly Christian, et al, have considered > something similar to this. The INTPARSE[1] option let's you use an internal > parser, instead of the standard Java parser. There are options in the > BaseX GUI to use the INTPARSE *and* expand entities from DTDs, but I don't > know if those switches are available in the Options. > >> >> >> Best regards, >> >> Fabrice Etanchaud >> >> >> > > Hope that sheds some light on this. I tried the MarkLogic example using > the INTPARSE (and no DTDs/entity parsing) and created a database that > contains `<foo/>` :). > > And on an additional test, again using the BaseX GUI, using the default Java Parser (both with and without the 'Parse DTDs and entities' option selected), databases were created that expanded the entity and inserted <foo> <thing> <one>ONE</one> </thing> </foo> into the db. > Best, > Bridger > > [1] http://docs.basex.org/wiki/Options#INTPARSE > > So... untrusted input? INTPARSE is your friend - unless you need to expand custom entities. Bridger > > >> *De :* general-bounces(a)developer.marklogic.com [mailto: >> general-bounces(a)developer.marklogic.com] *De la part de* Marcel de Kleine >> *Envoyé :* mercredi 14 mars 2018 13:43 >> *À :* general(a)developer.marklogic.com >> *Objet :* [MarkLogic Dev General] Marklogic XXE and XML Bomb prevention >> >> >> >> Hello, >> >> >> >> We have noticed Marklogic is vulnerable to xxe (entity expansion) and xml >> bomb attacks. When loading an malicious document using xdmp:document-insert >> it won’t catch these and cause either loading of unwanted external >> documents (xxe) and lockup of the system (xml bomb). >> >> >> >> For example, if I load this document : >> >> <?xml version="1.0" encoding="ISO-8859-1"?> >> >> <!DOCTYPE foo [ >> >> <!ELEMENT foo ANY > >> >> <!ENTITY xxe SYSTEM "file:///c:/text.xml" >]> >> >> <foo>&xxe;</foo> >> >> >> >> The file test.xml gets nicely added to the xml document. >> >> >> >> See OWASP and others for examples. >> >> >> >> This is clearly a xml processing issue so the question is : can we >> disable this? And if so, on what levels would this be possible. Best should >> be system-wide. >> >> ( And if you cannot disable this, I think this is something ML should >> address immediately. >> >> >> >> Thank you in advance, >> >> Marcel de Kleine, EPAM >> >> >> >> *Marcel de Kleine* >> >> *Senior Software Engineer* >> >> >> >> *Office: *+31 20 241 6134 *x* 30530 <+31%2020%20241%206134;ext=30530> >> *Cell: *+31 6 14806016 <+31%206%2014806016> *Email: * >> marcel_de_kleine(a)epam.com >> >> *Delft,* *Netherlands * *epam.com <http://www.epam.com>* >> >> >> >> CONFIDENTIALITY CAUTION AND DISCLAIMER >> This message is intended only for the use of the individual(s) or >> entity(ies) to which it is addressed and contains information that is >> legally privileged and confidential. If you are not the intended recipient, >> or the person responsible for delivering the message to the intended >> recipient, you are hereby notified that any dissemination, distribution or >> copying of this communication is strictly prohibited. All unintended >> recipients are obliged to delete this message and destroy any printed >> copies. >> >> >> > >

2 1

TR: Marklogic XXE and XML Bomb prevention
by Fabrice ETANCHAUD 14 Mar '18

14 Mar '18

Hello, I found this MarkLogic post interesting, So I forward it to the BaseX users. I do not remember loading data I did not trust, but did somebody experience this kind of issue ? Best regards, Fabrice Etanchaud De : general-bounces(a)developer.marklogic.com [mailto:general-bounces@developer.marklogic.com] De la part de Marcel de Kleine Envoyé : mercredi 14 mars 2018 13:43 À : general(a)developer.marklogic.com Objet : [MarkLogic Dev General] Marklogic XXE and XML Bomb prevention Hello, We have noticed Marklogic is vulnerable to xxe (entity expansion) and xml bomb attacks. When loading an malicious document using xdmp:document-insert it won't catch these and cause either loading of unwanted external documents (xxe) and lockup of the system (xml bomb). For example, if I load this document : <?xml version="1.0" encoding="ISO-8859-1"?> <!DOCTYPE foo [ <!ELEMENT foo ANY > <!ENTITY xxe SYSTEM "file:///c:/text.xml<file:///c:\text.xml>" >]> <foo>&xxe;</foo> The file test.xml gets nicely added to the xml document. See OWASP and others for examples. This is clearly a xml processing issue so the question is : can we disable this? And if so, on what levels would this be possible. Best should be system-wide. ( And if you cannot disable this, I think this is something ML should address immediately. Thank you in advance, Marcel de Kleine, EPAM Marcel de Kleine Senior Software Engineer Office: +31 20 241 6134 x 30530<tel:+31%2020%20241%206134;ext=30530> Cell: +31 6 14806016<tel:+31%206%2014806016> Email: marcel_de_kleine(a)epam.com<mailto:marcel_de_kleine@epam.com> Delft, Netherlands epam.com<http://www.epam.com> CONFIDENTIALITY CAUTION AND DISCLAIMER This message is intended only for the use of the individual(s) or entity(ies) to which it is addressed and contains information that is legally privileged and confidential. If you are not the intended recipient, or the person responsible for delivering the message to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. All unintended recipients are obliged to delete this message and destroy any printed copies.

2 1

Basex Standalone client for R?
by Ben Engbers 08 Mar '18

08 Mar '18

Hi, At the moment I am following a postgraduate course DataScience at Erasmus University Rotterdam. Part of this course is that I will have to do a final project. During the course, I noticed that datascience relies havily on the use of XML. These files are accessed with R and are read from a file-system. >From the past I remembered having worked with Basex once. So I wondered if there exists a stand-alone R-client to Basex so that I can store all the files in Basex and get access to all Basex functionalities. It shows that such a client does not exist. My question is how difficult it is to write such a client and which language should be used (I have some experience with Java)? Would this be a good subject for my project? Greetings, Ben Engbers

3 2

JTS Library homepage moved
by Omar Siam 08 Mar '18

08 Mar '18

Hi list! This has a new owner/website: https://projects.eclipse.org/projects/locationtech.jts If you have time please update the link Best regards Omar Siam

2 1

copy behavior ?
by Xavier-Laurent SALVADOR 06 Mar '18

06 Mar '18

Hi List, I was working on multi replacement in Word docx. I saw a strange behavior of "copy/modify". I modified a document.xml and zip my folder content twice when I realized my template was wrong. I had to do it again and, after a hundred tries, I realized that copy/modify clause restored every time the old version of my content. I had to close basexGui and relaunch it to have my changes taken in charge. Hi List, I was working on multi replacement in Word docx. I saw a strange behavior of "copy/modify". I modified a document.xml and zip my folder content twice when I realized my template was wrong. I had to do it again and, after a hundred tries, I realized that copy/modify clause restored every time the old version of my content. I had to close basexGui and relaunch it to have my changes taken in charge. Hi List, I was working on multi replacement in Word docx. I saw a strange behavior of "copy/modify". Here is the fact: I use Basex to modify document.xml then zip its folder in a docx file. After a few try, I needed to modify the original and start the process again. But it always zipped back the elder document.xml version I had corrected ... and deleted (I mean, the text inside $doc was the elder version, see infra). I had to close basexGui and relaunch it to have my new file taken in charge. And I have to do this every time I modify the original docx file. Here is the code : declare function local:mapp($path,$space){ for $x at $count in file:list($path) order by $x descending return if (file:is-dir($path||'/'||$x)) then let $s := count(analyze-string($path||$x,'/')//fn:match) return <zip:dir name = "{replace($x,'/$','')}"> {local:mapp($path||'/'||$x,$s)} </zip:dir> else <zip:entry name="{$x}" src="{$path}/{$x}"/> }; for $x in file:read-text-lines('liste.txt') return *copy $doc := doc("/document.xml")* modify for $w in $doc//w:t[.//text() contains text 'XXXXX'] return replace value of node $w with replace($w,'XXXXX',$x) return ( file:write("document.xml",$doc) , let $rep := "/docx" let $dest := "/tmp" return zip:zip-file( <file xmlns="http://expath.org/ns/zip" href="{$dest}/courrier{$count} .docx"> {local:mapp($rep,1)} </file>) ) All the best, X -- Ce message peut contenir des informations réservées exclusivement à son destinataire. Toute diffusion sans autorisation est interdite. Si vous n'en êtes pas le destinataire, merci de prendre contact avec l'expéditeur et de détruire ce message. *This email may contain material for the sole use of the intended recipient. Any forwarding without express permission is prohibited. If you are not the intended recipient, please contact the sender and delete all copies*. -- Ce message peut contenir des informations réservées exclusivement à son destinataire. Toute diffusion sans autorisation est interdite. Si vous n'en êtes pas le destinataire, merci de prendre contact avec l'expéditeur et de détruire ce message. *This email may contain material for the sole use of the intended recipient. Any forwarding without express permission is prohibited. If you are not the intended recipient, please contact the sender and delete all copies*. -- Ce message peut contenir des informations réservées exclusivement à son destinataire. Toute diffusion sans autorisation est interdite. Si vous n'en êtes pas le destinataire, merci de prendre contact avec l'expéditeur et de détruire ce message. *This email may contain material for the sole use of the intended recipient. Any forwarding without express permission is prohibited. If you are not the intended recipient, please contact the sender and delete all copies*.

2 1

Re: [basex-talk] Problem launching BaseX on fresh Fedora 27
by Kristian Kankainen 05 Mar '18

05 Mar '18

1 0

Problem launching BaseX on fresh Fedora 27
by Kristian Kankainen 04 Mar '18

04 Mar '18

I downloaded both the stable and latest release, they both give me the same error message: > java.awt.HeadlessException > at sun.awt.HeadlessToolkit.setDynamicLayout(HeadlessToolkit.java:294) > at org.basex.BaseXGUI.init(BaseXGUI.java:113) > at org.basex.BaseXGUI.<init>(BaseXGUI.java:67) > at org.basex.BaseXGUI.main(BaseXGUI.java:39) > Exception in thread "main" java.lang.ExceptionInInitializerError > at org.basex.gui.GUIMenuCmd.<init>(GUIMenuCmd.java:916) > at org.basex.gui.GUIMenuCmd.<init>(GUIMenuCmd.java:911) > at org.basex.gui.GUIMenuCmd$1.<init>(GUIMenuCmd.java:35) > at org.basex.gui.GUIMenuCmd.<clinit>(GUIMenuCmd.java:35) > at org.basex.gui.GUIConstants.<clinit>(GUIConstants.java:96) > at org.basex.BaseXGUI.<init>(BaseXGUI.java:69) > at org.basex.BaseXGUI.main(BaseXGUI.java:39) > Caused by: java.awt.HeadlessException > at > sun.awt.HeadlessToolkit.getMenuShortcutKeyMask(HeadlessToolkit.java:236 Do I have some missing Java stuff? I use the latest Fedora and I thought it comes well prepared with Java ... I just installed it a few days ago. Best regards Kristian K

2 2

9.0?
by Marco Lettere 01 Mar '18

01 Mar '18

Hi Christian & team, since XML Prague 18 is approaching I remembered that there used to be this wonderful habit of releasing a new BaseX version for the event :-). I am wondering whether there are any timing plans for the 9.0 release. In particular we are eager to test the new features [1] and [2]. Thank you for any updates on the matter and please continue with your incredible work! Best regards, Marco. [1] https://github.com/BaseXdb/basex/issues/1498 (persistent jobs) [2] https://github.com/BaseXdb/basex/issues/1449 (websocket support)

2 2

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

BaseX-Talk March 2018