BaseX-Talk

basex-talk@mailman.uni-konstanz.de

6 participants
5250 discussions

Re: [basex-talk] TR: Marklogic XXE and XML Bomb prevention
by Bridger Dyson-Smith 14 Mar '18

14 Mar '18

Forwarding/replying to the list, since I'm officially Bad At Email. On Wed, Mar 14, 2018 at 11:56 AM, Bridger Dyson-Smith <bdysonsmith(a)gmail.com > wrote: > Hi Fabrice - > > On Wed, Mar 14, 2018 at 11:28 AM, Fabrice ETANCHAUD < > fetanchaud(a)pch.cerfrance.fr> wrote: > >> Hello, >> >> >> >> I found this MarkLogic post interesting, >> >> So I forward it to the BaseX users. >> >> I do not remember loading data I did not trust, but did somebody >> experience this kind of issue ? >> >> > I certainly haven't :) but clearly Christian, et al, have considered > something similar to this. The INTPARSE[1] option let's you use an internal > parser, instead of the standard Java parser. There are options in the > BaseX GUI to use the INTPARSE *and* expand entities from DTDs, but I don't > know if those switches are available in the Options. > >> >> >> Best regards, >> >> Fabrice Etanchaud >> >> >> > > Hope that sheds some light on this. I tried the MarkLogic example using > the INTPARSE (and no DTDs/entity parsing) and created a database that > contains `<foo/>` :). > > And on an additional test, again using the BaseX GUI, using the default Java Parser (both with and without the 'Parse DTDs and entities' option selected), databases were created that expanded the entity and inserted <foo> <thing> <one>ONE</one> </thing> </foo> into the db. > Best, > Bridger > > [1] http://docs.basex.org/wiki/Options#INTPARSE > > So... untrusted input? INTPARSE is your friend - unless you need to expand custom entities. Bridger > > >> *De :* general-bounces(a)developer.marklogic.com [mailto: >> general-bounces(a)developer.marklogic.com] *De la part de* Marcel de Kleine >> *Envoyé :* mercredi 14 mars 2018 13:43 >> *À :* general(a)developer.marklogic.com >> *Objet :* [MarkLogic Dev General] Marklogic XXE and XML Bomb prevention >> >> >> >> Hello, >> >> >> >> We have noticed Marklogic is vulnerable to xxe (entity expansion) and xml >> bomb attacks. When loading an malicious document using xdmp:document-insert >> it won’t catch these and cause either loading of unwanted external >> documents (xxe) and lockup of the system (xml bomb). >> >> >> >> For example, if I load this document : >> >> <?xml version="1.0" encoding="ISO-8859-1"?> >> >> <!DOCTYPE foo [ >> >> <!ELEMENT foo ANY > >> >> <!ENTITY xxe SYSTEM "file:///c:/text.xml" >]> >> >> <foo>&xxe;</foo> >> >> >> >> The file test.xml gets nicely added to the xml document. >> >> >> >> See OWASP and others for examples. >> >> >> >> This is clearly a xml processing issue so the question is : can we >> disable this? And if so, on what levels would this be possible. Best should >> be system-wide. >> >> ( And if you cannot disable this, I think this is something ML should >> address immediately. >> >> >> >> Thank you in advance, >> >> Marcel de Kleine, EPAM >> >> >> >> *Marcel de Kleine* >> >> *Senior Software Engineer* >> >> >> >> *Office: *+31 20 241 6134 *x* 30530 <+31%2020%20241%206134;ext=30530> >> *Cell: *+31 6 14806016 <+31%206%2014806016> *Email: * >> marcel_de_kleine(a)epam.com >> >> *Delft,* *Netherlands * *epam.com <http://www.epam.com>* >> >> >> >> CONFIDENTIALITY CAUTION AND DISCLAIMER >> This message is intended only for the use of the individual(s) or >> entity(ies) to which it is addressed and contains information that is >> legally privileged and confidential. If you are not the intended recipient, >> or the person responsible for delivering the message to the intended >> recipient, you are hereby notified that any dissemination, distribution or >> copying of this communication is strictly prohibited. All unintended >> recipients are obliged to delete this message and destroy any printed >> copies. >> >> >> > >

2 1

TR: Marklogic XXE and XML Bomb prevention
by Fabrice ETANCHAUD 14 Mar '18

14 Mar '18

Hello, I found this MarkLogic post interesting, So I forward it to the BaseX users. I do not remember loading data I did not trust, but did somebody experience this kind of issue ? Best regards, Fabrice Etanchaud De : general-bounces(a)developer.marklogic.com [mailto:general-bounces@developer.marklogic.com] De la part de Marcel de Kleine Envoyé : mercredi 14 mars 2018 13:43 À : general(a)developer.marklogic.com Objet : [MarkLogic Dev General] Marklogic XXE and XML Bomb prevention Hello, We have noticed Marklogic is vulnerable to xxe (entity expansion) and xml bomb attacks. When loading an malicious document using xdmp:document-insert it won't catch these and cause either loading of unwanted external documents (xxe) and lockup of the system (xml bomb). For example, if I load this document : <?xml version="1.0" encoding="ISO-8859-1"?> <!DOCTYPE foo [ <!ELEMENT foo ANY > <!ENTITY xxe SYSTEM "file:///c:/text.xml<file:///c:\text.xml>" >]> <foo>&xxe;</foo> The file test.xml gets nicely added to the xml document. See OWASP and others for examples. This is clearly a xml processing issue so the question is : can we disable this? And if so, on what levels would this be possible. Best should be system-wide. ( And if you cannot disable this, I think this is something ML should address immediately. Thank you in advance, Marcel de Kleine, EPAM Marcel de Kleine Senior Software Engineer Office: +31 20 241 6134 x 30530<tel:+31%2020%20241%206134;ext=30530> Cell: +31 6 14806016<tel:+31%206%2014806016> Email: marcel_de_kleine(a)epam.com<mailto:marcel_de_kleine@epam.com> Delft, Netherlands epam.com<http://www.epam.com> CONFIDENTIALITY CAUTION AND DISCLAIMER This message is intended only for the use of the individual(s) or entity(ies) to which it is addressed and contains information that is legally privileged and confidential. If you are not the intended recipient, or the person responsible for delivering the message to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. All unintended recipients are obliged to delete this message and destroy any printed copies.

2 1

Basex Standalone client for R?
by Ben Engbers 08 Mar '18

08 Mar '18

Hi, At the moment I am following a postgraduate course DataScience at Erasmus University Rotterdam. Part of this course is that I will have to do a final project. During the course, I noticed that datascience relies havily on the use of XML. These files are accessed with R and are read from a file-system. >From the past I remembered having worked with Basex once. So I wondered if there exists a stand-alone R-client to Basex so that I can store all the files in Basex and get access to all Basex functionalities. It shows that such a client does not exist. My question is how difficult it is to write such a client and which language should be used (I have some experience with Java)? Would this be a good subject for my project? Greetings, Ben Engbers

3 2

JTS Library homepage moved
by Omar Siam 08 Mar '18

08 Mar '18

Hi list! This has a new owner/website: https://projects.eclipse.org/projects/locationtech.jts If you have time please update the link Best regards Omar Siam

2 1

copy behavior ?
by Xavier-Laurent SALVADOR 06 Mar '18

06 Mar '18

Hi List, I was working on multi replacement in Word docx. I saw a strange behavior of "copy/modify". I modified a document.xml and zip my folder content twice when I realized my template was wrong. I had to do it again and, after a hundred tries, I realized that copy/modify clause restored every time the old version of my content. I had to close basexGui and relaunch it to have my changes taken in charge. Hi List, I was working on multi replacement in Word docx. I saw a strange behavior of "copy/modify". I modified a document.xml and zip my folder content twice when I realized my template was wrong. I had to do it again and, after a hundred tries, I realized that copy/modify clause restored every time the old version of my content. I had to close basexGui and relaunch it to have my changes taken in charge. Hi List, I was working on multi replacement in Word docx. I saw a strange behavior of "copy/modify". Here is the fact: I use Basex to modify document.xml then zip its folder in a docx file. After a few try, I needed to modify the original and start the process again. But it always zipped back the elder document.xml version I had corrected ... and deleted (I mean, the text inside $doc was the elder version, see infra). I had to close basexGui and relaunch it to have my new file taken in charge. And I have to do this every time I modify the original docx file. Here is the code : declare function local:mapp($path,$space){ for $x at $count in file:list($path) order by $x descending return if (file:is-dir($path||'/'||$x)) then let $s := count(analyze-string($path||$x,'/')//fn:match) return <zip:dir name = "{replace($x,'/$','')}"> {local:mapp($path||'/'||$x,$s)} </zip:dir> else <zip:entry name="{$x}" src="{$path}/{$x}"/> }; for $x in file:read-text-lines('liste.txt') return *copy $doc := doc("/document.xml")* modify for $w in $doc//w:t[.//text() contains text 'XXXXX'] return replace value of node $w with replace($w,'XXXXX',$x) return ( file:write("document.xml",$doc) , let $rep := "/docx" let $dest := "/tmp" return zip:zip-file( <file xmlns="http://expath.org/ns/zip" href="{$dest}/courrier{$count} .docx"> {local:mapp($rep,1)} </file>) ) All the best, X -- Ce message peut contenir des informations réservées exclusivement à son destinataire. Toute diffusion sans autorisation est interdite. Si vous n'en êtes pas le destinataire, merci de prendre contact avec l'expéditeur et de détruire ce message. *This email may contain material for the sole use of the intended recipient. Any forwarding without express permission is prohibited. If you are not the intended recipient, please contact the sender and delete all copies*. -- Ce message peut contenir des informations réservées exclusivement à son destinataire. Toute diffusion sans autorisation est interdite. Si vous n'en êtes pas le destinataire, merci de prendre contact avec l'expéditeur et de détruire ce message. *This email may contain material for the sole use of the intended recipient. Any forwarding without express permission is prohibited. If you are not the intended recipient, please contact the sender and delete all copies*. -- Ce message peut contenir des informations réservées exclusivement à son destinataire. Toute diffusion sans autorisation est interdite. Si vous n'en êtes pas le destinataire, merci de prendre contact avec l'expéditeur et de détruire ce message. *This email may contain material for the sole use of the intended recipient. Any forwarding without express permission is prohibited. If you are not the intended recipient, please contact the sender and delete all copies*.

2 1

Re: [basex-talk] Problem launching BaseX on fresh Fedora 27
by Kristian Kankainen 05 Mar '18

05 Mar '18

1 0

Problem launching BaseX on fresh Fedora 27
by Kristian Kankainen 04 Mar '18

04 Mar '18

I downloaded both the stable and latest release, they both give me the same error message: > java.awt.HeadlessException > at sun.awt.HeadlessToolkit.setDynamicLayout(HeadlessToolkit.java:294) > at org.basex.BaseXGUI.init(BaseXGUI.java:113) > at org.basex.BaseXGUI.<init>(BaseXGUI.java:67) > at org.basex.BaseXGUI.main(BaseXGUI.java:39) > Exception in thread "main" java.lang.ExceptionInInitializerError > at org.basex.gui.GUIMenuCmd.<init>(GUIMenuCmd.java:916) > at org.basex.gui.GUIMenuCmd.<init>(GUIMenuCmd.java:911) > at org.basex.gui.GUIMenuCmd$1.<init>(GUIMenuCmd.java:35) > at org.basex.gui.GUIMenuCmd.<clinit>(GUIMenuCmd.java:35) > at org.basex.gui.GUIConstants.<clinit>(GUIConstants.java:96) > at org.basex.BaseXGUI.<init>(BaseXGUI.java:69) > at org.basex.BaseXGUI.main(BaseXGUI.java:39) > Caused by: java.awt.HeadlessException > at > sun.awt.HeadlessToolkit.getMenuShortcutKeyMask(HeadlessToolkit.java:236 Do I have some missing Java stuff? I use the latest Fedora and I thought it comes well prepared with Java ... I just installed it a few days ago. Best regards Kristian K

2 2

9.0?
by Marco Lettere 01 Mar '18

01 Mar '18

Hi Christian & team, since XML Prague 18 is approaching I remembered that there used to be this wonderful habit of releasing a new BaseX version for the event :-). I am wondering whether there are any timing plans for the 9.0 release. In particular we are eager to test the new features [1] and [2]. Thank you for any updates on the matter and please continue with your incredible work! Best regards, Marco. [1] https://github.com/BaseXdb/basex/issues/1498 (persistent jobs) [2] https://github.com/BaseXdb/basex/issues/1449 (websocket support)

2 2

Signature of db:add
by Tim Thompson 28 Feb '18

28 Feb '18

Hello, The two argument signature of the db:add function does not seem to work ( http://docs.basex.org/wiki/Database_Module#db:add). Is this still valid in BaseX? When I try something like db:add("test", <test/>), I get an error: [FODC0007] Resource path '' is invalid. Thanks! -- Tim A. Thompson Discovery Metadata Librarian Yale University Library

2 1

Whitespace chopping / option -w
by Hans-Juergen Rennau 27 Feb '18

27 Feb '18

Dear BaseX team, "whitespace chopping" (which happens by default unless command-line option -w is used) does not only remove "whitespace-only" text nodes, but also leading/trainling whitespace in element content, for example: doc.xml:<hello> BaseX is phantastic! </hello> basex -i doc.xml . => <hello>BaseX is phantastic!</hello> While the result is of course true, I regard leading/trailing whitespace in element content as information which has a different "level of significance" than whitespace-only text nodes. In other words: I think it is an important use case that leading/trailing whitespace must be preserved, while the "pretty print whitespace" should be discarded. Did I overlook a way to get rid of whitespace-only text nodes without touching leading/trailing whitespace in element content? If not, it would be wonderful if you added a new option doing just that. With kind regards -Hans-Jürgen

1 0

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

BaseX-Talk