- BaseX-Talk - mailman.uni-konstanz.de

XML in HTTP request body
by Jerome Chauveau 03 Jun '19

03 Jun '19

Hi, I am trying to check XML content (HTTP posted) with an hmac function in a RESTXQ webapp. My Java client sends an HTTP post request where XML data looks like : <data><user>doe</user><content><TEI xsi:schemaLocation="http://www.tei-c.org/ns/1.0" xml:id="idtest" ><teiHeader ><fileDesc ><titleStmt ><title >Test</title ><author xml:id="authortest" >XXX</author ></titleStmt ></teiHeader ><text ><front ></front ><body xml:space="preserve" ><listPlace ><place type="lieu" ><placeName xml:lang="la" >Burgunvilla</placeName ></place ></listPlace ></body ></text ></TEI ></content></data> This content is HMAC encrypted and the signature is added to the HTTP header : [...] String secret = "1234"; StringEntity entity = new StringEntity(xml, ContentType.create("text/xml", Consts.UTF_8)); String signature = HTTPUtils.hmacSHA1(xml, secret); httppost.setHeader("Authorization", signature); httppost.setEntity(entity); [...] Here is my RESTXQ function : declare %rest:POST("{$body}") %rest:consumes("application/xml", "text/xml") %rest:header-param("Authorization", "{$signature}", "none") %rest:path("/sf/testpost") function sf.test:testPost($signature,$body){ let $secret := 1234 crypto:hmac($body,$secret,'sha1','base64') = $signature } Equality check always fails with such an XML sequence but works fine when XML does not contain any carriage return : <data><user>doe</user><content><TEI><front></front><text><body>This is a test</body></text></TEI> It seems to come when "The body of a POST or PUT request is converted to an XQuery item". I tried to set serializer's options without any success ! How could I retrieve xml (in $body) exactly serialized as sent by the client ? Tests are running on BaseX922. Thank you for your time. Jérôme

2 2

Update of arbitrarily selected elements?
by Joshua Kordani 03 Jun '19

03 Jun '19

Greetings all. I am curious about how basex supports the idea of modifying any given (or set of given) elements, say a user makes a query to select a bunch of elements for editing and wishes to add an attribute to an arbitrary subset (as in, not selectable with a query), how do I reference a particular element for later updating? Sincerely, Josh DISCLAIMER: This email communication and any attached files are Speedcast Wireless proprietary and may be legally privileged. Export-controlled information shall not be disseminated without proper authorization and proper export-control markings, per Speedcast Wireless export policy. If you have received this transmission in error please notify the sender immediately and then delete this email and all its attachments. If you are not the addressee, any disclosure, reproduction, copying, distribution, or any other dissemination or use of this communication is strictly prohibited. Thank you.

3 2

archive:create - Out Of Main Memory
by Han (US), Paul P 29 May '19

29 May '19

Hello, We encountered Out Of Main Memory when trying to create archive of a 4GB XML file. I'm using version BaseX922.war. We allocated 20GB with -Xmx flag for tomcat. Please let me know if there is a way to overcome this issue. let $files := file:list('/tmp', false(), '4gb.xml') let $zip := archive:create($files ! element archive:entry { . }, file:read-binary('/tmp/4gb.xml')) return file:write-binary('4gb.xml.zip', $zip) Thanks, Paul

2 2

FW: Attempting to write bulk import of files is failing
by Joshua Kordani 27 May '19

27 May '19

CC group, my mistake. -----Original Message----- From: Joshua Kordani Sent: Wednesday, May 22, 2019 14:12 PM To: Christian Grün <christian.gruen(a)gmail.com> Subject: RE: [basex-talk] Attempting to write bulk import of files is failing Christian, Thank you for your reply. I don't understand why import-all.xq would throw this warning and import-and-process.xq would not. In the case of the latter, the nodes imported with that call are in fact updated properly, while in the former, no updates are seen. Can you tell me why -u would change the behavior of the one and not the other, or in other words, what is different about the two queries that makes this distinction important? Josh -----Original Message----- From: Christian Grün <christian.gruen(a)gmail.com> Sent: Wednesday, May 22, 2019 14:07 PM To: Joshua Kordani <joshua.kordani(a)speedcastwireless.com> Cc: BaseX <basex-talk(a)mailman.uni-konstanz.de> Subject: Re: [basex-talk] Attempting to write bulk import of files is failing Hi Joshua, > I write the function in lib.xqm as a standalone query. All it does it add some attributes to each testcase before importing the content into the database. Forgive me if this is hard to read, it was my first attempt at making a function which updates some records and then imports in the same query. Looks fine. Thanks for sending code that can be parsed out-of-the-box. > I am now trying to write a bulk import, which is import-all.xq and this is not working. I receive a message about updates not being written back. My first attempt to write that function inside of a for-each also failed. This warning is only shown on command line. It simply indicates that your updates won’t be propagated back to the original files. If that’s what you would like to achieve, you can use the command-line argument -u (see [1]). If something else is not working as expected, feel free to provide us with more details. Best, Christian [1] http://docs.basex.org/wiki/Options#WRITEBACK DISCLAIMER: This email communication and any attached files are Speedcast Wireless proprietary and may be legally privileged. Export-controlled information shall not be disseminated without proper authorization and proper export-control markings, per Speedcast Wireless export policy. If you have received this transmission in error please notify the sender immediately and then delete this email and all its attachments. If you are not the addressee, any disclosure, reproduction, copying, distribution, or any other dissemination or use of this communication is strictly prohibited. Thank you.

2 6

Attempting to write bulk import of files is failing
by Joshua Kordani 22 May '19

22 May '19

https://pastebin.com/BEA6XiE5 I write the function in lib.xqm as a standalone query. All it does it add some attributes to each testcase before importing the content into the database. Forgive me if this is hard to read, it was my first attempt at making a function which updates some records and then imports in the same query. I decided to break that query out into its own function, where I then call it from another query, import-and-process.xq, which works as expected. I am now trying to write a bulk import, which is import-all.xq and this is not working. I receive a message about updates not being written back. My first attempt to write that function inside of a for-each also failed. I've read all the documentation I can get my hands on regarding how to write the kinds of queries I need but there still seems to be gotchas I can't seem to see in advance. Any help would be greatly appreciated. Joshua Kordani DISCLAIMER: This email communication and any attached files are Speedcast Wireless proprietary and may be legally privileged. Export-controlled information shall not be disseminated without proper authorization and proper export-control markings, per Speedcast Wireless export policy. If you have received this transmission in error please notify the sender immediately and then delete this email and all its attachments. If you are not the addressee, any disclosure, reproduction, copying, distribution, or any other dissemination or use of this communication is strictly prohibited. Thank you.

2 1

[experr:HC0001] unknown protocol: localhost (Windows only)
by France Baril 21 May '19

21 May '19

Hi, We have two servers with similar jetty configuration, we find that an http:request call on static works on Linux but not on Windows. HOW TO REPLICATE Add an image test.png in webapp/static Try the attached .xqm (change img-url for your server path) Copy of the .xqm function for reference: declare %rest:path('/a-test') > %rest:GET > %output:method("html") > %output:html-version("5.0") > function a-test:http-req(){ > let $img-url := 'http://localhost:8972/static/test.png' > let $http-function := function() {http:send-request(<http:request > method='get' status-only='true'/>, $img-url)} > let $http-s := xquery:fork-join($http-function) > let $http-status := $http-s//data(@status) > return <div>{$http-status}</div> > }; Test: http://localhost:8972/a-test Result on Linux: 200 Result on Windows: [experr:HC0001] unknown protocol: localhost This seemed to be security issue, but tests in multiple browsers return the same results and enabling CORS doesn't help... technically we're calling from the same domain/port... but hey we tried! If we type the value of $img-url directly in the browser, we see the image on both platforms, so the issue is not with accessing the image itself, but with accessing it with the http:send-request function. -- France Baril Architecte documentaire / Documentation architect france.baril(a)architextus.com

2 6

db:replace() in DBA resulting in duplicates
by Zimmel, Daniel 20 May '19

20 May '19

Hello, something has gone awry in my database after a db:replace(): query: db:open('DB','/path/file.xml')/base-uri() result: /DB/path/file.xml This is as expected. The result is the replaced file. But when I query: db:open('DB')/root[@id='id']/base-uri() result is a sequence: /DB/path/file.xml /DB/path/file.xml The first result is the file before the db:replace(), the second after. Can somebody explain why db:open() with path parameter is returning one result, but without parameter two results? Database statistics say there are indeed two documents. But when I export the database, only one gets exported. This happens with db:replace() on 9.2 from the DBA browser interface. When I use the same db:replace() XQuery on 9.2 with my local database, all is well (document gets replaced, no duplicates). Settings should be the default ones. Thanks, Daniel

3 8

Found a little error in docs for 'db:attribute#3'
by Andreas Mixich 20 May '19

20 May '19

Hi, at http://docs.basex.org/wiki/Database_Module there is a typo for the signature of 'db:attribute#3': It's 'xs:strings' instead of 'xs:string'. *db:attribute*($db as xs:string, $strings as xs:strings*, $name as xs:string) as attribute()* -- Minden jót, all the best, Alles Gute, Andreas Mixich

2 1

XQUnit Test
by Markus Schmid 20 May '19

20 May '19

Hi is it possible to run multiple testsuites with one TEST command call from the basex console? Or is there always only one <testsuite> element in the <testsuites> result? Markus __________________________________________

2 7

Optimizer and JSON/XML response
by Marco Lettere 17 May '19

17 May '19

Hello everyone, we came accross an inconsistency which we are not able to find an explanation for and giving up after having spent on it the whole morning... From a restxq we call an external REST service (InfluxDB) like shown in [1]. This usually works as expected returning a response forged as in [2]. When the same RESTXQ is deployed in a much more complex application (with several RESTXQ modules and REPO modules) the first call (which takes a lot of time because it incurs the initial compiling delay) returns [3] instead of [2]. From the second call onwards the result stabilizes correctly to [2]. If I call any other RESTXQ function as a sort of warm-up (spending there all the initial delay) [1] behaves correctly returning always [2]. Do you please have a clue on why this happens? We noticed it for the first time because we started to use service jobs which cause this access pattern ... Thanks for any help. [1] module namespace test = 'test'; declare %rest:path("test") %rest:GET function test:test() { let $url := "http://localhost:8086/query?pretty=false&chunked=false&db=nestway&a…" let $results := http:send-request(<http:request method="get"/>, $url) return $results }; [2] <http:response xmlns:http="http://expath.org/ns/http-client" status="200" message="OK"> <http:header name="Transfer-Encoding" value="chunked"/> <http:header name="Connection" value="close"/> <http:header name="X-Influxdb-Version" value="1.1.1"/> <http:header name="Date" value="Thu, 16 May 2019 11:05:59 GMT"/> <http:header name="Request-Id" value="95b729c7-77ca-11e9-8065-000000000000"/> <http:header name="Content-Type" value="application/json"/> <http:body media-type="application/json"/> </http:response> <json type="object"> <results type="array"> <_ type="object"> <series type="array"> <_ type="object"> <name>frontoffice</name> <columns type="array"> <_>time</_> <_>consumable</_> <_>customer</_> <_>guests</_> <_>type</_> <_>user</_> </columns> <values type="array"> <_ type="array"> <_>2019-05-10T21:55:00Z</_> <_>access</_> <_>c9a83ce3-c938-4677-a0df-72bc96354509</_> <_ type="number">0</_> <_>starttimer</_> <_>d50de945-23ed-4b33-84e8-8ef533a96fb7</_> </_> </values> </_> </series> </_> </results> </json> [3] <http:response xmlns:http="http://expath.org/ns/http-client" status="200" message="OK"> <http:header name="Transfer-Encoding" value="chunked"/> <http:header name="Connection" value="close"/> <http:header name="X-Influxdb-Version" value="1.1.1"/> <http:header name="Date" value="Thu, 16 May 2019 11:19:30 GMT"/> <http:header name="Request-Id" value="7957030a-77cc-11e9-8066-000000000000"/> <http:header name="Content-Type" value="application/json"/> <http:body media-type="application/json"/> </http:response> map { "results": [map { "series": [map { "values": [["2019-05-10T21:55:00Z", "access", "c9a83ce3-c938-4677-a0df-72bc96354509", 0.0e0, "starttimer", "d50de945-23ed-4b33-84e8-8ef533a96fb7"]], "name": "frontoffice", "columns": ["time", "consumable", "customer", "guests", "type", "user"] }] }] }

2 3