Hi James,
There will always be an upper limit when uploading files via RESTXQ. The limit will be lower if multipart data is sent, as all single parts of your request body will additionally be wrapped into a map and (possibly) converted to another format.
You could try to interpret the incoming POST data with XQuery, but then you might also struggle with memory constraints.
But I rather recommend you to limit the number of bytes that users are allowed to send to a server (see e.g. [1]) and (if uploading larger files is an important requirement) increase the memory that’s assigned to the JVM.
If you have gigabytes of data to upload, you may need to resort to more flexible JavaScript libraries (that e.g. allow you to upload portions of large files) or use the plain REST API, which supports full streaming support when adding database resources.
Hope this helps, Christian
[1] https://www.eclipse.org/jetty/documentation/9.4.x/setting-form-size.html
On Wed, Mar 20, 2019 at 3:02 PM James Ball basex-talk@jamesball.co.uk wrote:
Hello Dirk,
Thank you for such a quick response.
However, what is the actual reason to put the binary into a map?
I don’t know - it’s BaseX that is doing it. For files uploaded via POST they are put into a map. I’m using the information here: http://docs.basex.org/wiki/RESTXQ#File_Uploads
"The file contents are placed in a map, with the filename serving as key.”
I would try to use the streaming capabilities and to store the binary as-is.
This would be perfect but I’m not sure how to get the binary from the POST request without using the map.
I have a function something like this:
declare %rest:POST("{$data}") %rest:path("/test2.htm") %rest:consumes("multipart/form-data") %rest:form-param("zip", "{$files}") function _:test($data,$files, $database) { ... };
If I just save $data to disk I don’t get a valid Zip file. $data type is xs:base64Binary but is not the same as the xs:base64Binary from $files. To save the file from $files I have to use the map and so get the error.
Is there a trick to get $data in the right format to match what I get from $files?
Many thanks, James