Re: [basex-talk] Send multipart binary with http client
Hi Max, Do you have an MCVE for us? Thanks in advance, Christian Am 30.01.2017 4:24 nachm. schrieb "Maximilian Gärber" <mgaerber@arcor.de>: Hi, this seems related to https://github.com/BaseXdb/basex/issues/1203 Problem 1: When setting Content-Transfer-Encoding manually (via <http:header>), it will appear twice in the request that is sent Problem 2: I cannot override base64 because it happens automatically based on the content I am sending and for some reason the web server that receives this request does not recognize the base64 encoding correctly Regards, Max 2017-01-30 15:35 GMT+01:00 Maximilian Gärber <mgaerber@arcor.de>:
Hi,
I can't figure out how to send a file in binary (not base64) when using multipart [1]
On the body, I've tried method="basex" because the docs says this would write raw bytes to the request
Regards, Max
[1]
declare function _:send-multipart($url, $file-name, $path) as item()* {
let $uri := file:path-to-uri($path || $file-name) let $type := fetch:content-type($uri)
let $binary := file:read-binary($path || $file-name) let $boundary_id := concat("batch_", random:uuid())
let $req := <http:request method="post" send-authorization="false" override-media-type="text/plain">
<http:header name="Accept" value="application/json"/> <http:header name="Accept-Encoding" value="gzip, deflate"/>
<http:multipart media-type="multipart/form-data" boundary="{$boundary_id}">
<http:header name="Content-Transfer-Encoding" value="binary"/> <http:header name="Content-Disposition" value="form-data; name='FILE'; filename='{$file-name}'"/> <http:header name="Content-Type" value="{$type}"/>
<!-- method="basex" method="adaptive" ? --> <http:body media-type="application/octet-stream" method="binary"> {$binary} </http:body>
</http:multipart> </http:request>
let $resp := http:send-request($req, $url) return json:parse($resp[2])
};
Hi, the decision for base64 is now hardcoded: https://github.com/BaseXdb/basex/commit/b48318abaf3a406d6e34253694c8e33e6628... Browser for example just send files in raw binary, so most other software (including jetty) does not know how to handle form-data with base64 encoding. Regards, Max 2017-01-30 19:16 GMT+01:00 Christian Grün <christian.gruen@gmail.com>:
Hi Max,
Do you have an MCVE for us?
Thanks in advance, Christian
Am 30.01.2017 4:24 nachm. schrieb "Maximilian Gärber" <mgaerber@arcor.de>:
Hi,
this seems related to https://github.com/BaseXdb/basex/issues/1203
Problem 1: When setting Content-Transfer-Encoding manually (via <http:header>), it will appear twice in the request that is sent
Problem 2: I cannot override base64 because it happens automatically based on the content I am sending and for some reason the web server that receives this request does not recognize the base64 encoding correctly
Regards,
Max
2017-01-30 15:35 GMT+01:00 Maximilian Gärber <mgaerber@arcor.de>:
Hi,
I can't figure out how to send a file in binary (not base64) when using multipart [1]
On the body, I've tried method="basex" because the docs says this would write raw bytes to the request
Regards, Max
[1]
declare function _:send-multipart($url, $file-name, $path) as item()* {
let $uri := file:path-to-uri($path || $file-name) let $type := fetch:content-type($uri)
let $binary := file:read-binary($path || $file-name) let $boundary_id := concat("batch_", random:uuid())
let $req := <http:request method="post" send-authorization="false" override-media-type="text/plain">
<http:header name="Accept" value="application/json"/> <http:header name="Accept-Encoding" value="gzip, deflate"/>
<http:multipart media-type="multipart/form-data" boundary="{$boundary_id}">
<http:header name="Content-Transfer-Encoding" value="binary"/> <http:header name="Content-Disposition" value="form-data; name='FILE'; filename='{$file-name}'"/> <http:header name="Content-Type" value="{$type}"/>
<!-- method="basex" method="adaptive" ? --> <http:body media-type="application/octet-stream" method="binary"> {$binary} </http:body>
</http:multipart> </http:request>
let $resp := http:send-request($req, $url) return json:parse($resp[2])
};
And the header is always written (no check if already set) https://github.com/BaseXdb/basex/commit/b48318abaf3a406d6e34253694c8e33e6628... It would be good if I could decide what Content-Transfer-Encoding was used. 2017-02-01 14:57 GMT+01:00 Maximilian Gärber <mgaerber@arcor.de>:
Hi,
the decision for base64 is now hardcoded: https://github.com/BaseXdb/basex/commit/b48318abaf3a406d6e34253694c8e33e6628...
Browser for example just send files in raw binary, so most other software (including jetty) does not know how to handle form-data with base64 encoding.
Regards, Max
2017-01-30 19:16 GMT+01:00 Christian Grün <christian.gruen@gmail.com>:
Hi Max,
Do you have an MCVE for us?
Thanks in advance, Christian
Am 30.01.2017 4:24 nachm. schrieb "Maximilian Gärber" <mgaerber@arcor.de>:
Hi,
this seems related to https://github.com/BaseXdb/basex/issues/1203
Problem 1: When setting Content-Transfer-Encoding manually (via <http:header>), it will appear twice in the request that is sent
Problem 2: I cannot override base64 because it happens automatically based on the content I am sending and for some reason the web server that receives this request does not recognize the base64 encoding correctly
Regards,
Max
2017-01-30 15:35 GMT+01:00 Maximilian Gärber <mgaerber@arcor.de>:
Hi,
I can't figure out how to send a file in binary (not base64) when using multipart [1]
On the body, I've tried method="basex" because the docs says this would write raw bytes to the request
Regards, Max
[1]
declare function _:send-multipart($url, $file-name, $path) as item()* {
let $uri := file:path-to-uri($path || $file-name) let $type := fetch:content-type($uri)
let $binary := file:read-binary($path || $file-name) let $boundary_id := concat("batch_", random:uuid())
let $req := <http:request method="post" send-authorization="false" override-media-type="text/plain">
<http:header name="Accept" value="application/json"/> <http:header name="Accept-Encoding" value="gzip, deflate"/>
<http:multipart media-type="multipart/form-data" boundary="{$boundary_id}">
<http:header name="Content-Transfer-Encoding" value="binary"/> <http:header name="Content-Disposition" value="form-data; name='FILE'; filename='{$file-name}'"/> <http:header name="Content-Type" value="{$type}"/>
<!-- method="basex" method="adaptive" ? --> <http:body media-type="application/octet-stream" method="binary"> {$binary} </http:body>
</http:multipart> </http:request>
let $resp := http:send-request($req, $url) return json:parse($resp[2])
};
Hi Max, Thanks for the hint. I’ll be glad to fix this, but I think I need an example, or step-by-step description, how to reproduce this quickly. Do you think that would be possible? Cheers from the other side of the lake, Christian On Wed, Feb 1, 2017 at 2:57 PM, Maximilian Gärber <mgaerber@arcor.de> wrote:
Hi,
the decision for base64 is now hardcoded: https://github.com/BaseXdb/basex/commit/b48318abaf3a406d6e34253694c8e33e6628...
Browser for example just send files in raw binary, so most other software (including jetty) does not know how to handle form-data with base64 encoding.
Regards, Max
2017-01-30 19:16 GMT+01:00 Christian Grün <christian.gruen@gmail.com>:
Hi Max,
Do you have an MCVE for us?
Thanks in advance, Christian
Am 30.01.2017 4:24 nachm. schrieb "Maximilian Gärber" <mgaerber@arcor.de>:
Hi,
this seems related to https://github.com/BaseXdb/basex/issues/1203
Problem 1: When setting Content-Transfer-Encoding manually (via <http:header>), it will appear twice in the request that is sent
Problem 2: I cannot override base64 because it happens automatically based on the content I am sending and for some reason the web server that receives this request does not recognize the base64 encoding correctly
Regards,
Max
2017-01-30 15:35 GMT+01:00 Maximilian Gärber <mgaerber@arcor.de>:
Hi,
I can't figure out how to send a file in binary (not base64) when using multipart [1]
On the body, I've tried method="basex" because the docs says this would write raw bytes to the request
Regards, Max
[1]
declare function _:send-multipart($url, $file-name, $path) as item()* {
let $uri := file:path-to-uri($path || $file-name) let $type := fetch:content-type($uri)
let $binary := file:read-binary($path || $file-name) let $boundary_id := concat("batch_", random:uuid())
let $req := <http:request method="post" send-authorization="false" override-media-type="text/plain">
<http:header name="Accept" value="application/json"/> <http:header name="Accept-Encoding" value="gzip, deflate"/>
<http:multipart media-type="multipart/form-data" boundary="{$boundary_id}">
<http:header name="Content-Transfer-Encoding" value="binary"/> <http:header name="Content-Disposition" value="form-data; name='FILE'; filename='{$file-name}'"/> <http:header name="Content-Type" value="{$type}"/>
<!-- method="basex" method="adaptive" ? --> <http:body media-type="application/octet-stream" method="binary"> {$binary} </http:body>
</http:multipart> </http:request>
let $resp := http:send-request($req, $url) return json:parse($resp[2])
};
Hi, just try to send for example an image file (e.g. attached) with the code below [1]. It will always be base64 (and when I uncomment the "Content-Transfer-Encoding" header it will appear twice) You would need something like Fiddler (https://www.telerik.com/download/fiddler) to intercept the request an see what is going on. This is the shortest example I could create. Best, Max [1] let $name := "grass.jpg" let $path := "upload/files/" let $url := concat("http://localhost:4567", "/file/meta") return _:send-multipart($url, $name, $path) declare function _:send-multipart($url, $file-name, $path) as item()* { let $uri := file:path-to-uri($path || $file-name) let $type := fetch:content-type($uri) let $binary := file:read-binary($path || $file-name) let $req := <http:request method="post" send-authorization="false" override-media-type="text/plain"> <http:header name="Accept" value="application/json"/> <http:header name="Accept-Encoding" value="gzip, deflate"/> <http:multipart media-type="multipart/form-data"> <!-- <http:header name="Content-Transfer-Encoding" value="binary"/> --> <http:header name="Content-Disposition" value='form-data; name="FILE"; filename="{$file-name}"'/> <http:body media-type="{$type}"/> </http:multipart> </http:request> let $resp := http:send-request($req, $url, $binary) return json:parse($resp[2]) }; 2017-02-01 15:01 GMT+01:00 Christian Grün <christian.gruen@gmail.com>:
Hi Max,
Thanks for the hint. I’ll be glad to fix this, but I think I need an example, or step-by-step description, how to reproduce this quickly. Do you think that would be possible?
Cheers from the other side of the lake, Christian
On Wed, Feb 1, 2017 at 2:57 PM, Maximilian Gärber <mgaerber@arcor.de> wrote:
Hi,
the decision for base64 is now hardcoded: https://github.com/BaseXdb/basex/commit/b48318abaf3a406d6e34253694c8e33e6628...
Browser for example just send files in raw binary, so most other software (including jetty) does not know how to handle form-data with base64 encoding.
Regards, Max
2017-01-30 19:16 GMT+01:00 Christian Grün <christian.gruen@gmail.com>:
Hi Max,
Do you have an MCVE for us?
Thanks in advance, Christian
Am 30.01.2017 4:24 nachm. schrieb "Maximilian Gärber" <mgaerber@arcor.de>:
Hi,
this seems related to https://github.com/BaseXdb/basex/issues/1203
Problem 1: When setting Content-Transfer-Encoding manually (via <http:header>), it will appear twice in the request that is sent
Problem 2: I cannot override base64 because it happens automatically based on the content I am sending and for some reason the web server that receives this request does not recognize the base64 encoding correctly
Regards,
Max
2017-01-30 15:35 GMT+01:00 Maximilian Gärber <mgaerber@arcor.de>:
Hi,
I can't figure out how to send a file in binary (not base64) when using multipart [1]
On the body, I've tried method="basex" because the docs says this would write raw bytes to the request
Regards, Max
[1]
declare function _:send-multipart($url, $file-name, $path) as item()* {
let $uri := file:path-to-uri($path || $file-name) let $type := fetch:content-type($uri)
let $binary := file:read-binary($path || $file-name) let $boundary_id := concat("batch_", random:uuid())
let $req := <http:request method="post" send-authorization="false" override-media-type="text/plain">
<http:header name="Accept" value="application/json"/> <http:header name="Accept-Encoding" value="gzip, deflate"/>
<http:multipart media-type="multipart/form-data" boundary="{$boundary_id}">
<http:header name="Content-Transfer-Encoding" value="binary"/> <http:header name="Content-Disposition" value="form-data; name='FILE'; filename='{$file-name}'"/> <http:header name="Content-Type" value="{$type}"/>
<!-- method="basex" method="adaptive" ? --> <http:body media-type="application/octet-stream" method="binary"> {$binary} </http:body>
</http:multipart> </http:request>
let $resp := http:send-request($req, $url) return json:parse($resp[2])
};
Hi Max, I think I still need some more information on the missing pieces. For example, which process is hidden behind localhost:4567? Is there some RESTXQ code as well?
On the body, I've tried method="basex" because the docs says this would write raw bytes to the request
Which docs do you refer to? Thanks in advance, Christian On Wed, Feb 1, 2017 at 3:47 PM, Maximilian Gärber <mgaerber@arcor.de> wrote:
Hi,
just try to send for example an image file (e.g. attached) with the code below [1]. It will always be base64 (and when I uncomment the "Content-Transfer-Encoding" header it will appear twice)
You would need something like Fiddler (https://www.telerik.com/download/fiddler) to intercept the request an see what is going on.
This is the shortest example I could create.
Best, Max
[1]
let $name := "grass.jpg" let $path := "upload/files/" let $url := concat("http://localhost:4567", "/file/meta") return _:send-multipart($url, $name, $path)
declare function _:send-multipart($url, $file-name, $path) as item()* {
let $uri := file:path-to-uri($path || $file-name) let $type := fetch:content-type($uri) let $binary := file:read-binary($path || $file-name)
let $req := <http:request method="post" send-authorization="false" override-media-type="text/plain">
<http:header name="Accept" value="application/json"/> <http:header name="Accept-Encoding" value="gzip, deflate"/>
<http:multipart media-type="multipart/form-data">
<!-- <http:header name="Content-Transfer-Encoding" value="binary"/> --> <http:header name="Content-Disposition" value='form-data; name="FILE"; filename="{$file-name}"'/>
<http:body media-type="{$type}"/>
</http:multipart> </http:request>
let $resp := http:send-request($req, $url, $binary) return json:parse($resp[2]) };
2017-02-01 15:01 GMT+01:00 Christian Grün <christian.gruen@gmail.com>:
Hi Max,
Thanks for the hint. I’ll be glad to fix this, but I think I need an example, or step-by-step description, how to reproduce this quickly. Do you think that would be possible?
Cheers from the other side of the lake, Christian
On Wed, Feb 1, 2017 at 2:57 PM, Maximilian Gärber <mgaerber@arcor.de> wrote:
Hi,
the decision for base64 is now hardcoded: https://github.com/BaseXdb/basex/commit/b48318abaf3a406d6e34253694c8e33e6628...
Browser for example just send files in raw binary, so most other software (including jetty) does not know how to handle form-data with base64 encoding.
Regards, Max
2017-01-30 19:16 GMT+01:00 Christian Grün <christian.gruen@gmail.com>:
Hi Max,
Do you have an MCVE for us?
Thanks in advance, Christian
Am 30.01.2017 4:24 nachm. schrieb "Maximilian Gärber" <mgaerber@arcor.de>:
Hi,
this seems related to https://github.com/BaseXdb/basex/issues/1203
Problem 1: When setting Content-Transfer-Encoding manually (via <http:header>), it will appear twice in the request that is sent
Problem 2: I cannot override base64 because it happens automatically based on the content I am sending and for some reason the web server that receives this request does not recognize the base64 encoding correctly
Regards,
Max
2017-01-30 15:35 GMT+01:00 Maximilian Gärber <mgaerber@arcor.de>:
Hi,
I can't figure out how to send a file in binary (not base64) when using multipart [1]
On the body, I've tried method="basex" because the docs says this would write raw bytes to the request
Regards, Max
[1]
declare function _:send-multipart($url, $file-name, $path) as item()* {
let $uri := file:path-to-uri($path || $file-name) let $type := fetch:content-type($uri)
let $binary := file:read-binary($path || $file-name) let $boundary_id := concat("batch_", random:uuid())
let $req := <http:request method="post" send-authorization="false" override-media-type="text/plain">
<http:header name="Accept" value="application/json"/> <http:header name="Accept-Encoding" value="gzip, deflate"/>
<http:multipart media-type="multipart/form-data" boundary="{$boundary_id}">
<http:header name="Content-Transfer-Encoding" value="binary"/> <http:header name="Content-Disposition" value="form-data; name='FILE'; filename='{$file-name}'"/> <http:header name="Content-Type" value="{$type}"/>
<!-- method="basex" method="adaptive" ? --> <http:body media-type="application/octet-stream" method="binary"> {$binary} </http:body>
</http:multipart> </http:request>
let $resp := http:send-request($req, $url) return json:parse($resp[2])
};
participants (2)
-
Christian Grün -
Maximilian Gärber