Content-Encoding: gzip on incoming RESTXQ POST
Hello, I have a RESTXQ function that receives an XML file via HTTP POST. This works perfectly. The client would now like to compress the data with gzip and set the Content-Encoding: gzip header. I set GZIP = true in my .basex file based on the documentation [1] But these gzipped post requests fail - my function is called but content seems to arrive at my function still compressed. (Responses from the server become compressed dynamically as expected - and that works great). Is there something else I need to do to make it work for requests? Many thanks, James [1] https://docs.basex.org/wiki/Options#GZIP
Hi James, This is due to the default gzip settings of Jetty, which exclude POST requests. There had been a similar question in the past, and I have now decided to add the POST and PUT method to the Jetty defaults [1,2]. A new snapshot is available! Cheers, Christian [1] https://www.mail-archive.com/basex-talk@mailman.uni-konstanz.de/msg12281.htm... [2] https://github.com/BaseXdb/basex/issues/1748 On Tue, Nov 2, 2021 at 6:29 PM James Ball <basex-talk@jamesball.co.uk> wrote:
Hello,
I have a RESTXQ function that receives an XML file via HTTP POST. This works perfectly.
The client would now like to compress the data with gzip and set the Content-Encoding: gzip header.
I set GZIP = true in my .basex file based on the documentation [1]
But these gzipped post requests fail - my function is called but content seems to arrive at my function still compressed.
(Responses from the server become compressed dynamically as expected - and that works great).
Is there something else I need to do to make it work for requests?
Many thanks, James
Hi Christian, My Google-fu was definitely letting me down. I completely missed that POST and PUT had to be explicitly included. Thank you for the snapshot - which I’ve downloaded and tried. Sadly - same error. I *think* I’ve found yet another default configuration for GZIPHandler getting in the way [1]: "Both Request uncompress and Response compress are gated by a configurable DispatcherType check on the GzipHandler … (Default: DispatcherType.REQUEST).” I understand this to mean that requests will NOT be decompressed by default. Would you read that the same way? Do you think adding a line like the below would be the answer? My Java is getting out of date but if you think I’m on the right track I will try it and prepare a pull request for you. Many thanks, James BaseXHTTP.java [2] import javax.servlet.* gzip.setDispatcherTypes(DispatcherType.REQUEST, DispatcherType.RESPONSE) [1] https://www.eclipse.org/jetty/javadoc/jetty-9/org/eclipse/jetty/server/handl......) [2] https://github.com/BaseXdb/basex/blob/5f4f4316c77f781275a3e8afd58c151792d20f...
On 4 Nov 2021, at 08:22, Christian Grün <christian.gruen@gmail.com> wrote:
Hi James,
This is due to the default gzip settings of Jetty, which exclude POST requests. There had been a similar question in the past, and I have now decided to add the POST and PUT method to the Jetty defaults [1,2].
A new snapshot is available!
Cheers, Christian
[1] https://www.mail-archive.com/basex-talk@mailman.uni-konstanz.de/msg12281.htm... [2] https://github.com/BaseXdb/basex/issues/1748
On Tue, Nov 2, 2021 at 6:29 PM James Ball <basex-talk@jamesball.co.uk> wrote:
Hello,
I have a RESTXQ function that receives an XML file via HTTP POST. This works perfectly.
The client would now like to compress the data with gzip and set the Content-Encoding: gzip header.
I set GZIP = true in my .basex file based on the documentation [1]
But these gzipped post requests fail - my function is called but content seems to arrive at my function still compressed.
(Responses from the server become compressed dynamically as expected - and that works great).
Is there something else I need to do to make it work for requests?
Many thanks, James
Hi James, Sounds like a good hint! I’ll be happy to have a look at your pull request. Thanks in advance, Christian On Fri, Nov 5, 2021 at 1:20 PM James Ball <basex-talk@jamesball.co.uk> wrote:
Hi Christian,
My Google-fu was definitely letting me down. I completely missed that POST and PUT had to be explicitly included.
Thank you for the snapshot - which I’ve downloaded and tried. Sadly - same error.
I *think* I’ve found yet another default configuration for GZIPHandler getting in the way [1]:
"Both Request uncompress and Response compress are gated by a configurable DispatcherType check on the GzipHandler … (Default: DispatcherType.REQUEST).”
I understand this to mean that requests will NOT be decompressed by default. Would you read that the same way?
Do you think adding a line like the below would be the answer? My Java is getting out of date but if you think I’m on the right track I will try it and prepare a pull request for you.
Many thanks, James
BaseXHTTP.java [2]
import javax.servlet.*
gzip.setDispatcherTypes(DispatcherType.REQUEST, DispatcherType.RESPONSE)
[1] https://www.eclipse.org/jetty/javadoc/jetty-9/org/eclipse/jetty/server/handl......) [2] https://github.com/BaseXdb/basex/blob/5f4f4316c77f781275a3e8afd58c151792d20f...
On 4 Nov 2021, at 08:22, Christian Grün <christian.gruen@gmail.com> wrote:
Hi James,
This is due to the default gzip settings of Jetty, which exclude POST requests. There had been a similar question in the past, and I have now decided to add the POST and PUT method to the Jetty defaults [1,2].
A new snapshot is available!
Cheers, Christian
[1] https://www.mail-archive.com/basex-talk@mailman.uni-konstanz.de/msg12281.htm... [2] https://github.com/BaseXdb/basex/issues/1748
On Tue, Nov 2, 2021 at 6:29 PM James Ball <basex-talk@jamesball.co.uk> wrote:
Hello,
I have a RESTXQ function that receives an XML file via HTTP POST. This works perfectly.
The client would now like to compress the data with gzip and set the Content-Encoding: gzip header.
I set GZIP = true in my .basex file based on the documentation [1]
But these gzipped post requests fail - my function is called but content seems to arrive at my function still compressed.
(Responses from the server become compressed dynamically as expected - and that works great).
Is there something else I need to do to make it work for requests?
Many thanks, James
Is it possible to encrypt the zip file for HTTP requests? I have some documents that are not freely licensed, and I need some way to protect the IP of the copyright holder if I want to enable their texts. Jonathan On Fri, Nov 5, 2021 at 8:32 AM Christian Grün <christian.gruen@gmail.com> wrote: > Hi James, > > Sounds like a good hint! I’ll be happy to have a look at your pull request. > > Thanks in advance, > Christian > > > > On Fri, Nov 5, 2021 at 1:20 PM James Ball <basex-talk@jamesball.co.uk> > wrote: > > > > Hi Christian, > > > > My Google-fu was definitely letting me down. I completely missed that > POST and PUT had to be explicitly included. > > > > Thank you for the snapshot - which I’ve downloaded and tried. Sadly - > same error. > > > > I *think* I’ve found yet another default configuration for GZIPHandler > getting in the way [1]: > > > > "Both Request uncompress and Response compress are gated by a > configurable DispatcherType check on the GzipHandler … (Default: > DispatcherType.REQUEST).” > > > > I understand this to mean that requests will NOT be decompressed by > default. Would you read that the same way? > > > > Do you think adding a line like the below would be the answer? My Java > is getting out of date but if you think I’m on the right track I will try > it and prepare a pull request for you. > > > > > > Many thanks, James > > > > > > BaseXHTTP.java [2] > > > > import javax.servlet.* > > > > gzip.setDispatcherTypes(DispatcherType.REQUEST, DispatcherType.RESPONSE) > > > > > > > > [1] > https://www.eclipse.org/jetty/javadoc/jetty-9/org/eclipse/jetty/server/handler/gzip/GzipHandler.html#setDispatcherTypes(javax.servlet.DispatcherType.. > .) > > [2] > https://github.com/BaseXdb/basex/blob/5f4f4316c77f781275a3e8afd58c151792d20f40/basex-api/src/main/java/org/basex/BaseXHTTP.java > > > > > On 4 Nov 2021, at 08:22, Christian Grün <christian.gruen@gmail.com> > wrote: > > > > > > Hi James, > > > > > > This is due to the default gzip settings of Jetty, which exclude POST > > > requests. There had been a similar question in the past, and I have > > > now decided to add the POST and PUT method to the Jetty defaults > > > [1,2]. > > > > > > A new snapshot is available! > > > > > > Cheers, > > > Christian > > > > > > [1] > https://www.mail-archive.com/basex-talk@mailman.uni-konstanz.de/msg12281.html > > > [2] https://github.com/BaseXdb/basex/issues/1748 > > > > > > > > > > > > On Tue, Nov 2, 2021 at 6:29 PM James Ball <basex-talk@jamesball.co.uk> > wrote: > > >> > > >> Hello, > > >> > > >> I have a RESTXQ function that receives an XML file via HTTP POST. > This works perfectly. > > >> > > >> The client would now like to compress the data with gzip and set the > Content-Encoding: gzip header. > > >> > > >> I set GZIP = true in my .basex file based on the documentation [1] > > >> > > >> But these gzipped post requests fail - my function is called but > content seems to arrive at my function still compressed. > > >> > > >> (Responses from the server become compressed dynamically as expected > - and that works great). > > >> > > >> Is there something else I need to do to make it work for requests? > > >> > > >> Many thanks, James > > >> > > >> > > >> [1] https://docs.basex.org/wiki/Options#GZIP > > >
Hi Jonathan, do you have a chance to use HTTPS? On Mon, Nov 8, 2021 at 3:40 PM Jonathan Robie <jonathan.robie@gmail.com> wrote:
Is it possible to encrypt the zip file for HTTP requests? I have some documents that are not freely licensed, and I need some way to protect the IP of the copyright holder if I want to enable their texts.
Jonathan
On Fri, Nov 5, 2021 at 8:32 AM Christian Grün <christian.gruen@gmail.com> wrote:
Hi James,
Sounds like a good hint! I’ll be happy to have a look at your pull request.
Thanks in advance, Christian
On Fri, Nov 5, 2021 at 1:20 PM James Ball <basex-talk@jamesball.co.uk> wrote:
Hi Christian,
My Google-fu was definitely letting me down. I completely missed that POST and PUT had to be explicitly included.
Thank you for the snapshot - which I’ve downloaded and tried. Sadly - same error.
I *think* I’ve found yet another default configuration for GZIPHandler getting in the way [1]:
"Both Request uncompress and Response compress are gated by a configurable DispatcherType check on the GzipHandler … (Default: DispatcherType.REQUEST).”
I understand this to mean that requests will NOT be decompressed by default. Would you read that the same way?
Do you think adding a line like the below would be the answer? My Java is getting out of date but if you think I’m on the right track I will try it and prepare a pull request for you.
Many thanks, James
BaseXHTTP.java [2]
import javax.servlet.*
gzip.setDispatcherTypes(DispatcherType.REQUEST, DispatcherType.RESPONSE)
[1] https://www.eclipse.org/jetty/javadoc/jetty-9/org/eclipse/jetty/server/handl......) [2] https://github.com/BaseXdb/basex/blob/5f4f4316c77f781275a3e8afd58c151792d20f...
On 4 Nov 2021, at 08:22, Christian Grün <christian.gruen@gmail.com> wrote:
Hi James,
This is due to the default gzip settings of Jetty, which exclude POST requests. There had been a similar question in the past, and I have now decided to add the POST and PUT method to the Jetty defaults [1,2].
A new snapshot is available!
Cheers, Christian
[1] https://www.mail-archive.com/basex-talk@mailman.uni-konstanz.de/msg12281.htm... [2] https://github.com/BaseXdb/basex/issues/1748
On Tue, Nov 2, 2021 at 6:29 PM James Ball <basex-talk@jamesball.co.uk> wrote:
Hello,
I have a RESTXQ function that receives an XML file via HTTP POST. This works perfectly.
The client would now like to compress the data with gzip and set the Content-Encoding: gzip header.
I set GZIP = true in my .basex file based on the documentation [1]
But these gzipped post requests fail - my function is called but content seems to arrive at my function still compressed.
(Responses from the server become compressed dynamically as expected - and that works great).
Is there something else I need to do to make it work for requests?
Many thanks, James
Yes, I do. And yeah, that's really the right solution ... Thanks! Jonathan On Mon, Nov 8, 2021 at 11:52 AM Christian Grün <christian.gruen@gmail.com> wrote:
Hi Jonathan, do you have a chance to use HTTPS?
On Mon, Nov 8, 2021 at 3:40 PM Jonathan Robie <jonathan.robie@gmail.com> wrote:
Is it possible to encrypt the zip file for HTTP requests? I have some
documents that are not freely licensed, and I need some way to protect the IP of the copyright holder if I want to enable their texts.
Jonathan
On Fri, Nov 5, 2021 at 8:32 AM Christian Grün <christian.gruen@gmail.com>
wrote:
Hi James,
Sounds like a good hint! I’ll be happy to have a look at your pull
request.
Thanks in advance, Christian
On Fri, Nov 5, 2021 at 1:20 PM James Ball <basex-talk@jamesball.co.uk>
wrote:
Hi Christian,
My Google-fu was definitely letting me down. I completely missed that
POST and PUT had to be explicitly included.
Thank you for the snapshot - which I’ve downloaded and tried. Sadly -
same error.
I *think* I’ve found yet another default configuration for
GZIPHandler getting in the way [1]:
"Both Request uncompress and Response compress are gated by a
configurable DispatcherType check on the GzipHandler … (Default: DispatcherType.REQUEST).”
I understand this to mean that requests will NOT be decompressed by
default. Would you read that the same way?
Do you think adding a line like the below would be the answer? My
Java is getting out of date but if you think I’m on the right track I will try it and prepare a pull request for you.
Many thanks, James
BaseXHTTP.java [2]
import javax.servlet.*
gzip.setDispatcherTypes(DispatcherType.REQUEST,
DispatcherType.RESPONSE)
[1]
https://www.eclipse.org/jetty/javadoc/jetty-9/org/eclipse/jetty/server/handl..... .)
[2] https://github.com/BaseXdb/basex/blob/5f4f4316c77f781275a3e8afd58c151792d20f...
On 4 Nov 2021, at 08:22, Christian Grün <christian.gruen@gmail.com> wrote:
Hi James,
This is due to the default gzip settings of Jetty, which exclude POST requests. There had been a similar question in the past, and I have now decided to add the POST and PUT method to the Jetty defaults [1,2].
A new snapshot is available!
Cheers, Christian
[1] https://www.mail-archive.com/basex-talk@mailman.uni-konstanz.de/msg12281.htm... [2] https://github.com/BaseXdb/basex/issues/1748
On Tue, Nov 2, 2021 at 6:29 PM James Ball < basex-talk@jamesball.co.uk> wrote:
Hello,
I have a RESTXQ function that receives an XML file via HTTP POST.
This works perfectly.
The client would now like to compress the data with gzip and set
the Content-Encoding: gzip header.
I set GZIP = true in my .basex file based on the documentation [1]
But these gzipped post requests fail - my function is called but
content seems to arrive at my function still compressed.
(Responses from the server become compressed dynamically as
expected - and that works great).
Is there something else I need to do to make it work for requests?
Many thanks, James
To close this for the archive and for anyone who needs this in the future: In Jetty the GzipHandler has a number of defaults that will limit what it does unless explicitly overridden. For example it will compress responses only with GET requests and will not decompress requests. The configuration of the GzipHandler in BaseX has been changed (thank you Christian!): • It now explicitly includes POST and PUT methods • The inflate buffer size is set greater than 0 - which enables transparent decompression of gzipped requests So something like: echo '<Test/>' | gzip | curl -v -H "Content-Type: text/xml" -H "Content-Encoding: gzip" --data-binary @- http://localhost:8984/yourFunction will now deliver XML to yourFunction on a system that has GZIP set to true. Regards, James
On 4 Nov 2021, at 08:22, Christian Grün <christian.gruen@gmail.com> wrote:
Hi James,
This is due to the default gzip settings of Jetty, which exclude POST requests. There had been a similar question in the past, and I have now decided to add the POST and PUT method to the Jetty defaults [1,2].
A new snapshot is available!
Cheers, Christian
[1] https://www.mail-archive.com/basex-talk@mailman.uni-konstanz.de/msg12281.htm... [2] https://github.com/BaseXdb/basex/issues/1748
On Tue, Nov 2, 2021 at 6:29 PM James Ball <basex-talk@jamesball.co.uk> wrote:
Hello,
I have a RESTXQ function that receives an XML file via HTTP POST. This works perfectly.
The client would now like to compress the data with gzip and set the Content-Encoding: gzip header.
I set GZIP = true in my .basex file based on the documentation [1]
But these gzipped post requests fail - my function is called but content seems to arrive at my function still compressed.
(Responses from the server become compressed dynamically as expected - and that works great).
Is there something else I need to do to make it work for requests?
Many thanks, James
participants (3)
-
Christian Grün -
James Ball -
Jonathan Robie