Issue with HTTP client and authentication
Hello, I'm trying to post a SPARQL query to an endpoint using Digest authentication with the HTTP client. The query works fine using curl: curl --digest --user user:pass -X POST -d@'test.rq' \ -H "Content-type: application/sparql-query" \ 'http://example.org/sparql' But the equivalent request in BaseX fails with 401 Unauthorized: let $endpoint := "http://example.org/sparql" let $user := "user" let $pass := "pass" let $type := "application/sparql-query" let $response := ( http:send-request( <http:request method="POST" href="{$endpoint}" username="{$user}" password="{$pass}" auth-method="Digest" send-authorization="true"> <http:header name="Content-Type" value="{$type}; charset=utf-8"/> <http:body media-type="{$type}">{ ``[ select * where {?s ?p ?o} limit 1 ]`` }</http:body> </http:request> ) ) return $response Any ideas about what might be causing the BaseX HTTP client to be denied here? Thanks in advance, Tim -- Tim A. Thompson Metadata Librarian Yale University Library
Hi Tim, Difficult to tell; both the curl and the XQuery variants give me 404. Do you think you could provide us with an example that works out of the box? Best, Christian On Wed, Feb 2, 2022 at 12:54 AM Tim Thompson <timathom@gmail.com> wrote:
Hello,
I'm trying to post a SPARQL query to an endpoint using Digest authentication with the HTTP client.
The query works fine using curl:
curl --digest --user user:pass -X POST -d@'test.rq' \ -H "Content-type: application/sparql-query" \ 'http://example.org/sparql'
But the equivalent request in BaseX fails with 401 Unauthorized:
let $endpoint := "http://example.org/sparql" let $user := "user" let $pass := "pass" let $type := "application/sparql-query"
let $response := ( http:send-request( <http:request method="POST" href="{$endpoint}" username="{$user}" password="{$pass}" auth-method="Digest" send-authorization="true"> <http:header name="Content-Type" value="{$type}; charset=utf-8"/> <http:body media-type="{$type}">{ ``[ select * where {?s ?p ?o} limit 1 ]`` }</http:body> </http:request> ) ) return $response
Any ideas about what might be causing the BaseX HTTP client to be denied here?
Thanks in advance, Tim
-- Tim A. Thompson Metadata Librarian Yale University Library
Thanks, Christian. Right, that wasn't a working example. The server I'm querying is behind a firewall, so I can't really reproduce the issue unfortunately. The BaseX documentation states that the HTTP client does support Digest authentication. However, I did some poking around using Wireshark and limited knowledge of network traffic. When I run the request in curl, the expected protocol is played out, and I see the evidence in Wireshark: (1) The server responds with "401 Unauthorized" and provides the nonce value. (2) The client (curl) then does its business (supplies the username and password, resends the request, etc.) But when I submit the request via BaseX, it never gets past step (1). It never seems to send the username and password at all. Is Digest authentication still supported? Best, Tim -- Tim A. Thompson Metadata Librarian Yale University Library On Wed, Feb 2, 2022 at 10:13 AM Christian Grün <christian.gruen@gmail.com> wrote:
Hi Tim,
Difficult to tell; both the curl and the XQuery variants give me 404. Do you think you could provide us with an example that works out of the box?
Best, Christian
On Wed, Feb 2, 2022 at 12:54 AM Tim Thompson <timathom@gmail.com> wrote:
Hello,
I'm trying to post a SPARQL query to an endpoint using Digest
authentication with the HTTP client.
The query works fine using curl:
curl --digest --user user:pass -X POST -d@'test.rq' \ -H "Content-type: application/sparql-query" \ 'http://example.org/sparql'
But the equivalent request in BaseX fails with 401 Unauthorized:
let $endpoint := "http://example.org/sparql" let $user := "user" let $pass := "pass" let $type := "application/sparql-query"
let $response := ( http:send-request( <http:request method="POST" href="{$endpoint}" username="{$user}" password="{$pass}" auth-method="Digest" send-authorization="true"> <http:header name="Content-Type" value="{$type}; charset=utf-8"/> <http:body media-type="{$type}">{ ``[ select * where {?s ?p ?o} limit 1 ]`` }</http:body> </http:request> ) ) return $response
Any ideas about what might be causing the BaseX HTTP client to be denied
here?
Thanks in advance, Tim
-- Tim A. Thompson Metadata Librarian Yale University Library
Yes, it should still be supported. I assume it doesn't make a difference if you use "Digest" or "digest"? Tim Thompson <timathom@gmail.com> schrieb am Mi., 2. Feb. 2022, 17:48:
Thanks, Christian. Right, that wasn't a working example. The server I'm querying is behind a firewall, so I can't really reproduce the issue unfortunately.
The BaseX documentation states that the HTTP client does support Digest authentication. However, I did some poking around using Wireshark and limited knowledge of network traffic. When I run the request in curl, the expected protocol is played out, and I see the evidence in Wireshark:
(1) The server responds with "401 Unauthorized" and provides the nonce value. (2) The client (curl) then does its business (supplies the username and password, resends the request, etc.)
But when I submit the request via BaseX, it never gets past step (1). It never seems to send the username and password at all. Is Digest authentication still supported?
Best, Tim
-- Tim A. Thompson Metadata Librarian Yale University Library
On Wed, Feb 2, 2022 at 10:13 AM Christian Grün <christian.gruen@gmail.com> wrote:
Hi Tim,
Difficult to tell; both the curl and the XQuery variants give me 404. Do you think you could provide us with an example that works out of the box?
Best, Christian
On Wed, Feb 2, 2022 at 12:54 AM Tim Thompson <timathom@gmail.com> wrote:
Hello,
I'm trying to post a SPARQL query to an endpoint using Digest
authentication with the HTTP client.
The query works fine using curl:
curl --digest --user user:pass -X POST -d@'test.rq' \ -H "Content-type: application/sparql-query" \ 'http://example.org/sparql'
But the equivalent request in BaseX fails with 401 Unauthorized:
let $endpoint := "http://example.org/sparql" let $user := "user" let $pass := "pass" let $type := "application/sparql-query"
let $response := ( http:send-request( <http:request method="POST" href="{$endpoint}" username="{$user}" password="{$pass}" auth-method="Digest" send-authorization="true"> <http:header name="Content-Type" value="{$type}; charset=utf-8"/> <http:body media-type="{$type}">{ ``[ select * where {?s ?p ?o} limit 1 ]`` }</http:body> </http:request> ) ) return $response
Any ideas about what might be causing the BaseX HTTP client to be
denied here?
Thanks in advance, Tim
-- Tim A. Thompson Metadata Librarian Yale University Library
With "digest" I get: "[experr:HC0004] Invalid authentication method: digest." -- Tim A. Thompson Metadata Librarian Yale University Library On Wed, Feb 2, 2022 at 11:52 AM Christian Grün <christian.gruen@gmail.com> wrote:
Yes, it should still be supported. I assume it doesn't make a difference if you use "Digest" or "digest"?
Tim Thompson <timathom@gmail.com> schrieb am Mi., 2. Feb. 2022, 17:48:
Thanks, Christian. Right, that wasn't a working example. The server I'm querying is behind a firewall, so I can't really reproduce the issue unfortunately.
The BaseX documentation states that the HTTP client does support Digest authentication. However, I did some poking around using Wireshark and limited knowledge of network traffic. When I run the request in curl, the expected protocol is played out, and I see the evidence in Wireshark:
(1) The server responds with "401 Unauthorized" and provides the nonce value. (2) The client (curl) then does its business (supplies the username and password, resends the request, etc.)
But when I submit the request via BaseX, it never gets past step (1). It never seems to send the username and password at all. Is Digest authentication still supported?
Best, Tim
-- Tim A. Thompson Metadata Librarian Yale University Library
On Wed, Feb 2, 2022 at 10:13 AM Christian Grün <christian.gruen@gmail.com> wrote:
Hi Tim,
Difficult to tell; both the curl and the XQuery variants give me 404. Do you think you could provide us with an example that works out of the box?
Best, Christian
On Wed, Feb 2, 2022 at 12:54 AM Tim Thompson <timathom@gmail.com> wrote:
Hello,
I'm trying to post a SPARQL query to an endpoint using Digest
authentication with the HTTP client.
The query works fine using curl:
curl --digest --user user:pass -X POST -d@'test.rq' \ -H "Content-type: application/sparql-query" \ 'http://example.org/sparql'
But the equivalent request in BaseX fails with 401 Unauthorized:
let $endpoint := "http://example.org/sparql" let $user := "user" let $pass := "pass" let $type := "application/sparql-query"
let $response := ( http:send-request( <http:request method="POST" href="{$endpoint}" username="{$user}" password="{$pass}" auth-method="Digest" send-authorization="true"> <http:header name="Content-Type" value="{$type}; charset=utf-8"/> <http:body media-type="{$type}">{ ``[ select * where {?s ?p ?o} limit 1 ]`` }</http:body> </http:request> ) ) return $response
Any ideas about what might be causing the BaseX HTTP client to be
denied here?
Thanks in advance, Tim
-- Tim A. Thompson Metadata Librarian Yale University Library
Good to hear; so at least Digest is detected at this stage ;) Tim Thompson <timathom@gmail.com> schrieb am Mi., 2. Feb. 2022, 17:54:
With "digest" I get: "[experr:HC0004] Invalid authentication method: digest."
-- Tim A. Thompson Metadata Librarian Yale University Library
On Wed, Feb 2, 2022 at 11:52 AM Christian Grün <christian.gruen@gmail.com> wrote:
Yes, it should still be supported. I assume it doesn't make a difference if you use "Digest" or "digest"?
Tim Thompson <timathom@gmail.com> schrieb am Mi., 2. Feb. 2022, 17:48:
Thanks, Christian. Right, that wasn't a working example. The server I'm querying is behind a firewall, so I can't really reproduce the issue unfortunately.
The BaseX documentation states that the HTTP client does support Digest authentication. However, I did some poking around using Wireshark and limited knowledge of network traffic. When I run the request in curl, the expected protocol is played out, and I see the evidence in Wireshark:
(1) The server responds with "401 Unauthorized" and provides the nonce value. (2) The client (curl) then does its business (supplies the username and password, resends the request, etc.)
But when I submit the request via BaseX, it never gets past step (1). It never seems to send the username and password at all. Is Digest authentication still supported?
Best, Tim
-- Tim A. Thompson Metadata Librarian Yale University Library
On Wed, Feb 2, 2022 at 10:13 AM Christian Grün < christian.gruen@gmail.com> wrote:
Hi Tim,
Difficult to tell; both the curl and the XQuery variants give me 404. Do you think you could provide us with an example that works out of the box?
Best, Christian
On Wed, Feb 2, 2022 at 12:54 AM Tim Thompson <timathom@gmail.com> wrote:
Hello,
I'm trying to post a SPARQL query to an endpoint using Digest
authentication with the HTTP client.
The query works fine using curl:
curl --digest --user user:pass -X POST -d@'test.rq' \ -H "Content-type: application/sparql-query" \ 'http://example.org/sparql'
But the equivalent request in BaseX fails with 401 Unauthorized:
let $endpoint := "http://example.org/sparql" let $user := "user" let $pass := "pass" let $type := "application/sparql-query"
let $response := ( http:send-request( <http:request method="POST" href="{$endpoint}" username="{$user}" password="{$pass}" auth-method="Digest" send-authorization="true"> <http:header name="Content-Type" value="{$type}; charset=utf-8"/> <http:body media-type="{$type}">{ ``[ select * where {?s ?p ?o} limit 1 ]`` }</http:body> </http:request> ) ) return $response
Any ideas about what might be causing the BaseX HTTP client to be
denied here?
Thanks in advance, Tim
-- Tim A. Thompson Metadata Librarian Yale University Library
I find httpbin.org a useful resource to test this kind of thing [1] and [2] /Andy [1] http://httpbin.org/#/Auth/get_basic_auth__user___passwd_ [2] http://httpbin.org/basic-auth/user1/mypass On Wed, 2 Feb 2022 at 16:58, Christian Grün <christian.gruen@gmail.com> wrote:
Good to hear; so at least Digest is detected at this stage ;)
Tim Thompson <timathom@gmail.com> schrieb am Mi., 2. Feb. 2022, 17:54:
With "digest" I get: "[experr:HC0004] Invalid authentication method: digest."
-- Tim A. Thompson Metadata Librarian Yale University Library
On Wed, Feb 2, 2022 at 11:52 AM Christian Grün <christian.gruen@gmail.com> wrote:
Yes, it should still be supported. I assume it doesn't make a difference if you use "Digest" or "digest"?
Tim Thompson <timathom@gmail.com> schrieb am Mi., 2. Feb. 2022, 17:48:
Thanks, Christian. Right, that wasn't a working example. The server I'm querying is behind a firewall, so I can't really reproduce the issue unfortunately.
The BaseX documentation states that the HTTP client does support Digest authentication. However, I did some poking around using Wireshark and limited knowledge of network traffic. When I run the request in curl, the expected protocol is played out, and I see the evidence in Wireshark:
(1) The server responds with "401 Unauthorized" and provides the nonce value. (2) The client (curl) then does its business (supplies the username and password, resends the request, etc.)
But when I submit the request via BaseX, it never gets past step (1). It never seems to send the username and password at all. Is Digest authentication still supported?
Best, Tim
-- Tim A. Thompson Metadata Librarian Yale University Library
On Wed, Feb 2, 2022 at 10:13 AM Christian Grün < christian.gruen@gmail.com> wrote:
Hi Tim,
Difficult to tell; both the curl and the XQuery variants give me 404. Do you think you could provide us with an example that works out of the box?
Best, Christian
On Wed, Feb 2, 2022 at 12:54 AM Tim Thompson <timathom@gmail.com> wrote:
Hello,
I'm trying to post a SPARQL query to an endpoint using Digest
authentication with the HTTP client.
The query works fine using curl:
curl --digest --user user:pass -X POST -d@'test.rq' \ -H "Content-type: application/sparql-query" \ 'http://example.org/sparql'
But the equivalent request in BaseX fails with 401 Unauthorized:
let $endpoint := "http://example.org/sparql" let $user := "user" let $pass := "pass" let $type := "application/sparql-query"
let $response := ( http:send-request( <http:request method="POST" href="{$endpoint}" username="{$user}" password="{$pass}" auth-method="Digest" send-authorization="true"> <http:header name="Content-Type" value="{$type}; charset=utf-8"/> <http:body media-type="{$type}">{ ``[ select * where {?s ?p ?o} limit 1 ]`` }</http:body> </http:request> ) ) return $response
Any ideas about what might be causing the BaseX HTTP client to be
denied here?
Thanks in advance, Tim
-- Tim A. Thompson Metadata Librarian Yale University Library
Thanks, Andy! That's very handy. So, when I test against httpbin, the same thing happens. curl request: curl --digest --user user:pw -X GET ' http://httpbin.org/digest-auth/auth/user/pw' response: { "authenticated": true, "user": "user" } BaseX request: let $endpoint := "http://httpbin.org/digest-auth/auth/user/pw" let $response := ( http:send-request( <http:request method="GET" href="{$endpoint}" username="user" password="pw" auth-method="Digest" send-authorization="true"> <http:header name="Content-Type" value="{$type}; charset=utf-8"/> </http:request> ) ) return $response response: <http:response xmlns:http="http://expath.org/ns/http-client" status="401" message="UNAUTHORIZED"> <http:header name="Server" value="gunicorn/19.9.0"/> <http:header name="Access-Control-Allow-Origin" value="*"/> <http:header name="Access-Control-Allow-Credentials" value="true"/> <http:header name="WWW-Authenticate" value="Digest realm=& quot;me@kennethreitz.com", nonce="36d59385e96e1689595f4f14fa2921ca", qop="auth", opaque="a21df3287b5cb9cbf8b6a68b2c15fa1d", algorithm=MD5, stale=FALSE"/> <http:header name="Connection" value="keep-alive"/> <http:header name="Set-Cookie" value="fake=fake_value; Path=/"/> <http:header name="Set-Cookie" value="stale_after=never; Path=/"/> <http:header name="Content-Length" value="0"/> <http:header name="Date" value="Wed, 02 Feb 2022 17:14:13 GMT"/> <http:header name="Content-Type" value="text/html; charset=utf-8"/> </http:response> -- Tim A. Thompson Metadata Librarian Yale University Library On Wed, Feb 2, 2022 at 12:06 PM Andy Bunce <bunce.andy@gmail.com> wrote:
I find httpbin.org a useful resource to test this kind of thing [1] and [2] /Andy
[1] http://httpbin.org/#/Auth/get_basic_auth__user___passwd_ [2] http://httpbin.org/basic-auth/user1/mypass
On Wed, 2 Feb 2022 at 16:58, Christian Grün <christian.gruen@gmail.com> wrote:
Good to hear; so at least Digest is detected at this stage ;)
Tim Thompson <timathom@gmail.com> schrieb am Mi., 2. Feb. 2022, 17:54:
With "digest" I get: "[experr:HC0004] Invalid authentication method: digest."
-- Tim A. Thompson Metadata Librarian Yale University Library
On Wed, Feb 2, 2022 at 11:52 AM Christian Grün < christian.gruen@gmail.com> wrote:
Yes, it should still be supported. I assume it doesn't make a difference if you use "Digest" or "digest"?
Tim Thompson <timathom@gmail.com> schrieb am Mi., 2. Feb. 2022, 17:48:
Thanks, Christian. Right, that wasn't a working example. The server I'm querying is behind a firewall, so I can't really reproduce the issue unfortunately.
The BaseX documentation states that the HTTP client does support Digest authentication. However, I did some poking around using Wireshark and limited knowledge of network traffic. When I run the request in curl, the expected protocol is played out, and I see the evidence in Wireshark:
(1) The server responds with "401 Unauthorized" and provides the nonce value. (2) The client (curl) then does its business (supplies the username and password, resends the request, etc.)
But when I submit the request via BaseX, it never gets past step (1). It never seems to send the username and password at all. Is Digest authentication still supported?
Best, Tim
-- Tim A. Thompson Metadata Librarian Yale University Library
On Wed, Feb 2, 2022 at 10:13 AM Christian Grün < christian.gruen@gmail.com> wrote:
Hi Tim,
Difficult to tell; both the curl and the XQuery variants give me 404. Do you think you could provide us with an example that works out of the box?
Best, Christian
On Wed, Feb 2, 2022 at 12:54 AM Tim Thompson <timathom@gmail.com> wrote: > > Hello, > > I'm trying to post a SPARQL query to an endpoint using Digest authentication with the HTTP client. > > The query works fine using curl: > > curl --digest --user user:pass -X POST -d@'test.rq' \ > -H "Content-type: application/sparql-query" \ > 'http://example.org/sparql' > > But the equivalent request in BaseX fails with 401 Unauthorized: > > let $endpoint := "http://example.org/sparql" > let $user := "user" > let $pass := "pass" > let $type := "application/sparql-query" > > let $response := ( > http:send-request( > <http:request > method="POST" > href="{$endpoint}" > username="{$user}" > password="{$pass}" > auth-method="Digest" > send-authorization="true"> > <http:header > name="Content-Type" > value="{$type}; charset=utf-8"/> > <http:body > media-type="{$type}">{ > ``[ > select * where {?s ?p ?o} limit 1 > ]`` > }</http:body> > </http:request> > ) > ) > return > $response > > Any ideas about what might be causing the BaseX HTTP client to be denied here? > > Thanks in advance, > Tim > > > -- > Tim A. Thompson > Metadata Librarian > Yale University Library >
Thanks! I’ll have a look at that soon. Tim Thompson <timathom@gmail.com> schrieb am Mi., 2. Feb. 2022, 18:19:
Thanks, Andy! That's very handy. So, when I test against httpbin, the same thing happens.
curl request: curl --digest --user user:pw -X GET ' http://httpbin.org/digest-auth/auth/user/pw'
response: { "authenticated": true, "user": "user" }
BaseX request: let $endpoint := "http://httpbin.org/digest-auth/auth/user/pw"
let $response := ( http:send-request( <http:request method="GET" href="{$endpoint}" username="user" password="pw" auth-method="Digest" send-authorization="true"> <http:header name="Content-Type" value="{$type}; charset=utf-8"/> </http:request> ) ) return $response
response: <http:response xmlns:http="http://expath.org/ns/http-client" status="401" message="UNAUTHORIZED"> <http:header name="Server" value="gunicorn/19.9.0"/> <http:header name="Access-Control-Allow-Origin" value="*"/> <http:header name="Access-Control-Allow-Credentials" value="true"/> <http:header name="WWW-Authenticate" value="Digest realm=& quot;me@kennethreitz.com", nonce="36d59385e96e1689595f4f14fa2921ca", qop="auth", opaque="a21df3287b5cb9cbf8b6a68b2c15fa1d", algorithm=MD5, stale=FALSE"/> <http:header name="Connection" value="keep-alive"/> <http:header name="Set-Cookie" value="fake=fake_value; Path=/"/> <http:header name="Set-Cookie" value="stale_after=never; Path=/"/> <http:header name="Content-Length" value="0"/> <http:header name="Date" value="Wed, 02 Feb 2022 17:14:13 GMT"/> <http:header name="Content-Type" value="text/html; charset=utf-8"/> </http:response>
-- Tim A. Thompson Metadata Librarian Yale University Library
On Wed, Feb 2, 2022 at 12:06 PM Andy Bunce <bunce.andy@gmail.com> wrote:
I find httpbin.org a useful resource to test this kind of thing [1] and [2] /Andy
[1] http://httpbin.org/#/Auth/get_basic_auth__user___passwd_ [2] http://httpbin.org/basic-auth/user1/mypass
On Wed, 2 Feb 2022 at 16:58, Christian Grün <christian.gruen@gmail.com> wrote:
Good to hear; so at least Digest is detected at this stage ;)
Tim Thompson <timathom@gmail.com> schrieb am Mi., 2. Feb. 2022, 17:54:
With "digest" I get: "[experr:HC0004] Invalid authentication method: digest."
-- Tim A. Thompson Metadata Librarian Yale University Library
On Wed, Feb 2, 2022 at 11:52 AM Christian Grün < christian.gruen@gmail.com> wrote:
Yes, it should still be supported. I assume it doesn't make a difference if you use "Digest" or "digest"?
Tim Thompson <timathom@gmail.com> schrieb am Mi., 2. Feb. 2022, 17:48:
Thanks, Christian. Right, that wasn't a working example. The server I'm querying is behind a firewall, so I can't really reproduce the issue unfortunately.
The BaseX documentation states that the HTTP client does support Digest authentication. However, I did some poking around using Wireshark and limited knowledge of network traffic. When I run the request in curl, the expected protocol is played out, and I see the evidence in Wireshark:
(1) The server responds with "401 Unauthorized" and provides the nonce value. (2) The client (curl) then does its business (supplies the username and password, resends the request, etc.)
But when I submit the request via BaseX, it never gets past step (1). It never seems to send the username and password at all. Is Digest authentication still supported?
Best, Tim
-- Tim A. Thompson Metadata Librarian Yale University Library
On Wed, Feb 2, 2022 at 10:13 AM Christian Grün < christian.gruen@gmail.com> wrote:
> Hi Tim, > > Difficult to tell; both the curl and the XQuery variants give me 404. > Do you think you could provide us with an example that works out of > the box? > > Best, > Christian > > > > On Wed, Feb 2, 2022 at 12:54 AM Tim Thompson <timathom@gmail.com> > wrote: > > > > Hello, > > > > I'm trying to post a SPARQL query to an endpoint using Digest > authentication with the HTTP client. > > > > The query works fine using curl: > > > > curl --digest --user user:pass -X POST -d@'test.rq' \ > > -H "Content-type: application/sparql-query" \ > > 'http://example.org/sparql' > > > > But the equivalent request in BaseX fails with 401 Unauthorized: > > > > let $endpoint := "http://example.org/sparql" > > let $user := "user" > > let $pass := "pass" > > let $type := "application/sparql-query" > > > > let $response := ( > > http:send-request( > > <http:request > > method="POST" > > href="{$endpoint}" > > username="{$user}" > > password="{$pass}" > > auth-method="Digest" > > send-authorization="true"> > > <http:header > > name="Content-Type" > > value="{$type}; charset=utf-8"/> > > <http:body > > media-type="{$type}">{ > > ``[ > > select * where {?s ?p ?o} limit 1 > > ]`` > > }</http:body> > > </http:request> > > ) > > ) > > return > > $response > > > > Any ideas about what might be causing the BaseX HTTP client to be > denied here? > > > > Thanks in advance, > > Tim > > > > > > -- > > Tim A. Thompson > > Metadata Librarian > > Yale University Library > > >
Hi Tim, Your request should be successful if you omit the send-authorization attribute: “If send-authorization is true (default value is false) and the authentication method supports generating the header Authorization without challenge, the request contains this header. The default value is to send a non-authenticated request, and if the response is an authentication challenge, then only send the credentials in a second message.” [1] If you use Basic authentication, you can reduce the communication to a single request by sending the authorization with the first request. With Digest authentication, there will always be two requests. With the (still to be finalized) HTTP Client module 2.0, send-authorization will disappear, and it will be up to the implementation to decide how many requests will be sent [2]. Hope this helps, Christian [1] http://expath.org/spec/http-client#d2e430 [2] https://expath.github.io/expath-cg/specs/http-client-2/ On Wed, Feb 2, 2022 at 6:19 PM Tim Thompson <timathom@gmail.com> wrote:
Thanks, Andy! That's very handy. So, when I test against httpbin, the same thing happens.
curl request: curl --digest --user user:pw -X GET 'http://httpbin.org/digest-auth/auth/user/pw'
response: { "authenticated": true, "user": "user" }
BaseX request: let $endpoint := "http://httpbin.org/digest-auth/auth/user/pw"
let $response := ( http:send-request( <http:request method="GET" href="{$endpoint}" username="user" password="pw" auth-method="Digest" send-authorization="true"> <http:header name="Content-Type" value="{$type}; charset=utf-8"/> </http:request> ) ) return $response
response: <http:response xmlns:http="http://expath.org/ns/http-client" status="401" message="UNAUTHORIZED"> <http:header name="Server" value="gunicorn/19.9.0"/> <http:header name="Access-Control-Allow-Origin" value="*"/> <http:header name="Access-Control-Allow-Credentials" value="true"/> <http:header name="WWW-Authenticate" value="Digest realm="me@kennethreitz.com", nonce="36d59385e96e1689595f4f14fa2921ca", qop="auth", opaque="a21df3287b5cb9cbf8b6a68b2c15fa1d", algorithm=MD5, stale=FALSE"/> <http:header name="Connection" value="keep-alive"/> <http:header name="Set-Cookie" value="fake=fake_value; Path=/"/> <http:header name="Set-Cookie" value="stale_after=never; Path=/"/> <http:header name="Content-Length" value="0"/> <http:header name="Date" value="Wed, 02 Feb 2022 17:14:13 GMT"/> <http:header name="Content-Type" value="text/html; charset=utf-8"/> </http:response>
-- Tim A. Thompson Metadata Librarian Yale University Library
On Wed, Feb 2, 2022 at 12:06 PM Andy Bunce <bunce.andy@gmail.com> wrote:
I find httpbin.org a useful resource to test this kind of thing [1] and [2] /Andy
[1] http://httpbin.org/#/Auth/get_basic_auth__user___passwd_ [2] http://httpbin.org/basic-auth/user1/mypass
On Wed, 2 Feb 2022 at 16:58, Christian Grün <christian.gruen@gmail.com> wrote:
Good to hear; so at least Digest is detected at this stage ;)
Tim Thompson <timathom@gmail.com> schrieb am Mi., 2. Feb. 2022, 17:54:
With "digest" I get: "[experr:HC0004] Invalid authentication method: digest."
-- Tim A. Thompson Metadata Librarian Yale University Library
On Wed, Feb 2, 2022 at 11:52 AM Christian Grün <christian.gruen@gmail.com> wrote:
Yes, it should still be supported. I assume it doesn't make a difference if you use "Digest" or "digest"?
Tim Thompson <timathom@gmail.com> schrieb am Mi., 2. Feb. 2022, 17:48:
Thanks, Christian. Right, that wasn't a working example. The server I'm querying is behind a firewall, so I can't really reproduce the issue unfortunately.
The BaseX documentation states that the HTTP client does support Digest authentication. However, I did some poking around using Wireshark and limited knowledge of network traffic. When I run the request in curl, the expected protocol is played out, and I see the evidence in Wireshark:
(1) The server responds with "401 Unauthorized" and provides the nonce value. (2) The client (curl) then does its business (supplies the username and password, resends the request, etc.)
But when I submit the request via BaseX, it never gets past step (1). It never seems to send the username and password at all. Is Digest authentication still supported?
Best, Tim
-- Tim A. Thompson Metadata Librarian Yale University Library
On Wed, Feb 2, 2022 at 10:13 AM Christian Grün <christian.gruen@gmail.com> wrote: > > Hi Tim, > > Difficult to tell; both the curl and the XQuery variants give me 404. > Do you think you could provide us with an example that works out of > the box? > > Best, > Christian > > > > On Wed, Feb 2, 2022 at 12:54 AM Tim Thompson <timathom@gmail.com> wrote: > > > > Hello, > > > > I'm trying to post a SPARQL query to an endpoint using Digest authentication with the HTTP client. > > > > The query works fine using curl: > > > > curl --digest --user user:pass -X POST -d@'test.rq' \ > > -H "Content-type: application/sparql-query" \ > > 'http://example.org/sparql' > > > > But the equivalent request in BaseX fails with 401 Unauthorized: > > > > let $endpoint := "http://example.org/sparql" > > let $user := "user" > > let $pass := "pass" > > let $type := "application/sparql-query" > > > > let $response := ( > > http:send-request( > > <http:request > > method="POST" > > href="{$endpoint}" > > username="{$user}" > > password="{$pass}" > > auth-method="Digest" > > send-authorization="true"> > > <http:header > > name="Content-Type" > > value="{$type}; charset=utf-8"/> > > <http:body > > media-type="{$type}">{ > > ``[ > > select * where {?s ?p ?o} limit 1 > > ]`` > > }</http:body> > > </http:request> > > ) > > ) > > return > > $response > > > > Any ideas about what might be causing the BaseX HTTP client to be denied here? > > > > Thanks in advance, > > Tim > > > > > > -- > > Tim A. Thompson > > Metadata Librarian > > Yale University Library > >
Ah! Thanks for clarifying that. So, now it is working for GET requests in BaseX, but still not for POST. So, this works: http:send-request( <http:request method="GET" href="{$endpoint}?query={encode-for-uri('select * where {?s ?p ?o} limit 1')}" username="{$user}" password="{$pass}" auth-method="Digest"> </http:request> ) But when I try to POST a request body with Digest auth, it is still failing (httpbin doesn't seem to support testing Digest with POST, unfortunately). The request hangs for a while, then fails with 401 Unauthorized. Wireshark reveals that the first network response from the server is actually a 502 Bad Gateway. Attached are two screenshots from Wireshark: first, the request/response sequence with curl, then with BaseX. Here's what Wireshark shows for the first request via curl: Hypertext Transfer Protocol POST /v1/graphs/sparql HTTP/1.1\r\n [Expert Info (Chat/Sequence): POST /v1/graphs/sparql HTTP/1.1\r\n] [POST /v1/graphs/sparql HTTP/1.1\r\n] [Severity level: Chat] [Group: Sequence] Request Method: POST Request URI: /v1/graphs/sparql Request Version: HTTP/1.1 Host: example.org\r\n User-Agent: curl/7.71.1\r\n Accept: */*\r\n Content-type: application/sparql-query\r\n Content-Length: 0\r\n [Content length: 0] \r\n [Full request URI: example.org/v1/graphs/sparql] Then via BaseX: Hypertext Transfer Protocol POST /v1/graphs/sparql HTTP/1.1\r\n [Expert Info (Chat/Sequence): POST /v1/graphs/sparql HTTP/1.1\r\n] [POST /v1/graphs/sparql HTTP/1.1\r\n] [Severity level: Chat] [Group: Sequence] Request Method: POST Request URI: /v1/graphs/sparql Request Version: HTTP/1.1 Content-type: application/sparql-query\r\n Authorization: Digest\r\n User-Agent: Java/1.8.0_121\r\n Host: example.org\r\n Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2\r\n Connection: keep-alive\r\n \r\n [Full request URI: example.org/v1/graphs/sparql] I notice that curl doesn't include the Authorization header on the first request, but I am out of my depth here. Best, Tim -- Tim A. Thompson Metadata Librarian Yale University Library On Wed, Feb 2, 2022 at 12:40 PM Christian Grün <christian.gruen@gmail.com> wrote:
Hi Tim,
Your request should be successful if you omit the send-authorization attribute:
“If send-authorization is true (default value is false) and the authentication method supports generating the header Authorization without challenge, the request contains this header. The default value is to send a non-authenticated request, and if the response is an authentication challenge, then only send the credentials in a second message.” [1]
If you use Basic authentication, you can reduce the communication to a single request by sending the authorization with the first request. With Digest authentication, there will always be two requests.
With the (still to be finalized) HTTP Client module 2.0, send-authorization will disappear, and it will be up to the implementation to decide how many requests will be sent [2].
Hope this helps, Christian
[1] http://expath.org/spec/http-client#d2e430 [2] https://expath.github.io/expath-cg/specs/http-client-2/
On Wed, Feb 2, 2022 at 6:19 PM Tim Thompson <timathom@gmail.com> wrote:
Thanks, Andy! That's very handy. So, when I test against httpbin, the
same thing happens.
curl request: curl --digest --user user:pw -X GET '
http://httpbin.org/digest-auth/auth/user/pw'
response: { "authenticated": true, "user": "user" }
BaseX request: let $endpoint := "http://httpbin.org/digest-auth/auth/user/pw"
let $response := ( http:send-request( <http:request method="GET" href="{$endpoint}" username="user" password="pw" auth-method="Digest" send-authorization="true"> <http:header name="Content-Type" value="{$type}; charset=utf-8"/> </http:request> ) ) return $response
response: <http:response xmlns:http="http://expath.org/ns/http-client"
<http:header name="Server" value="gunicorn/19.9.0"/> <http:header name="Access-Control-Allow-Origin" value="*"/> <http:header name="Access-Control-Allow-Credentials" value="true"/> <http:header name="WWW-Authenticate" value="Digest realm=& quot;me@kennethreitz.com", nonce="36d59385e96e1689595f4f14fa2921ca", qop="auth", opaque="a21df3287b5cb9cbf8b6a68b2c15fa1d", algorithm=MD5, stale=FALSE"/> <http:header name="Connection" value="keep-alive"/> <http:header name="Set-Cookie" value="fake=fake_value; Path=/"/> <http:header name="Set-Cookie" value="stale_after=never; Path=/"/> <http:header name="Content-Length" value="0"/> <http:header name="Date" value="Wed, 02 Feb 2022 17:14:13 GMT"/> <http:header name="Content-Type" value="text/html; charset=utf-8"/> </http:response>
-- Tim A. Thompson Metadata Librarian Yale University Library
On Wed, Feb 2, 2022 at 12:06 PM Andy Bunce <bunce.andy@gmail.com> wrote:
I find httpbin.org a useful resource to test this kind of thing [1]
and [2]
/Andy
[1] http://httpbin.org/#/Auth/get_basic_auth__user___passwd_ [2] http://httpbin.org/basic-auth/user1/mypass
On Wed, 2 Feb 2022 at 16:58, Christian Grün <christian.gruen@gmail.com> wrote:
Good to hear; so at least Digest is detected at this stage ;)
Tim Thompson <timathom@gmail.com> schrieb am Mi., 2. Feb. 2022, 17:54:
With "digest" I get: "[experr:HC0004] Invalid authentication method:
digest."
-- Tim A. Thompson Metadata Librarian Yale University Library
On Wed, Feb 2, 2022 at 11:52 AM Christian Grün <
christian.gruen@gmail.com> wrote:
Yes, it should still be supported. I assume it doesn't make a
difference if you use "Digest" or "digest"?
Tim Thompson <timathom@gmail.com> schrieb am Mi., 2. Feb. 2022,
17:48:
> > Thanks, Christian. Right, that wasn't a working example. The server I'm querying is behind a firewall, so I can't really reproduce the issue unfortunately. > > The BaseX documentation states that the HTTP client does support Digest authentication. However, I did some poking around using Wireshark and limited knowledge of network traffic. When I run the request in curl, the expected protocol is played out, and I see the evidence in Wireshark: > > (1) The server responds with "401 Unauthorized" and provides the nonce value. > (2) The client (curl) then does its business (supplies the username and password, resends the request, etc.) > > But when I submit the request via BaseX, it never gets past step (1). It never seems to send the username and password at all. Is Digest authentication still supported? > > Best, > Tim > > > -- > Tim A. Thompson > Metadata Librarian > Yale University Library > > > On Wed, Feb 2, 2022 at 10:13 AM Christian Grün < christian.gruen@gmail.com> wrote: >> >> Hi Tim, >> >> Difficult to tell; both the curl and the XQuery variants give me
status="401" message="UNAUTHORIZED"> 404.
>> Do you think you could provide us with an example that works out of >> the box? >> >> Best, >> Christian >> >> >> >> On Wed, Feb 2, 2022 at 12:54 AM Tim Thompson <timathom@gmail.com> wrote: >> > >> > Hello, >> > >> > I'm trying to post a SPARQL query to an endpoint using Digest authentication with the HTTP client. >> > >> > The query works fine using curl: >> > >> > curl --digest --user user:pass -X POST -d@'test.rq' \ >> > -H "Content-type: application/sparql-query" \ >> > 'http://example.org/sparql' >> > >> > But the equivalent request in BaseX fails with 401 Unauthorized: >> > >> > let $endpoint := "http://example.org/sparql" >> > let $user := "user" >> > let $pass := "pass" >> > let $type := "application/sparql-query" >> > >> > let $response := ( >> > http:send-request( >> > <http:request >> > method="POST" >> > href="{$endpoint}" >> > username="{$user}" >> > password="{$pass}" >> > auth-method="Digest" >> > send-authorization="true"> >> > <http:header >> > name="Content-Type" >> > value="{$type}; charset=utf-8"/> >> > <http:body >> > media-type="{$type}">{ >> > ``[ >> > select * where {?s ?p ?o} limit 1 >> > ]`` >> > }</http:body> >> > </http:request> >> > ) >> > ) >> > return >> > $response >> > >> > Any ideas about what might be causing the BaseX HTTP client to be denied here? >> > >> > Thanks in advance, >> > Tim >> > >> > >> > -- >> > Tim A. Thompson >> > Metadata Librarian >> > Yale University Library >> >
I notice that curl doesn't include the Authorization header on the first request, but I am out of my depth here.
That could be an interesting hint. In the latest snapshot [1], I have removed the Authorization header from the first request. Can you check if it makes a difference? If not, could you share the headers of the second requests with us? [1] https://files.basex.org/releases/latest/
Thanks for that. Still no luck, I'm afraid. Attached is the full HTTP info for the requests from BaseX (hostname info replaced with "example.org"). I should be able to stick with GET requests for now, but would be nice to have access to POST as well. All best, Tim -- Tim A. Thompson Metadata Librarian Yale University Library On Wed, Feb 2, 2022 at 2:20 PM Christian Grün <christian.gruen@gmail.com> wrote:
I notice that curl doesn't include the Authorization header on the first request, but I am out of my depth here.
That could be an interesting hint. In the latest snapshot [1], I have removed the Authorization header from the first request. Can you check if it makes a difference?
If not, could you share the headers of the second requests with us?
[Time since request: 30.071527000 seconds] possible cause (just a wild guess): the AWS load balancer encountered the classic BaseX timeout of 30 seconds. On 02.02.2022 21:22, Tim Thompson wrote:
Thanks for that. Still no luck, I'm afraid. Attached is the full HTTP info for the requests from BaseX (hostname info replaced with "example.org <http://example.org>").
I should be able to stick with GET requests for now, but would be nice to have access to POST as well.
All best, Tim
Dommage. Do you know more about the server you are communicating with? Maybe it rejects http (POST?) requests sent by Java? You could try to supply the curl-specific request headers (such as User-Agent, and others) with http:send-request. Could you additionally forward the full Wireshark logs for curl? I remember that Adam (Retter) added the http-version option to the new version of the HTTP Client Module (it’s not available in 1.0 yet). In the given case, I’d assume that http-version=1.1 should be okay, because curl uses it as well. Tim Thompson <timathom@gmail.com> schrieb am Mi., 2. Feb. 2022, 21:22:
Thanks for that. Still no luck, I'm afraid. Attached is the full HTTP info for the requests from BaseX (hostname info replaced with "example.org").
I should be able to stick with GET requests for now, but would be nice to have access to POST as well.
All best, Tim
-- Tim A. Thompson Metadata Librarian Yale University Library
On Wed, Feb 2, 2022 at 2:20 PM Christian Grün <christian.gruen@gmail.com> wrote:
I notice that curl doesn't include the Authorization header on the first request, but I am out of my depth here.
That could be an interesting hint. In the latest snapshot [1], I have removed the Authorization header from the first request. Can you check if it makes a difference?
If not, could you share the headers of the second requests with us?
The server is MarkLogic, s maybe they are biased against BaseX ;) I'm not sure about the configuration, but replacing the User-Agent header with "curl/7.71.1" didn't help. Attached is the HTTP info for the curl requests. I also tried it in the Postman GUI, which worked as well. I noticed that in both curl and Postman the first request says, "[HTTP request 1/2]," whereas BaseX says, "[HTTP request 1/1]" Tim On Wed, Feb 2, 2022 at 3:38 PM Christian Grün <christian.gruen@gmail.com> wrote:
Dommage.
Do you know more about the server you are communicating with? Maybe it rejects http (POST?) requests sent by Java?
You could try to supply the curl-specific request headers (such as User-Agent, and others) with http:send-request.
Could you additionally forward the full Wireshark logs for curl?
I remember that Adam (Retter) added the http-version option to the new version of the HTTP Client Module (it’s not available in 1.0 yet). In the given case, I’d assume that http-version=1.1 should be okay, because curl uses it as well.
Tim Thompson <timathom@gmail.com> schrieb am Mi., 2. Feb. 2022, 21:22:
Thanks for that. Still no luck, I'm afraid. Attached is the full HTTP info for the requests from BaseX (hostname info replaced with " example.org").
I should be able to stick with GET requests for now, but would be nice to have access to POST as well.
All best, Tim
-- Tim A. Thompson Metadata Librarian Yale University Library
On Wed, Feb 2, 2022 at 2:20 PM Christian Grün <christian.gruen@gmail.com> wrote:
I notice that curl doesn't include the Authorization header on the first request, but I am out of my depth here.
That could be an interesting hint. In the latest snapshot [1], I have removed the Authorization header from the first request. Can you check if it makes a difference?
If not, could you share the headers of the second requests with us?
Thanks.
The server is MarkLogic, s maybe they are biased against BaseX ;) I'm not sure about the configuration, but replacing the User-Agent header with "curl/7.71.1" didn't help.
Fraudulent competition ;·) I’m particularly confused by the response code: 502 usually indicates a server-side configuration error, which should not be triggerable from a client. I noticed it would be more correct if BaseX had not sent a second request if the response code is unequal to 401. In our case, it might have proven to be helpful, as we can see that a) the response to the second request is 401, and b) it contains the Digest WWW-Authenticate header. In addition, the second request contains a Content-Length header, which is missing in the first request. Presumably, due to the missing Content-Length:0 header, the server tries to retrieve a body, but gives up after the timeout and returns 502. With the latest snapshot [1], I’m now creating an explicit 0-bytes-body for all methods except for GET and TRACE. Does this help?
I noticed that in both curl and Postman the first request says, "[HTTP request 1/2]," whereas BaseX says, "[HTTP request 1/1]"
My hope is that numerous of these irregularities will disappear once we’ll have switched to the new JDK 11 HTTP Client [2] with BaseX 10. [1] https://files.basex.org/releases/latest/ [2] https://openjdk.java.net/groups/net/httpclient/intro.html
That solved it! Thank you, Christian, for the outstanding support, as always. I always panic a bit when I have to think about using something other than BaseX for my programming needs ;) Tim On Wed, Feb 2, 2022 at 6:08 PM Christian Grün <christian.gruen@gmail.com> wrote:
Thanks.
The server is MarkLogic, s maybe they are biased against BaseX ;) I'm not sure about the configuration, but replacing the User-Agent header with "curl/7.71.1" didn't help.
Fraudulent competition ;·)
I’m particularly confused by the response code: 502 usually indicates a server-side configuration error, which should not be triggerable from a client.
I noticed it would be more correct if BaseX had not sent a second request if the response code is unequal to 401. In our case, it might have proven to be helpful, as we can see that
a) the response to the second request is 401, and b) it contains the Digest WWW-Authenticate header.
In addition, the second request contains a Content-Length header, which is missing in the first request. Presumably, due to the missing Content-Length:0 header, the server tries to retrieve a body, but gives up after the timeout and returns 502.
With the latest snapshot [1], I’m now creating an explicit 0-bytes-body for all methods except for GET and TRACE. Does this help?
I noticed that in both curl and Postman the first request says, "[HTTP request 1/2]," whereas BaseX says, "[HTTP request 1/1]"
My hope is that numerous of these irregularities will disappear once we’ll have switched to the new JDK 11 HTTP Client [2] with BaseX 10.
[1] https://files.basex.org/releases/latest/ [2] https://openjdk.java.net/groups/net/httpclient/intro.html
Hi Tim, glad to hear! And thanks for your excellent feedback, which helped a lot. I’ve incorporated the insights into the code, a finalized version is available [1,2]. BaseX 9.6.5 will be released this month. – Best, Christian [1] https://files.basex.org/releases/latest/ [2] https://github.com/BaseXdb/basex/commit/b1ae50b13ba5912ad0bb7c149409f882cabe... On Thu, Feb 3, 2022 at 3:04 AM Tim Thompson <timathom@gmail.com> wrote:
That solved it! Thank you, Christian, for the outstanding support, as always. I always panic a bit when I have to think about using something other than BaseX for my programming needs ;)
Tim
On Wed, Feb 2, 2022 at 6:08 PM Christian Grün <christian.gruen@gmail.com> wrote:
Thanks.
The server is MarkLogic, s maybe they are biased against BaseX ;) I'm not sure about the configuration, but replacing the User-Agent header with "curl/7.71.1" didn't help.
Fraudulent competition ;·)
I’m particularly confused by the response code: 502 usually indicates a server-side configuration error, which should not be triggerable from a client.
I noticed it would be more correct if BaseX had not sent a second request if the response code is unequal to 401. In our case, it might have proven to be helpful, as we can see that
a) the response to the second request is 401, and b) it contains the Digest WWW-Authenticate header.
In addition, the second request contains a Content-Length header, which is missing in the first request. Presumably, due to the missing Content-Length:0 header, the server tries to retrieve a body, but gives up after the timeout and returns 502.
With the latest snapshot [1], I’m now creating an explicit 0-bytes-body for all methods except for GET and TRACE. Does this help?
I noticed that in both curl and Postman the first request says, "[HTTP request 1/2]," whereas BaseX says, "[HTTP request 1/1]"
My hope is that numerous of these irregularities will disappear once we’ll have switched to the new JDK 11 HTTP Client [2] with BaseX 10.
[1] https://files.basex.org/releases/latest/ [2] https://openjdk.java.net/groups/net/httpclient/intro.html
In case it helps: I had a similar problem in eXist some years ago. My fix was to put the attribute `http-version="1.0"` on the request element. The HTTP client defaults to HTTP v1.1, but for whatever reason, only HTTP 1.0 worked for me. ~Ash ________________________________ From: BaseX-Talk <basex-talk-bounces@mailman.uni-konstanz.de> on behalf of Tim Thompson <timathom@gmail.com> Sent: Wednesday, February 2, 2022 1:59 PM To: Christian Grün <christian.gruen@gmail.com>; BaseX <basex-talk@mailman.uni-konstanz.de> Subject: Re: [basex-talk] Issue with HTTP client and authentication Ah! Thanks for clarifying that. So, now it is working for GET requests in BaseX, but still not for POST. So, this works: http:send-request( <http:request method="GET" href="{$endpoint}?query={encode-for-uri('select * where {?s ?p ?o} limit 1')}" username="{$user}" password="{$pass}" auth-method="Digest"> </http:request> ) But when I try to POST a request body with Digest auth, it is still failing (httpbin doesn't seem to support testing Digest with POST, unfortunately). The request hangs for a while, then fails with 401 Unauthorized. Wireshark reveals that the first network response from the server is actually a 502 Bad Gateway. Attached are two screenshots from Wireshark: first, the request/response sequence with curl, then with BaseX. Here's what Wireshark shows for the first request via curl: Hypertext Transfer Protocol POST /v1/graphs/sparql HTTP/1.1\r\n [Expert Info (Chat/Sequence): POST /v1/graphs/sparql HTTP/1.1\r\n] [POST /v1/graphs/sparql HTTP/1.1\r\n] [Severity level: Chat] [Group: Sequence] Request Method: POST Request URI: /v1/graphs/sparql Request Version: HTTP/1.1 Host: example.org<https://nam12.safelinks.protection.outlook.com/?url=http%3A%2F%2Fexample.org%2F&data=04%7C01%7Cas.clark%40northeastern.edu%7Cdf41115443c24ad1a67a08d9e67e4565%7Ca8eec281aaa34daeac9b9a398b9215e7%7C0%7C0%7C637794253163659084%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=j3Wipf2PLgF3njCsb6BTN1J%2BwrOOfe1%2Fcm%2F%2BUiGTMQk%3D&reserved=0>\r\n User-Agent: curl/7.71.1\r\n Accept: */*\r\n Content-type: application/sparql-query\r\n Content-Length: 0\r\n [Content length: 0] \r\n [Full request URI: example.org/v1/graphs/sparql<https://nam12.safelinks.protection.outlook.com/?url=http%3A%2F%2Fexample.org%2Fv1%2Fgraphs%2Fsparql&data=04%7C01%7Cas.clark%40northeastern.edu%7Cdf41115443c24ad1a67a08d9e67e4565%7Ca8eec281aaa34daeac9b9a398b9215e7%7C0%7C0%7C637794253163659084%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=PeCGztKkndWthFeKyUrYUljM2oq4ivkyP1ugAYL64QU%3D&reserved=0>] Then via BaseX: Hypertext Transfer Protocol POST /v1/graphs/sparql HTTP/1.1\r\n [Expert Info (Chat/Sequence): POST /v1/graphs/sparql HTTP/1.1\r\n] [POST /v1/graphs/sparql HTTP/1.1\r\n] [Severity level: Chat] [Group: Sequence] Request Method: POST Request URI: /v1/graphs/sparql Request Version: HTTP/1.1 Content-type: application/sparql-query\r\n Authorization: Digest\r\n User-Agent: Java/1.8.0_121\r\n Host: example.org<https://nam12.safelinks.protection.outlook.com/?url=http%3A%2F%2Fexample.org%2F&data=04%7C01%7Cas.clark%40northeastern.edu%7Cdf41115443c24ad1a67a08d9e67e4565%7Ca8eec281aaa34daeac9b9a398b9215e7%7C0%7C0%7C637794253163659084%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=j3Wipf2PLgF3njCsb6BTN1J%2BwrOOfe1%2Fcm%2F%2BUiGTMQk%3D&reserved=0>\r\n Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2\r\n Connection: keep-alive\r\n \r\n [Full request URI: example.org/v1/graphs/sparql<https://nam12.safelinks.protection.outlook.com/?url=http%3A%2F%2Fexample.org%2Fv1%2Fgraphs%2Fsparql&data=04%7C01%7Cas.clark%40northeastern.edu%7Cdf41115443c24ad1a67a08d9e67e4565%7Ca8eec281aaa34daeac9b9a398b9215e7%7C0%7C0%7C637794253163659084%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=PeCGztKkndWthFeKyUrYUljM2oq4ivkyP1ugAYL64QU%3D&reserved=0>] I notice that curl doesn't include the Authorization header on the first request, but I am out of my depth here. Best, Tim -- Tim A. Thompson Metadata Librarian Yale University Library On Wed, Feb 2, 2022 at 12:40 PM Christian Grün <christian.gruen@gmail.com<mailto:christian.gruen@gmail.com>> wrote: Hi Tim, Your request should be successful if you omit the send-authorization attribute: “If send-authorization is true (default value is false) and the authentication method supports generating the header Authorization without challenge, the request contains this header. The default value is to send a non-authenticated request, and if the response is an authentication challenge, then only send the credentials in a second message.” [1] If you use Basic authentication, you can reduce the communication to a single request by sending the authorization with the first request. With Digest authentication, there will always be two requests. With the (still to be finalized) HTTP Client module 2.0, send-authorization will disappear, and it will be up to the implementation to decide how many requests will be sent [2]. Hope this helps, Christian [1] http://expath.org/spec/http-client#d2e430<https://nam12.safelinks.protection.outlook.com/?url=http%3A%2F%2Fexpath.org%2Fspec%2Fhttp-client%23d2e430&data=04%7C01%7Cas.clark%40northeastern.edu%7Cdf41115443c24ad1a67a08d9e67e4565%7Ca8eec281aaa34daeac9b9a398b9215e7%7C0%7C0%7C637794253163659084%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=RREH6BeyWOV8juK%2Brf0UuofvBiC2JOv7B1%2BWXLYCjUQ%3D&reserved=0> [2] https://expath.github.io/expath-cg/specs/http-client-2/<https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fexpath.github.io%2Fexpath-cg%2Fspecs%2Fhttp-client-2%2F&data=04%7C01%7Cas.clark%40northeastern.edu%7Cdf41115443c24ad1a67a08d9e67e4565%7Ca8eec281aaa34daeac9b9a398b9215e7%7C0%7C0%7C637794253163659084%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=owfYuuSXSpkkPTnQhBIEJ1wBcKPCs4oFnaaVEvndvMA%3D&reserved=0> On Wed, Feb 2, 2022 at 6:19 PM Tim Thompson <timathom@gmail.com<mailto:timathom@gmail.com>> wrote:
Thanks, Andy! That's very handy. So, when I test against httpbin, the same thing happens.
curl request: curl --digest --user user:pw -X GET 'http://httpbin.org/digest-auth/auth/user/pw<https://nam12.safelinks.protection.outlook.com/?url=http%3A%2F%2Fhttpbin.org%2Fdigest-auth%2Fauth%2Fuser%2Fpw&data=04%7C01%7Cas.clark%40northeastern.edu%7Cdf41115443c24ad1a67a08d9e67e4565%7Ca8eec281aaa34daeac9b9a398b9215e7%7C0%7C0%7C637794253163659084%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=SWqtnqsC98VlfYr7TeIUggFdoR7dJDZtkNJxz9051jw%3D&reserved=0>'
response: { "authenticated": true, "user": "user" }
let $response := ( http:send-request( <http:request method="GET" href="{$endpoint}" username="user" password="pw" auth-method="Digest" send-authorization="true"> <http:header name="Content-Type" value="{$type}; charset=utf-8"/> </http:request> ) ) return $response
response: <http:response xmlns:http="http://expath.org/ns/http-client<https://nam12.safelinks.protection.outlook.com/?url=http%3A%2F%2Fexpath.org%2Fns%2Fhttp-client&data=04%7C01%7Cas.clark%40northeastern.edu%7Cdf41115443c24ad1a67a08d9e67e4565%7Ca8eec281aaa34daeac9b9a398b9215e7%7C0%7C0%7C637794253163659084%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=%2FNS%2BIceZOx6yP%2Fp4ikxkNqAhYq44F49nUaE3u8iijr0%3D&reserved=0>" status="401" message="UNAUTHORIZED"> <http:header name="Server" value="gunicorn/19.9.0"/> <http:header name="Access-Control-Allow-Origin" value="*"/> <http:header name="Access-Control-Allow-Credentials" value="true"/> <http:header name="WWW-Authenticate" value="Digest realm="me@kennethreitz.com<mailto:quot%3Bme@kennethreitz.com>", nonce="36d59385e96e1689595f4f14fa2921ca", qop="auth", opaque="a21df3287b5cb9cbf8b6a68b2c15fa1d", algorithm=MD5, stale=FALSE"/> <http:header name="Connection" value="keep-alive"/> <http:header name="Set-Cookie" value="fake=fake_value; Path=/"/> <http:header name="Set-Cookie" value="stale_after=never; Path=/"/> <http:header name="Content-Length" value="0"/> <http:header name="Date" value="Wed, 02 Feb 2022 17:14:13 GMT"/> <http:header name="Content-Type" value="text/html; charset=utf-8"/> </http:response>
-- Tim A. Thompson Metadata Librarian Yale University Library
On Wed, Feb 2, 2022 at 12:06 PM Andy Bunce <bunce.andy@gmail.com<mailto:bunce.andy@gmail.com>> wrote:
I find httpbin.org<https://nam12.safelinks.protection.outlook.com/?url=http%3A%2F%2Fhttpbin.org%2F&data=04%7C01%7Cas.clark%40northeastern.edu%7Cdf41115443c24ad1a67a08d9e67e4565%7Ca8eec281aaa34daeac9b9a398b9215e7%7C0%7C0%7C637794253163659084%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=5ZA6A7rJL%2B2YD%2BPvtA9wtLUjJVf3EAZw3Fb5ifo4H%2BU%3D&reserved=0> a useful resource to test this kind of thing [1] and [2] /Andy
[1] http://httpbin.org/#/Auth/get_basic_auth__user___passwd_<https://nam12.safelinks.protection.outlook.com/?url=http%3A%2F%2Fhttpbin.org%2F%23%2FAuth%2Fget_basic_auth__user___passwd_&data=04%7C01%7Cas.clark%40northeastern.edu%7Cdf41115443c24ad1a67a08d9e67e4565%7Ca8eec281aaa34daeac9b9a398b9215e7%7C0%7C0%7C637794253163659084%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=TU9Itw0Bs4Wbk51bnStj2oFqWMq%2BrdIOxgUwq5fFFrM%3D&reserved=0> [2] http://httpbin.org/basic-auth/user1/mypass<https://nam12.safelinks.protection.outlook.com/?url=http%3A%2F%2Fhttpbin.org%2Fbasic-auth%2Fuser1%2Fmypass&data=04%7C01%7Cas.clark%40northeastern.edu%7Cdf41115443c24ad1a67a08d9e67e4565%7Ca8eec281aaa34daeac9b9a398b9215e7%7C0%7C0%7C637794253163659084%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=lH8HeRKkVu%2Fev4Er8lokTCZ0xqyS3HOYtBBqgTSw0PY%3D&reserved=0>
On Wed, 2 Feb 2022 at 16:58, Christian Grün <christian.gruen@gmail.com<mailto:christian.gruen@gmail.com>> wrote:
Good to hear; so at least Digest is detected at this stage ;)
Tim Thompson <timathom@gmail.com<mailto:timathom@gmail.com>> schrieb am Mi., 2. Feb. 2022, 17:54:
With "digest" I get: "[experr:HC0004] Invalid authentication method: digest."
-- Tim A. Thompson Metadata Librarian Yale University Library
On Wed, Feb 2, 2022 at 11:52 AM Christian Grün <christian.gruen@gmail.com<mailto:christian.gruen@gmail.com>> wrote:
Yes, it should still be supported. I assume it doesn't make a difference if you use "Digest" or "digest"?
Tim Thompson <timathom@gmail.com<mailto:timathom@gmail.com>> schrieb am Mi., 2. Feb. 2022, 17:48:
Thanks, Christian. Right, that wasn't a working example. The server I'm querying is behind a firewall, so I can't really reproduce the issue unfortunately.
The BaseX documentation states that the HTTP client does support Digest authentication. However, I did some poking around using Wireshark and limited knowledge of network traffic. When I run the request in curl, the expected protocol is played out, and I see the evidence in Wireshark:
(1) The server responds with "401 Unauthorized" and provides the nonce value. (2) The client (curl) then does its business (supplies the username and password, resends the request, etc.)
But when I submit the request via BaseX, it never gets past step (1). It never seems to send the username and password at all. Is Digest authentication still supported?
Best, Tim
-- Tim A. Thompson Metadata Librarian Yale University Library
On Wed, Feb 2, 2022 at 10:13 AM Christian Grün <christian.gruen@gmail.com<mailto:christian.gruen@gmail.com>> wrote: > > Hi Tim, > > Difficult to tell; both the curl and the XQuery variants give me 404. > Do you think you could provide us with an example that works out of > the box? > > Best, > Christian > > > > On Wed, Feb 2, 2022 at 12:54 AM Tim Thompson <timathom@gmail.com<mailto:timathom@gmail.com>> wrote: > > > > Hello, > > > > I'm trying to post a SPARQL query to an endpoint using Digest authentication with the HTTP client. > > > > The query works fine using curl: > > > > curl --digest --user user:pass -X POST -d@'test.rq' \ > > -H "Content-type: application/sparql-query" \ > > 'http://example.org/sparql<https://nam12.safelinks.protection.outlook.com/?url=http%3A%2F%2Fexample.org%2Fsparql&data=04%7C01%7Cas.clark%40northeastern.edu%7Cdf41115443c24ad1a67a08d9e67e4565%7Ca8eec281aaa34daeac9b9a398b9215e7%7C0%7C0%7C637794253163659084%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=8uhboyxkhGWHk8HbZRKgDXOAbDRa2UwIigI1H5vWxt0%3D&reserved=0>' > > > > But the equivalent request in BaseX fails with 401 Unauthorized: > > > > let $endpoint := "http://example.org/sparql<https://nam12.safelinks.protection.outlook.com/?url=http%3A%2F%2Fexample.org%2Fsparql&data=04%7C01%7Cas.clark%40northeastern.edu%7Cdf41115443c24ad1a67a08d9e67e4565%7Ca8eec281aaa34daeac9b9a398b9215e7%7C0%7C0%7C637794253163659084%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=8uhboyxkhGWHk8HbZRKgDXOAbDRa2UwIigI1H5vWxt0%3D&reserved=0>" > > let $user := "user" > > let $pass := "pass" > > let $type := "application/sparql-query" > > > > let $response := ( > > http:send-request( > > <http:request > > method="POST" > > href="{$endpoint}" > > username="{$user}" > > password="{$pass}" > > auth-method="Digest" > > send-authorization="true"> > > <http:header > > name="Content-Type" > > value="{$type}; charset=utf-8"/> > > <http:body > > media-type="{$type}">{ > > ``[ > > select * where {?s ?p ?o} limit 1 > > ]`` > > }</http:body> > > </http:request> > > ) > > ) > > return > > $response > > > > Any ideas about what might be causing the BaseX HTTP client to be denied here? > > > > Thanks in advance, > > Tim > > > > > > -- > > Tim A. Thompson > > Metadata Librarian > > Yale University Library > >
Thanks, Ash! That attribute doesn't seem to be available in BaseX, as far as I can tell. -- Tim A. Thompson Metadata Librarian Yale University Library On Wed, Feb 2, 2022 at 2:21 PM Clark, Ash <as.clark@northeastern.edu> wrote:
In case it helps: I had a similar problem in eXist some years ago. My fix was to put the attribute `http-version="1.0"` on the request element. The HTTP client defaults to HTTP v1.1, but for whatever reason, only HTTP 1.0 worked for me.
~Ash ------------------------------ *From:* BaseX-Talk <basex-talk-bounces@mailman.uni-konstanz.de> on behalf of Tim Thompson <timathom@gmail.com> *Sent:* Wednesday, February 2, 2022 1:59 PM *To:* Christian Grün <christian.gruen@gmail.com>; BaseX < basex-talk@mailman.uni-konstanz.de> *Subject:* Re: [basex-talk] Issue with HTTP client and authentication
Ah! Thanks for clarifying that. So, now it is working for GET requests in BaseX, but still not for POST.
So, this works: http:send-request( <http:request method="GET" href="{$endpoint}?query={encode-for-uri('select * where {?s ?p ?o} limit 1')}" username="{$user}" password="{$pass}" auth-method="Digest"> </http:request> )
But when I try to POST a request body with Digest auth, it is still failing (httpbin doesn't seem to support testing Digest with POST, unfortunately). The request hangs for a while, then fails with 401 Unauthorized. Wireshark reveals that the first network response from the server is actually a 502 Bad Gateway.
Attached are two screenshots from Wireshark: first, the request/response sequence with curl, then with BaseX.
Here's what Wireshark shows for the first request via curl:
Hypertext Transfer Protocol POST /v1/graphs/sparql HTTP/1.1\r\n [Expert Info (Chat/Sequence): POST /v1/graphs/sparql HTTP/1.1\r\n] [POST /v1/graphs/sparql HTTP/1.1\r\n] [Severity level: Chat] [Group: Sequence] Request Method: POST Request URI: /v1/graphs/sparql Request Version: HTTP/1.1 Host: example.org <https://nam12.safelinks.protection.outlook.com/?url=http%3A%2F%2Fexample.org%2F&data=04%7C01%7Cas.clark%40northeastern.edu%7Cdf41115443c24ad1a67a08d9e67e4565%7Ca8eec281aaa34daeac9b9a398b9215e7%7C0%7C0%7C637794253163659084%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=j3Wipf2PLgF3njCsb6BTN1J%2BwrOOfe1%2Fcm%2F%2BUiGTMQk%3D&reserved=0> \r\n User-Agent: curl/7.71.1\r\n Accept: */*\r\n Content-type: application/sparql-query\r\n Content-Length: 0\r\n [Content length: 0] \r\n [Full request URI: example.org/v1/graphs/sparql <https://nam12.safelinks.protection.outlook.com/?url=http%3A%2F%2Fexample.org%2Fv1%2Fgraphs%2Fsparql&data=04%7C01%7Cas.clark%40northeastern.edu%7Cdf41115443c24ad1a67a08d9e67e4565%7Ca8eec281aaa34daeac9b9a398b9215e7%7C0%7C0%7C637794253163659084%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=PeCGztKkndWthFeKyUrYUljM2oq4ivkyP1ugAYL64QU%3D&reserved=0> ]
Then via BaseX:
Hypertext Transfer Protocol POST /v1/graphs/sparql HTTP/1.1\r\n [Expert Info (Chat/Sequence): POST /v1/graphs/sparql HTTP/1.1\r\n] [POST /v1/graphs/sparql HTTP/1.1\r\n] [Severity level: Chat] [Group: Sequence] Request Method: POST Request URI: /v1/graphs/sparql Request Version: HTTP/1.1 Content-type: application/sparql-query\r\n Authorization: Digest\r\n User-Agent: Java/1.8.0_121\r\n Host: example.org <https://nam12.safelinks.protection.outlook.com/?url=http%3A%2F%2Fexample.org%2F&data=04%7C01%7Cas.clark%40northeastern.edu%7Cdf41115443c24ad1a67a08d9e67e4565%7Ca8eec281aaa34daeac9b9a398b9215e7%7C0%7C0%7C637794253163659084%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=j3Wipf2PLgF3njCsb6BTN1J%2BwrOOfe1%2Fcm%2F%2BUiGTMQk%3D&reserved=0> \r\n Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2\r\n Connection: keep-alive\r\n \r\n [Full request URI: example.org/v1/graphs/sparql <https://nam12.safelinks.protection.outlook.com/?url=http%3A%2F%2Fexample.org%2Fv1%2Fgraphs%2Fsparql&data=04%7C01%7Cas.clark%40northeastern.edu%7Cdf41115443c24ad1a67a08d9e67e4565%7Ca8eec281aaa34daeac9b9a398b9215e7%7C0%7C0%7C637794253163659084%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=PeCGztKkndWthFeKyUrYUljM2oq4ivkyP1ugAYL64QU%3D&reserved=0> ]
I notice that curl doesn't include the Authorization header on the first request, but I am out of my depth here.
Best, Tim
-- Tim A. Thompson Metadata Librarian Yale University Library
On Wed, Feb 2, 2022 at 12:40 PM Christian Grün <christian.gruen@gmail.com> wrote:
Hi Tim,
Your request should be successful if you omit the send-authorization attribute:
“If send-authorization is true (default value is false) and the authentication method supports generating the header Authorization without challenge, the request contains this header. The default value is to send a non-authenticated request, and if the response is an authentication challenge, then only send the credentials in a second message.” [1]
If you use Basic authentication, you can reduce the communication to a single request by sending the authorization with the first request. With Digest authentication, there will always be two requests.
With the (still to be finalized) HTTP Client module 2.0, send-authorization will disappear, and it will be up to the implementation to decide how many requests will be sent [2].
Hope this helps, Christian
[1] http://expath.org/spec/http-client#d2e430 <https://nam12.safelinks.protection.outlook.com/?url=http%3A%2F%2Fexpath.org%2Fspec%2Fhttp-client%23d2e430&data=04%7C01%7Cas.clark%40northeastern.edu%7Cdf41115443c24ad1a67a08d9e67e4565%7Ca8eec281aaa34daeac9b9a398b9215e7%7C0%7C0%7C637794253163659084%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=RREH6BeyWOV8juK%2Brf0UuofvBiC2JOv7B1%2BWXLYCjUQ%3D&reserved=0> [2] https://expath.github.io/expath-cg/specs/http-client-2/ <https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fexpath.github.io%2Fexpath-cg%2Fspecs%2Fhttp-client-2%2F&data=04%7C01%7Cas.clark%40northeastern.edu%7Cdf41115443c24ad1a67a08d9e67e4565%7Ca8eec281aaa34daeac9b9a398b9215e7%7C0%7C0%7C637794253163659084%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=owfYuuSXSpkkPTnQhBIEJ1wBcKPCs4oFnaaVEvndvMA%3D&reserved=0>
On Wed, Feb 2, 2022 at 6:19 PM Tim Thompson <timathom@gmail.com> wrote:
Thanks, Andy! That's very handy. So, when I test against httpbin, the
same thing happens.
curl request: curl --digest --user user:pw -X GET '
http://httpbin.org/digest-auth/auth/user/pw <https://nam12.safelinks.protection.outlook.com/?url=http%3A%2F%2Fhttpbin.org%2Fdigest-auth%2Fauth%2Fuser%2Fpw&data=04%7C01%7Cas.clark%40northeastern.edu%7Cdf41115443c24ad1a67a08d9e67e4565%7Ca8eec281aaa34daeac9b9a398b9215e7%7C0%7C0%7C637794253163659084%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=SWqtnqsC98VlfYr7TeIUggFdoR7dJDZtkNJxz9051jw%3D&reserved=0> '
response: { "authenticated": true, "user": "user" }
BaseX request: let $endpoint := "http://httpbin.org/digest-auth/auth/user/pw
let $response := ( http:send-request( <http:request method="GET" href="{$endpoint}" username="user" password="pw" auth-method="Digest" send-authorization="true"> <http:header name="Content-Type" value="{$type}; charset=utf-8"/> </http:request> ) ) return $response
response: <http:response xmlns:http="http://expath.org/ns/http-client
<http:header name="Server" value="gunicorn/19.9.0"/> <http:header name="Access-Control-Allow-Origin" value="*"/> <http:header name="Access-Control-Allow-Credentials" value="true"/> <http:header name="WWW-Authenticate" value="Digest realm=& quot;me@kennethreitz.com", nonce="36d59385e96e1689595f4f14fa2921ca", qop="auth", opaque="a21df3287b5cb9cbf8b6a68b2c15fa1d", algorithm=MD5, stale=FALSE"/> <http:header name="Connection" value="keep-alive"/> <http:header name="Set-Cookie" value="fake=fake_value; Path=/"/> <http:header name="Set-Cookie" value="stale_after=never; Path=/"/> <http:header name="Content-Length" value="0"/> <http:header name="Date" value="Wed, 02 Feb 2022 17:14:13 GMT"/> <http:header name="Content-Type" value="text/html; charset=utf-8"/> </http:response>
-- Tim A. Thompson Metadata Librarian Yale University Library
On Wed, Feb 2, 2022 at 12:06 PM Andy Bunce <bunce.andy@gmail.com> wrote:
I find httpbin.org
<https://nam12.safelinks.protection.outlook.com/?url=http%3A%2F%2Fhttpbin.org%2F&data=04%7C01%7Cas.clark%40northeastern.edu%7Cdf41115443c24ad1a67a08d9e67e4565%7Ca8eec281aaa34daeac9b9a398b9215e7%7C0%7C0%7C637794253163659084%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=5ZA6A7rJL%2B2YD%2BPvtA9wtLUjJVf3EAZw3Fb5ifo4H%2BU%3D&reserved=0> a useful resource to test this kind of thing [1] and [2]
/Andy
[1] http://httpbin.org/#/Auth/get_basic_auth__user___passwd_ <https://nam12.safelinks.protection.outlook.com/?url=http%3A%2F%2Fhttpbin.org%2F%23%2FAuth%2Fget_basic_auth__user___passwd_&data=04%7C01%7Cas.clark%40northeastern.edu%7Cdf41115443c24ad1a67a08d9e67e4565%7Ca8eec281aaa34daeac9b9a398b9215e7%7C0%7C0%7C637794253163659084%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=TU9Itw0Bs4Wbk51bnStj2oFqWMq%2BrdIOxgUwq5fFFrM%3D&reserved=0> [2] http://httpbin.org/basic-auth/user1/mypass <https://nam12.safelinks.protection.outlook.com/?url=http%3A%2F%2Fhttpbin.org%2Fbasic-auth%2Fuser1%2Fmypass&data=04%7C01%7Cas.clark%40northeastern.edu%7Cdf41115443c24ad1a67a08d9e67e4565%7Ca8eec281aaa34daeac9b9a398b9215e7%7C0%7C0%7C637794253163659084%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=lH8HeRKkVu%2Fev4Er8lokTCZ0xqyS3HOYtBBqgTSw0PY%3D&reserved=0>
On Wed, 2 Feb 2022 at 16:58, Christian Grün <christian.gruen@gmail.com> wrote:
Good to hear; so at least Digest is detected at this stage ;)
Tim Thompson <timathom@gmail.com> schrieb am Mi., 2. Feb. 2022, 17:54:
With "digest" I get: "[experr:HC0004] Invalid authentication method:
digest."
-- Tim A. Thompson Metadata Librarian Yale University Library
On Wed, Feb 2, 2022 at 11:52 AM Christian Grün <
christian.gruen@gmail.com> wrote:
Yes, it should still be supported. I assume it doesn't make a
difference if you use "Digest" or "digest"?
Tim Thompson <timathom@gmail.com> schrieb am Mi., 2. Feb. 2022,
17:48:
> > Thanks, Christian. Right, that wasn't a working example. The server I'm querying is behind a firewall, so I can't really reproduce the issue unfortunately. > > The BaseX documentation states that the HTTP client does support Digest authentication. However, I did some poking around using Wireshark and limited knowledge of network traffic. When I run the request in curl, the expected protocol is played out, and I see the evidence in Wireshark: > > (1) The server responds with "401 Unauthorized" and provides the nonce value. > (2) The client (curl) then does its business (supplies the username and password, resends the request, etc.) > > But when I submit the request via BaseX, it never gets past step (1). It never seems to send the username and password at all. Is Digest authentication still supported? > > Best, > Tim > > > -- > Tim A. Thompson > Metadata Librarian > Yale University Library > > > On Wed, Feb 2, 2022 at 10:13 AM Christian Grün < christian.gruen@gmail.com> wrote: >> >> Hi Tim, >> >> Difficult to tell; both the curl and the XQuery variants give me
>> Do you think you could provide us with an example that works out of >> the box? >> >> Best, >> Christian >> >> >> >> On Wed, Feb 2, 2022 at 12:54 AM Tim Thompson <timathom@gmail.com> wrote: >> > >> > Hello, >> > >> > I'm trying to post a SPARQL query to an endpoint using Digest authentication with the HTTP client. >> > >> > The query works fine using curl: >> > >> > curl --digest --user user:pass -X POST -d@'test.rq' \ >> > -H "Content-type: application/sparql-query" \ >> > 'http://example.org/sparql <https://nam12.safelinks.protection.outlook.com/?url=http%3A%2F%2Fexample.org%2Fsparql&data=04%7C01%7Cas.clark%40northeastern.edu%7Cdf41115443c24ad1a67a08d9e67e4565%7Ca8eec281aaa34daeac9b9a398b9215e7%7C0%7C0%7C637794253163659084%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=8uhboyxkhGWHk8HbZRKgDXOAbDRa2UwIigI1H5vWxt0%3D&reserved=0> ' >> > >> > But the equivalent request in BaseX fails with 401 Unauthorized: >> > >> > let $endpoint := "http://example.org/sparql <https://nam12.safelinks.protection.outlook.com/?url=http%3A%2F%2Fexample.org%2Fsparql&data=04%7C01%7Cas.clark%40northeastern.edu%7Cdf41115443c24ad1a67a08d9e67e4565%7Ca8eec281aaa34daeac9b9a398b9215e7%7C0%7C0%7C637794253163659084%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=8uhboyxkhGWHk8HbZRKgDXOAbDRa2UwIigI1H5vWxt0%3D&reserved=0> " >> > let $user := "user" >> > let $pass := "pass" >> > let $type := "application/sparql-query" >> > >> > let $response := ( >> > http:send-request( >> > <http:request >> > method="POST" >> > href="{$endpoint}" >> > username="{$user}" >> > password="{$pass}" >> > auth-method="Digest" >> > send-authorization="true"> >> > <http:header >> > name="Content-Type" >> > value="{$type}; charset=utf-8"/> >> > <http:body >> > media-type="{$type}">{ >> > ``[ >> > select * where {?s ?p ?o} limit 1 >> > ]`` >> > }</http:body> >> > </http:request> >> > ) >> > ) >> > return >> > $response >> > >> > Any ideas about what might be causing the BaseX HTTP client to be denied here? >> > >> > Thanks in advance, >> > Tim >> > >> > >> > -- >> > Tim A. Thompson >> > Metadata Librarian >> > Yale University Library >> >
participants (5)
-
Andy Bunce -
Christian Grün -
Clark, Ash -
Imsieke, Gerrit, le-tex -
Tim Thompson