Hi list,
while debugging my application, I noticed that when I send a request to the HTTP API of basex, the request is then sent by basex to itself via a localhost connection. That is, the HTTP API part of basex sends a message to the BaseX native API through TCP. Now, I wonder, is this really necessary? After all, both reside in the same process, so going through the OS feels a bit wasteful here. I noticed that I can change this behaviour if I pass the "-l" flag to basexhttp ("local mode"), so the code to directly use the in-process DB obviously exists. I wonder why it can only be used in the local mode which entirely disables the "native" BaseX API?
Kind regards, Ralf
Hi Ralf,
yes, that's a good point. We'd probably have to check somewhere (e.g. in the HTTPSession.local() method) if a server exists in the same JVM. If you have a spontaneous idea how to perform this check.. Feel free to tell us ;)
Thanks, Christian
On Sat, Oct 15, 2011 at 2:21 PM, Ralf Jung ralfjung-e@gmx.de wrote:
Hi list,
while debugging my application, I noticed that when I send a request to the HTTP API of basex, the request is then sent by basex to itself via a localhost connection. That is, the HTTP API part of basex sends a message to the BaseX native API through TCP. Now, I wonder, is this really necessary? After all, both reside in the same process, so going through the OS feels a bit wasteful here. I noticed that I can change this behaviour if I pass the "-l" flag to basexhttp ("local mode"), so the code to directly use the in-process DB obviously exists. I wonder why it can only be used in the local mode which entirely disables the "native" BaseX API?
Kind regards, Ralf _______________________________________________ BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
Hi,
yes, that's a good point. We'd probably have to check somewhere (e.g. in the HTTPSession.local() method) if a server exists in the same JVM. If you have a spontaneous idea how to perform this check.. Feel free to tell us ;)
Is using LocalSession safe if there can be concurrent queries through the native interface, i.e. concurrent ClientSession (and of course other LocalSession) accesses?
The decision (Local vs. Client) is complicated by the fact that the logic when a native BaseX server is launched, is quite complicated: It is started if the -l flag is NOT present AND the HOST property is set to something different from "localhost". I would have expected to control that directly via some flags, for example: * "basexhttp" starts the API plus a local native BaseX server * "basexhttp -l" starts just the API (local mode) * "basexhttp -r" starts the API and connects to a remote BaseX server to execute the queries etc. on (remote mode) In the first two modes, LocalSession could be used (if it can be used concurrently, see above). It must not be possible to pass both the -l and -r flag. This would keep the default behaviour, and would remove the "magic" of the server launch happening or not. IMHO that's a quite significant difference, because it decides which data is going to be used. But maybe I just have a strange intuition ;-)
Kind regards, Ralf
Hi again,
to clarify what I mean, I roughly implemented it in attached patch... or at least I think I did, it is *COMPLETELY UNTESTED* and just serves the purpose of stating more precisely what I described.
Kind regards, Ralf
On Monday 17 October 2011 15:26:15 Ralf Jung wrote:
Hi,
yes, that's a good point. We'd probably have to check somewhere (e.g. in the HTTPSession.local() method) if a server exists in the same JVM. If you have a spontaneous idea how to perform this check.. Feel free to tell us ;)
Is using LocalSession safe if there can be concurrent queries through the native interface, i.e. concurrent ClientSession (and of course other LocalSession) accesses?
The decision (Local vs. Client) is complicated by the fact that the logic when a native BaseX server is launched, is quite complicated: It is started if the -l flag is NOT present AND the HOST property is set to something different from "localhost". I would have expected to control that directly via some flags, for example:
- "basexhttp" starts the API plus a local native BaseX server
- "basexhttp -l" starts just the API (local mode)
- "basexhttp -r" starts the API and connects to a remote BaseX server to
execute the queries etc. on (remote mode) In the first two modes, LocalSession could be used (if it can be used concurrently, see above). It must not be possible to pass both the -l and -r flag. This would keep the default behaviour, and would remove the "magic" of the server launch happening or not. IMHO that's a quite significant difference, because it decides which data is going to be used. But maybe I just have a strange intuition ;-)
Kind regards, Ralf _______________________________________________ BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
Is using LocalSession safe if there can be concurrent queries through the native interface, i.e. concurrent ClientSession (and of course other LocalSession) accesses?
Yes; both LocalSession and ClientSession instances take advantage of our locking architecture and can be used in parallel (as long as the server instance and the LocalSession use the same instance of the Context class).
- "basexhttp -l" starts just the API (local mode)
- "basexhttp -r" starts the API and connects to a remote BaseX server to
execute the queries etc. on (remote mode)
Your proposal is worth discussing. I'm still not sure, though, how to name such an additional option, and how to make it as self-explanatory as possible: if no alternative host name will be specified by the user, "remote" is rather confusing imho, as the database server will be located on "localhost", i.e., the same machine. – After all, the reason for the current approach was to reduce the number of options that would otherwise need more explanation, and could make things even more complicated (we even thought about completely discarding the local flag).
But maybe I just have a strange intuition ;-)
Not at all.. And I still like the idea of directly communicating to the server and avoiding the ClientSession if it resides in the same JVM (it's mainly a question of how to implement it most properly).
Thanks for caring, Christian
Hi,
Yes; both LocalSession and ClientSession instances take advantage of our locking architecture and can be used in parallel (as long as the server instance and the LocalSession use the same instance of the Context class).
Great!
- "basexhttp -l" starts just the API (local mode)
- "basexhttp -r" starts the API and connects to a remote BaseX server to
execute the queries etc. on (remote mode)
Your proposal is worth discussing. I'm still not sure, though, how to name such an additional option, and how to make it as self-explanatory as possible: if no alternative host name will be specified by the user, "remote" is rather confusing imho, as the database server will be located on "localhost", i.e., the same machine. – After all, the reason for the current approach was to reduce the number of options that would otherwise need more explanation, and could make things even more complicated (we even thought about completely discarding the local flag).
From the point of view of the server, it is still "remote" as it is accessed via TCP, even if that connection is a localhost connection. From my experience, it is quite unusual to treat a TCP connection to localhost differently than a TCP connection to any other host - and technically, after all, there is no difference. But then, I know how this stuff works, other who don't might find it less intuitive. Maybe "external" would be a better name? (as in, external to the JVM/OS process the API is running in). The current setup however makes it impossible to start a BaseX native server on a host, and then start an API frontend on the same machine but in a different process that connects to it - the remote IP would be "localhost" and the API would start a 2nd database server instance. So this would require work-arounds like adding a name to the /etc/hosts that resolves to 127.0.0.1 but is not recognized by BaseX as localhost. You might find that a strange setup, but together with SSH tunnelling it even becomes useful: The server runs on a machine far away that you can't directly connect to (or don't want to, unencrypted, as the network in between is untrusted). So you SSH in that machine and create a tunnel from the local BaseX port to the one on the other machine, and let the API connect to that local port (which in reality is a remote BaseX server). Okay, this might still be strange, but I think you get the point ;-)
But maybe I just have a strange intuition ;-)
Not at all.. And I still like the idea of directly communicating to the server and avoiding the ClientSession if it resides in the same JVM (it's mainly a question of how to implement it most properly).
Well, I could easily come up with a patch that essentially duplicates the "start server in JVM or not" logic and passes the info on to HTTPSession (I actually already had that patch done before I came up with the other one). But that would still be confusing to me, and the problem mentioned above would persist. At the very least, this magic behaviour should be documented (in the wiki and maybe also in the -h help text). I only learned how it works by looking at the source.
Kind regards, Ralf
Hi again,
Well, I could easily come up with a patch that essentially duplicates the "start server in JVM or not" logic and passes the info on to HTTPSession (I actually already had that patch done before I came up with the other one).
You can find that patch attached, again *COMPLETELY UNTESTED*. It should get rid of the pointless TCP overhead, but it does not fix the "magic" behaviour.
Kind regards, Ralf
Looks reasonable (and could surely be worse for being "completely untested" ;). I'll give you feedback some time later today. Thanks again, Christian ___________________________
Well, I could easily come up with a patch that essentially duplicates the "start server in JVM or not" logic and passes the info on to HTTPSession (I actually already had that patch done before I came up with the other one).
You can find that patch attached, again *COMPLETELY UNTESTED*. It should get rid of the pointless TCP overhead, but it does not fix the "magic" behaviour.
Kind regards, Ralf
Hi Ralf,
I've just committed a new solution that should reflect your change request. We decided to call the flag "client" (-c), in order to indicate that the HTTP Server talks to the database server as just another client.
Feedback welcome, Christian ___________________________
On Tue, Oct 18, 2011 at 10:09 AM, Ralf Jung ralfjung-e@gmx.de wrote:
Hi again,
Well, I could easily come up with a patch that essentially duplicates the "start server in JVM or not" logic and passes the info on to HTTPSession (I actually already had that patch done before I came up with the other one).
You can find that patch attached, again *COMPLETELY UNTESTED*. It should get rid of the pointless TCP overhead, but it does not fix the "magic" behaviour.
Kind regards, Ralf
basex-talk@mailman.uni-konstanz.de
-
Christian Grün -
Ralf Jung