Re: [basex-talk] Too Many Open Files
Hello Andreas, thank you very much for your valuable hints. sudo lsof -p basex-ps-no delivers many lines of java 11489 root 962u sock 0,6 0t0 10449645 can't identify protocol and (1) sudo lsof -p 11489 | wc -l gets reports an increasing number of used "files". The increase of the number in (1) in a given time interval correlates to the number of requests that the LVM load balancer sends to the basex port 1984 in that interval. There are about 4 requests from the LVM load balancer per minute. At the time of this writing (1) has the value of 1076 ... The parameter keepalive has the value KEEPALIVE = 600 but it does not seem to stop the increase of the value in (1). This information opens the way to workarounds - decrease the number of requests from the LVM load balancer, - increase the ulimit -n - restart the basex -server before the number in one reaches ulimt -n. But we really would prefer solutions to the increase of the figure in (1) The almost identical settings of the load balancer do work for a MySQL-Database without any problems. And without the load balancer an instance of the basex runs since April 10 without hitting ulimit -n = 1024. Any hints on this are very welcome! With kind regards, Andreas
I just searched our mailing list, cause there was something about too many open files some time ago:
On typical Linux installations, die open filedescriptor limit is 1024:
$ grep 'open files' /proc/self/limits Max open files 1024 1024 files
In Java, if a file-based object (FileWriter, FileReader, etc.) is not closed, the underlying file descriptor is not closed. Bug detectors (FindBugs) check for that intraprocedurally. I recommend you run both BaseX as well as this application through it.
If an open file-based object becomes unreachable, the finalizer will eventually close it - but it's possible to run out of open file descriptors simply due to unreachable, but not yet finalized objects. (Of course, if the object is leaked, it won't be closed ever.)
In Linux, use 'ps' to find out the pid of the Java JVM process, then do a ls -l /proc/<pid>/fds to see which file descriptors the process in question has open; or use the 'lsof' command.
- Godmar Probably that helps to investigate the issue.
-- Nexoma GmbH Theodorus Weg 7 59755 Arnsberg Tel. + 49 (0) 52 51 1613-0 Fax + 49 (0) 52 51 1613-99 mailto:andreas.rulle@nexoma.de Geschäftsführer: Guido Sauerland Sitz der Gesellschaft: Arnsberg Registergericht: Arnsberg, HRB 9365
Hello Andreas, this email informs you that * with the tool mon (1), * that is often used together, see (3), the LVS load balancer (2) it has been able to reproduce the java.net.SocketException: Too many open files with the following configuration watch basex service ping description Responses to ping interval 2s monitor tcp.monitor -p 1984 period It seems to be that basex does not close the sockets when monitored with tcp.monitor -p 1984. The basex-server crashes when ulimit -n is reached. With best regards, Andreas (1) https://mon.wiki.kernel.org/index.php/Monitors (2) http://www.linuxvirtualserver.org/ (3) http://www.austintek.com/LVS/LVS-HOWTO/HOWTO/LVS-HOWTO.realserver_failure.ht... Am 11.05.2012 18:21, schrieb Andreas Rulle:
Hello Andreas,
thank you very much for your valuable hints.
sudo lsof -p basex-ps-no
delivers many lines of
java 11489 root 962u sock 0,6 0t0 10449645 can't identify protocol
and
(1) sudo lsof -p 11489 | wc -l gets
reports an increasing number of used "files". The increase of the number in (1) in a given time interval correlates to the number of requests that the LVM load balancer sends to the basex port 1984 in that interval. There are about 4 requests from the LVM load balancer per minute.
At the time of this writing (1) has the value of 1076 ...
The parameter keepalive has the value
KEEPALIVE = 600
but it does not seem to stop the increase of the value in (1).
This information opens the way to workarounds
- decrease the number of requests from the LVM load balancer, - increase the ulimit -n - restart the basex -server before the number in one reaches ulimt -n.
But we really would prefer solutions to the increase of the figure in (1)
The almost identical settings of the load balancer do work for a MySQL-Database without any problems. And without the load balancer an instance of the basex runs since April 10 without hitting ulimit -n = 1024.
Any hints on this are very welcome!
With kind regards,
Andreas
I just searched our mailing list, cause there was something about too many open files some time ago:
On typical Linux installations, die open filedescriptor limit is 1024:
$ grep 'open files' /proc/self/limits Max open files 1024 1024 files
In Java, if a file-based object (FileWriter, FileReader, etc.) is not closed, the underlying file descriptor is not closed. Bug detectors (FindBugs) check for that intraprocedurally. I recommend you run both BaseX as well as this application through it.
If an open file-based object becomes unreachable, the finalizer will eventually close it - but it's possible to run out of open file descriptors simply due to unreachable, but not yet finalized objects. (Of course, if the object is leaked, it won't be closed ever.)
In Linux, use 'ps' to find out the pid of the Java JVM process, then do a ls -l /proc/<pid>/fds to see which file descriptors the process in question has open; or use the 'lsof' command.
- Godmar Probably that helps to investigate the issue.
-- Nexoma GmbH Theodorus Weg 7 59755 Arnsberg Tel. + 49 (0) 52 51 1613-0 Fax + 49 (0) 52 51 1613-99 mailto:andreas.rulle@nexoma.de Geschäftsführer: Guido Sauerland Sitz der Gesellschaft: Arnsberg Registergericht: Arnsberg, HRB 9365
Hello Andreas, the only file that i can think of that is opened for each client is the log file of BaseX. Normally the log file should just be opened by the server and the clients are referring to it, however, you should give it a try and start the basexserver with the "-z" option (http://docs.basex.org/wiki/Startup_Options) for suppressing the logging mechanism. -- Andreas Am 12.05.2012 um 00:41 schrieb Andreas Rulle:
Hello Andreas,
this email informs you that
* with the tool mon (1), * that is often used together, see (3), the LVS load balancer (2)
it has been able to reproduce the
java.net.SocketException: Too many open files
with the following configuration
watch basex service ping description Responses to ping interval 2s monitor tcp.monitor -p 1984 period
It seems to be that basex does not close the sockets when monitored with tcp.monitor -p 1984. The basex-server crashes when ulimit -n is reached.
With best regards,
Andreas
(1) https://mon.wiki.kernel.org/index.php/Monitors
(2) http://www.linuxvirtualserver.org/
(3) http://www.austintek.com/LVS/LVS-HOWTO/HOWTO/LVS-HOWTO.realserver_failure.ht...
Am 11.05.2012 18:21, schrieb Andreas Rulle:
Hello Andreas,
thank you very much for your valuable hints.
sudo lsof -p basex-ps-no
delivers many lines of
java 11489 root 962u sock 0,6 0t0 10449645 can't identify protocol
and
(1) sudo lsof -p 11489 | wc -l gets
reports an increasing number of used "files". The increase of the number in (1) in a given time interval correlates to the number of requests that the LVM load balancer sends to the basex port 1984 in that interval. There are about 4 requests from the LVM load balancer per minute.
At the time of this writing (1) has the value of 1076 ...
The parameter keepalive has the value
KEEPALIVE = 600
but it does not seem to stop the increase of the value in (1).
This information opens the way to workarounds
- decrease the number of requests from the LVM load balancer, - increase the ulimit -n - restart the basex -server before the number in one reaches ulimt -n.
But we really would prefer solutions to the increase of the figure in (1)
The almost identical settings of the load balancer do work for a MySQL-Database without any problems. And without the load balancer an instance of the basex runs since April 10 without hitting ulimit -n = 1024.
Any hints on this are very welcome!
With kind regards,
Andreas
I just searched our mailing list, cause there was something about too many open files some time ago:
On typical Linux installations, die open filedescriptor limit is 1024:
$ grep 'open files' /proc/self/limits Max open files 1024 1024 files
In Java, if a file-based object (FileWriter, FileReader, etc.) is not closed, the underlying file descriptor is not closed. Bug detectors (FindBugs) check for that intraprocedurally. I recommend you run both BaseX as well as this application through it.
If an open file-based object becomes unreachable, the finalizer will eventually close it - but it's possible to run out of open file descriptors simply due to unreachable, but not yet finalized objects. (Of course, if the object is leaked, it won't be closed ever.)
In Linux, use 'ps' to find out the pid of the Java JVM process, then do a ls -l /proc/<pid>/fds to see which file descriptors the process in question has open; or use the 'lsof' command.
- Godmar Probably that helps to investigate the issue.
-- Nexoma GmbH Theodorus Weg 7 59755 Arnsberg
Tel. + 49 (0) 52 51 1613-0 Fax + 49 (0) 52 51 1613-99
mailto:andreas.rulle@nexoma.de
Geschäftsführer: Guido Sauerland Sitz der Gesellschaft: Arnsberg Registergericht: Arnsberg, HRB 9365
_______________________________________________ BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
Hallo Andreas, thank you for your hint to start basex with the -z option. But I'm sorry to say that it did not change the basex servers reactions. 1. lsof does report the following lines: COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME java 23767 userXY 30u sock 0,6 0t0 92598 can't identify protocol *u* for a read and write lock of any length ''sock'' for a socket of unknown domain see the lsof man page (1). It shall be pointed out that TYPE has the value sock in contrast to REG which stands for regular file, see also the attached image. 2. Therefore it is very likely that not a regular file remains open, but a socket. 3. The tcpdump - viewed with wireshark - shows that the healthy check of the load balancer - mon and ldirectoryd do the same - sends a RST (Connection reset) message TCP to the BaseX database. Each RST message from the mon or ldirectord seem to lead to a new socket that remains open ... One question is: What can be done one the BaxeX-side to make it compatible with a load balancer? (Especially with ldirectord, that is used in the productive environment for healthy check.) With best regards Andreas (1) http://linux.die.net/man/8/lsof
Hello Andreas,
the only file that i can think of that is opened for each client is the log file of BaseX. Normally the log file should just be opened by the server and the clients are referring to it, however, you should give it a try and start the basexserver with the "-z" option (http://docs.basex.org/wiki/Startup_Options) for suppressing the logging mechanism.
-- Andreas
Am 12.05.2012 um 00:41 schrieb Andreas Rulle:
Hello Andreas,
this email informs you that
* with the tool mon (1), * that is often used together, see (3), the LVS load balancer (2)
it has been able to reproduce the
java.net.SocketException: Too many open files
with the following configuration
watch basex service ping description Responses to ping interval 2s monitor tcp.monitor -p 1984 period
It seems to be that basex does not close the sockets when monitored with tcp.monitor -p 1984. The basex-server crashes when ulimit -n is reached.
With best regards,
Andreas
(1) https://mon.wiki.kernel.org/index.php/Monitors
(2) http://www.linuxvirtualserver.org/
(3) http://www.austintek.com/LVS/LVS-HOWTO/HOWTO/LVS-HOWTO.realserver_failure.ht...
Am 11.05.2012 18:21, schrieb Andreas Rulle:
Hello Andreas,
thank you very much for your valuable hints.
sudo lsof -p basex-ps-no
delivers many lines of
java 11489 root 962u sock 0,6 0t0 10449645 can't identify protocol
and
(1) sudo lsof -p 11489 | wc -l gets
reports an increasing number of used "files". The increase of the number in (1) in a given time interval correlates to the number of requests that the LVM load balancer sends to the basex port 1984 in that interval. There are about 4 requests from the LVM load balancer per minute.
At the time of this writing (1) has the value of 1076 ...
The parameter keepalive has the value
KEEPALIVE = 600
but it does not seem to stop the increase of the value in (1).
This information opens the way to workarounds
- decrease the number of requests from the LVM load balancer, - increase the ulimit -n - restart the basex -server before the number in one reaches ulimt -n.
But we really would prefer solutions to the increase of the figure in (1)
The almost identical settings of the load balancer do work for a MySQL-Database without any problems. And without the load balancer an instance of the basex runs since April 10 without hitting ulimit -n = 1024.
Any hints on this are very welcome!
With kind regards,
Andreas
I just searched our mailing list, cause there was something about too many open files some time ago:
On typical Linux installations, die open filedescriptor limit is 1024:
$ grep 'open files' /proc/self/limits Max open files 1024 1024 files
In Java, if a file-based object (FileWriter, FileReader, etc.) is not closed, the underlying file descriptor is not closed. Bug detectors (FindBugs) check for that intraprocedurally. I recommend you run both BaseX as well as this application through it.
If an open file-based object becomes unreachable, the finalizer will eventually close it - but it's possible to run out of open file descriptors simply due to unreachable, but not yet finalized objects. (Of course, if the object is leaked, it won't be closed ever.)
In Linux, use 'ps' to find out the pid of the Java JVM process, then do a ls -l /proc/<pid>/fds to see which file descriptors the process in question has open; or use the 'lsof' command.
- Godmar Probably that helps to investigate the issue.
-- Nexoma GmbH Theodorus Weg 7 59755 Arnsberg
Tel. + 49 (0) 52 51 1613-0 Fax + 49 (0) 52 51 1613-99
mailto:andreas.rulle@nexoma.de
Geschäftsführer: Guido Sauerland Sitz der Gesellschaft: Arnsberg Registergericht: Arnsberg, HRB 9365
_______________________________________________ BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
-- Nexoma GmbH Theodorus Weg 7 59755 Arnsberg
Tel. + 49 (0) 52 51 1613-0 aktuell + 49 (0) 29 32 99 400 52 Fax + 49 (0) 52 51 1613-99
mailto:andreas.rulle@nexoma.de
Geschäftsführer: Guido Sauerland Sitz der Gesellschaft: Arnsberg Registergericht: Arnsberg, HRB 9365
On May 14, 2012, at 3:22 PM, Andreas Rulle wrote:
3. The tcpdump - viewed with wireshark - shows that the healthy check of the load balancer - mon and ldirectoryd do the same - sends a RST (Connection reset) message TCP to the BaseX database.
Each RST message from the mon or ldirectord seem to lead to a new socket that remains open ...
Any chance to take a look at that TCP dump? Regards, Dimitar
Hi there, i guess the problem is that the load balancer is just sending a message over the socket connection to the BaseX server, which does not correspond to the BaseX client-server protocol. So the client thread on the BaseX server side is waiting for the credentials in the appropriate form. The keepalive property does not work in this case, cause there is no "real" BaseX server-client connection to close in this moment. The BaseX server is not able to close the connection to the client cause the client thread on the server side is still in the reading loop for the incoming credentials. Regards, Andreas Am 14.05.2012 um 17:35 schrieb Dimitar Popov:
On May 14, 2012, at 3:22 PM, Andreas Rulle wrote:
3. The tcpdump - viewed with wireshark - shows that the healthy check of the load balancer - mon and ldirectoryd do the same - sends a RST (Connection reset) message TCP to the BaseX database.
Each RST message from the mon or ldirectord seem to lead to a new socket that remains open ...
Any chance to take a look at that TCP dump?
Regards, Dimitar _______________________________________________ BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
Hi all, I just proposed a fix (see pull request). Connections to clients in authentification phase are closed by the server now after `KEEPALIVE` seconds have passed. Kind regards from Lake Constance, Jens -- Jens Erat [phone]: tel:+49-151-56961126 [mail]: mailto:email@jenserat.de [jabber]: xmpp:jabber@jenserat.de [web]: http://www.jenserat.de PGP: 350E D9B6 9ADC 2DED F5F2 8549 CBC2 613C D745 722B Am 15.05.2012 um 09:37 schrieb Andreas Weiler:
Hi there,
i guess the problem is that the load balancer is just sending a message over the socket connection to the BaseX server, which does not correspond to the BaseX client-server protocol. So the client thread on the BaseX server side is waiting for the credentials in the appropriate form.
The keepalive property does not work in this case, cause there is no "real" BaseX server-client connection to close in this moment. The BaseX server is not able to close the connection to the client cause the client thread on the server side is still in the reading loop for the incoming credentials.
Regards, Andreas
Am 14.05.2012 um 17:35 schrieb Dimitar Popov:
On May 14, 2012, at 3:22 PM, Andreas Rulle wrote:
3. The tcpdump - viewed with wireshark - shows that the healthy check of the load balancer - mon and ldirectoryd do the same - sends a RST (Connection reset) message TCP to the BaseX database.
Each RST message from the mon or ldirectord seem to lead to a new socket that remains open ...
Any chance to take a look at that TCP dump?
Regards, Dimitar _______________________________________________ BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
_______________________________________________ BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
Hello Jens, your email has astonished me, that is a really fast reaction!
Hi all,
I just proposed a fix (see pull request). Connections to clients in authentification phase are closed by the server now after `KEEPALIVE` seconds have passed.
Kind regards from Lake Constance, Jens
1. If you want to do some further testing and need some support you are welcome to contact me. Just for your information the source code of the Ldirectord can be found at (1). The check by Ldirectord on the productive system is done with checkcommand=connect, so the actual check is performed in the method "sub check_connect" at (1). A tcpdump is attached and can be viewed with wireshark (2). 2. One of the questions that remain is when your fix might be available in a BaseX release version. 3. As a workaround, until the fix is in BaseX release, the Ldirectord can be used with the options ... checktype=external-perl checkcommand="/home/nexadmin/perl/Test.pl" ... And Test.pl could be a script like the following: # Script to test whether the BaseX-database is available # It return 1 in the case of failure, 0 otherwise use lib "/home/nexadmin/perl"; use BaseXClient; use warnings; use strict; eval { # create session my $session = Session->new("XXX.XXX.XXX.XXX", 1984, "admin", "admin"); # close session $session->close(); }; # print exception # print $@ if $@; exit 1 if$@; If there are comments from Perl experts to this version of the script, they are highly appreciated! With best regards Andreas (1) http://horms.net/projects/ldirectord/download/ldirectord-latest (2) http://www.wireshark.org/ -- Nexoma GmbH Theodorus Weg 7 59755 Arnsberg Tel. + 49 (0) 52 51 1613-0 aktuell + 49 (0) 29 32 99 400 52 Fax + 49 (0) 52 51 1613-99 mailto:andreas.rulle@nexoma.de Geschäftsführer: Guido Sauerland Sitz der Gesellschaft: Arnsberg Registergericht: Arnsberg, HRB 9365
Hi Andreas, seems you managed to get half of our team up and running.. ;) We have added a new option to limit the maximum time for connecting to the server [1]. By default, it is currently set to 10 seconds; after that, a client will be dismissed. As the implementation is still subject to change, your feedback is welcome (the changes have been discussed in Jens’ pull request [2]). Christian [1] http://docs.basex.org/wiki/Options#AUTHTIMEOUT [2] https://github.com/BaseXdb/basex/pull/501 ___________________________ On Tue, May 15, 2012 at 4:37 PM, Andreas Rulle <andreas.rulle@nexoma.de> wrote:
Hello Jens,
your email has astonished me, that is a really fast reaction!
Hi all,
I just proposed a fix (see pull request). Connections to clients in authentification phase are closed by the server now after `KEEPALIVE` seconds have passed.
Kind regards from Lake Constance, Jens
1. If you want to do some further testing and need some support you are welcome to contact me. Just for your information the source code of the Ldirectord can be found at (1).
The check by Ldirectord on the productive system is done with checkcommand=connect, so the actual check is performed in the method "sub check_connect" at (1). A tcpdump is attached and can be viewed with wireshark (2).
2. One of the questions that remain is when your fix might be available in a BaseX release version.
3. As a workaround, until the fix is in BaseX release, the Ldirectord can be used with the options ... checktype=external-perl checkcommand="/home/nexadmin/perl/Test.pl" ...
And Test.pl could be a script like the following:
# Script to test whether the BaseX-database is available # It return 1 in the case of failure, 0 otherwise use lib "/home/nexadmin/perl"; use BaseXClient; use warnings; use strict;
eval { # create session my $session = Session->new("XXX.XXX.XXX.XXX", 1984, "admin", "admin");
# close session $session->close(); };
# print exception # print $@ if $@; exit 1 if$@;
If there are comments from Perl experts to this version of the script, they are highly appreciated!
With best regards
Andreas
(1) http://horms.net/projects/ldirectord/download/ldirectord-latest (2) http://www.wireshark.org/
-- Nexoma GmbH Theodorus Weg 7 59755 Arnsberg
Tel. + 49 (0) 52 51 1613-0 aktuell + 49 (0) 29 32 99 400 52 Fax + 49 (0) 52 51 1613-99
mailto:andreas.rulle@nexoma.de
Geschäftsführer: Guido Sauerland Sitz der Gesellschaft: Arnsberg Registergericht: Arnsberg, HRB 9365
_______________________________________________ BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
Dear Andreas, have you been successful with the latest snapshots? Christian ___________________________ On Wed, May 16, 2012 at 1:11 AM, Christian Grün <christian.gruen@gmail.com> wrote:
Hi Andreas,
seems you managed to get half of our team up and running.. ;) We have added a new option to limit the maximum time for connecting to the server [1]. By default, it is currently set to 10 seconds; after that, a client will be dismissed. As the implementation is still subject to change, your feedback is welcome (the changes have been discussed in Jens’ pull request [2]).
Christian
[1] http://docs.basex.org/wiki/Options#AUTHTIMEOUT [2] https://github.com/BaseXdb/basex/pull/501 ___________________________
On Tue, May 15, 2012 at 4:37 PM, Andreas Rulle <andreas.rulle@nexoma.de> wrote:
Hello Jens,
your email has astonished me, that is a really fast reaction!
Hi all,
I just proposed a fix (see pull request). Connections to clients in authentification phase are closed by the server now after `KEEPALIVE` seconds have passed.
Kind regards from Lake Constance, Jens
1. If you want to do some further testing and need some support you are welcome to contact me. Just for your information the source code of the Ldirectord can be found at (1).
The check by Ldirectord on the productive system is done with checkcommand=connect, so the actual check is performed in the method "sub check_connect" at (1). A tcpdump is attached and can be viewed with wireshark (2).
2. One of the questions that remain is when your fix might be available in a BaseX release version.
3. As a workaround, until the fix is in BaseX release, the Ldirectord can be used with the options ... checktype=external-perl checkcommand="/home/nexadmin/perl/Test.pl" ...
And Test.pl could be a script like the following:
# Script to test whether the BaseX-database is available # It return 1 in the case of failure, 0 otherwise use lib "/home/nexadmin/perl"; use BaseXClient; use warnings; use strict;
eval { # create session my $session = Session->new("XXX.XXX.XXX.XXX", 1984, "admin", "admin");
# close session $session->close(); };
# print exception # print $@ if $@; exit 1 if$@;
If there are comments from Perl experts to this version of the script, they are highly appreciated!
With best regards
Andreas
(1) http://horms.net/projects/ldirectord/download/ldirectord-latest (2) http://www.wireshark.org/
-- Nexoma GmbH Theodorus Weg 7 59755 Arnsberg
Tel. + 49 (0) 52 51 1613-0 aktuell + 49 (0) 29 32 99 400 52 Fax + 49 (0) 52 51 1613-99
mailto:andreas.rulle@nexoma.de
Geschäftsführer: Guido Sauerland Sitz der Gesellschaft: Arnsberg Registergericht: Arnsberg, HRB 9365
_______________________________________________ BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
Hello Christian, sorry for the delay of the response. 1. Tests with the BaseX722-20120526.024641.zip show that a telnet connection is effectively interrupted with the new parameter AUTHTIMEOUT and no socket remains open, in contrast to version 7.2. 2. But I'm sorry to say Basex when tested by Ldirectord with the following checktimeout=10 negotiatetimeout=10 checkinterval=15 autoreload=yes quiescent=no logfile="local0" virtual=192.168.100.65:1984 real=192.168.100.62:1984 gate 10 scheduler=wlc quiescent=no protocol=tcp checktype=connect parameters and AUTHTIMEOUT = 10 it still leaves the sockets open. 3. If a telnet session is opened to the basex server and interrupted before AUTHTIMEOUT has expired then the socket remains open as well. 4. Since the productive system uses an outdated version of Ldirectord that does not support a connection test with an external (perl) script, we at the moment start * the basexserver on port 1984 * and the rest server on port 80 with login and password (This avoid http authentifaction of the ldirectord that it does not support.) The Ldirectord checks as a workaround the liveness of the database with an http service request to rest interface on port 80. 5. With this workaround the BaseX database can be monitored for the LVS load balancer. And this enables us to go productive ... We wish to thank you and your team for all dedication to his point and for the quality of the BaseX database in general. Please let me know if I can support in further analysing and testing for this topic. With best regards, Andreas
Dear Andreas, have you been successful with the latest snapshots? Christian ___________________________
On Wed, May 16, 2012 at 1:11 AM, Christian Grün <christian.gruen@gmail.com> wrote:
Hi Andreas,
seems you managed to get half of our team up and running.. ;) We have added a new option to limit the maximum time for connecting to the server [1]. By default, it is currently set to 10 seconds; after that, a client will be dismissed. As the implementation is still subject to change, your feedback is welcome (the changes have been discussed in Jens’ pull request [2]).
Christian
[1] http://docs.basex.org/wiki/Options#AUTHTIMEOUT [2] https://github.com/BaseXdb/basex/pull/501 ___________________________
On Tue, May 15, 2012 at 4:37 PM, Andreas Rulle<andreas.rulle@nexoma.de> wrote:
Hello Jens,
your email has astonished me, that is a really fast reaction!
Hi all,
I just proposed a fix (see pull request). Connections to clients in authentification phase are closed by the server now after `KEEPALIVE` seconds have passed.
Kind regards from Lake Constance, Jens
1. If you want to do some further testing and need some support you are welcome to contact me. Just for your information the source code of the Ldirectord can be found at (1).
The check by Ldirectord on the productive system is done with checkcommand=connect, so the actual check is performed in the method "sub check_connect" at (1). A tcpdump is attached and can be viewed with wireshark (2).
2. One of the questions that remain is when your fix might be available in a BaseX release version.
3. As a workaround, until the fix is in BaseX release, the Ldirectord can be used with the options ... checktype=external-perl checkcommand="/home/nexadmin/perl/Test.pl" ...
And Test.pl could be a script like the following:
# Script to test whether the BaseX-database is available # It return 1 in the case of failure, 0 otherwise use lib "/home/nexadmin/perl"; use BaseXClient; use warnings; use strict;
eval { # create session my $session = Session->new("XXX.XXX.XXX.XXX", 1984, "admin", "admin");
# close session $session->close(); };
# print exception # print $@ if $@; exit 1 if$@;
If there are comments from Perl experts to this version of the script, they are highly appreciated!
With best regards
Andreas
(1) http://horms.net/projects/ldirectord/download/ldirectord-latest (2) http://www.wireshark.org/
-- Nexoma GmbH Theodorus Weg 7 59755 Arnsberg
Tel. + 49 (0) 52 51 1613-0 aktuell + 49 (0) 29 32 99 400 52 Fax + 49 (0) 52 51 1613-99
mailto:andreas.rulle@nexoma.de
Geschäftsführer: Guido Sauerland Sitz der Gesellschaft: Arnsberg Registergericht: Arnsberg, HRB 9365
_______________________________________________ BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
-- Nexoma GmbH Theodorus Weg 7 59755 Arnsberg Tel. + 49 (0) 52 51 1613-0 aktuell + 49 (0) 29 32 99 400 52 Fax + 49 (0) 52 51 1613-99 mailto:andreas.rulle@nexoma.de Geschäftsführer: Guido Sauerland Sitz der Gesellschaft: Arnsberg Registergericht: Arnsberg, HRB 9365
Hi Andreas, just a short one: you may also want to experiment with the KEEPALIVE option [1], which specifies how long a session is kept open. Hope this helps, Christian [1] http://docs.basex.org/wiki/Options#KEEPALIVE On Sat, May 26, 2012 at 8:05 AM, Andreas Rulle <andreas.rulle@nexoma.de> wrote:
Hello Christian,
sorry for the delay of the response.
1. Tests with the BaseX722-20120526.024641.zip show that a telnet connection is effectively interrupted with the new parameter AUTHTIMEOUT and no socket remains open, in contrast to version 7.2.
2. But I'm sorry to say Basex when tested by Ldirectord with the following
checktimeout=10 negotiatetimeout=10 checkinterval=15 autoreload=yes quiescent=no logfile="local0"
virtual=192.168.100.65:1984 real=192.168.100.62:1984 gate 10 scheduler=wlc quiescent=no protocol=tcp checktype=connect
parameters and
AUTHTIMEOUT = 10
it still leaves the sockets open.
3. If a telnet session is opened to the basex server and interrupted before AUTHTIMEOUT has expired then the socket remains open as well.
4. Since the productive system uses an outdated version of Ldirectord that does not support a connection test with an external (perl) script, we at the moment start
* the basexserver on port 1984 * and the rest server on port 80 with login and password (This avoid http authentifaction of the ldirectord that it does not support.)
The Ldirectord checks as a workaround the liveness of the database with an http service request to rest interface on port 80.
5. With this workaround the BaseX database can be monitored for the LVS load balancer. And this enables us to go productive ...
We wish to thank you and your team for all dedication to his point and for the quality of the BaseX database in general. Please let me know if I can support in further analysing and testing for this topic.
With best regards,
Andreas
Dear Andreas, have you been successful with the latest snapshots? Christian ___________________________
On Wed, May 16, 2012 at 1:11 AM, Christian Grün <christian.gruen@gmail.com> wrote:
Hi Andreas,
seems you managed to get half of our team up and running.. ;) We have added a new option to limit the maximum time for connecting to the server [1]. By default, it is currently set to 10 seconds; after that, a client will be dismissed. As the implementation is still subject to change, your feedback is welcome (the changes have been discussed in Jens’ pull request [2]).
Christian
[1] http://docs.basex.org/wiki/Options#AUTHTIMEOUT [2] https://github.com/BaseXdb/basex/pull/501 ___________________________
On Tue, May 15, 2012 at 4:37 PM, Andreas Rulle<andreas.rulle@nexoma.de> wrote:
Hello Jens,
your email has astonished me, that is a really fast reaction!
Hi all,
I just proposed a fix (see pull request). Connections to clients in authentification phase are closed by the server now after `KEEPALIVE` seconds have passed.
Kind regards from Lake Constance, Jens
1. If you want to do some further testing and need some support you are welcome to contact me. Just for your information the source code of the Ldirectord can be found at (1).
The check by Ldirectord on the productive system is done with checkcommand=connect, so the actual check is performed in the method "sub check_connect" at (1). A tcpdump is attached and can be viewed with wireshark (2).
2. One of the questions that remain is when your fix might be available in a BaseX release version.
3. As a workaround, until the fix is in BaseX release, the Ldirectord can be used with the options ... checktype=external-perl checkcommand="/home/nexadmin/perl/Test.pl" ...
And Test.pl could be a script like the following:
# Script to test whether the BaseX-database is available # It return 1 in the case of failure, 0 otherwise use lib "/home/nexadmin/perl"; use BaseXClient; use warnings; use strict;
eval { # create session my $session = Session->new("XXX.XXX.XXX.XXX", 1984, "admin", "admin");
# close session $session->close(); };
# print exception # print $@ if $@; exit 1 if$@;
If there are comments from Perl experts to this version of the script, they are highly appreciated!
With best regards
Andreas
(1) http://horms.net/projects/ldirectord/download/ldirectord-latest (2) http://www.wireshark.org/
-- Nexoma GmbH Theodorus Weg 7 59755 Arnsberg
Tel. + 49 (0) 52 51 1613-0 aktuell + 49 (0) 29 32 99 400 52 Fax + 49 (0) 52 51 1613-99
mailto:andreas.rulle@nexoma.de
Geschäftsführer: Guido Sauerland Sitz der Gesellschaft: Arnsberg Registergericht: Arnsberg, HRB 9365
_______________________________________________ BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
-- Nexoma GmbH Theodorus Weg 7 59755 Arnsberg
Tel. + 49 (0) 52 51 1613-0 aktuell + 49 (0) 29 32 99 400 52 Fax + 49 (0) 52 51 1613-99
mailto:andreas.rulle@nexoma.de
Geschäftsführer: Guido Sauerland Sitz der Gesellschaft: Arnsberg Registergericht: Arnsberg, HRB 9365
participants (6)
-
Andreas Rulle -
Andreas Rulle -
Andreas Weiler -
Christian Grün -
Dimitar Popov -
Jens Erat