Hello,
I am importing BaseX, with tons of XML files. Currently I have roughly 1600 databases, I am starting basexhttp service, to access it over a web service endpoint, thru a xquery file. Using BaseX 8.2.3.
I am receiving below error:
[bxerr:BXDB0002] Resource "/veracode/msheth/BaseXDB/bi_output_713/inf.basex (Too many open files)" not found.
basexhttp, is running with 10240M virtual memory.
I can share the xquery file, if thats needed.
Has anyone experienced this before ? Is there a limit on no of databases supported by BaseX ? Is there some configuration option, which I can use to close already queried database ?
Thanks, - Mansi
Hi Mansi,
If you are working with Linux, you may need to increase the maximum file limit with "ulimit -n" [1].
Hope this helps, Christian
[1] http://www.linuxhowtos.org/Tips%20and%20Tricks/ulimit.htm
On Fri, Dec 4, 2015 at 7:52 PM, Mansi Sheth mansi.sheth@gmail.com wrote:
Hello,
I am importing BaseX, with tons of XML files. Currently I have roughly 1600 databases, I am starting basexhttp service, to access it over a web service endpoint, thru a xquery file. Using BaseX 8.2.3.
I am receiving below error:
[bxerr:BXDB0002] Resource "/veracode/msheth/BaseXDB/bi_output_713/inf.basex (Too many open files)" not found.
basexhttp, is running with 10240M virtual memory.
I can share the xquery file, if thats needed.
Has anyone experienced this before ? Is there a limit on no of databases supported by BaseX ? Is there some configuration option, which I can use to close already queried database ?
Thanks,
- Mansi
Thanks Christian,
I had already set open files limit on the OS:
ubuntu@ip-10-0-0-83:~$ ulimit -Hn
9999
However, I still face exact same problem. Process breaks at the same db count
[bxerr:BXDB0002] Resource "/veracode/msheth/BaseXDB/bi_output_715/inf.basex (Too many open files)" not found.
On Sat, Dec 5, 2015 at 7:53 AM, Christian Grün christian.gruen@gmail.com wrote:
Hi Mansi,
If you are working with Linux, you may need to increase the maximum file limit with "ulimit -n" [1].
Hope this helps, Christian
[1] http://www.linuxhowtos.org/Tips%20and%20Tricks/ulimit.htm
On Fri, Dec 4, 2015 at 7:52 PM, Mansi Sheth mansi.sheth@gmail.com wrote:
Hello,
I am importing BaseX, with tons of XML files. Currently I have roughly
1600
databases, I am starting basexhttp service, to access it over a web
service
endpoint, thru a xquery file. Using BaseX 8.2.3.
I am receiving below error:
[bxerr:BXDB0002] Resource
"/veracode/msheth/BaseXDB/bi_output_713/inf.basex
(Too many open files)" not found.
basexhttp, is running with 10240M virtual memory.
I can share the xquery file, if thats needed.
Has anyone experienced this before ? Is there a limit on no of databases supported by BaseX ? Is there some configuration option, which I can use
to
close already queried database ?
Thanks,
- Mansi
Hi Mansi,
The results of ulimit can be misleading. See this article - which really helped me when I encountered this issue (though not with BaseX):
https://underyx.me/2015/05/18/raising-the-maximum-number-of-file-descriptors...
Joe
On Mon, Dec 7, 2015 at 1:22 PM, Mansi Sheth mansi.sheth@gmail.com wrote:
Thanks Christian,
I had already set open files limit on the OS:
ubuntu@ip-10-0-0-83:~$ ulimit -Hn
9999
However, I still face exact same problem. Process breaks at the same db count
[bxerr:BXDB0002] Resource "/veracode/msheth/BaseXDB/bi_output_715/inf.basex (Too many open files)" not found.
On Sat, Dec 5, 2015 at 7:53 AM, Christian Grün christian.gruen@gmail.com wrote:
Hi Mansi,
If you are working with Linux, you may need to increase the maximum file limit with "ulimit -n" [1].
Hope this helps, Christian
[1] http://www.linuxhowtos.org/Tips%20and%20Tricks/ulimit.htm
On Fri, Dec 4, 2015 at 7:52 PM, Mansi Sheth mansi.sheth@gmail.com wrote:
Hello,
I am importing BaseX, with tons of XML files. Currently I have roughly
1600
databases, I am starting basexhttp service, to access it over a web
service
endpoint, thru a xquery file. Using BaseX 8.2.3.
I am receiving below error:
[bxerr:BXDB0002] Resource
"/veracode/msheth/BaseXDB/bi_output_713/inf.basex
(Too many open files)" not found.
basexhttp, is running with 10240M virtual memory.
I can share the xquery file, if thats needed.
Has anyone experienced this before ? Is there a limit on no of databases supported by BaseX ? Is there some configuration option, which I can
use to
close already queried database ?
Thanks,
- Mansi
--
- Mansi
Thanks Joe for your input. I haven't tried all the options yet, but will surely go thru.
I guess, what I was trying to see is, if there is a way I can optimize my XQUERIES, to close open databases which it not longer needs. Currently, my queries are something of below nature. I am thinking, if there is a better way to deal with *bold* piece of code below.
declare variable $n as xs:string external; declare option output:item-separator "
";
let $queryData := for $db in db:list()
* let $query := "declare variable $db external; " || "db:open($db)" || $n* * return xquery:eval($query,map { 'db': $db, 'query': $n })*
return distinct-values($queryData)
On Mon, Dec 7, 2015 at 1:30 PM, Joe Wicentowski joewiz@gmail.com wrote:
Hi Mansi,
The results of ulimit can be misleading. See this article - which really helped me when I encountered this issue (though not with BaseX):
https://underyx.me/2015/05/18/raising-the-maximum-number-of-file-descriptors...
Joe
On Mon, Dec 7, 2015 at 1:22 PM, Mansi Sheth mansi.sheth@gmail.com wrote:
Thanks Christian,
I had already set open files limit on the OS:
ubuntu@ip-10-0-0-83:~$ ulimit -Hn
9999
However, I still face exact same problem. Process breaks at the same db count
[bxerr:BXDB0002] Resource "/veracode/msheth/BaseXDB/bi_output_715/inf.basex (Too many open files)" not found.
On Sat, Dec 5, 2015 at 7:53 AM, Christian Grün <christian.gruen@gmail.com
wrote:
Hi Mansi,
If you are working with Linux, you may need to increase the maximum file limit with "ulimit -n" [1].
Hope this helps, Christian
[1] http://www.linuxhowtos.org/Tips%20and%20Tricks/ulimit.htm
On Fri, Dec 4, 2015 at 7:52 PM, Mansi Sheth mansi.sheth@gmail.com wrote:
Hello,
I am importing BaseX, with tons of XML files. Currently I have roughly
1600
databases, I am starting basexhttp service, to access it over a web
service
endpoint, thru a xquery file. Using BaseX 8.2.3.
I am receiving below error:
[bxerr:BXDB0002] Resource
"/veracode/msheth/BaseXDB/bi_output_713/inf.basex
(Too many open files)" not found.
basexhttp, is running with 10240M virtual memory.
I can share the xquery file, if thats needed.
Has anyone experienced this before ? Is there a limit on no of
databases
supported by BaseX ? Is there some configuration option, which I can
use to
close already queried database ?
Thanks,
- Mansi
--
- Mansi
So, increasing ulimit, based on below blog helped.
Thanks,
- Mansi
On Dec 7, 2015, at 1:30 PM, Joe Wicentowski joewiz@gmail.com wrote:
Hi Mansi,
The results of ulimit can be misleading. See this article - which really helped me when I encountered this issue (though not with BaseX):
https://underyx.me/2015/05/18/raising-the-maximum-number-of-file-descriptors...
Joe
On Mon, Dec 7, 2015 at 1:22 PM, Mansi Sheth mansi.sheth@gmail.com wrote: Thanks Christian,
I had already set open files limit on the OS:
ubuntu@ip-10-0-0-83:~$ ulimit -Hn 9999
However, I still face exact same problem. Process breaks at the same db count [bxerr:BXDB0002] Resource "/veracode/msheth/BaseXDB/bi_output_715/inf.basex (Too many open files)" not found.
On Sat, Dec 5, 2015 at 7:53 AM, Christian Grün christian.gruen@gmail.com wrote: Hi Mansi,
If you are working with Linux, you may need to increase the maximum file limit with "ulimit -n" [1].
Hope this helps, Christian
[1] http://www.linuxhowtos.org/Tips%20and%20Tricks/ulimit.htm
On Fri, Dec 4, 2015 at 7:52 PM, Mansi Sheth mansi.sheth@gmail.com wrote:
Hello,
I am importing BaseX, with tons of XML files. Currently I have roughly 1600 databases, I am starting basexhttp service, to access it over a web service endpoint, thru a xquery file. Using BaseX 8.2.3.
I am receiving below error:
[bxerr:BXDB0002] Resource "/veracode/msheth/BaseXDB/bi_output_713/inf.basex (Too many open files)" not found.
basexhttp, is running with 10240M virtual memory.
I can share the xquery file, if thats needed.
Has anyone experienced this before ? Is there a limit on no of databases supported by BaseX ? Is there some configuration option, which I can use to close already queried database ?
Thanks,
- Mansi
--
- Mansi
basex-talk@mailman.uni-konstanz.de
-
Christian Grün -
Joe Wicentowski -
Mansi -
Mansi Sheth