BaseX-Talk May 2012

basex-talk@mailman.uni-konstanz.de

48 participants
64 discussions

Question about Servlet accessing BaseX
by Anupam Bakshi 29 May '12

29 May '12

Hello, I'm creating my first BaseX based app and have a few questions about the best possible way to architect it around BaseX. I need to use BaseX as a service in a Tomcat container on the same server machine. I have the following doubts: 1. I need to validate the XML coming into the database and do XSLT transformation. Should this be done in a servlet that connects to BaseX? 2. If so, should the servlet use REST too, or is there a better way for direct connection? 3. Since BaseX is running in Tomcat context, is it possible to access it using a local client on the server machine- for debug etc on the server machine? Thank you for your help. Best regards, Anupam

2 3

Who is using...?
by Christian Grün 28 May '12

28 May '12

Hi all, we are interested in your opinion! As one of our main focus is to keep BaseX small and cute (..sad but true: the BaseX core library already exceeds 2mb.. ;), we're currently contemplating the removal of two existing BaseX features, namely.. 1) the BaseXHTTP client mode (flag -c). Currently, we offer three operation modes: default, local, and client [1]. In client mode, all requests are supposed to be passed on to yet another BaseX server instance. Currently, however, WebDAV is the only service which is 100% compatible with this mode, while RESTXQ isn't tailored at all to handle remote database requests. Instead, we are contemplating on adding a light-weight XQuery Client API Module, which can then be used to explicitly address remote BaseX instances from within XQuery whenever databases are not stored locally anyway. 2) the current ZIP Module. The functionality won't disappear, though: we will replace it with a new, more flexible proposal specified by Matthias Branter (28msec) and me [2]. In a nutshell: who of you is using either 1) or 2) and would run into big troubles if those features were ousted and replaced with the approaches we believe are more promising for future development? Thanks as usual for your feedback, either to the list (preferred) or in private, Christian [1] http://docs.basex.org/wiki/Startup_Options#BaseX_HTTP_Server [2] http://spex.basex.org/index.php?title=ZIP_Module

6 8

bindings command in GUI not functioning as expected
by Brown, David (SSC-A) 28 May '12

28 May '12

Running 7.2.2 beta The standalone interface takes the following input: "set bindings $date=20120507, $envir=PRD_Win" and returns "bindings: $date=20120507, $envir=PRD_Win" The GUI interface on the command bar takes the following input: "set bindings $date=20120507" and returns "bindings: $date=20120507" "set bindings $date=20120507, $envir=PRD_Win" and returns "Error: Syntax: SET [option] ([value]) Set global options." This is a problem if you have more than one binding required. v/r, David Brown Fully Qualified Navy Certification Agent LSS Certified Blackbelt, CISSP, NSA-IAM/IEM, MSCIS SPAWARSYSCEN ATLANTIC ISS LOB SSC Lead Email: <mailto:david.k.brown1@navy.mil> david.k.brown1(a)navy.mil Phone: 843.218.6227 DSN: 588.6227 Fax: 843.218.5461 "Let us not look back in anger or forward in fear, but around in awareness." James Thurber

2 1

Connecting to BaseX running as a Tomcat servlet
by Anupam Bakshi 28 May '12

28 May '12

Hello, I'm using "net.xqj.basex.BaseXXQDataSource" to connect to a BaseX servlet running in Tomcat but it doesn't connect. I can run the following command : http://localhost:8080/BaseX721/rest?command=info Main Options DBPATH: C:\Apache Tomcat 6.0.20\webapps\BaseX721\\data DEBUG: false EVENTPORT: 1985 HOST: localhost HTTPPATH: C:\Apache Tomcat 6.0.20\webapps\BaseX721\ HTTPPORT: 8984 KEEPALIVE: 0 LANG: English LANGKEYS: false NONPROXYHOSTS: PARALLEL: 8 PORT: 1984 PROXYHOST: PROXYPORT: 80 REPOPATH: C:\Apache Tomcat 6.0.20\webapps\BaseX721\\repo SERVERHOST: SERVERPORT: 1984 STOPPORT: 8985 TIMEOUT: 0 But when I run : private static final String DRIVER = "net.xqj.basex.BaseXXQDataSource"; XQDataSource xqds = (XQDataSource) Class.forName(DRIVER).newInstance(); xqds.setProperty("serverName", "localhost"); xqds.setProperty("port", "1985"); I get : Exception in thread "main" javax.xml.xquery.XQException: XQJNC001 - Connection refused: connect Root Cause: java.net.ConnectException: Connection refused: connect Is this even possible? Any pointers will be greatly appreciated. Thanks, Anupam

2 1

error message
by Brown, David (SSC-A) 28 May '12

28 May '12

The first 8 characters of the root is a folder name. I want to group by folder name. I simplified the xquery to eliminate the collateral interactions, but still receive this error. p.s. I was able to get bindings working using quotes: open C4;set QUERYPATH c:\dev\xq;set bindings "$date=20120507, $envir=PRD_Win" Source scheme: <20120507/PRD_Win.nessus> <NessusClientData_v2> <Policy> <Report> <ReportHost> < ReportItem> ------------------- Query: declare variable $date external; <results> { for $x in //Report/ReportHost/ReportItem let $src := fn:root($x) return <summary> <source>substring(fn:name({$src}),1,8)</source> </summary> }</results> Query plan: <QueryPlan> <VarStack> <Var name="$date" id="0"> <Atm value="20120507" type="xs:untypedAtomic"/> </Var> <Var name="$envir" id="1"> <Atm value="PRD_Win" type="xs:untypedAtomic"/> </Var> </VarStack> <CElem> <QNm value="results" type="xs:QName"/> <FLWR> <For var="$x"> <AxisPath> <DBNodeSeq size="28"> <DBNode name="C4" pre="0"/> <DBNode name="C4" pre="21934"/> <DBNode name="C4" pre="74441"/> <DBNode name="C4" pre="121967"/> <DBNode name="C4" pre="145433"/> </DBNodeSeq> <IterStep axis="descendant" test="Report"/> <IterStep axis="child" test="ReportHost"/> <IterStep axis="child" test="ReportItem"/> </AxisPath> </For> <Let var="$src"> <FNNode name="root([node])"> <VarRef> <Var name="$x" id="3"/> </VarRef> </FNNode> </Let> <Return> <CElem> <QNm value="summary" type="xs:QName"/> <CElem> <QNm value="source" type="xs:QName"/> <Str value="substring(fn:name(" type="xs:string"/> <VarRef> <Var name="$src" id="4"/> </VarRef> <Str value="),1,8)" type="xs:string"/> </CElem> </CElem> </Return> </FLWR> </CElem> </QueryPlan> Error: Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk(a)mailman.uni-konstanz.de Version: BaseX 7.2.2 beta Java: Oracle Corporation, 1.7.0 OS: Windows 7, amd64 Stack Trace: java.lang.ArrayIndexOutOfBoundsException v/r, David Brown Fully Qualified Navy Certification Agent LSS Certified Blackbelt, CISSP, NSA-IAM/IEM, MSCIS SPAWARSYSCEN ATLANTIC ISS LOB SSC Lead Email: <mailto:david.k.brown1@navy.mil> david.k.brown1(a)navy.mil Phone: 843.218.6227 DSN: 588.6227 Fax: 843.218.5461 "Let us not look back in anger or forward in fear, but around in awareness." James Thurber

2 1

Output xquery to xml file
by Brown, David (SSC-A) 27 May '12

27 May '12

Sorry to ask such a basic question, but: I am using your product to read through a large set of xml files and output a simplified xml file. It works well manually, but would like to automate this process. I want all the query files to resolve to an output, each their own patterned file name. I have configured the query file to accept external variables, but as is stated in my previous message the bind function is not working for me just now. given that bind works I can use the variable values to create the output file name value. if I could get the query to create an xml file instead of output to console. v/r, David Brown Fully Qualified Navy Certification Agent LSS Certified Blackbelt, CISSP, NSA-IAM/IEM, MSCIS SPAWARSYSCEN ATLANTIC ISS LOB SSC Lead Email: <mailto:david.k.brown1@navy.mil> david.k.brown1(a)navy.mil Phone: 843.218.6227 DSN: 588.6227 Fax: 843.218.5461 "Let us not look back in anger or forward in fear, but around in awareness." James Thurber

2 1

XQuery Client API. WAS: whitespace handling
by Dimitar Popov 27 May '12

27 May '12

Am Freitag, 25. Mai 2012, 17:37:03 schrieb Christian Grün: > ..fertig ;) Zumindest mit Version 1; Tests are welcome: > > http://docs.basex.org/wiki/Client_Module > > > und ein paar weitere Punkte die man vlt. später umsetzen könnte: > > 1. Connection Registry: die Idee ist die Einzelheiten (Adresse, Username, > > Passwort) für eine Verbindung irgendwo zentral zu speichern und dann > > mittels eines Identifikators eine neue Verbindung zu erstellen, z.B. > > Richtig; momentan stört mich auch, dass das Passwort im Klartext in > den Queries steht. Fraglich bei der Registry-Lösung ist vermutlich, > wie man das am einfachsten umsetzt. Hast Du Ideen hierzu? > > Christian spontant würde ich sagen, irgendwo die Connection-Strings zu speichern (z.B. in einer Systemdatenbank - so könnte man ohne SSH oder Kommando- zeile BaseX einstellen... aber das ist ein anderes Thema). Man könnte auch die MD5 Summen statt die Passwörte speichern oder sogar an die Funktion angeben. Grüße, Dimitar

3 2

Who is using... ?
by Fabrice Etanchaud 26 May '12

26 May '12

Dear Christian, 1. Will BaseX clients still able to connect to the BaseX server serving HHTP requests ? This XQuery Client API Module is a very good idea ! 2. IMHO, BaseX should concentrate on processing xml, I stopped using ZIP module when I realized db:add() was doing the job, in streaming mode. Great job ! Best regards, Fabrice ________________________________ Hi all, we are interested in your opinion! As one of our main focus is to keep BaseX small and cute (..sad but true: the BaseX core library already exceeds 2mb.. ;), we're currently contemplating the removal of two existing BaseX features, namely.. 1) the BaseXHTTP client mode (flag -c). Currently, we offer three operation modes: default, local, and client [1]. In client mode, all requests are supposed to be passed on to yet another BaseX server instance. Currently, however, WebDAV is the only service which is 100% compatible with this mode, while RESTXQ isn't tailored at all to handle remote database requests. Instead, we are contemplating on adding a light-weight XQuery Client API Module, which can then be used to explicitly address remote BaseX instances from within XQuery whenever databases are not stored locally anyway. 2) the current ZIP Module. The functionality won't disappear, though: we will replace it with a new, more flexible proposal specified by Matthias Branter (28msec) and me [2]. In a nutshell: who of you is using either 1) or 2) and would run into big troubles if those features were ousted and replaced with the approaches we believe are more promising for future development? Thanks as usual for your feedback, either to the list (preferred) or in private, Christian [1] http://docs.basex.org/wiki/Startup_Options#BaseX_HTTP_Server [2] http://spex.basex.org/index.php?title=ZIP_Module Fabrice ETANCHAUD Senior Software Engineer edital Berkenlaan 1 B-1831 Brussels, Belgium <skype:fabrice.etanchaud?call> +32 2 716 32 32 general +32 2 716 32 20 fax fetanchaud(a)edital.com corsearch.com <http://www.corsearch.com/> | edital.com <http://www.edital.com/> <outbind://899-00000000E439C83B50BF17409D8C27B7AE0425AF07005AA868536216F 543BAB9422EAEA5338B00000011020900005AA868536216F543BAB9422EAEA5338B00000 13AA36B0000/cid:image001.gif@01CC0F02.2C91B6C0> Confidentiality Notice: This email and its attachments (if any) contain confidential information of the sender. The information is intended only for the use by the direct addressees of the original sender of this email. If you are not an intended recipient of the original sender (or responsible for delivering the message to such person), you are hereby notified that any review, disclosure, copying, distribution or the taking of any action in reliance of the contents of and attachments to this email is strictly prohibited. If you have received this email in error, please immediately notify the sender at the address shown herein and permanently delete any copies of this email (digital or paper) in your possession.

2 1

Re: [basex-talk] Too Many Open Files
by Andreas Rulle 26 May '12

26 May '12

Hello Andreas, thank you very much for your valuable hints. sudo lsof -p basex-ps-no delivers many lines of java 11489 root 962u sock 0,6 0t0 10449645 can't identify protocol and (1) sudo lsof -p 11489 | wc -l gets reports an increasing number of used "files". The increase of the number in (1) in a given time interval correlates to the number of requests that the LVM load balancer sends to the basex port 1984 in that interval. There are about 4 requests from the LVM load balancer per minute. At the time of this writing (1) has the value of 1076 ... The parameter keepalive has the value KEEPALIVE = 600 but it does not seem to stop the increase of the value in (1). This information opens the way to workarounds - decrease the number of requests from the LVM load balancer, - increase the ulimit -n - restart the basex -server before the number in one reaches ulimt -n. But we really would prefer solutions to the increase of the figure in (1) The almost identical settings of the load balancer do work for a MySQL-Database without any problems. And without the load balancer an instance of the basex runs since April 10 without hitting ulimit -n = 1024. Any hints on this are very welcome! With kind regards, Andreas > I just searched our mailing list, cause there was something about too many open files some time ago: > >> On typical Linux installations, die open filedescriptor limit is 1024: >> >> $ grep 'open files' /proc/self/limits >> Max open files 1024 1024 files >> >> In Java, if a file-based object (FileWriter, FileReader, etc.) is not closed, the underlying file descriptor is not closed. Bug detectors (FindBugs) check for that intraprocedurally. I recommend you run both BaseX as well as this application through it. >> >> If an open file-based object becomes unreachable, the finalizer will eventually close it - but it's possible to run out of open file descriptors simply due to unreachable, but not yet finalized objects. (Of course, if the object is leaked, it won't be closed ever.) >> >> In Linux, use 'ps' to find out the pid of the Java JVM process, then do a ls -l /proc/<pid>/fds to see which file descriptors the process in question has open; or use the 'lsof' command. >> >> - Godmar > Probably that helps to investigate the issue. > > -- Nexoma GmbH Theodorus Weg 7 59755 Arnsberg Tel. + 49 (0) 52 51 1613-0 Fax + 49 (0) 52 51 1613-99 mailto:andreas.rulle@nexoma.de Geschäftsführer: Guido Sauerland Sitz der Gesellschaft: Arnsberg Registergericht: Arnsberg, HRB 9365

6 11

XML file with xmlns attribute and XPATH
by Philippe Rathé 25 May '12

25 May '12

Hello there, I just want to do a simple xpath expression but I can't get it works when I use this XML document <?xml version="1.0" encoding='utf-8'?> <feed xmlns="http://example.org" xmlns:abc="http://abc.example.org"> <title>Projects</title> </feed> The expression "exquery //title" does not return anything from the CLI once I open the database containing only that file. But it works with this file <?xml version="1.0" encoding='utf-8'?> <feed xmlns="http://example.org"> <title>Projects</title> </feed> It looks like having both xmlns and xmlns:abc namespace attribute creates a problem. My real use case is atompub feed document which the 2 first lines looks like this: <?xml version="1.0" encoding='utf-8'?> <feed xmlns="http://www.w3.org/2005/Atom" xmlns:app="http://www.w3.org/2007/app"> Any idea? I'm using a 7.2.2 snapshot, thanks.

3 6

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

BaseX-Talk May 2012