July 2012 - BaseX-Talk - mailman.uni-konstanz.de

Lucene with BaseX !!!
by John Best 18 Jul '12

18 Jul '12

Hi there, I have gone through BaseX and used it. But the of application which I am going to develop has 10+ GB XML data. When I indexed this using BaseX and tried certain searches, I found that the time taken to complete the search was quite big. Is there any way to lower the time and when are you going to module Lucene? -- Have a nice day JBest

2 1

Counting groups
by Pascal Heus 17 Jul '12

17 Jul '12

All: In a group by query expression like for $varGrp at $pos in local:getVariables($from,$to) group by $name := string($varGrp/@name) return <Something/> is there any way to get a value or position for the group count? In my particular use case, I used this to generate an HTML table and I would like to repeat the header every 20-30 rows (group). The "at $pos" variable can't be used as it holds a sequence of positions of the group members (which is what it's supposed to). Any suggestion appreciated. best *P

2 2

BaseX 7.3 CREATE DB command fails with: "database locked"
by Godmar Back 13 Jul '12

13 Jul '12

Hi, I"m using BaseX 7.3 using the client/server interface with the org.basex.ClientSession driver. After working for a while, BaseX starts failing with: Fri Jul 13 14:39:23 EDT 2012: 198.82.2.41:B38B8E21E30111A56E8A828453F4E768:z_orj: Unexpected exception: org.basex.core.BaseXException: Database 'PUB_feed_libx_dot_editions_at_gmail_dot_com_core' is currently opened by another client. org.basex.core.BaseXException: Database 'PUB_feed_libx_dot_editions_at_gmail_dot_com_core' is currently opened by another client. at org.basex.server.ClientSession.receive(ClientSession.java:259) at org.basex.server.ClientSession.send(ClientSession.java:246) at org.basex.server.ClientSession.send(ClientSession.java:284) at org.basex.server.ClientSession.create(ClientSession.java:129) This happens when I attempt a 'CREATE DB' command. The log shows: 14:39:23.094 [127.0.0.1:47974]: ts_20120713_dot_143922_dot_654_feed_libx_dot_editions_at_gmail_dot_com_core CREATE DB PUB_feed_libx_dot_editions_at_gmail_dot_com_core [...] 14:39:23.098 [127.0.0.1:47974] Error: Database 'PUB_feed_libx_dot_editions_at_gmail_dot_com_core' is currently opened by another client. 4.05 ms I don't understand what 'locking' or "opened by another client" refers to in this context - certainly not something I'm doing. I'm exclusively using the commands documented at http://docs.basex.org/wiki/Commands none of which is an 'open' or 'lock' command. Again, this makes BaseX unusable - we can't create a reliable server if database operations start failing after a while. This happened after a series of operations - it's not something I can easily create a test case for (due to the fact that BaseX's log are incomplete so I can't simply replay the dozens or so operations I did before BaseX locked up in this way.) It happens regularly, and has happened in BaseX 6.6.3 (I've waited to report it after I upgraded to 7.3). - Godmar

1 0

recusive subdirectory suggestion
by Kurt Conrad 13 Jul '12

13 Jul '12

First, I just downloaded and used BaseX to audit and report a variety of parent contexts across a set of about 2,500 XML documents. It was very painless. Many thanks. Second, I couldn't find a way to prevent recursive indexing of subdirectories. If the feature doesn't exist, it would be handy. In my case, I had the current set of files in the selected directory and old versions in subdirectories. Thanks, again. /s/ kwc 2012.07.12 22:34 ___________________________________________________________________ Kurt Conrad mailto:conrad@SagebrushGroup.com www.SagebrushGroup.com

2 1

can i edit xml-files?
by kaeptn haddock 12 Jul '12

12 Jul '12

Dear List Please forgive me for asking a stupid question: is it possible to not only browse, but also edit content of an xml-file with basex? When I browse a file I can right-click node-content, but the edit/delete etc. commands are all greyed out. IMHO it would be a great feature to be able to edit stuff... Best regards, Paull

3 3

How to contribute to basex? (Python 3.x and Python 2.7.3 client examples.)
by 秘湯 11 Jul '12

11 Jul '12

Hi all. Because no client examples for Python 3.x, I wrote it. The code is for Python 3.x, and further more, for Python 2.7.3 with improving performance (a little). I want to contribute, but I had not found entry to contribute. I don't know basex-talk is fit for the purpse, but i'll post, sorry. I hope these will be in https://github.com/BaseXdb/basex-api... (my work is not perfect especially to process binaries, so, additional reconstruction are wellcome. In addition, as you will see, i'm not native English speaker, so docstring and comments are fairly horrible...fix up, please.)

2 3

Fwd: Replication with BaseX
by Dirk Kirsten 11 Jul '12

11 Jul '12

Once again forwarding to the list as I am apparently unable to correctly use this Gmail-client... So for the sake of completeness see attached mails ---------- Forwarded message ---------- From: Dirk Kirsten <dk(a)basex.org> Date: Wed, Jul 11, 2012 at 12:20 PM Subject: Re: [basex-talk] Replication with BaseX To: Usama Dar <munir.usama(a)gmail.com> Oh, yeah, sure. You are absolutely right, sorry for misguiding you there. Yes, the indexes are on disk and will be transferred, so using rsync should be pretty performant. Cheers, Dirk On Wed, Jul 11, 2012 at 9:31 AM, Usama Dar <munir.usama(a)gmail.com> wrote: > Thanks Dirk. I noticed that all data structures are stored in the data > directory , so if i copy over the full data directory on a remote machine > and then continuously rsync them , do you think there would still be some > index re-building required? i thought the indexes like everything else are > on disk already? > > Best Regards, > Usama Dar > > > On Wed, Jul 11, 2012 at 9:02 AM, Dirk Kirsten <dk(a)basex.org> wrote: > >> Dear Usama, >> >> Currently there is no recommended strategy how to this. So there is no >> single right answer to your question. >> >> I think your strategy using rsync sounds very reasonable. Rsync has the >> nice advantage that it will be able to just transfer the incremental >> change, so you don't have to transfer the whole 500K documents all the >> time. However, you then still have to update the indexes, so depending on >> your data and your update interval this could take some time. >> >> Another very simple approach could be by using the BaseX Client-Server >> Infrastructure to send all commands and queries to both servers and trying >> to keep them synchronized. This could be more efficient, but this highly >> depends on your data and your queries (i.e. you use case). >> >> Cheers, >> Dirk >> >> On Tue, Jul 10, 2012 at 8:12 PM, Usama Dar <munir.usama(a)gmail.com> wrote: >> >>> Hi Guys, >>> >>> I am new to BaseX and i am going to use it to store 500K XML documents >>> which will be edited, Queried and stuff. I have already built the main >>> application. I have need to be able to run BaseX in a >>> high availability environment. For that i will initially have two servers >>> and i plan to use rsync on baseX data directory to replicate data. >>> >>> My question basically is are there any recommended strategies to do >>> this? is using rsync going to do the job? i can copy files over in the data >>> directory in a running instance? or the other server should be shutdown >>> while i copy files? any tool/ strategy you might know of? >>> >>> Thanks for your help >>> >>> Usama Dar. >>> >>> _______________________________________________ >>> BaseX-Talk mailing list >>> BaseX-Talk(a)mailman.uni-konstanz.de >>> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >>> >>> >> >

1 0

Re: [basex-talk] Displaying transformed XML into JEditorPane
by Anand Chiney (AIR) 11 Jul '12

11 Jul '12

Dear Christian, Yes there is SIMPLE GENERAL answer to the question asked by Sandeep. :-) Actually we are storing thousands of resumes in BaseX. These are stored as a single XML document with following format. <Resumes> <Resume>No 1</Resume> <Resume>No 2</Resume> <Resume>No </Resume> <Resume>No 4</Resume> <Resume></Resume> <Resumes> There are various child elements for <Resume>, which I think is not essential for this generalized answer. Till now I had explained about the Data. Now coming to front-end. We are using Swing based application to search and view these resumes. We want to display the resumes one-by-one in JEditorPane. For this we first transformed the XML to HTML using a XSLT. The BaseX editor can do this using XSLT Module. Christian, do you remember that, earlier there were 3 examples on this and we faced trouble using them. You then corrected these and it helped us a lot, But these were to work with BaseX editor. We need to perform this transformation using Java. Yesterday, Sandeep asked the question on this. We did found a good solution in these 2 examples, here https://github.com/BaseXdb/basex/blob/master/src/test/java/org/basex/test/qu ery/func/FNXsltTest.java https://github.com/BaseXdb/basex/blob/master/src/test/java/org/basex/test/qu ery/AdvancedQueryTest.java But this was not exactly what we were looking for. We need to modify this as per our requirement. And we did it :-) I am not going into deep to explain how we did and why due to less time. Rather I would simply paste the method which worked for us. ----------------------------- Method from the above Example (FNXsltTest.java )------------------------------------------------------------ @Test public void xsltTransform() { check(_XSLT_TRANSFORM); final String doc = "<a/>"; String style = wrap("<xsl:template match='/'><X/></xsl:template>"); query(_XSLT_TRANSFORM.args(doc, style), "<X/>"); query(_XSLT_TRANSFORM.args(doc, '"' + style + '"'), "<X/>"); style = wrap("<xsl:param name='t'/><xsl:template match='/'>" + "<X><xsl:value-of select='$t'/></X></xsl:template>"); final String param = "<xslt:parameters><xslt:t>1</xslt:t></xslt:parameters>"; query(_XSLT_TRANSFORM.args(doc, style, param), "<X>1</X>"); } ---------------------------------------------------------------------------- ---------------------------------------------------------------------------- ---------------- ----------------------------- Our Method ------------------------------------------------------------ public String xsltTransform() { return query(_XSLT_TRANSFORM.args(getDoc(), getStyleSheet())); } Explanation- getDoc() will send the query which will retrieve the required Resumes based on user search getStyleSheet() for resume.xslt let $in := /Resumes/Resume[WorkExpr/Java > "5"] let $style := doc('resume.xslt') return (xslt:transform($in, $style)) ---------------------------------------------------------------------------- ---------------------------------------------------------------------------- ---------------- ----------------------------- Method from the above Example (AdvancedQueryTest.java)------------------------------------------------- protected static String query(final String query) { final QueryProcessor qp = new QueryProcessor(query, CONTEXT); qp.ctx.sc.baseURI("."); try { return qp.execute().toString(); } catch (QueryException ex) { Logger.getLogger(DBTransform.class.getName()).log(Level.SEVERE, null, ex); return ex.getMessage(); } } We used this method as it is rather copy and paste Other methods after certain R&D, we found not useful in our case (!!) ---------------------------------------------------------------------------- ---------------------------------------------------------------------------- ---------------- So wrote a class with these 2 methods (for transformation) and other methods which were useful and it did work. But with a little trouble. This piece of code is NOT PRESERVING THE SPACES which we are giving through our stylesheet. Now for this either you can help us or we need to find a way out of it. Thanks for your kind and timely help. And we will trouble you more in future ;) Regards Anand Chiney _____ From: Sandeep Yohans [mailto:sandeepy.air@gmail.com] Sent: Wednesday, July 11, 2012 4:24 PM To: Anand Chiney Subject: Fwd: [basex-talk] Displaying transformed XML into JEditorPane ---------- Forwarded message ---------- From: Christian Grün <christian.gruen(a)gmail.com> Date: Wed, Jul 11, 2012 at 4:09 PM Subject: Re: [basex-talk] Displaying transformed XML into JEditorPane To: Sandeep Yohans <sandeepy.air(a)gmail.com> Cc: basex-talk(a)mailman.uni-konstanz.de Dear Sandeep, there is no simple general answer to your question, so I'd be glad to if you could prepare a little example [1] that demonstrates the problem. It often takes some time to generate a good and minimized example, but it's often helpful for other readers, too. Thanks in advance (and sorry for all the wise instructions ;), Christian [1] http://mindprod.com/jgloss/sscce.html _____________________________ On Tue, Jul 10, 2012 at 6:31 AM, Sandeep Yohans <sandeepy.air(a)gmail.com> wrote: > Hello, > > Below is the code to transform XML file to HTML. When tried with BaseX > working fine. > > > > let $in := /Resumes/Resume[WorkExpr/Java > "5"] > > let $style := doc('resume.xslt') > > return (xslt:transform($in, $style)) > > > > The aobve XSLT file will tramsform the output to HTML. > > Now I want to display this HTML output in JEditorPane. > > > > > -- > Thanks and Regards > > Sandeep Yohans > > _______________________________________________ > BaseX-Talk mailing list > BaseX-Talk(a)mailman.uni-konstanz.de > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk > -- Thanks and Regards Sandeep Yohans Software Developer Systems - AIR Infotech Nagpur - India www.airinfotech.in <http://www.airinfotech.in/> Mobile: +91-997-086-5520

2 1

Fwd: Displaying transformed XML into JEditorPane
by Dirk Kirsten 11 Jul '12

11 Jul '12

By mistake I answered privately to Sandeep, so for the sake of documentation here is my answer as well on the mailing list. ---------- Forwarded message ---------- From: Dirk Kirsten <dk(a)basex.org> Date: Wed, Jul 11, 2012 at 8:45 AM Subject: Re: [basex-talk] Displaying transformed XML into JEditorPane To: Sandeep Yohans <sandeepy.air(a)gmail.com> Dear Sandeep, I don't see how your question relates to BaseX? If the transformation is working fine and you have well-formed HTML output, then the rest is up to the application. Seems like you want to do some Swing-programming; displaying HTML in JEditorPane should work out-of-the-box, but please have a look at the Java-API, e.g.: http://docs.oracle.com/javase/7/docs/api/javax/swing/JEditorPane.html Cheers, Dirk On Tue, Jul 10, 2012 at 6:31 AM, Sandeep Yohans <sandeepy.air(a)gmail.com>wrote: > Hello, > > Below is the code to transform XML file to HTML. When tried with BaseX > working fine. > > > > let $in := /Resumes/Resume[WorkExpr/Java > "5"] > > let $style := doc('resume.xslt') > > return (xslt:transform($in, $style)) > > > > *The aobve XSLT file will tramsform the output to HTML.* > > *Now I want to display this HTML output in JEditorPane.* > > * * > ** > -- > Thanks and Regards > > Sandeep Yohans > > _______________________________________________ > BaseX-Talk mailing list > BaseX-Talk(a)mailman.uni-konstanz.de > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk > >

1 0

Displaying transformed XML into JEditorPane
by Sandeep Yohans 11 Jul '12

11 Jul '12

Hello, Below is the code to transform XML file to HTML. When tried with BaseX working fine. let $in := /Resumes/Resume[WorkExpr/Java > "5"] let $style := doc('resume.xslt') return (xslt:transform($in, $style)) *The aobve XSLT file will tramsform the output to HTML.* *Now I want to display this HTML output in JEditorPane.* * * ** -- Thanks and Regards Sandeep Yohans

2 1