- BaseX-Talk - mailman.uni-konstanz.de

Re: [basex-talk] BaseX and Git versioning of XML at the same
by Enea Parimbelli 10 Feb '16

10 Feb '16

Thanks Dirk and Eliot, Dirk's suggestion about the WRITEBACK option looks very promising, I'm gonna try it out but I guess it would fit my needs. Update: just tried it and it seems to work also on docs previously imported in a database and retrieved using db:open("DBname", "filename") so that's even better than providing the full path. I was well aware of the bad performances caused by working outside the optimized basex data model (hard to beat a key-value optimized datastore) but performances are not an issue for my specific application. Eliot's project sounds also very cool but I think it's a bit too much of an effort for the initial proof-of-concept tests that I have in mind. Thanks for the great suggestions and timely support... that was FAST :) Best, Enea 2016-02-10 13:55 GMT+01:00 <basex-talk-request(a)mailman.uni-konstanz.de>: > Send BaseX-Talk mailing list submissions to > basex-talk(a)mailman.uni-konstanz.de > > To subscribe or unsubscribe via the World Wide Web, visit > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk > or, via email, send a message with subject or body 'help' to > basex-talk-request(a)mailman.uni-konstanz.de > > You can reach the person managing the list at > basex-talk-owner(a)mailman.uni-konstanz.de > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of BaseX-Talk digest..." > > > Today's Topics: > > 1. BaseX and Git versioning of XML at the same time (Enea Parimbelli) > 2. Re: BaseX and Git versioning of XML at the same time > (Dirk Kirsten) > 3. Re: BaseX and Git versioning of XML at the same time > (Eliot Kimber) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Wed, 10 Feb 2016 13:33:58 +0100 > From: Enea Parimbelli <enea.parimbelli(a)gmail.com> > To: basex-talk(a)mailman.uni-konstanz.de > Subject: [basex-talk] BaseX and Git versioning of XML at the same time > Message-ID: > <CABz3M_P4mWEsG20tws+UiCXoTbzXTEQqzX7-Ew=+kJJq= > pZsYg(a)mail.gmail.com> > Content-Type: text/plain; charset="utf-8" > > Dear all, > I'm Enea from Italy (working at the university of Pavia) and this is my > first email to this list and I hope you'll forgive my dumb question. > For one of my projects I was planning to use BaseX to be able to run > Xqueries to manipulate a bunch of XMLs that I have on my filesystem (i.e. > to take advantage of what XQuery can do vs manual editing of the xml > files). > At the same time I would like to have a version control system (Git in my > case) control my files to be able to diff, branch, revert changes etc. > However the way BaseX internally stores files is preventing me from doing > this easily. > > Here is an example of what I have in mind: > 0. I have a bunch of xmls in a folder (versioned with a git repo) > 1. I create a baseX database from these > 2. Run a bunch of updates/edits on the files using Xquery processor from > basex > 3. move back to git to diff what I've accomplished, commit, etc. > 4. (possibly more advanced, not sure if I really need this) I want to be > able to checkout a previous version of the files and have basex do the same > (i.e. if I run a query now I want it to run on the version of the xmls that > I have currently checked-out in my filesystem) > > Here are the (dumb) questions: > 1. Is there a way for basex (maybe some config settings when I create the > database?) to work keeping on the plain xml files? In a way that every > update query would actually change the original xml files (which I can view > with a plain text editor even "outside" the basex environment)? > 2. Is the only way of immediately seeing the changes I made to the xml > files to export the database after every update? > 3. (assuming 1 is doable) Would it be possible to have baseX recognize the > changes made to the xml files using an external editor (i.e. not through > xqueries run in basex)? > 4. From your experience is BaseX the proper tool for my purposes? It feels > to me that I merely want to use its xquery processor capabilities (and I > don't need the full-fledged database) while keeping the files plainly > stored in my filesystem... any suggestion on alternative options if baseX > doesn't sound like the right one? > > Thanks in advance for the help, > and congratulations on the great work (I used basex in the past in a more > orthodox way and was impressed by the numerous nice features... including > being the nice GUI and the hassle-free integration with tomcat!) > > Best regards, > Enea Parimbelli > > -- > Dr. Enea Parimbelli > Post-doctoral research fellow > Laboratory for Biomedical Informatics "Mario Stefanelli" > Department of Electrical, Computer and Biomedical Engineering > University of Pavia, Italy > e-mail: enea.parimbelli(a)gmail.com > phone: +39-0382-985057 +39-0382-985981 > http://www.labmedinfo.org/people/parimbelli >

1 0

BaseX and Git versioning of XML at the same time
by Enea Parimbelli 10 Feb '16

10 Feb '16

Dear all, I'm Enea from Italy (working at the university of Pavia) and this is my first email to this list and I hope you'll forgive my dumb question. For one of my projects I was planning to use BaseX to be able to run Xqueries to manipulate a bunch of XMLs that I have on my filesystem (i.e. to take advantage of what XQuery can do vs manual editing of the xml files). At the same time I would like to have a version control system (Git in my case) control my files to be able to diff, branch, revert changes etc. However the way BaseX internally stores files is preventing me from doing this easily. Here is an example of what I have in mind: 0. I have a bunch of xmls in a folder (versioned with a git repo) 1. I create a baseX database from these 2. Run a bunch of updates/edits on the files using Xquery processor from basex 3. move back to git to diff what I've accomplished, commit, etc. 4. (possibly more advanced, not sure if I really need this) I want to be able to checkout a previous version of the files and have basex do the same (i.e. if I run a query now I want it to run on the version of the xmls that I have currently checked-out in my filesystem) Here are the (dumb) questions: 1. Is there a way for basex (maybe some config settings when I create the database?) to work keeping on the plain xml files? In a way that every update query would actually change the original xml files (which I can view with a plain text editor even "outside" the basex environment)? 2. Is the only way of immediately seeing the changes I made to the xml files to export the database after every update? 3. (assuming 1 is doable) Would it be possible to have baseX recognize the changes made to the xml files using an external editor (i.e. not through xqueries run in basex)? 4. From your experience is BaseX the proper tool for my purposes? It feels to me that I merely want to use its xquery processor capabilities (and I don't need the full-fledged database) while keeping the files plainly stored in my filesystem... any suggestion on alternative options if baseX doesn't sound like the right one? Thanks in advance for the help, and congratulations on the great work (I used basex in the past in a more orthodox way and was impressed by the numerous nice features... including being the nice GUI and the hassle-free integration with tomcat!) Best regards, Enea Parimbelli -- Dr. Enea Parimbelli Post-doctoral research fellow Laboratory for Biomedical Informatics "Mario Stefanelli" Department of Electrical, Computer and Biomedical Engineering University of Pavia, Italy e-mail: enea.parimbelli(a)gmail.com phone: +39-0382-985057 +39-0382-985981 http://www.labmedinfo.org/people/parimbelli

3 2

BaseX 8.4: The XML Prague 2016 Edition
by Christian Grün 08 Feb '16

08 Feb '16

Dear all, Aligned with the XML Prague 2016 Conference, we are more than happy to provide you with a great new version of BaseX, your XML database system and XQuery processor! These are the new features: GUI - Project View: all modules of the project are now parsed in the background - Create Database, Properties: easy access to advanced database options - Sort Dialog: order by column positions and locale collation INDEXING - New Token Index: speedy token lookups in DITA and HTML documents - Including full support for incremental and main-memory databases - XQuery optimizations for fn:contains-token, fn:tokenize, fn:idref WEB APPLICATIONS - Cancel server-side queries via %rest:single annotation - DBA query editor: faster querying, better responsivity XQUERY - Database Module: ADDCACHE support (caching of large results) - Direct output of binary data with new 'basex' serialization method - Better XQuery 3.1 compliance: string constructors, new functions - Java bindings: address specific Java function signatures - User Module: create users and grant permissions in one step DOCUMENTATION - Many Wiki articles have been revised and updated A big thank you to our sponsors, which made some new features possible and available to everyone, and a special shout-out to Jens Erat, who realized the new token index! Visit http://basex.org to find the latest release. We are looking forward to your feedback, Christian BaseX Team

1 0

XProc 2 ...
by Marco Lettere 07 Feb '16

07 Feb '16

Hi all, sorry for the small OT but this sounds really interesting: http://lists.xml.org/archives/xml-dev/201602/msg00001.html Another reason why I should have been there ... :-(. Anyone from the BaseX community planning to look into it? Ciao, M.

3 2

Output from BaseX 8.4 GUI contains invalid characters
by Matti Lassila 07 Feb '16

07 Feb '16

Hello, I noticed that the current snapshot of BaseX GUI (8.4 beta 1e13ab6) generates erroneus values to the end of the result when the result is saved via Save-button. Using the same data with BaseX 8.3.1 GUI gives a clear output without errors. Strangely, When the result is cut-and-pasted, erroneous values are not included. I included erroneus and correct output files. I'm running OSX 10.11.2 but this happens also in Ubuntu 14.04. Best regards, Matti Lassila

2 1

Re: [basex-talk] XProc 2
by Christophe Marchand 05 Feb '16

05 Feb '16

Hello ! We have used XProc on a project, where we pipeline many XSL. XProc 1 is difficult to learn, and so very expensive to maintain. We have quit XProc and write our own xsl-pipeline tool, which is much simpler to use, and quite more quick. But it only covers XSL pipelining, and no other rich features that are present in XProc. So, I am very interested in this new version, but won't be there, for date constraints. Best regards, Christophe Le 05/02/2016 12:00, basex-talk-request(a)mailman.uni-konstanz.de a écrit : > Hi all, > sorry for the small OT but this sounds really interesting: > > http://lists.xml.org/archives/xml-dev/201602/msg00001.html > > Another reason why I should have been there ... :-(. > Anyone from the BaseX community planning to look into it? > > Ciao, > M. > > > ------------------------------ > > Message: 3 > Date: Fri, 5 Feb 2016 10:19:36 +0100 > From: Marc van Grootel <marc.van.grootel(a)gmail.com> > To: Marco Lettere <marco.lettere(a)dedalus.eu> > Cc: BaseX <basex-talk(a)mailman.uni-konstanz.de> > Subject: Re: [basex-talk] XProc 2 ... > Message-ID: > <CA+bQNCss6ELfR8ohpPNMYCzSc2ToOEEJAAiyFUNWBrY1Cfj0yg(a)mail.gmail.com> > Content-Type: text/plain; charset=UTF-8 > > Well, I looked at it for a bit when Norm demonstrated his XProc > "dialect" as he called it on his laptop at XML Amsterdam. Looked > really good. > > I think this is a good thing. I always found XProc a fantastic idea > but never got into it. Tried some simple things but found it too much > work and didn't wrap my head around it enough. It just didn't gel. The > XML is not helping in the case of XProc. I already went through > something like this going from Ant to Gradle where you can write your > builds in Groovy. Maybe XProc is on the same route and it becames a > library of functions that we can put together using XQuery. > > Definitely going to look into it again. > > Cheers, > --Marc > >

1 0

Fwd: Re: BaseX, Zorba, e-mail module
by Christian Grün 05 Feb '16

05 Feb '16

Hi Vincent, Sorry, I overlooked that my mail with the sources did not reach the list. Here it is. Christian ---------- Weitergeleitete Nachricht ---------- Von: "Christian Grün" <christian.gruen(a)gmail.com> Datum: 23.09.2015 3:51 nachm. Betreff: Re: [basex-talk] BaseX, Zorba, e-mail module An: "Tim Thompson" <timathom(a)gmail.com> Cc: > Is there an XSD for the e-mail document format being used? (And this module > is compatible with the BaseX open source license, correct?) No, sorry, there is no XSD available. However, you are invited to create one; I have attached the Java sources to this mail, which should contain all information you need. The Java code is based on the standard Java JavaMail API [1,2]. Hope this helps, Christian [1] http://www.oracle.com/technetwork/java/javamail/index.html [2] https://java.net/projects/javamail/pages/License > Tim A. Thompson > Metadata Librarian (Spanish/Portuguese Specialty) > Princeton University Library > > > On Wed, Sep 23, 2015 at 3:24 AM, Christian Grün <christian.gruen(a)gmail.com > > wrote: >> >> Hi Tim, >> >> We have implemented an XQuery/Java e-mail module quite some time ago, >> which we are using in various commercial projects. The main reason why >> we didn't make it public yet is that we didn't have time to document >> it properly. I have just uploaded this module, and you can install it >> via: >> >> repo install >> http://files.basex.org/modules/org/basex/modules/email/EMail.jar >> >> The jar file contains two simple example for sending mails (in the >> xquery sub-directory). >> >> Looking forward to your feedback. >> Christian >> ___________________________________ >> >> On Wed, Sep 23, 2015 at 7:04 AM, Tim Thompson <timathom(a)gmail.com> wrote: >> > Hello, >> > >> > I would like to be able to send e-mail from a Web app I am developing in >> > BaseX and was wondering whether anyone else had implemented this >> > functionality (or whether a BaseX e-mail module might be in the works?). >> > >> > I see that Zorba has an e-mail module with support for SMTP and IMAP >> > (not to >> > mention many other useful modules), which leads me to wonder whether >> > anyone >> > has worked on a BaseX connector for Zorba. >> > >> > In the meantime, I decided to try using the BaseX process module to call >> > Zorba, like so: >> > >> > proc:system( >> > 'zorba', ( >> > '-q', convert:binary-to-string( >> > db:retrieve( >> > 'out', 'email.xq' >> > ) >> > ) >> > ) >> > ) >> > >> > I stored the XQuery code for Zorba as a raw file in the BaseX database, >> > since storing it in the webapp directory caused an error, due to the >> > unrecognized Zorba module declarations. This approach seems to work >> > adequately, but does anyone have any suggestions for improvement, or a >> > different approach? >> > >> > Thanks in advance, >> > Tim >> > >> > >> > -- >> > Tim A. Thompson >> > Metadata Librarian (Spanish/Portuguese Specialty) >> > Princeton University Library >> > > >

1 0

BaseX, Zorba, e-mail module
by Tim Thompson 05 Feb '16

05 Feb '16

Hello, I would like to be able to send e-mail from a Web app I am developing in BaseX and was wondering whether anyone else had implemented this functionality (or whether a BaseX e-mail module might be in the works?). I see that Zorba has an e-mail module with support for SMTP and IMAP (not to mention many other useful modules), which leads me to wonder whether anyone has worked on a BaseX connector for Zorba. In the meantime, I decided to try using the BaseX process module to call Zorba, like so: proc:system( 'zorba', ( '-q', convert:binary-to-string( db:retrieve( 'out', 'email.xq' ) ) ) ) I stored the XQuery code for Zorba as a raw file in the BaseX database, since storing it in the webapp directory caused an error, due to the unrecognized Zorba module declarations. This approach seems to work adequately, but does anyone have any suggestions for improvement, or a different approach? Thanks in advance, Tim -- Tim A. Thompson Metadata Librarian (Spanish/Portuguese Specialty) Princeton University Library

3 4

Multithread lock
by Bondeson, Carl 04 Feb '16

04 Feb '16

I have a highly multithreaded application that uses BaseX as a translation/transformation engine. It has been functioning in production for almost a year. From time to time I see the following in the nightly report the application generates which is sent from the server hosting the application. It requires a restart of JBoss, used as the application server. Any ideas? Name: Embedded EIP BaseX Server Thread : blocked Blocked on: java.net.SocksSocketImpl@5915a689 Blocked by: 1428 (Thread-1368) Blocked Count: 1 Lock Info: java.net.SocksSocketImpl@5915a689 Stack Trace: java.net.PlainSocketImpl.accept(PlainSocketImpl.java:194) java.net.ServerSocket.implAccept(ServerSocket.java:545) java.net.ServerSocket.accept(ServerSocket.java:513) org.basex.BaseXServer.run(BaseXServer.java:142) com.pilotfish.eip.basex.BaseXServerRunner$1.run(BaseXServerRunner.java:93) java.lang.Thread.run(Thread.java:745) Name: C3P0PooledConnectionPoolManager[identityToken->2szqjh9e17wbsc31kua7k9|58166f21]-HelperThread-#2 : timed_waiting Thank you Carl R Bondeson Systems Developer Department of Public Health Data Processing 410 Capitol Ave Hartford, CT 06134 Phone: 860-509-7434 carl.bondeson(a)ct.gov<mailto:carl.bondeson@ct.gov> [logosmall]

3 11

Rép: Re: Rép: Re: Slow XQueries on a quite small database but great number of docs
by ansel.jm 02 Feb '16

02 Feb '16

Hi I don't know why but the problem is present even with the basex simple client (not client/server ). It seems java (tried many versions) doesn't execute normally : display or computation is halting, jolting.... The result of the query is quite fast but the display of the result very very halting. I'm on a readynas server running on an ARM processor. I think the io library of java (buffered io or something like that) has a problem. I'm very disappointed because that database seems very well adapted for my project... Jm Christian Grün <christian.gruen(a)gmail.com> a écrit : >> I can't run the GUI, I don't know why, I'm over a VPN and when I try the administrator login on the 1984 port, the GUI answers : access denied... >> >> I have tried basexclient -o output.xml : Same result, the query you know takes 11s... > >So I’m pretty sure it must be your connection, and not the query >processor of BaseX, that slows down the query. > >What about running the GUI locally? It’s not based on the >client/server architecture [1]. > >Christian > >[1] http://docs.basex.org/wiki/GUI > > > >> >> ----- Mail original ----- >> De: "Christian Grün" <christian.gruen(a)gmail.com> >> À: "Jean Marc Ansel" <ansel.jm(a)free.fr>, "BaseX" <basex-talk(a)mailman.uni-konstanz.de> >> Envoyé: Dimanche 31 Janvier 2016 17:49:00 >> Objet: Re: Rép: Re: [basex-talk] Slow XQueries on a quite small database but great number of docs >> >> Hi Jean Marc (cc to the list), >> >> What happens if you run the query in the GUI (how long does it take)? >> How much time is spent if you write the client output to a file >> (basexclient -o output.xml)? Which client do you use? >> >> >> >> >> On Sun, Jan 31, 2016 at 5:41 PM, <ansel.jm(a)free.fr> wrote: >>> Many many thanks for your quickness ! >>> Yes I optimized the database (OPTIMIZE ALL). I don't undestand where the pb is. >>> The query I ran is the one I have transmitted to you in the last mail. So there is a pb with the server I'm afraid. May be with java ? >> >>> When I try to execute the query from basexclient, I'have a immediate answer but a very very very slow display : lines are displayed in blocks of one or two screen terminals... There is a problem with the data exchange between the daemon and the client I think...

2 1