- BaseX-Talk - mailman.uni-konstanz.de

WebDAV Lock error
by Andreas Jung 22 Feb '15

22 Feb '15

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi there, some more WebDAV issues (tested against the 8.1 snapshot from yesterday). - - connecting with cadaver and locking a file twice causes a HTTP 500 error on the cadaver side and BaseX fails with this exception: [qtp2029686239-17] INFO com.bradmcevoy.http.HttpManager - PROPFIND :: http://dev1.veit-schiele.de:8984/webdav/scripts/backup.xql - http://dev1.veit-schiele.de:8984/webdav/scripts/backup.xql [qtp2029686239-16] INFO com.bradmcevoy.http.HttpManager - LOCK :: http://dev1.veit-schiele.de:8984/webdav/scripts/backup.xql - http://dev1.veit-schiele.de:8984/webdav/scripts/backup.xql [qtp2029686239-16] ERROR com.bradmcevoy.http.StandardFilter - process java.lang.NullPointerException at com.bradmcevoy.http.webdav.LockHandler.respondWithToken(LockHandler.java:274) at com.bradmcevoy.http.webdav.LockHandler.processRefresh(LockHandler.java:256) at com.bradmcevoy.http.webdav.LockHandler.processExistingResource(LockHandler.java:112) at com.bradmcevoy.http.webdav.LockHandler.process(LockHandler.java:86) at com.bradmcevoy.http.StandardFilter.process(StandardFilter.java:52) at com.bradmcevoy.http.FilterChain.process(FilterChain.java:40) at com.bradmcevoy.http.HttpManager.process(HttpManager.java:228) at org.basex.http.webdav.WebDAVServlet.run(WebDAVServlet.java:43) at org.basex.http.BaseXServlet.service(BaseXServlet.java:64) at javax.servlet.http.HttpServlet.service(HttpServlet.java:848) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:684) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:503) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1086) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:429) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1020) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116) at org.eclipse.jetty.server.Server.handle(Server.java:370) at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:494) at org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:982) at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1043) at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:865) at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240) at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82) at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:696) at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:53) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) at java.lang.Thread.run(Unknown Source) . 2nd problem: locking the same file twice works when I disconnect and reconnect with cadaver: ..... dav:/webdav/scripts/> lock backup.xql Locking `backup.xql': succeeded. dav:/webdav/scripts/> Connection to `dev1.veit-schiele.de' closed. Suxmac:~ ajung$ cadaver http://dev1.veit-schiele.de:8984/webdav Authentication required for BaseX on server `dev1.veit-schiele.de': Username: admin Password: dav:/webdav/> cd scripts dav:/webdav/scripts/> lock backup.xql Locking `backup.xql': succeeded. It is nice that the 2nd lock call works however this indicates that locking does not work at all since a lock should persist across connections. Andreas -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (Darwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQGUBAEBAgAGBQJU6FRtAAoJEADcfz7u4AZjmOILv0B5T2l3tMIhLzCOljWLMw7R aC8H0dqOo/KtBV750eNAGpzmxGpFSd59SB35C2lrBOleCOIjI9fm/JzhCa0Mw2/J m1lLRAn9uK9i0uq7B7TKVlXf4N6vatZ8eecVs9KPBs0PB7D/G/GBl6ws9MfVY8QX g3OEa3A+Wi/FlMy0VDuVysYtvfs9CqGWe1etemYSA1u9z1dMN54ZkQKWKuoEi7mZ 9WPUNSF6A9kw5u6ACaTt2Qx7chzPreRckw9JmipVmv4xdZL1xWkW8yZGADL5jf94 egb0qzWrYSjSkrV/4lNkCgDoW0/+IebXq1lDC2/nQtwgyumi0jyNzWy9jKtziM6G WqUt1XM+A1buttMQ5+Z3WGZ1QdCd2yhoOT75bQTkhjuDBZTCiQwHabYgIPgfWHZX BNYE90SrBdRYIR/9n0q2+aG0oPEKlMWeDCVMAS35Q9naczYdl6jtVAJuiJ7KsvU4 QAQ3as+p2rAuUF2S1RXdIZ6m3VGL0Uw= =xLzv -----END PGP SIGNATURE-----

2 2

Fwd: Re: schema validation
by Dirk Kirsten 21 Feb '15

21 Feb '15

After Christians mail I realized I pressed the wrong reply button, so I sent I forward my response to the mailing list. But good to see Christian says basically the same :) Cheers, Dirk -------- Forwarded Message -------- Subject: Re: [basex-talk] schema validation Date: Fri, 20 Feb 2015 11:06:53 +0100 From: Dirk Kirsten <dk(a)basex.org> To: pmenso57(a)comcast.net Hi Paul, no, not directly. You would have to update via your own update functions by using our validation module. You could invoke all updates through such functions, e.g. by using RESTXQ. BaseX itself is not schema-aware. Cheers, Dirk On 02/20/2015 03:33 AM, pmenso57(a)comcast.net wrote: > Hi all. > > Is it possible to enforce schema validation on update in BaseX? > > Regards, > Paul Mensonides > -- Dirk Kirsten, BaseX GmbH, http://basexgmbh.de |-- Firmensitz: Blarerstrasse 56, 78462 Konstanz |-- Registergericht Freiburg, HRB: 708285, Geschäftsführer: | Dr. Christian Grün, Dr. Alexander Holupirek, Michael Seiferle `-- Phone: 0049 7531 28 28 676, Fax: 0049 7531 20 05 22

2 2

Lock issue while editing a .xql script over WebDAV
by Andreas Jung 20 Feb '15

20 Feb '15

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi there, the following bug occurs while re-editing a .xql script. - - start cadaver against the WebDAV port - - inside cadaver: edit some.xsql and save an empty file - - re-edit the same file again and this LOCK related issue pops up Cheers Andreas [qtp43256131-15] INFO com.bradmcevoy.http.HttpManager - PROPFIND :: http://localhost:8984/webdav/scripts/foo.xql - http://localhost:8984/webdav/scripts/foo.xql [qtp43256131-15] INFO com.bradmcevoy.http.HttpManager - PROPFIND :: http://localhost:8984/webdav/scripts/foo.xql - http://localhost:8984/webdav/scripts/foo.xql [qtp43256131-15] INFO com.bradmcevoy.http.HttpManager - OPTIONS :: http://localhost:8984/webdav/scripts/foo.xql - http://localhost:8984/webdav/scripts/foo.xql [qtp43256131-20] INFO com.bradmcevoy.http.HttpManager - LOCK :: http://localhost:8984/webdav/scripts/foo.xql - http://localhost:8984/webdav/scripts/foo.xql com.bradmcevoy.http.exceptions.BadRequestException [qtp43256131-20] ERROR com.bradmcevoy.http.StandardFilter - process java.lang.NullPointerException at com.bradmcevoy.http.webdav.LockHandler.processNewLock(LockHandler.java:234) at com.bradmcevoy.http.webdav.LockHandler.processExistingResource(LockHandler.java:110) at com.bradmcevoy.http.webdav.LockHandler.process(LockHandler.java:86) at com.bradmcevoy.http.StandardFilter.process(StandardFilter.java:52) at com.bradmcevoy.http.FilterChain.process(FilterChain.java:40) at com.bradmcevoy.http.HttpManager.process(HttpManager.java:228) at org.basex.http.webdav.WebDAVServlet.run(WebDAVServlet.java:43) at org.basex.http.BaseXServlet.service(BaseXServlet.java:64) at javax.servlet.http.HttpServlet.service(HttpServlet.java:848) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:684) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:503) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1086) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:429) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1020) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116) at org.eclipse.jetty.server.Server.handle(Server.java:370) at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:494) at org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:982) at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1043) at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:865) at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240) at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82) at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:696) at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:53) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) at java.lang.Thread.run(Unknown Source) -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (Darwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQGUBAEBAgAGBQJU4NDtAAoJEADcfz7u4AZjrYYLvA5ssGdFWLFhxjcH+h+2HzUd Q/GXKDuHoz1cpzBxYahaur2VbAuVysTWbb1Cr5PkOQMtXgqoNh0IbP/FDprk8VZW mqtpIVfPqSEhYtsxzr+5lo2iuppco+K5WOo16UV4+R2tRKi9yOtq5vwhp8mNOBZ1 Ottv6Q2iBON6G1DBg6qStNGBJWl1AD+HWEgvc6p0SKL8IWUnp+iaalowK00AFb6y Ie2xXZd/5iF9ukSKEs2TeSXyXF6Hx6dZqKRDUFjDZSdvIqcR4w3Z/3AQdseM3oh4 gdPMTM7HVen5mqX0DNookE4OPQY2yurNMHWrhAiHNdAisU0PkjdqZ9wjN2zl8rMy 7NnSiZwkCRL3Fk0SP2F8YrQTMFhZTUaJbIwOHclHIUZw/Jl6ULOJTQSVu34YCjBK cmwe7h9k2TBsq/LchHYLWwjtppoUNKmlMkbaKvH+7cSdwMumX7Vg2M5GApZ7xeYD 0ZjV2xXCil+Y4eyNE/Dwk+QKFO4rxp8= =rjkk -----END PGP SIGNATURE-----

2 3

Design pattern ...
by Marco Lettere 20 Feb '15

20 Feb '15

Hi all, I start from the following condition: - 2 dbs (let's say A and B) with around 200K documents each - every document in B refers to exactly one document in A (many to one is possible) - I have to apply a filter on every document of A and depending on the boolean outcome "move" the document to anoter db A'. - According to this the correlated documents of B need to be "moved" to a database B'. - The solution should be made automatic in the mid term What is the best design pattern and tool (I use the GUI by now) for this finding the right compromise for limiting "Out of memory (huge PUL)", "too much time", "complexity introduced by chunking" and "absence of a _db:move-document_ operation for moving one document in one single operation? Every suggestion is more then warmly welcome! Thanks and best regards, Marco

2 2

schema validation
by pmenso57＠comcast.net 20 Feb '15

20 Feb '15

Hi all. Is it possible to enforce schema validation on update in BaseX? Regards, Paul Mensonides

2 1

Performance on batch processes with http requests.
by France Baril 20 Feb '15

20 Feb '15

Hi, I have an item that I would like to bring to attention. We have developed a web controller to let users manages translation processes for BaseX content. Our process is something like this: - Users select content to translation (1 to 500 small files) + languages to translation to (1 to 32 selections). - For each language, for each files: - The system transforms the content to xliff and sets the segments to translate="yes" if they have changed since the last translation for this language. Content is saved because we'll need to query it, we redirect to the next task. - File without new segment to translate, content is processed and saved to the target languages (because attributes might have changed and/or segments might have been deleted). The xliff-file is deleted, we redirect to the next task because we'll need the new information for the next query. - We query the server to offer the users stats on items to translate. This is a simplification of the process, but it shows the logics. Redirects occur after each step for each language. We have grouped operations to limit commits/redirects to a minimum. We apply them: - After each language to avoid running out of memory. - Before each operation that needs to query files based on the changes from the previous steps. We have also split groups of tasks into smaller groups where too many tasks have led us to run out of memory in the past. Our request would be for a way to force changes to commit without having to redirect. Refreshing the browser has a big impact on performance. Or maybe you have suggestions to improve batch processing when using a web interface for process management. Thank you in advance for you input! -- France Baril Architecte documentaire / Documentation architect france.baril(a)architextus.com

2 4

Repo and development workflow
by Marc van Grootel 19 Feb '15

19 Feb '15

Hi, As part of the basex-gradle-starter scripts [1] I am now looking at how to deal with the repo. Until now I have always developed by putting modules under webapp. It allows to just edit a file and query away. I like to have the same workflow during development and take advantage of having modules not depend on a fixed import path. When working with the repo modules have to be installed first. Just copying them under the repo doesn't make basex aware of them. Of course I could arrange it so that I run a build task that does this via repo:install but that involves the extra step of copying files from the install location into the repo instead of just registering the files that are already in the repo location. I would like to have, for example, a 'src' directory with the repo module directory structure. Is there a way to say to basex that my repo lives in 'src' and that it auto registers what it finds there. Say a command like repo:register('dir') or maybe even an auto-register option. Once a module is already installed editing and saving is enough for basex to pick up the change. No issue there. Of course for production I probably want to compile the code to jar or xar in order to install it and share it. But during development I would like to be able to just have the code be picked up so that I can edit, save and query. I am not sure if what I'm asking is a feature or if I just have to do things differently. BTW one more thing I noticed. When I sometimes get into a situation where individual empty directories are also recognized as packages (but I cannot delete them from the GUI package manager). With repo:list() I then see something like this. <package name="org." type="Internal"/> <package name="org.basex." type="Internal"/> <package name="org.xokomola." type="Internal"/> <package name="org.xokomola.xq.Hello" type="Internal"/> [1] https://github.com/theapsgroup/basex-gradle-starter --Marc

2 8

Static files in other places than /static/* without modifying WEB-INF/web.xml ?
by Maud Ingarao 19 Feb '15

19 Feb '15

Hi everybody Thanks a lot for this wonderful BaseX 8 release and for distributing the dba module : Not only is it very practical, but it is also very inspiring to look carefully at how you guys wrote it and it gives us thousands of ideas to complete our own SynopsX module :) But it doesn't mean we always manage to imitate you... For example, exactly as you do, we want to distribute our static files inside our module, in a synopsx/files/ directory, and of course we don't want to ask people to modify the static path declaration in their WEB-INF/web.xml file to use our module. It works perfectly in your dba module : for exemple in dba/modules/tmpl.xqm, the created html element calls files/js.js or files/style.css and it simply works... But not in ours : we get this "No function matches the request" error because these files pathes are interpreted as restxq pathes (which sounds reasonable, but why not in yours ?) We are know trying more and more complicated things that don't work neither, like in https://github.com/mingarao/synopsx/blob/dev/webapp/synopsx/_restxq/files.x… line 37... So I think we should better stop there and ask you : what's the trick in dba ? All the best ! Maud and the AHN/SynopsX people -- /« Ne travaillez jamais »/ *Maud Ingarao* IHPC (UMR 5037) Institut d’histoire de la pensée classique, de l’Humanisme aux Lumières Ecole Normale Supérieure de Lyon - Site Descartes - Bureau R322 15 Parvis René Descartes - BP7000 - 69342 Lyon CEDEX 07 +33 4 37 37 65 79 - maud.ingarao(a)ens-lyon.fr /Je suis absente le jeudi | Out of office on Thursdays/ http://pensee-classique.ens-lyon.fr http://institutdesanti.ens-lyon.fr http://ahn.ens-lyon.fr http://www.mutec-shs.fr http://cahier.hypotheses.org

3 4

Issue with dtd validation
by France Baril 18 Feb '15

18 Feb '15

Hi, Since my last upgrade to BaseX 8.0 beta 9cb3a37 (also just tried BaseX 8.0.1 beta 161c4fd), I noticed that the dtd validation errors are cut when to long, so I'm getting messages like this one, that don't help my users much: File 'html-review/dh4/device-help/3338/50/global/en-us/T1401261601.html' doesn't validate against BaseX-DTDs/xhtml/xhtml1-transitional.dtd. Error [bxerr:BXVA0001]: Validation failed. "Error:archie, 5:28: Attribute "... Plus, in previous versions, I was removing some errors that we don't care about, which I can't do anymore. Is there a new option to get the full messages like I used to with $err:description? Here is my code snippet (sorry for the ugly color coding in there... never had time to upgrade them to @class): *let* *$errors* := <ul>{ *for **$file* *in **$files* *return try*{ *let **$validate* := *validate:dtd*(*$file*, *$dtd-file-path*) *return *<li *style*=*"color:green"*>Ye!</li> } *catch* *** { *let **$error-description* := *$err:description* *return if* (*contains* (*normalize-space*( *$error-description*), *normalize-space*('Attribute ""xmlns"" must be declared for element type')) or *contains* (*normalize-space*( *$error-description*), *normalize-space*('Attribute ""xmlns:xsi"" must be declared for element type'))) (: The 'xmlns' error is introduced by BaseX, and does not get exported to static content, so we ignore it :) *then *() *else if* (*contains* (*normalize-space*( *$error-description*), *normalize-space*('Attribute ""content"" is required and must be specified for element type ""meta""'))) *then *() (: No longer true in HTML5 :) *else *<li *style*=*"color:red;"*> <p>File '{*$file*/*base-uri*()}' doesn't validate against {*$dtd-file-path*}.</p> <p>{'Error [' || *$err:code* || ']: ' || *$err:description*}</p> </li> } }</*ul*> (:let $debug := file:write('debug-in-fix-validation.xml', $errors):) *return* <ul>{*$errors*//*li*[*contains*(*@style*, 'color:red;')]}</ul> -- France Baril Architecte documentaire / Documentation architect france.baril(a)architextus.com

3 4

Bug in XQuery archive module
by alxarch 18 Feb '15

18 Feb '15

Hi, I stumbled upon a bug in the archive module on BaseX 8.0. The zip files created by the archive module are not valid. ``` let $found := file:list("/tmp/test/media/") return file:write-binary("/tmp/test.zip", archive:create( $found, $found ! file:read-binary( "/tmp/test/media/" || .) , map { "format" : "zip", "algorithm" : "deflate"})) ``` The resultting zip file is invalid. Trying to unzip in cli returns: ``` Archive: test.zip End-of-central-directory signature not found. Either this file is not a zipfile, or it constitutes one disk of a multi-part archive. In the latter case the central directory and zipfile comment will be found on the last disk(s) of this archive. unzip: cannot find zipfile directory in one of test.zip or test.zip.zip, and cannot find test.zip.ZIP, period. ``` The same code works in BaseX7.9 Cheers, Alex

2 1