- BaseX-Talk - mailman.uni-konstanz.de

Re: [basex-talk] Writeback option
by Christian Grün 09 Nov '13

09 Nov '13

…I’ve made it an GitHub issue: https://github.com/BaseXdb/basex/issues/786 On Fri, Nov 8, 2013 at 5:31 PM, Christian Grün <christian.gruen(a)gmail.com> wrote: >> - Concerning your comment on using the WRITEBACK option only for single >> file data bases: debugging the basex server yields that when using more >> than one xml document (via the ADD command), the "original" field in the >> MetaData class is not set, thus the WRITEBACK operation does not write >> anything in these situations. > > …exactly. The original URI of added resources will not be stored in > the database, and the optional "original" string you mentioned refers > to the path of a single resource, which was added in the database > creation process. > >> - When raw files reside in the database (added via STORE commands) in >> addition to the single initial xml data file, then an error is thrown >> upon every xquery write statement stating: >> Stopped at .../backend/restxq/ajax/addmodel.xqm, 10/10: [FOUP0002] >> ".../backend/data/gallery.xml" could not be written. > > I agree that it would make sense to reduce the number of unwanted side > effects caused by the WRITEBACK option. I even think the best solution > is to only write back files when updating documents that have been > created within a query, e.g... > > delete node doc("file-on-disk.xml")//abc > > and disallow updates on database instances. > > Thanks, > Christian > > >> At the same time the file gallery.xml is in fact updated to reflect the >> write operations changes. >> >> >> Than you >> Stefan >> >> Am 06.11.13 19:58, schrieb Christian Grün: >>> Dear Stefan, >>> >>>> - My impression is that the WRITEBACK option only works for databases >>>> directly created from an initial input xml file (CREATE DB <name> >>>> <input.xml>). Is that true? My other tests involving ADD <file.xml> >>>> commands failed to write back inserted or deleted nodes. >>> >>> that’s correct: the WRITEBACK option is only recommendable if you are >>> working with single files. Usually, the EXPORT command is the better >>> choice for writing database contents back to files. >>> >>>> - I checked out the latest git version and noticed that the WRITEBACK >>>> option is not read from the .basex file anymore. In version 7.7.2 that I >>>> used for the other tests it was read. >>> >>> Thanks for the observation; I have committed a fix and uploaded a new >>> snapshot, which should fix the problem [1]. >>> >>> Christian >>> >>> [1] https://github.com/BaseXdb/basex/commit/2d618a18f4b2ad14cc17b15d155320a088f… >>> >>> >>>> Am 25.10.13 18:04, schrieb Dirk Kirsten: >>>>> Dear Stefan, >>>>> >>>>> please be aware that this is a english-speaking mailing list. As we want >>>>> most of our users to benefit from all questions and answers I'll stick >>>>> to english in my response (and aa your web page is in English as well I >>>>> guess it is no obstacle for you). >>>>> >>>>> Regarding your question about saving plain xml files: You can use the >>>>> Export Command (see https://docs.basex.org/wiki/Commands#EXPORT) to >>>>> export all your documents. You can e.g. use a cron job to export them >>>>> regularly. There also is a writeback option >>>>> (https://docs.basex.org/wiki/Options#WRITEBACK) which will update your >>>>> original input nodes. >>>>> >>>>> Secondly, regarding your question about empty WebDAV files: Please take >>>>> a look at a recent discussion on the list (see >>>>> https://mailman.uni-konstanz.de/pipermail/basex-talk/2013-September/005453.…) >>>>> which is about the same topic. Short answer: With another viewer it >>>>> should work. Computing the size is quite difficult as we store the data >>>>> in a special storage format to speed up querying. Seems like some >>>>> viewers don't like it if the file size isn't correct. >>>>> >>>>> Cheers, >>>>> Dirk >>>>> >>>>> On 25/10/13 17:27, Stefan Sechelmann wrote: >>>>>> Hallo, >>>>>> ich bin dabei BaseX für unser aktuelles Projekt zu evaluieren. Bis jetzt >>>>>> sieht alles danach aus als ob BaseX genau das ist wonach wir gesucht >>>>>> haben. Hier meine Fragen dazu: >>>>>> >>>>>> - Der Datenbankserver speichert seine XML-Daten optimiert in .basex >>>>>> Files. Gibt es die Möglichkeit den Server auch XML Files schreiben zu >>>>>> lassen (So wie er das mit raw Files tut), so dass ich alles auch ohne >>>>>> BaseX ansehen kann? Gibt es vielleicht einen automatischen Export? >>>>>> >>>>>> - Der Webdav-Support kommt mir unstabil vor. Wenn ich XML-Dateien >>>>>> kopiere sind diese danach teilweise leer oder werden vom Server als RAW >>>>>> erkannt. Ich benutze einen Mac und den Finder als Client. Sind das >>>>>> eventuell bekannte Probleme oder muss ich bei mir suchen? >>>>>> >>>>>> Vielen Dank >>>>>> Stefan Sechelmann >>>>>> >>>>> >>>> >>>> >>>> -- >>>> Stefan Sechelmann >>>> DFG-Forschungszentrum Matheon >>>> Mathematik für Schlüsseltechnologien >>>> Technische Universität Berlin >>>> Sekretariat MA 8-3 Tel. 030/314 29 486 >>>> Straße des 17. Juni 136 Fax 030/314 79 282 >>>> 10623 Berlin sechel(a)math.tu-berlin.de >>>> http://www.math.tu-berlin.de/~sechel >>>> >>>> >>>> _______________________________________________ >>>> BaseX-Talk mailing list >>>> BaseX-Talk(a)mailman.uni-konstanz.de >>>> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >>>> >> >> >> -- >> Stefan Sechelmann >> DFG-Forschungszentrum Matheon >> Mathematik für Schlüsseltechnologien >> Technische Universität Berlin >> Sekretariat MA 8-3 Tel. 030/314 29 486 >> Straße des 17. Juni 136 Fax 030/314 79 282 >> 10623 Berlin sechel(a)math.tu-berlin.de >> http://www.math.tu-berlin.de/~sechel

1 1

~webdav locks + server stop = conflicted locks
by France Baril 08 Nov '13

08 Nov '13

This Issue occurs when the server stops while files are open/locked: When basex starts it doesn't show lock icons on files the user had locked before the server stopped. When users tries to open a file oxygen says the file is locked, even though the current user owns the lock. In other words, the lock seems to be tracking something that doesn't survive sessions. To recover, we delete the webdav db + restart Oxygen. Granted, doing all check-ins prior to stopping the server for maintenance would be a best practice. But in case of an unexpected failure, since the lock are saved in ~webdav, is there anyway we could get back to the locks as set in ~webdav? -- France Baril Architecte documentaire / Documentation architect france.baril(a)architextus.com (514) 572-0341

3 3

Re: [basex-talk] xquery and hash:md5
by James Wright 07 Nov '13

07 Nov '13

Would this work $someDoc//@*/text() --- Original Message --- From: "buzz chopra" <buzz.chopra(a)gmail.com> Sent: November 2, 2013 11:38 PM To: "Dirk Kirsten" <dk(a)basex.org> Cc: "BaseX" <basex-talk(a)mailman.uni-konstanz.de> Subject: Re: [basex-talk] xquery and hash:md5 Hello Dirk, Yes in that case, the issue exists in the nature of our data - and my poor choice of the return value of fn:string as the content to hash. Our data looks like: <test guid="blah"> <e1 attr0="a" attr1="b"/> <e2 attr0="c" attr1="d" attr2="e"/> </test> As you state, fn:string behaves the way http://www.w3.org/TR/xpath-functions-30/#func-string says it should. i.e. the empty string is returned for the document above. I'll have to try and find something that returns all attribute values in each document. Thanks again for your help. Buzz On Fri, Nov 1, 2013 at 10:37 AM, Dirk Kirsten <dk(a)basex.org> wrote: > Hello Buzz, > > The result of string() depends on the kind of element as an input. For > an attribute value it will simply be the value of the attribute. For an > element node it will be a concatination of the node content and all > descendant nodes. So if string($r) for $r being an element node is > always empty I would guess it is an empty node (something like <reps > />), which will always yield an empty result. > > If you want to create different hashs for empty elements with different > attribute values you might want to do something like the following > > let $reps := fn:collection($coll)//reps > for $r at $lcv in $reps > let $l := string-join($r/@*, '') > let $hash := xs:hexBinary(hash:md5(fn:string($l))) > return <hash i="{$lcv}">{$hash}</hash> > > If this is not what you actually want to achieve it would be very > helpful to actually get a small example of your data (especiall the reps > nodes) to get an idea how it looks like. > > Cheers, > Dirk > > On 01/11/13 16:19, buzz chopra wrote: > > Dirk, > > > > Thank you for your quick response. > > > > Upon your suggestion (and prior to that) I checked that the values for $r > > are different. Indeed they are. But, what I didn't check until your > > suggestion is that the values of fn:string($r) are different. They in > fact > > are the same - thank you for your suggestion. > > > > Is it possible that there is some sort of type casting related issue > going > > on where I call fn:string($r)? > > > > I get different values for the hash of each attribute value when I select > > attributes in the collection like: > > > > let $reps := fn:collection($coll)//reps/@att0 > > for $r at $lcv in $reps > > let $hash := xs:hexBinary(hash:md5(fn:string($r))) > > return <hash i="{$lcv}">{$hash}</hash> > > > > > > when I select elements in the collection like: > > > > let $reps := fn:collection($coll)//reps > > for $r at $lcv in $reps > > let $hash := xs:hexBinary(hash:md5(fn:string($r))) > > return <hash i="{$lcv}">{$hash}</hash> > > > > I found that the value of fn:string($r) is blank for all values of $r > > (which are different) which is why the hash is the same. > > > > I appreciate your help. > > > > Buzz > > > > > > On Fri, Nov 1, 2013 at 2:52 AM, Dirk Kirsten <dk(a)basex.org> wrote: > > > >> Hello buzz, > >> > >> it is ok to also ask more general XQuery questions on this list. > >> > >> For testing I executed the following query: > >> > >> for $r in ("123", "234", "345") > >> let $hash := xs:hexBinary(hash:md5(fn:string($r))) > >> return $hash > >> > >> which yields different results for each value of $r. Are you absolutely > >> sure you have different values for $r (you can check using trace())? > >> Also, given they query I would be very suprised if indeed it shows the > >> same result. > >> > >> Cheers, > >> Dirk > >> > >> On 01/11/13 08:24, buzz chopra wrote: > >>> This may be a question better suited for an xquery forum but I'll start > >>> here since I'm using basex: > >>> > >>> Can someone suggest why I'd be getting the same result for the hash > call > >>> even though > >>> the content of $r should be changing? > >>> > >>> let $reps := fn:collection($coll)/Rep > >>> for $r at $lcv in $reps > >>> let $hash := xs:hexBinary(hash:md5(fn:string($r))) > >>> return <hash i="{$lcv}">{$hash}</hash> > >>> }; > >>> > >>> Please let me know if I should post this elsewhere (and please include > >>> where). > >>> > >>> Thanks in advance for any suggestions. > >>> > >>> Buzz > >>> > >>> > >>> > >>> _______________________________________________ > >>> BaseX-Talk mailing list > >>> BaseX-Talk(a)mailman.uni-konstanz.de > >>> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk > >>> > >> > >> -- > >> Dirk Kirsten, BaseX GmbH, http://basex.org > >> |-- Firmensitz: Blarerstrasse 56, 78462 Konstanz > >> |-- Registergericht Freiburg, HRB: 708285, Geschäftsführer: > >> | Dr. Christian Grün, Dr. Alexander Holupirek, Michael Seiferle > >> `-- Phone: 0049 7531 28 28 676, Fax: 0049 7531 20 05 22 > >> _______________________________________________ > >> BaseX-Talk mailing list > >> BaseX-Talk(a)mailman.uni-konstanz.de > >> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk > >> > > > > -- > Dirk Kirsten, BaseX GmbH, http://basex.org > |-- Firmensitz: Blarerstrasse 56, 78462 Konstanz > |-- Registergericht Freiburg, HRB: 708285, Geschäftsführer: > | Dr. Christian Grün, Dr. Alexander Holupirek, Michael Seiferle > `-- Phone: 0049 7531 28 28 676, Fax: 0049 7531 20 05 22 > _______________________________________________ BaseX-Talk mailing list BaseX-Talk(a)mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk

1 0

Receiving user feedback (too much traffic for BaseX?)
by France Baril 07 Nov '13

07 Nov '13

Hi, Our online help offers a form to collect comments from end-users. We have a lot of users and when a new product hits the market, we get a lot of feedback. We have found that BaseX is unable to handle so many write operations. Is there a suggested setup/configuration to handle multiple write operations coming from http requests. Is it possible to queue them somehow? Or is it better to use a more traditional approach to feedback collection? I.e. use a different toolset? Right now, BaseX just goes blank (freeze) with all the feedback coming in. Expectations: Average 5 comments/minute, but bursts of 2 per second should not break anything. Dropping comments during bursts is acceptable to us for our current business case because we hope to identify the most important issues after a new product release/software upgrade. Major issues should come up often enough. Any suggestion would be appreciated. -- France Baril Architecte documentaire / Documentation architect france.baril(a)architextus.com (514) 572-0341

2 1

BaseX
by Stefan Sechelmann 06 Nov '13

06 Nov '13

Hallo, ich bin dabei BaseX für unser aktuelles Projekt zu evaluieren. Bis jetzt sieht alles danach aus als ob BaseX genau das ist wonach wir gesucht haben. Hier meine Fragen dazu: - Der Datenbankserver speichert seine XML-Daten optimiert in .basex Files. Gibt es die Möglichkeit den Server auch XML Files schreiben zu lassen (So wie er das mit raw Files tut), so dass ich alles auch ohne BaseX ansehen kann? Gibt es vielleicht einen automatischen Export? - Der Webdav-Support kommt mir unstabil vor. Wenn ich XML-Dateien kopiere sind diese danach teilweise leer oder werden vom Server als RAW erkannt. Ich benutze einen Mac und den Finder als Client. Sind das eventuell bekannte Probleme oder muss ich bei mir suchen? Vielen Dank Stefan Sechelmann -- Stefan Sechelmann DFG-Forschungszentrum Matheon Mathematik für Schlüsseltechnologien Technische Universität Berlin Sekretariat MA 8-3 Tel. 030/314 29 486 Straße des 17. Juni 136 Fax 030/314 79 282 10623 Berlin sechel(a)math.tu-berlin.de http://www.math.tu-berlin.de/~sechel

3 3

Adding WKT/WKB To Geo Module
by Zachary DeLuca 03 Nov '13

03 Nov '13

Hi All, apologies in advance for the long message, I have many documents that store WKB (base64-encoded) geometry data (going for speed not readability). I want to be able to do queries where I can use the geo predicate functions with the stored WKB geometry and a provided WKT geometry. Example: Sample document structure: <root><geoWKBdata type=xs:base64Binary/></root> Sample query: collection()/root[geo:intersects(*geo:fromWKB*(geoWKBData), * geo:fromWKT*("LINESTRING((0,0),(1,1))")) I have looked at the module code and the only problem is that all methods assume an input of GML. I don't want to just make functions to convert from WKB/WKT to GML on input because I think that would be very inefficient (slow). I could just hack something together for my own needs, but I would like some input so that ideally I can produce something good enough to contribute back to the project. Options: 1) Add copies of all methods to accept each possible encoding type (GML, WKB, WKT). I don't like this approach because it doesn't scale well, and I don't even know if it's possible? (Pretty sure you can't overload functions in XPath/XQuery) 2) Leave the Geo module the way it is and add "sister" modules for the additional encoding types. Yuck. 3) The only thing that makes sense, really. Define 3 new functions, *fromGML *, *fromBinary*, and *fromText* and change the signatures of all module functions to just accept Geometry objects. That way you can load your geometry however you want and then pass it to the desired function. I will also modify the functions *asText *and *asBinary *to accept a Geometry object, and add an additional *asGML* so that the many existing functions that already return GML (envelope, boundary, etc.) can instead return Geometry, and can then be converted as needed to the desired format. This should scale well too because if any future encoding formats are desired all you need to do is add *fromXYZ* and *asXYZ *functions. With these functions, you could also string together 2 calls to convert between any of the supported encoding types. Does this make sense? It seems easy enough, am I missing anything? Also, assume for a second that I implement #3. Refer to my sample document/query above. As I understand it, for each document in the collection, the back end will have to parse "LINESTRING((0,0),(1,1))" into a Geometry first in order to compare. Is there any way to make that happen only once for the duration of the query? Is it as easy as this?: let $data = geo:fromText("LINESTRING((0,0),(1,1))"); return collection()/root[geo:intersects(*geo:fromWKB*(geoWKBData), $data) This would work, right? (I'm new to XQuery, sorry) So, am I on the right track with this? Have I overlooked some absurdly simple solution to my problem? I would appreciate any and all input. As I said, I would like to go beyond meeting my own needs and actually get this into the code base. I'll leave you with this snippet taken from the EXPath Geo Module Spec: "The data model envisioned by EXPath Geo module is abstract in the sense that it may be possible for module implementers to support more than one encoding on input. Output data types are specified in GML and XML Schema, where appropriate. All that is necessary is that the input geometry be transformable to the model specified by OGC SF and supported by the EXPath implementation. -SNIP- *Implementations are not limited to GML, KML, nor even XML, on input. It is possible that geometries encoded as “Well Known Text”, GeoJSON, or even “Well Known Binary”, among other possibilities, could be supported by an implementation.*" Thanks, Zach

3 9

xquery and hash:md5
by buzz chopra 03 Nov '13

03 Nov '13

This may be a question better suited for an xquery forum but I'll start here since I'm using basex: Can someone suggest why I'd be getting the same result for the hash call even though the content of $r should be changing? let $reps := fn:collection($coll)/Rep for $r at $lcv in $reps let $hash := xs:hexBinary(hash:md5(fn:string($r))) return <hash i="{$lcv}">{$hash}</hash> }; Please let me know if I should post this elsewhere (and please include where). Thanks in advance for any suggestions. Buzz

2 4

Error message
by Lars Johnsen 31 Oct '13

31 Oct '13

When trying to create a database from a folder of approximately 5000 collections (each containing between 200 and 500 files) the processed stopped after two hours with an error something like this: Line too long for a single database {filename} and {line number}. The file itself looks ok, with a single tag in the referenced position. What is the recommended procedure from here? All the best Lars G Johnsen National Library of Norway

2 1

request:http() and servlet context
by Alexander Ryabov 29 Oct '13

29 Oct '13

Hello guys. Are you going to implement request:http() function from http://exquery.github.io/expath-specs-playground/request-module-1.0-specifi… ? What I need is java's request.getContextPath(). Any workarounds? Thanks for your product, Alexander Ryabov

2 2

Fundamental problems with BaseX 7.7
by Werner Dworaczek 29 Oct '13

29 Oct '13

Hi Christian, after migrating to 7.7 I get the error message "No context item defined to evaluate 'root()'". I get this message every time when I am going to access the root node within any of my xQuery functions. This is bad since many of my functions does do exactly that, which wasn’t a problem yet. Why does the old style does not work any more and is there a comfortable workaround to avoid the need for re-writing all my xQuery code? Of course I could pass the root node as an argument to avoid this kind of error, which could improve flexibility, but it’s a lot of work. For test purposes I do have re-written some stuff to get it working with version 7.7 and when I tried to test it with the basex GUI I get "out of memory" errors and my scripts are aborted - actually all my scripts does obviously not work any more. Why does my stuff works great with BaseX 7.6 and not at all with BaseX 7.7. The upgrade seems to change some very general stuff under the hood and it cannot be reduced to some new functionality and minor signature changes for some xQuery built-in-functions. Is there something special with my stuff? I don’t think so. What I do in principle is the following: I first add an XML file info the DB, then I load and start an import xQuery which analyses the XML file just added and adds nodes into the database (update operations). When the import has finished I delete the XML file added originally. Executing the import xQuery works fine with 7.6 and the memory consumption is fortunately low, but the same script aborts after a few seconds with an "out of memory“ error when using BaseX 7.7 - same basic configuration used. It does not make any difference whether I test all stuff with the GUI or let it run automatically by using the client lib. By the way, I do use the perl client lib. It would be great if you could give me some hints how to proceed with 7.7. I have tried even 7.7.2 with no success. Thanks in advance, Regards, Werner

2 1