BaseX-Talk

basex-talk@mailman.uni-konstanz.de

6 participants
5250 discussions

Is there an XML based collaborative curation environment?
by Martin Mueller 29 Aug '14

29 Aug '14

I have sent this post simultaneously to the eXist, baseX, and TEI lists in the hope that I will get some useful advice on how to cobble together an XML-based collaborative curation platform. If there is something "out there" that meets some of my requirements I would love to hear about it. Phil Burns, Craig Berry, and I have been engaged in an informal project that I call Shakespeare His Contemporaries (SHC). The source texts for SHC are the TCP transcriptions, transformed into TEI P5 with Abbot and linguistically with MorphAdorner. To goal is to produce "good enough" texts (in original and standardized spellings) through collaborative curation, enlisting the help of educated amateurs as well as professional scholars. A group of Northwestern undergraduate on summer internships fixed about 70% of some 45,000 known errors in almost 500 Early Modern non-Shakespearean plays and demonstrated that you don't need to have a Ph.D. to make valuable contributions to scholarship. My curation framework has been a mixture of XML and SQL. MorphAdorner can spit out its data as a TEI file or in a tabular format.(http://morphadorner.northwestern.edu). The latter is easily turned into a MySQL database. AnnoLex, our curation tool (http://annolex.at.northwestern.edu), was built by Craig Berry. It is a Django web site that talks to an underlying MySQL database and lets registered users make emendations that are kept in a separate table for subsequent review and integration into the XML files. MorphAdorner includes scripts for updating XML files in the light of approved emendations. It also keeps track of changes made. This system works well in environments where curators operate in a controlled and work-like environment, cycling through textual phenomena selected by some criteria likely to return incomplete or incorrect text. It works well because most of the errors in the TCP transcriptions are of a very simple kind that lends itself to an 'atomic' word by word treatment.The combination of Django and a relational database makes it very easy to keep track of who did what where and when, and who approved or rejected an emendation. These are non-trivial virtues, and there are many TCP errors that can be fixed by this method before you run up against its limits. On the other hand, this method does not support what I call "curation en passant," readers reading "with a pencil" or its digital equivalent, stopping here or there to question a word or passages and offering a diagnosis or emendation as a marginal gloss. I would like to have an curation environment that looks like a page that supports reading of the common garden variety but through a click on a word shows a pop-up window that lists the properties of a token and offers a template for structured and free-form annotation. The current model also fails when it comes to making changes in the XML structure. In terms of the immediate needs of a project that focus on EEBO-TCP drama texts, the most common errors involve just the correction of a tag name: turning <stage> into <l> or the other way round. There are ways in which these errors could be corrected by extending AnnoLex, but it would be clumsy, and it would not support more substantial changes in the XML structure. So you want a curation environment that lets you manipulate the XML structure as well as the words inside elements. For instance, half the plays in the EEBO-TCP collections are not divided into acts and scenes because such a division is lacking in the printed source. Adding scene divisions (with an appropriate notation that they are supplied by an editor) is an important step towards making the texts more "machine actionable," and a proper digital corpus should be both human readable and machine actionable. The best curation environment would be XML based, but I wonder about scale. As I understand it, 'scale' in an XML application is a function of the number of documents, their size, and the complexity of encoding. The SHC corpus has about 500 documents and could grow to 1,000 if one interpreted "contemporaries" more generously or decided to include Shakespeare's Restoration heirs. Most plays fall in a range between 15,000 and 25,000 words. In a linguistically annotated corpus the typical leaf node is a <w> or <pc> element with a set of attributes, such as <w xml:id="A04632_0500750" lemma ="Cleopatra", ana="#npg1">Cleopatraes</w> The modal XPath of such a leaf node goes like TEI/text/body/div@act/div@scene/sp/l/w There are about 12 million Xpaths in the SHC corpus, and it is important for readers to be able to move quickly from one to the other within and across plays so as to support what I call "cascading curation," where the diagnosis of an error in one play leads to the question whether there are similar errors elsewhere. So much for scale, and I don't know whether it is a big problem, a small problem, or no longer a problem at all. If you want to extend a curation model from 500 plays to the eventually 70,000 texts in the EEBO TCP corpus there might be a scale problem. On the other hand, if you think of collaborative curation in a humanities environment and you think of scholarly data communities that organize themselves around some body of texts, it may be that something like 1,000 texts or 50 millon words words is a good enough upper limit: breaking down the big city of a large corpus into "corpus villages" (perhaps with some overlap) may be technically and socially preferable. I don't have a very clear idea how a permissions regime would work in an XML environment. It is clear to me that a change of any kind should take the form of an annotation and that the integration of such an annotation into the source text would be the work of an editor with special privileges. There needs to be a clear trail of the 'who, what, when, and where' of any textual change, and as much a possible of this detail should be logged automatically. The scholarly communities I'm familiar with are unlikely to accept texts as curated unless there are familiar seals of good housekeeping that they recognize as the equivalent of the assurances that come with a reputable printed text. I could go on in considerable detail about the needs of our project (and similar ones), but I hope I have outlined enough of the requirements in sufficient detail. Thanks in advance for any advice Martin Mueller Professor emeritus of English and Classics Northwestern University

2 1

Reg : Over Laping Exception
by Chandrasekhar 29 Aug '14

29 Aug '14

Dear Team We are getting the below exception wen xquery executing.Please do the needful org.basex.core.BaseXException: Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk(a)mailman.uni-konstanz.de Version: BaseX 7.9 Java: Oracle Corporation, 1.7.0_07 OS: Windows 7, x86 Stack Trace: java.nio.channels.OverlappingFileLockException at sun.nio.ch.SharedFileLockTable.checkList(FileLockTable.java:255) at sun.nio.ch.SharedFileLockTable.add(FileLockTable.java:152) at sun.nio.ch.FileChannelImpl.tryLock(FileChannelImpl.java:1017) at org.basex.io.random.TableDiskAccess.lck(TableDiskAccess.java:188) at org.basex.io.random.TableDiskAccess.sharedLock(TableDiskAccess.java:175) at org.basex.io.random.TableDiskAccess.<init>(TableDiskAccess.java:88) at org.basex.data.DiskData.init(DiskData.java:122) at org.basex.data.DiskData.<init>(DiskData.java:92) at org.basex.core.cmd.Open.open(Open.java:68) at org.basex.query.QueryResources.open(QueryResources.java:263) The information contained in this e-mail and any accompanying documents may contain information that is confidential or otherwise protected from disclosure. If you are not the intended recipient of this message, or if this message has been addressed to you in error, please immediately alert the sender by reply e-mail and then delete this message, including any attachments. Any dissemination, distribution or other use of the contents of this message by anyone other than the intended recipient is strictly prohibited.

2 5

query across multiple databases.
by Kunal Chauhan 28 Aug '14

28 Aug '14

Hello, I want to fire an XQuery across multiple databases. how to achieve this ? -- *Kunal Chauhan* mail4ck(a)gmail.com [+918655517141]

2 3

Looking for something that may not exist...
by France Baril 27 Aug '14

27 Aug '14

Hi, I'm working on improving our tracking capabilities. I was hoping to find a function that would do the equivalent of what the code bellow does, but for internal functions that are not called through the rest interface. *string-join*(*for **$name* *in **request:parameter-names*() *return **$name* || ': ' || *string-join*(*request:parameter*(*$name*), '; '), '
') In short, I was wondering if there is way I can read the function's parameters and their values without listing each one explicitly. I searched the documentation without success. -- France Baril Architecte documentaire / Documentation architect france.baril(a)architextus.com

4 13

Cannot add schema (xsd) files
by irisDeveloper 27 Aug '14

27 Aug '14

Hi all Command Add /tmp/schemaAndXml/ above command only adds .xml files not xsd files. Can you suggest to make it add xsd files also Thanks Samby

2 2

Error while adding
by irisDeveloper 27 Aug '14

27 Aug '14

Hi all, Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk(a)mailman.uni-konstanz.de Version: BaseX 7.8 Java: Sun Microsystems Inc., 1.6.0_30 OS: Linux, amd64 Stack Trace: java.lang.RuntimeException: Data Access out of bounds: - pre value: 48280236 - #used blocks: 188573 - #total locks: 188573 - access: 188572 (188573 > 188572] at org.basex.util.Util.notExpected(Unknown Source) at org.basex.io.random.TableDiskAccess.cursor(Unknown Source) at org.basex.io.random.TableDiskAccess.read5(Unknown Source) at org.basex.data.Data.textOff(Unknown Source) at org.basex.data.DiskData.text(Unknown Source) at org.basex.index.resource.Docs.paths(Unknown Source) at org.basex.index.resource.Docs.insert(Unknown Source) at org.basex.index.resource.Resources.insert(Unknown Source) at org.basex.data.Data.insert(Unknown Source) at org.basex.core.cmd.Add.run(Unknown Source) at org.basex.core.Command.run(Unknown Source) at org.basex.core.Command.execute(Unknown Source) at org.basex.server.LocalSession.execute(Unknown Source) at org.basex.server.Session.execute(Unknown Source) at org.basex.core.Main.execute(Unknown Source) at org.basex.core.Main.execute(Unknown Source) at org.basex.core.Main.console(Unknown Source) at org.basex.BaseX.<init>(Unknown Source) at org.basex.BaseX.main(Unknown Source) Thanks Samby

2 2

Lucene implementation?
by Rameshwar Nagpure 27 Aug '14

27 Aug '14

Hi there >From this link https://github.com/BaseXdb/basex/pull/986 , I cam to know that hereafter BaseX will have Lucene XQuery Module. Thanks a lot. Please tell me, with which version can I use it ?

2 1

request:http()
by Ioan Fericel 27 Aug '14

27 Aug '14

Hi, congratulations for the job well done on this project! I'm having a problem that I hope you can help me with some guidance. A form made with XForms send data from a instance - with submission, POST method - in a function xq from BaseX, to be saved in the database, and can not find how to retrieve data from the request. We found here ( http://exquery.github.io/expath-specs-playground/request-modules-1.0-specif… ) specifications for request:http() function, that seems to be taking all content submitted by post, but not yet implemented. Are likely to be implemented soon, and really do this? What other methods would be to get body content? I found this guidance ( https://www.mail-archive.com/basex-talk%40mailman.uni-konstanz.de/msg03171.… ) to function http: send-request, but I did not understand how they could solve this. Many thanks, Ioan

2 2

fn:trace improvement
by Marc van Grootel 26 Aug '14

26 Aug '14

Hi, Would it be possible to add automatic serialization of map and array types in fn:trace? When using it should be transparent but currently providing a map or array fails. To trace it I now use awkward code to do it since I have to call map:serialize and array:serialize myself. E.g. (fn:trace(map:serialize($map, 'X: ')), func($map))[2] Cheers, --Marc

2 2

Apply variable argument list to anonymous function
by Marc van Grootel 26 Aug '14

26 Aug '14

Hi, I am trying to call an anonymous function with a variable argument list. Maybe I'm overlooking something within XQuery but I cannot figure out how to do this. This is the code I would like to get working. declare function local:apply($fn, $args) { $fn($args) }; declare variable $concat := fn:concat#3; declare variable $args := ('a', 'b', 'c'); (: needs to handle any list of strings :) local:apply($concat, $args) (: => 'abc' :) What I want local:apply to do is similar to apply in some function languages such as Clojure ( http://clojuredocs.org/clojure_core/clojure.core/apply) In the code above I have two problems. a) how to bind the fn:concat function to a variable. Arity? #1, #2 ... ? b) how to "apply" a variable argument list to the function passed in to local:apply. I figure I could do something maybe with some other higher order functions but I cannot see the solution. I don't mind rtfm responses, I may have missed it. I keep bombarding the list with questions so it is only fair to give a bit of context. I am porting parts of a few small Clojure libraries to XQuery, most important libraries are Ring and Compojure which are libraries for use in web frameworks. My implementation currently builds on top of RESTXQ but eventually can provide an alternative way of handling HTTP routing requests through function handlers (without function annotations). This is similar to WSGI in Python and Rack in Ruby. I would've liked to release this sooner but it is more work than anticipated and I want to provide something that does these ideas justice. So it's ready when it's ready ;-) but I'm pretty close. I'm currently finalizing the routing part and need to work some more on middleware handlers. Until then go look at https://github.com/ring-clojure to see what this is about. --Marc

5 15

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

BaseX-Talk