BaseX-Talk

basex-talk@mailman.uni-konstanz.de

13 participants
5244 discussions

performance question
by Cerstin Mahlow 18 Jan '12

18 Jan '12

Hi, I would like to set up a collection of TEI-annotated texts (novels, dramas, poems, etc.). In total, it would be around 3 GB XML data in some 1000 files, the text size varies from 29 KB to 94 MB. I have a server running with Java 1.6.0_07 on CentOS 5.7 on a Virtual Machine with 1 GB RAM. I started to add files to the database and wrote a preliminary query interface (<http://oldphras.unibas.ch/cgi-bin/basex-client.pl>). Since we want to look for examples of multi-word units, I would like to use queries like: //(p|l) [text() contains text "Korb geben" using stemming using language "de"] (In the end, queries will be more complex to allow users to search for several words in different word order within a sentence using stemming or fuzzy) To make inspection of results easier, I added ft:mark. A collection with only a dozen of texts of about 71 MB with full text index for German, optimized, etc. works quite well. However, the example query needs more than 9s, which is rather slow. What is worse: Adding more files, resulting in about 323 MB, causes a timeout when running the query. I already set the memory for the Java VM to 1024, but it does not help. I tried it with the GUI on my iMac with 4 GB RAM and got a time out when the collection size is above 900 MB (which is still only a small part of my data). Is there any recommendation for size of RAM or specific settings when processing collections of about 3 GB? Is there a better way to write queries when looking for inflected forms of several words and allowing for spelling errors? Thank you in advance Cerstin -- Dr. phil. Cerstin Mahlow Universität Basel Deutsches Seminar Nadelberg 4 4051 Basel Schweiz Tel: +41 61 267 07 65 Fax: +41 61 267 34 40 Mail: cerstin.mahlow(a)unibas.ch Web: http://www.oldphras.net ---------------------------------------------------------------- This message was sent using IMP, the Internet Messaging Program.

3 12

workaround for lack of http_proxy support
by jidanni＠jidanni.org 17 Jan '12

17 Jan '12

Whilst waiting for Debian's basex to support http(s)_proxy, I found a workaround: installing the caching proxy WWWOFFLE, one then can use $ basex -q 'doc("http://localhost:8080/http/example.org/api?xyz")' without which I would very fast exceed daily API usage limits for mere repetitive tests.

1 0

BaseX 7.1, XMLPrague, Review
by Christian Grün 16 Jan '12

16 Jan '12

Hi all, thanks for your continuous feedback and support (such as contributing to our Wiki, to our code base, or giving feedback on this list). For those who have noticed that BaseX 7.1 was announced for the middle of January: we will be postponing the release a little and align it with the upcoming XMLPrague conference. Next, our XMLPrague BaseX Meetup has now been confirmed. It will take place on Feb 10 (Friday) from 09:00-11:30 am. Please find more information on the XMLPrague and on our homepage soon [1]. Last but not least, we are glad to report that 2011 was a very successful year for BaseX, both regarding the commercial branch as well as the open source development! Rest assured that 2012 will get exciting again.. Stay tuned, Christian [1] http://www.xmlprague.cz/

1 0

Re: [basex-talk] URI for xquery : collection()
by Christian Grün 16 Jan '12

16 Jan '12

Dear Francesco, in oXygen, if your databases are locally stored, you can directly address the database you are working with by using e.g. the db:open() function: for $text in db:open("lettres_20120109", "Info2085.xml")/tei:TEI .... Currently, however, you'lll not be able to traverse collections on a remote server via oXygen; in this case, you'll have to resort to one of the other APIs/tools, or wait for an updated version of our XQJ implementation, which will also support remote connections. Hope this helps, more questions are welcome, Christian On Mon, Jan 16, 2012 at 7:12 PM, Francesco Beretta <Francesco.Beretta(a)ish-lyon.cnrs.fr> wrote: > Hi, > > as a beginner with BaseX, I'm trying to execute a xquery via an OxygenXML > transformation scenario on an external server. > > I created the data source and connection as explained in the BaseX wiki. > > The xquery functions well on a single document, with following connection > syntax : > > "for $text in > doc('http://admin:*****@10.211.55.5:8984/webdav/lettres_20120109/Info2085.xml')/tei:TEI > ..." > > But it doesn't work against the whole database if I write : > "for $text in > collection('http://admin:*****@10.211.55.5:8984/webdav/lettres_20120109')/tei:TEI > ...". > > It works fine if I use the baseX GUI on the 10.211.55.5 server (which > becomes a localhost) : > "for $text in collection('lettres_20120109')/tei:TEI" > > What I'm doing wrong ? > > Many thanks and greetings > > Francesco Beretta

1 0

webapps
by Manuel Müller 15 Jan '12

15 Jan '12

Hallo, habe heute meine Datenbank mit der neuesten BaseX Version erstellt. Seit dem bekomm ich folgenden Fehler: Problem accessing /app/books/. Reason: 20 Caused by: java.lang.ArrayIndexOutOfBoundsException: 20 at org.basex.data.StatsKey.<init>(StatsKey.java:63) at org.basex.index.Names.<init>(Names.java:47) at org.basex.data.DiskData.<init>(DiskData.java:60) at org.basex.core.cmd.Open.open(Open.java:81) at org.basex.core.cmd.Check.check(Check.java:86) Weiss jemand woran das liegt? Grüße Manuel

2 1

Мобильная Дисконтная Карта
by Discountka.ru 15 Jan '12

15 Jan '12

Мобильная Дисконтная Карта в Вашем телефоне Мобильная Дисконтная Карта?!! Все просто, Вы подаете заявление ( http://79.174.79.242/kabinet/73 ), регистрируетесь ( http://79.174.79.242/kabinet/73 ) в системе и скачиваете в "Личном кабинете" Единую Дисконтную Карту на свой мобильный телефон. Обращаясь к Партнеру Компании DISCOUNTKA, Вам достаточно просто предъявить Мобильную Дисконтную Карту в своем телефоне и получайте обещанные скидки. Кто Партнеры? Организации которые размещены на сайте www.discountka.ru, у них же Вы сможете скачать их собственные дисконтные карты, купоны, сертификаты Партнера Компании DISCOUNTKA, которые так же предоставляют право на скидки, акции, распродажи. Сколько это стоит? Нисколько... Ровным счетом ничего, кроме того Вы можете получить подарки от Партнеров Программы. Какие? Следите за новостями на сайте, там Вы найдете ответы на Ваши вопросы. И не только... Вопросы? Все вопросы по телефону: +7 (843) 520-86-41 на сайте www.discountka.ru или в m.discountka.ru ( http://m.discountka.ru/ )

1 0

http:request post parameter bug? basex 7.01
by Andy Bunce 15 Jan '12

15 Jan '12

Hi, Could there be an encoding issue with http:request post requests? I send.. let $r:=<http:request method='post' > <http:body media-type="application/x-www-form-urlencoded" method="text">a=3&b=4</http:body> </http:request> return http:send-request($r,"http://posttestserver.com/post.php")[2] http://www.posttestserver.com/ reports the parameters as.. Post Params: key: 'a' value: '3' key: 'amp;b' value: '4' Empty post body. /Andy

2 7

Re: [basex-talk] line number and column
by Dave Glick 15 Jan '12

15 Jan '12

I had a similar requirement a while back and ended up solving it in two ways: The first attempt I wrote a routine that would take a node and scan for all text nodes along the preceding axis. I would then add the length of each text node to a counter (I wanted offset), but counting new lines shouldn't be much different. I used the API, but I don't think it would be too tough to write an XQuery expression or function that does the same thing. Alternatively you could use the BaseX functionality to call Java code from XQuery to write it up in code. This worked okay in isolated cases, but I needed the info very frequently (I was writing a text editor backed by XML, so I was constantly getting offsets for the caret) and scanning the preceding axis each time took too long. I ended up going through every node in sequence for a new document and caching the information. The idea was the same though - every time I hit a text node, I incremented a counter and used the value for following nodes (until I hit another text node). Not sure if that helps - hopefully it does... Dave -----Original message----- From: "Christian Grün" <christian.gruen(a)gmail.com> To: Anders Hessellund <anders.hessellund(a)gmail.com> Cc: "basex-talk(a)mailman.uni-konstanz.de" <basex-talk(a)mailman.uni-konstanz.de> Sent: Sat, Jan 14, 2012 11:38:06 GMT+00:00 Subject: Re: [basex-talk] line number and column Dear Anders, maybe there's another way to achieve what you want; what do you exactly need line and column information for? Christian ___________________________ On Sat, Jan 14, 2012 at 12:21 PM, Anders Hessellund <anders.hessellund(a)gmail.com> wrote: > Thanks for your replies, > > the line number and column info is pretty important for us. Is there perhaps > some part of the parser code that I could hack/fork? > > -- Anders > > > On Thu, Jan 12, 2012 at 11:59 AM, Christian Grün <christian.gruen(a)gmail.com> > wrote: >> >> Dear Anders, >> >> I'm sorry: information on the original line and columns gets lost as >> soon as a database, or a main memory instance of a document, is >> created. If you look for ways to address the same XML nodes multiple >> times, you may want to have a look at the fn:path(); it returns an >> XPath expression to the adressed nodes [1]. A simple example: >> >> for $item in doc('http://files.basex.org/xml/xmark.xml')//item >> return path($item) >> >> If you have create a database from your files, you can also use the >> BaseX-specific db:node-pre() and db:open-pre() functions to directly >> address nodes [2]. >> >> Hope this helps, >> Christian >> >> [1] http://www.w3.org/TR/xpath-functions-30/#func-path >> [2] http://docs.basex.org/wiki/Database_Module >> ___________________________ >> >> On Wed, Jan 11, 2012 at 11:28 PM, Anders Hessellund >> <anders.hessellund(a)gmail.com> wrote: >> > Hi, >> > >> > we are experimenting with BaseX as a database for a large set of local >> > XML >> > files (3000-4000). When we query for individual elements and attributes >> > in >> > this portfolio of XML documents, is there any way to get location >> > information about concrete elements and attributes. Specifically, we >> > need to >> > know filename, line number and column of element (start tags) and >> > attributes. Is this possible? And if so, how? >> > >> > Thanks, >> > Anders > > > > _______________________________________________ > BaseX-Talk mailing list > BaseX-Talk(a)mailman.uni-konstanz.de > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk > _______________________________________________ BaseX-Talk mailing list BaseX-Talk(a)mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk

2 1

xquerydoc support for BaseX
by Andy Bunce 14 Jan '12

14 Jan '12

Hi, The https://github.com/xquery/xquerydoc project is looking like a good tool to generate Xquery documentation. Currently it is failing on some BaseX extensions. See https://github.com/xquery/xquerydoc/issues/6#issuecomment-3491893. Is there a EBNF source for the map module syntax available somewhere? /Andy

2 1

line number and column
by Anders Hessellund 14 Jan '12

14 Jan '12

Hi, we are experimenting with BaseX as a database for a large set of local XML files (3000-4000). When we query for individual elements and attributes in this portfolio of XML documents, is there any way to get location information about concrete elements and attributes. Specifically, we need to know filename, line number and column of element (start tags) and attributes. Is this possible? And if so, how? Thanks, Anders

4 5

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

BaseX-Talk