June 2010 - BaseX-Talk - mailman.uni-konstanz.de

Student project based on BaseX
by Antonis Thomas 22 Aug '10

22 Aug '10

Hello all, we are two master students at the university of Utrecht. We have a DB Architecture class that includes a small project (15%*7.5 ECTS= 1.125 ECTS). So, BaseX looks like a good chance to work on such a project. Our idea so far is based on some benchmarking experiments and small summary on how it works (maybe its relation with Monet) including any experiments we run (5-6 pages). Do you have anything better for us to do (maybe something you can get advantage of yourselves) or any guidelines/hints/whatever? Take into account that out time limit is approximately a week (full-time) and that we cannot do more work that what deserves 2.25ECTS for the 2 of us. I look forward to an answer (so we can decide and start working -the deadline is 9/7-. Best regards, -- Antonis Thomas Information & Computing Sciences Utrecht University Utrecht, The Netherlands

2 13

Data limit?
by judith.risse＠wur.nl 09 Aug '10

09 Aug '10

Hello, I'm trying to load a large dataset (Medline, 16.7 million records, 66Gb data on disk) into BaseX. The input data is divided into 563 files of 30000 records each. When I try to load the data into a database with the following command: create db path/dir medline_full I get the following error message: Error: "medline08n0507.xml" (Line 2620386): Document is too large for being processed. The document itself can be loaded into a seperate database without problems and I also managed to load a merged file containing 1 million records, so the file size itself seems not to be the problem. I used the gui implementation running on a quadcore server with SuSe 10.2 64bit and 8Gb of memory, although the java version is 32bit 1.6.0_18 Therefore my question, does BaseX have a datalimit, either on file-, record- or nodelevel? Or are there hardware limitiations that could result in such an error message? With kind regards Judith

4 3

Re: [basex-talk] Abstürze basex/DeepFS bei vielen Dateien
by Christian Grün 01 Jul '10

01 Jul '10

Hallo Markus, im aktuellen DeepFS-Code kenne ich mich nicht bis ins Detail aus. Ich leite Deine Mail mal an Bastian L. und Alex weiter; vielleicht haben sie ein paar Ideen? Grüße, Christian ___________________________ 2010/6/17 Markus Roth <markus.roth(a)uni-konstanz.de>: > Hallo Christian, > > ich bin derzeit ein wenig am Rumspielen mit DeepFS und es tritt > kontinuierlich ein Fehler auf (immer zum gleichen Zeitpunkt): too many open > files. > Wenn ich das Limit für offene file handles hochsetze per ulimit -n (standard > in der bash sind 1024), dann kommt der Fehler später, aber trotzdem. > Siehe dazu weiter unten. > Warum sollte baseX aber so viele offene file handles haben? Mit dem > Lesen/Schreiben zusammen sollten es ja nicht so arg viele sein. Oder werden > u.U. (da die Traversierungs-Fkt. wohl rekursiv sein dürfte) je nach Tiefe > des aktuellen Pfades die Ascendants offen gelassen? Da der Fehler in der > Konsole mehrfach ausgegeben wird bis zum Abbruch: gibt es mehrere parallele > "crawler"/Indexer? > Inwiefern die Volltext-Indexierung bzw. inkludieren der Textdaten Einfluß > hat habe ich noch nicht betrachtet (also die folgenden Daten/Fehler sind mit > den Standardvoreinstellungen von DeepFS OHNE Volltext-Indexierung und > Inklusion von Text entstanden). > > > Gruß, > Markus > > > Linux (CentOS 5.4 final) || basex 6.1 & 6.1.5 -> too many open files > ca. 115k Dateien > XML-Nodes: ulimit: > 1,74Mio 1024 > 1,75Mio 2048 > 1,77Mio 4096 > 1,91Mio 16384 > 3,38Mio 200000 > > Windows XP SP3 hatte mit ca. 73k Dateien keine Probleme (konnte auch nicht > rausfinden wie hoch da das OFH ist) > > > Auszug aus der Fehlermeldung: > > java.io.FileNotFoundException: /root/BaseXData/DeepFS/tbli.basex (Too many > open files) > at java.io.FileOutputStream.open(Native Method) > at java.io.FileOutputStream.<init>(FileOutputStream.java:209) > at java.io.FileOutputStream.<init>(FileOutputStream.java:160) > at org.basex.io.DataOutput.<init>(DataOutput.java:49) > at org.basex.io.DataOutput.<init>(DataOutput.java:38) > at org.basex.io.TableDiskAccess.flush(TableDiskAccess.java:77) > at org.basex.data.DiskData.flush(DiskData.java:122) > at org.basex.data.DiskData.cls(DiskData.java:134) > at org.basex.data.Data.close(Data.java:128) > at org.basex.core.proc.Close.close(Close.java:46) > at org.basex.core.proc.Close.run(Close.java:29) > at org.basex.core.Proc.run(Proc.java:139) > at org.basex.core.Proc.run(Proc.java:163) > at org.basex.core.proc.Open.run(Open.java:30) > at org.basex.core.Proc.run(Proc.java:139) > at org.basex.core.Proc.run(Proc.java:163) > at org.basex.core.proc.CreateFS.run(CreateFS.java:54) > at org.basex.core.Proc.run(Proc.java:139) > at org.basex.core.Proc.exec(Proc.java:109) > at org.basex.core.Proc.exec(Proc.java:123) > at org.basex.gui.GUICommands$49.run(GUICommands.java:931) > /root/.basexwin could not be written. > >

2 1

basex
by evan＠wingonwing.com 28 Jun '10

28 Jun '10

Hello, I would like to do some development with basex. I downloaded the project and am looking for a book or documentation to print out. What sort of references do you suggest. I am looking for more documentation than I found on the main page documentation from your website. Thanks, Evan

2 1

Reminder about your invitation from sayali Kulkarni
by sayali Kulkarni (LinkedIn Invitations) 28 Jun '10

28 Jun '10

LinkedIn ------------ This is a reminder that on June 21, sayali Kulkarni sent you an invitation to become part of his or her professional network at LinkedIn. Follow this link to accept sayali Kulkarni's invitation. https://www.linkedin.com/e/87foxj-gaz2pgqu-5s/doi/1401339739/ZCnLLJ4a/gir_8… Signing up is free and takes less than a minute. On June 21, sayali Kulkarni wrote: > To: [basex-talk(a)mailman.uni-konstanz.de] > From: sayali Kulkarni [sayalikulkarni7(a)gmail.com] > Subject: sayali Kulkarni wants to stay in touch on LinkedIn > > I'd like to add you to my professional network on LinkedIn. > > - sayali Kulkarni The only way to get access to sayali Kulkarni's professional network on LinkedIn is through the following link: https://www.linkedin.com/e/87foxj-gaz2pgqu-5s/doi/1401339739/ZCnLLJ4a/gir_8… You can remove yourself from sayali Kulkarni's network at any time. -------------- ------ (c) 2010, LinkedIn Corporation

1 0

Small project based on BaseX
by iorulezz＠gmail.com 27 Jun '10

27 Jun '10

Hello all, we are two master students at the university of Utrecht. We have a DB Architecture class that includes a small project (15%*7.5 ECTS= 1.125 ECTS). So, BaseX looks like a good chance to work on such a project. Our idea so far is based on some benchmarking experiments and small summary on how it works (maybe its relation with Monet) including any experiments we run (5-6 pages). Do you have anything better for us to do (maybe something you can get advantage of yourselves) or any guidelines/hints/whatever? Take into account that out time limit is approximately a week (full-time) and that we cannot do more work that what deserves 2.25ECTS for the 2 of us. I look forward to an answer (so we can decide and start working -the deadline is 9/7-. Best regards, Antonis

1 0

Small project based on BaseX
by iorulezz＠gmail.com 27 Jun '10

27 Jun '10

Hello all, we are two master students at the university of Utrecht. We have a DB Architecture class that includes a small project (15%*7.5 ECTS= 1.125 ECTS). So, BaseX looks like a good chance to work on such a project. Our idea so far is based on some benchmarking experiments and small summary on how it works (maybe its relation with Monet) including any experiments we run (5-6 pages). Do you have anything better for us to do (maybe something you can get advantage of yourselves) or any guidelines/hints/whatever? Take into account that out time limit is approximately a week (full-time) and that we cannot do more work that what deserves 2.25ECTS for the 2 of us. I look forward to an answer (so we can decide and start working -the deadline is 9/7-. Best regards, Antonis

1 0

Small project based on BaseX
by iorulezz＠gmail.com 27 Jun '10

27 Jun '10

Hello all, we are two master students at the university of Utrecht. We have a DB Architecture class that includes a small project (15%*7.5 ECTS= 1.125 ECTS). So, BaseX looks like a good chance to work on such a project. Our idea so far is based on some benchmarking experiments and small summary on how it works (maybe its relation with Monet) including any experiments we run (5-6 pages). Do you have anything better for us to do (maybe something you can get advantage of yourselves) or any guidelines/hints/whatever? Take into account that out time limit is approximately a week (full-time) and that we cannot do more work that what deserves 2.25ECTS for the 2 of us. I look forward to an answer (so we can decide and start working -the deadline is 9/7-. Best regards, Antonis

1 0

Small project based on BaseX
by iorulezz＠gmail.com 27 Jun '10

27 Jun '10

Hello all, we are two master students at the university of Utrecht. We have a DB Architecture class that includes a small project (15%*7.5 ECTS= 1.125 ECTS). So, BaseX looks like a good chance to work on such a project. Our idea so far is based on some benchmarking experiments and small summary on how it works (maybe its relation with Monet) including any experiments we run (5-6 pages). Do you have anything better for us to do (maybe something you can get advantage of yourselves) or any guidelines/hints/whatever? Take into account that out time limit is approximately a week (full-time) and that we cannot do more work that what deserves 2.25ECTS for the 2 of us. I look forward to an answer (so we can decide and start working -the deadline is 9/7-. Best regards, Antonis

1 0

Small project based on BaseX
by iorulezz＠gmail.com 27 Jun '10

27 Jun '10

Hello all, we are two master students at the university of Utrecht. We have a DB Architecture class that includes a small project (15%*7.5 ECTS= 1.125 ECTS). So, BaseX looks like a good chance to work on such a project. Our idea so far is based on some benchmarking experiments and small summary on how it works (maybe its relation with Monet) including any experiments we run (5-6 pages). Do you have anything better for us to do (maybe something you can get advantage of yourselves) or any guidelines/hints/whatever? Take into account that out time limit is approximately a week (full-time) and that we cannot do more work that what deserves 2.25ECTS for the 2 of us. I look forward to an answer (so we can decide and start working -the deadline is 9/7-. Best regards, Antonis

1 0