BaseX-Talk July 2016

basex-talk@mailman.uni-konstanz.de

33 participants
52 discussions

db:open('db') update {}
by Maximilian Gärber 17 Jul '16

17 Jul '16

from another thread is a big one (at least for me): you can force a db to be fully loaded to memory I've added it here, maybe another location would be better but until then: http://docs.basex.org/wiki/Databases#In_Memory_Database Regards, Max

1 0

Dynamic lookup of namespaces
by buddyonweb-software＠yahoo.com 17 Jul '16

17 Jul '16

Is there a way to get a list of the namespaces declared/used? I am wanting to right a function that does a function lookup, but I want to right it in such a way that I don't have to know the namespaces ahead of time. Are there any functions that will return a list of the namespaces being used? Thanks in advance, Buddy

3 2

question on document size limitation
by Wang, Genneva 13 Jul '16

13 Jul '16

Hi Basex gurus, I was wondering if basex has publish any recommended document size limitation ? (for both document file and binary file) ? Thanks, Genneva

2 1

Off topic: on leave
by Bram Vanroy | KU Leuven 13 Jul '16

13 Jul '16

Hello everyone I will be leaving on vacation soon, from 13th til 25th of July. I set an automated reply for my email account, but I'm not sure if it is smart enough to detect mailing lists. So if the programme is not smart enough, and you keep getting automated replies, feel free to manually unsubscribe me. I haven't done that myself, because I'd like to read the list when I get back. Good bye!

2 1

BaseX EMail-Module
by Ulrike Schaper - SideStep Business Solutions GmbH 13 Jul '16

13 Jul '16

Hi everybody, I know this issue has been discussed already, but I still have problems concerning BaseX and sending E-Mails: I downloaded Email.jar which has been described in earlier posts, and put it into the basex/lib folder. I found examples in the jars xquery subdirectory and tried to execute them. I get the following error: EMail:send(element()): org.basex.query.func.FuncOptions.parse(Lorg/basex/query/value/item/Item;Lorg/basex/util/options/Options;)Lorg/basex/util/options/Options;. Can you help? Best regards, Ulrike.

3 4

Who uses XQuery update? Dedicated wiki page for index updates?
by Maximilian Gärber 12 Jul '16

12 Jul '16

Hi, I was just thinking there could be other users that stumble over the details of index updates and optimization. Maybe this deserves a top-level page in the wiki? Then we would have: * Indexes: Explains about what is there * XQuery Update: How to use * Index Configuration: How to configure and good practice Regards, Max

4 4

UPDINDEX + AUTOOPTIMIZE
by Maximilian Gärber 12 Jul '16

12 Jul '16

Hi, the docs do not say anything about combining those options. Does it mean that optimiize now only has to create fresh statistics? Or does AUTOOPTIMIZE make the UPDINDEX pointless? As well, I used "statistics" here because this term is used in the docs. What does it actually meean? Is it the structural indexes? Lastly, UPDINDEX will leave the UPTODATE flag set to false after updates. What effect does this have? Regards, Max

2 2

Re: [basex-talk] Removing the xmlns attribute and/or adding the prefix
by Henning Phan 11 Jul '16

11 Jul '16

Hi, Emailed to announce by mistake. The stacktrace as requested > xquery for $e in //* let $m := $e/@mark return if($m) then ( delete node $m, (: [.] is same as: where $d ne "" :) for $d in data($m)[.] return ( delete node $e/text() ) ) else ( delete node $e ) java.lang.OutOfMemoryError: Java heap space at java.util.Arrays.copyOf(Unknown Source) at org.basex.util.hash.IntSet.rehash(IntSet.java:125) at org.basex.util.hash.IntObjMap.rehash(IntObjMap.java:55) at org.basex.util.hash.ASet.checkSize(ASet.java:82) at org.basex.util.hash.IntSet.index(IntSet.java:109) at org.basex.util.hash.IntSet.put(IntSet.java:49) at org.basex.util.hash.IntObjMap.put(IntObjMap.java:29) at org.basex.query.up.DataUpdates.add(DataUpdates.java:77) at org.basex.query.up.ContextModifier.add(ContextModifier.java:56) at org.basex.query.up.DatabaseModifier.add(DatabaseModifier.java:40) at org.basex.query.up.Updates.add(Updates.java:96) at org.basex.query.up.expr.Delete.item(Delete.java:44) at org.basex.query.expr.ParseExpr.iter(ParseExpr.java:44) at org.basex.query.QueryContext.iter(QueryContext.java:409) at org.basex.query.expr.If.iter(If.java:126) at org.basex.query.expr.gflwor.GFLWOR$1.next(GFLWOR.java:100) at org.basex.query.MainModule.cache(MainModule.java:100) at org.basex.query.QueryContext.iter(QueryContext.java:345) at org.basex.query.QueryProcessor.iter(QueryProcessor.java:81) at org.basex.core.cmd.AQuery.query(AQuery.java:90) at org.basex.core.cmd.XQuery.run(XQuery.java:22) at org.basex.core.Command.run(Command.java:398) at org.basex.core.Command.execute(Command.java:100) at org.basex.api.client.LocalSession.execute(LocalSession.java:132) at org.basex.api.client.Session.execute(Session.java:36) at org.basex.core.CLI.execute(CLI.java:103) at org.basex.core.CLI.execute(CLI.java:87) at org.basex.BaseX.console(BaseX.java:191) at org.basex.BaseX.<init>(BaseX.java:166) at org.basex.BaseX.main(BaseX.java:42) org.basex.core.BaseXException: Out of Main Memory. at org.basex.core.Command.execute(Command.java:101) at org.basex.api.client.LocalSession.execute(LocalSession.java:132) at org.basex.api.client.Session.execute(Session.java:36) at org.basex.core.CLI.execute(CLI.java:103) at org.basex.core.CLI.execute(CLI.java:87) at org.basex.BaseX.console(BaseX.java:191) at org.basex.BaseX.<init>(BaseX.java:166) at org.basex.BaseX.main(BaseX.java:42) Out of Main Memory. ________________________________ Från: Christian Grün <christian.gruen(a)gmail.com> Skickat: den 9 juli 2016 22:24 Till: Henning Phan Ämne: Re: [basex-talk] Removing the xmlns attribute and/or adding the prefix > for $e in //* let $m := $e/@mark return if($m) then ( delete node > $m, (: [.] is same as: where $d ne "" :) for $d in data($m)[.] > return ( delete node $e/text() ) ) else ( delete node $e ) > > Runs out of memory. Thanks for trying. Could you please pass me on the stack trace? If it doesn't show up immediately, you may need to switch on debugging (via SET DEBUG true, or -d on command line).

3 3

Improving performance in a 40GB database
by James Sears 11 Jul '16

11 Jul '16

Hi - I'm wondering what sort of recommendations anyone has for making my queries "go faster". I created a database by loading 21GB of xml data - that represents the OCR'd contents of 5000 books - into a new basex 8.4.4 instance. I run optimize after the import. basexgui> Properties> Information says it's sized at 40GB and contains around 1.5 billion nodes; there are no binaries. The sorts of query I'm interested in running are those that search each article on each page of each book - for example: for $book in //book return <result> <book id="{$book/id/text()}"/> { for $page in $book/page return <page id="{$page/id/text()}"> { for $article in $page/article return <article id="{$article/id/text()}"/> } </page> } </result> On a reasonably powerful i5 laptop with SSD + plenty of RAM, this query takes around 148550.00 ms - I'd like to significantly reduce this. Individual OCR'd words on pages maybe comprise around 85% of the data - and I don't actually care about this data. So maybe if I just don't load these OCR'd words it will help? I haven't tried that yet, but ideally I'd like not to have to do it. So, if anyone has any tips or ideas for reducing the query time then I'd be very interested in hearing what you have to say.

4 8

db:attribute and eq unexpected behaviour
by Alex Muir 09 Jul '16

09 Jul '16

Hi, I have XML data with attributes that I've set in attribute indexes with the following consistent format but with different values form13FFileNumber="028-17473" form13FFileNumber="028-16573" When I query the data with following without indexes let $data := db:open('13F')//data[@form13FFileNumber eq $form13FFileNumber] or with the db:attribute method using indexes let $data := db:attribute("13F", $form13FFileNumber, "form13FFileNumber")/.. with both queries I get extra data and the same extra data So if I try to extract form13FFileNumber 028-11185 for example I get 028-11185 as the first result but also 028-13947 and others... Similarly for 028-10601 I get data for 028-10601 and 028-17017 and others does the dash with numbers cause a problem with the indexing? XQUERY Getting result xquery version "3.0"; declare namespace db="http://basex.org/modules/db"; declare namespace file="http://expath.org/ns/file"; declare variable $form13FFileNumber as xs:string external; (:let $data := db:open('13F')//data[@form13FFileNumber eq $form13FFileNumber] :) let $data := db:attribute("13F", $form13FFileNumber, "form13FFileNumber")/.. let $fields := <form13FFile> { for $c in $data return $c } </form13FFile> return file:write(concat('../OUTPUT/form13Restatment3rd_' , $form13FFileNumber , '.xml'), $fields) BD CREATE COMMANDS: <?xml version="1.0" encoding="UTF-8"?> <commands> <set option='attrinclude'>xml:id,filingDate,type,cik,sequence,form13FFileNumber,periodOfReport</set> <create-db name='13F'/> <info-db/> </commands> The xml element that has all the indexes <data xml:id="0001534270-16-000006" filingDate="2016-06-08" type="13F-HR" cik="1534270" sequence="000006" form13FFileNumber="028-17473" periodOfReport="03-31-2016"> The basex 744 db is around 8gb and data was added with this xquery let $docs := collection($path) for $doc in $docs return let $path := concat($doc/data/@periodOfReport,'/',$doc/data/@form13FFileNumber,'/',$doc/data/@xml:id,'.xml') return db:add('13F', $doc, $path) Regards Alex tech.jahtoe.com bafila.jahtoe.com

2 2

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

BaseX-Talk July 2016