BaseX-Talk

basex-talk@mailman.uni-konstanz.de

9 participants
5241 discussions

Serialization ... one attribute per line ...
by Jonathan Robie 29 Mar '22

29 Mar '22

I have a set of syntax trees that I would like to serialize with one attribute per line, like this: https://github.com/biblicalhumanities/greek-new-testament/blob/master/synta… The current serialization looks more like this: https://github.com/biblicalhumanities/greek-new-testament/blob/master/synta… Is there a way to get there using just serialization parameters, or do I need to do something fancier? Jonathan

1 0

fn:count() performance
by Josselin Morvan 29 Mar '22

29 Mar '22

Hi everyone, We are experimenting a small issue with the count() function, and I wanted to know if you have any idea to reduce the response time of the server : We have a database containing almost 5.000 descriptions of expert reports from the 18th century. As a report can take place over several years, we want to list all the years mentioned in the reports and then count for each year how many reports we have. Our goal with this query is to produce a filter by year for our web application. But, if the first part of the query is quite fast, it is not the case for the second part… here is a simplified sample of our code : xquery version "3.1"; declare default element namespace "xpr" ; (: to create the xprDB ;) db:create('xpr', 'https://raw.githubusercontent.com/anrExperts/data/master/db/xpr.xml') :) let $years := fn:distinct-values(db:open('xpr')/xpr/expertises/expertise/description/sessions/date[@when castable as xs:date]/fn:year-from-date(@when)) for $year in $years return $year || ' : ' (:this first part of the query is quite fast 0.09sec on my old computer:) || fn:count(db:open('xpr')/xpr/expertises/expertise[description/sessions/date[fn:matches(@when, xs:string($year))]]) (:it takes 5sec to execute the second part of the query:) Do you see anything we are doing wrong or we can improve to reduce the server response time ? We thank you in advance! Best, Josselin

2 2

(no subject)
by Charles Bearden 29 Mar '22

29 Mar '22

I have loaded all the Pubmed baseline XML records into a series of 20 BaseX databases, 55 or 56 of the baseline files per database, each of which is about 12GB in size and has between 520 and 530 million nodes in 55 or 56 documents. Text, Token, and Attribute indices are enabled, but with 6GB RAM allocated to the Java VM it would not create a full text index. Each of the 55 or 56 documents has 30000 article records in it under a root PubmedArticleSet element. I typically use basexgui for interactive work and basex for scripted loads & queries, and I allocate 6G to the Java VM in each case: BASEX_JVM="-Xmx6g $BASEX_JVM" I'm exploring ways to search the data in a moderately performant way, starting with the realtively simple lookup by PubMed ID: /PubmedArticleSet/PubmedArticle[MedlineCitation/PMID[text()=$pmid]] I generated a seriers of five index XML files that pair PubMed IDs with database names, like so: <index> <entry> <dbname>pmed_baseline_a</dbname> <pmid>579614</pmid> </entry> … </index> Each file contains entries for four of the 20 BaseX databases. I loaded the five files into a single database. My hope was that I could quickly lookup the name of the database that contained a record by that record's PMID, and that I could then open that collection and quickly obtain that record, but it isn't working the way I had hoped. If I query the index database by PMID, I get the answer in 156ms: let $pmid := '22345065' let $icoll := collection('idx_pmed_baseline') let $pmid_lookup := $icoll/index/entry/pmid let $entry := $pmid_lookup[text()=$pmid] let $dbname := $entry/parent::entry/dbname/text() return $dbname (: returns 'pmed_baseline_s' :) If I open and query that collection by XPath for that PMID, also get the answer quickly, in about 420ms: let $coll := collection('pmed_baseline_s') let $pmid := '22345065' let $wanted := $coll/PubmedArticleSet/PubmedArticle[MedlineCitation/PMID/text()=$pmid] return $wanted (: returns desired XML record :) But if I combine the code to run in a single execution, it takes about 40s: let $pmid := '22345065' let $icoll := collection('idx_pmed_baseline') let $pmid_lookup := $icoll/index/entry/pmid let $entry := $pmid_lookup[text()=$pmid] let $dbname := $entry/parent::entry/dbname/text() let $coll := collection($dbname) let $wanted := $coll/PubmedArticleSet/PubmedArticle[MedlineCitation/PMID/text()=$pmid] return $wanted I feel like I must be doing some simple thing wrong, but the only difference I see in my code between the two separate steps and the single execution version is that I'm passing the db name in a variable instead of as a string literal to the collection() function, and I'm running the whole thing in a single execution. Note that before each execution of an XQuery, I exited basexgui and restarted it to avoid any caching effect in memory at least. The VM I'm running on is modest (spinning drives in RAID 1, four modest AMD CPU cores, dynamic memory growth up to 32GB). But these factors would not explain the difference in speed between the two steps in separate executions and both steps in a single execution. Can anyone point out what I'm doing wrong? And is there a better way to go about this? Many thanks & all the best, Chuck

2 5

attribute-range()
by Johannes Bauer 29 Mar '22

29 Mar '22

Hello, when I use the db:attribute-range function with the attribute name filter I always get all attributes: db:attribute-range("db", "1", "1000", "id") I would expect that it only returns the "id" attributes. It seems the attribute name does not matter at all. The results are always the same even for a bogus attribute name. Regards Johannes

2 1

BaseX for Newbies Tutorial
by Tamara Marnell 28 Mar '22

28 Mar '22

Hello everyone, When I began developing our finding aids website last summer, I was a complete newbie to XML databases. I lacked the basic knowledge needed to understand the documentation, or even how to navigate it. Even Paul Swennenhuis's "BaseX for Dummies" (2014) was too advanced for me to follow. "FLWOR?" "Indexes"? I just want to do a keyword search of some files; why is this so hard?! After extensive Googling, Stack Overflowing, and reading every conversation in this list, I finally got a handle on the basics and completed our website. I've written what I learned in a foundational primer, "BaseX for Newbies," which contains basic definitions of concepts like "client/server architecture" and "modules," XQuery examples using U.S. Library of Congress finding aids, and solutions for problems that tripped me up: https://drive.google.com/file/d/19A5twBhA6OT3YdlcII7fazlwXewUSNkX/view -Tamara -- Tamara Marnell Program Manager, Systems Orbis Cascade Alliance (orbiscascade.org <https://www.orbiscascade.org/>) Pronouns: she/her/hers

3 2

RBaseX-client
by Ben Engbers 25 Mar '22

25 Mar '22

Hi, Version 1.1.1 is finally stable and available at CRAN (https://cran.r-project.org/package=RBaseX). Hopefully this blog "https://r-posts.com/rbasex-a-basex-client-written-in-r/" will contribute to more support for BaseX from the R community. Ben Engbers

1 0

Storing maps in BaseX
by Jonathan Robie 24 Mar '22

24 Mar '22

Is there a way to store a map in a database and use it in other queries without recomputing it each time? Jonathan

3 5

BaseX 9.7 • Ukraine Edition
by Christian Grün 24 Mar '22

24 Mar '22

Dear all, Just next to us, millions of people are currently being torn out of their home country, and we desperately hope that the situation in Ukraine will calm down soon. For those of you who are planning to donate money to us in the upcoming weeks, we kindly ask you this time to redirect it to Médecins Sans Frontières or any NGO of your choice. – We are back to provide you with a new and fresh version of BaseX. It comes with a big bunch of advanced performance tweaks and optimizations and some new XQuery features: • Support for xsl:messages via xslt:transform-report: • Support for formatting integers and dates in French • XSD validation: Improved support for XML catalogs • Remove namespaces from elements via util:strip-namespaces For a more comprehensive list of added and updated features, look into our documentation (docs.basex.org) and check out the GitHub issues (github.com/BaseXdb/basex/issues). And one more pleasant announcement: Today, our company BaseX GmbH Our celebrates its 10th anniversary! Thanks to all of you who have keeping us busy with challenging projects. Rest assured we’ll be happy to continue to give you advice and support for the upcoming decades. All the best, Your BaseX Team

6 6

Local deployment of a database
by Jonathan Robie 24 Mar '22

24 Mar '22

We are creating server instances, but we also have users who want to work locally. Ideally, I would like to be able to create an installer that would add BaseX and a set of databases to a user's system. How hard would it be to do that? Jonathan

3 5

GUI syntax highlighting
by marc＠crosseyed.nl 23 Mar '22

23 Mar '22

The syntax highlighting doesn't change when changing the look and feel in the BaseX GUI (9.6.2). I noticed this also happens in earlier versions. After changing to a dark background this results in blue on black, for instance. This happens with the JTattoo lib, but also GTK under a dark Linux theme. This occurs in Linux, all kinds of distro's, both Debian and Fedora based. Open JDK 11.0.6. In BaseX GUI 8.6.x all is fine, running under the same conditions. Regards, Marc

3 2

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

BaseX-Talk