BaseX-Talk

basex-talk@mailman.uni-konstanz.de

8 participants
5255 discussions

BaseX YAML Parser?
by Eliot Kimber 26 Jul '22

26 Jul '22

I found myself needing to operate on some YAML documents and tried to find an XQuery YAML parser and did not find one, at least not one of sufficient completeness. It seems like kind of an obvious missing thing in the XQuery world so just curious if I missed something? I don’t have time myself to write one—it can’t be that hard but it’s not that critical of a need for me. I ended up using the Python PyYAML library, which let me quickly convert YAML to JSON, which then I could of course process just fine with XQuery. Thanks, E. _____________________________________________ Eliot Kimber Sr Staff Content Engineer O: 512 554 9368 M: 512 554 9368 servicenow.com<https://www.servicenow.com> LinkedIn<https://www.linkedin.com/company/servicenow> | Twitter<https://twitter.com/servicenow> | YouTube<https://www.youtube.com/user/servicenowinc> | Facebook<https://www.facebook.com/servicenow>

6 6

Fwd: Available Integration with Apache POI for Excel Generation?
by Tim Thompson 26 Jul '22

26 Jul '22

Resending to the list (forgot to reply all). ---------- Forwarded message --------- From: Tim Thompson <timathom(a)gmail.com> Date: Mon, Jul 25, 2022 at 5:04 PM Subject: Re: [basex-talk] Available Integration with Apache POI for Excel Generation? To: Eliot Kimber <eliot.kimber(a)servicenow.com> Hi, Eliot, I'm not aware of any integration with Apache POI (which would be great), but here's a project--it seems unfinished, but may be enough to get started with? https://github.com/eliudmeza/OOXML-Library-XQuery-BaseXdb All best, Tim -- Tim A. Thompson (he, him) Librarian for Applied Metadata Research Yale University Library On Mon, Jul 25, 2022 at 2:29 PM Eliot Kimber <eliot.kimber(a)servicenow.com> wrote: > I tried searching for “BaseX Excel generation” but it didn’t get me > farther than a reference to the CSV module, which of course I am familiar > with. > > > > I need to add the ability to generate non-trivial Excel spreadsheets, > which I know how to do using the Apache POI library, but being lazy I was > hoping somebody had already done it in a shareable form. > > > > So my question: Does anyone have or know of a shareable integration of > Apache POI with BaseX for generating Excel spreadsheets or some other > Excel-generation solution that is not just downloading CSV files? > > > > Thanks, > > > > E. > > _____________________________________________ > > *Eliot Kimber* > > Sr Staff Content Engineer > > O: 512 554 9368 > > M: 512 554 9368 > > servicenow.com <https://www.servicenow.com> > > LinkedIn <https://www.linkedin.com/company/servicenow> | Twitter > <https://twitter.com/servicenow> | YouTube > <https://www.youtube.com/user/servicenowinc> | Facebook > <https://www.facebook.com/servicenow> >

2 1

Available Integration with Apache POI for Excel Generation?
by Eliot Kimber 26 Jul '22

26 Jul '22

I tried searching for “BaseX Excel generation” but it didn’t get me farther than a reference to the CSV module, which of course I am familiar with. I need to add the ability to generate non-trivial Excel spreadsheets, which I know how to do using the Apache POI library, but being lazy I was hoping somebody had already done it in a shareable form. So my question: Does anyone have or know of a shareable integration of Apache POI with BaseX for generating Excel spreadsheets or some other Excel-generation solution that is not just downloading CSV files? Thanks, E. _____________________________________________ Eliot Kimber Sr Staff Content Engineer O: 512 554 9368 M: 512 554 9368 servicenow.com<https://www.servicenow.com> LinkedIn<https://www.linkedin.com/company/servicenow> | Twitter<https://twitter.com/servicenow> | YouTube<https://www.youtube.com/user/servicenowinc> | Facebook<https://www.facebook.com/servicenow>

2 1

What Would "Server is running or permission was denied." Indicate?
by Eliot Kimber 18 Jul '22

18 Jul '22

One my RESTXQ pages constructs an HTML report that takes a few seconds to construct. The page works fine in my local dev environment but when I deploy the code on my production server the process fails after constructing the report markup with the log messages: Server is running or permission was denied. Server is running or permission was denied. Server is running or permission was denied. That is, the message is emitted three times in a row with about 50ms between and the page generation fails (page never appears in the browser). Any idea what condition this might indicate and how I might track it down? It must be something I’m doing in my code but I can’t find any obvious logic bugs so wondering what kind of thing I might look for. Thanks, Eliot _____________________________________________ Eliot Kimber Sr Staff Content Engineer O: 512 554 9368 M: 512 554 9368 servicenow.com<https://www.servicenow.com> LinkedIn<https://www.linkedin.com/company/servicenow> | Twitter<https://twitter.com/servicenow> | YouTube<https://www.youtube.com/user/servicenowinc> | Facebook<https://www.facebook.com/servicenow>

2 1

First steps with client/server on Linux - bulk import; error message in dba
by Florian Schmitt 18 Jul '22

18 Jul '22

Hi all, i'm currently evaluating BaseX running on a headless Linux Server. It took me quite a while to get it running (the docs seem to be focused on Windows environments)... Is there a way to contribute to the documentation? First question: After importing some of documents nicely organized in collections, i've found that the REST client will display all documents on the same "level" instead of showing just the first level of collections. This isn't nice if the db contains a great number of documents.... When accessing the same db using WebDAV (Oxygen), one can nicelly navigate using the collection structure. With REST, adding the collection names to the URL will restrict the output to the db content of those collections. But without such a restriction, it would be great if the REST web interface would just return the first level of collections without any XML resource entries. Second question: What's the "best practice" for bulk imports while retaining the directory structure of the imported files using collections? I've found that Oxygen "translates" subdirectories into collections when using the "import directories" feature. Is there somehing similar using the command line client? And another issue: The "Logs" page of the DBA webapp shows the following error warning: "Unexpected error: Improper use? Potential bug? Your feedback is welcome: Contact..." Is this a critical error? How to debug this? So far, i'found the wrapper-basex log, but it doesn't contain anything useful. Since i'm using the Yajsw Wrapper, there's no debug output or debug option (?). Yours sincerely. Florian

4 8

Adding Attributes After the Fact: How To?
by Eliot Kimber 16 Jul '22

16 Jul '22

In my Mirabel project I’m loading large quantities of DITA content (40K topics per content database). I’m turning off DTD parsing because otherwise it takes hours to load the content (I haven’t had bandwidth to figure out how to use Xerces grammar caching with BaseX, which would solve the problem). DITA uses an attribute-based mechanism (@class) to associate elements with their base types. For example, a <concept> element is a kind of <topic>, as indicated by the @class value “- topic/topic concept/concept “. To find DITA elements you always search on @class values rather than tag names. In XSLT 3 this is best expressed as match=”*[contains-token(@class, ‘topic/topic’)]” to handle any topic element. These attributes are normally defaulted in grammars so they are not explicit in the source as imported and thus do not exist in my no-DTD content databases. To work around this I’ve implemented a function that hard codes the element-to-class-value mapping to enable selecting elements by @class value. (For any set of content you can know what the @class value is for any element type as its statically declared in the governing grammar, so it is possible to have this kind of hard-coded element-to-class mapping.) However, I now realize that if the attributes are in the data as stored in BaseX that lookup will be much faster using an attribute index. That is, rather than using a predicate like [dutils:class(., ‘topic/topic’)] I could do db:token($database, ‘topic/topic’, ‘class’). Because I have the element-to-class mapping I can add the attributes to the content after the initial database load. My question: How best to do that? I can have an updating function that simply adds the @class attribute to all the elements but I suspect there are some performance implications for doing that kind of bulk update to millions of nodes. So any guidance on this kind of bulk update would be helpful. Thanks, Eliot _____________________________________________ Eliot Kimber Sr Staff Content Engineer O: 512 554 9368 M: 512 554 9368 servicenow.com<https://www.servicenow.com> LinkedIn<https://www.linkedin.com/company/servicenow> | Twitter<https://twitter.com/servicenow> | YouTube<https://www.youtube.com/user/servicenowinc> | Facebook<https://www.facebook.com/servicenow>

1 0

RESTXQ: How to Reflect Progress of Long-Running Querries
by Eliot Kimber 16 Jul '22

16 Jul '22

In my Mirabel system, I have multiple BaseX instances to handle long-running processes while a primary instance serves the web pages. For these background processes I’d like to be able to show progress in the main web app. My question: what are the options for doing this? I can imagine having a database to write progress data to and then read from that in my RESTXQ page (maybe using a simple JavaScript to poll the app). Are there better techniques? I’m trying to limit the amount of JavaScript I have to write but I can do it if I have to. Thanks, E. _____________________________________________ Eliot Kimber Sr Staff Content Engineer O: 512 554 9368 M: 512 554 9368 servicenow.com<https://www.servicenow.com> LinkedIn<https://www.linkedin.com/company/servicenow> | Twitter<https://twitter.com/servicenow> | YouTube<https://www.youtube.com/user/servicenowinc> | Facebook<https://www.facebook.com/servicenow>

2 1

Possible to Speed Up This Lookup?
by Eliot Kimber 16 Jul '22

16 Jul '22

I generate a “where used” index as an XML doc in a separate database. I’m then doing lookups against this index like this where $node is the element I want to find the index entry for: collection($rkDatabase)/doc-where-used-index/where-used-entry[string(@key) eq string(db:node-id($node))] Where the markup is shown in [1] below. Using prof:track() I’m measuring a consistent 0.036 seconds per lookup, which seems very slow. Db:node-id() is instantaneous so the cost must be in the XPath itself. I’ve tried different variations on the predicate but nothing makes a measurable difference. There must be a more efficient way to do this kind of lookup or a better way to structure my index XML so that the lookup is optimized. Or is this something that could be affected by my index settings at the database level? Thanks, Eliot [1] Where-used index markup: <doc-where-used-index> <where-used-entry key="9151416" tagname="map" class="map/map" id=""> <title> <ph keyref="workplace-safety-management"/> </title> <conrefs/> <topicrefs> <noderef node-id="672471" database="tokyo" tagname="mapref" baseuri="/tokyo/qa-sushma-sundareswaran.ditamap" href="workplace-service-delivery-core.ditamap"/> <noderef node-id="898684" database="tokyo" tagname="mapref" baseuri="/tokyo/qa-ankit-jaiswal.ditamap" href="workplace-service-delivery-core.ditamap"/> <noderef node-id="9524996" database="tokyo" tagname="mapref" baseuri="/tokyo/workplace-service-delivery-suite.ditamap" href="workplace-service-delivery-core.ditamap"/> </topicrefs> <doc> <noderef node-id="9151416" database="tokyo" tagname="map" baseuri="/tokyo/workplace-service-delivery-core.ditamap"/> </doc> <xrefs/> <referencing-maps> <noderef node-id="672452" database="tokyo" tagname="map" baseuri="/tokyo/qa-sushma-sundareswaran.ditamap"/> <noderef node-id="898667" database="tokyo" tagname="map" baseuri="/tokyo/qa-ankit-jaiswal.ditamap"/> <noderef node-id="9524980" database="tokyo" tagname="map" baseuri="/tokyo/workplace-service-delivery-suite.ditamap"/> </referencing-maps> </where-used-entry> … </doc-where-used-index> _____________________________________________ Eliot Kimber Sr Staff Content Engineer O: 512 554 9368 M: 512 554 9368 servicenow.com<https://www.servicenow.com> LinkedIn<https://www.linkedin.com/company/servicenow> | Twitter<https://twitter.com/servicenow> | YouTube<https://www.youtube.com/user/servicenowinc> | Facebook<https://www.facebook.com/servicenow>

5 11

file:resolve-path, ".." steps
by Hans-Juergen Rennau 11 Jul '22

11 Jul '22

Dear BaseX people, extension function file:resolve-path() shows a behaviour which appears to me inconsistent and probably not intended: - if called with a single parameter, a path containing ".." is resolved as expected- if called with two parameters, the ".." steps are not resolved. I would expect the resolution of ".." steps to be always a part of path resolution. Example: cd C:\a\b\c file:resolve-path('../x') =>C:\a\b\x file:resolve-path('../x', 'file:///C:/a/b/c/') =>C:\a\b\c\..\x file:resolve-path('../x ', 'file:///C:/a/b/c/' ! file:resolve-path(.) =>C:\a\b\x Greetings, Hans-Jürgen

2 2

Explicit cacheing with store: vs. implicit cacheing in RESTXQ ?
by Majewski, Steven Dennis (sdm7g) 11 Jul '22

11 Jul '22

Thinking about use cases for the new store: cacheing module, and wondering if I understand the cacheing already provided in RESTXQ. https://docs.basex.org/wiki/RESTXQ#Preliminaries <https://docs.basex.org/wiki/RESTXQ#Preliminaries> So in a production environment where I have disabled monitoring for file changes by setting PARSERESTXQ to -1, variables declared in prolog of library modules have a long lifetime and are only recalculated when the modules are reloaded and parsed — which would, I assume, imply that if modifications are made to the database, I should call rest:init() to force a reload if I have declared values that are calculated from a collection that may have been changed. And that would also imply that I could, in many cases, use that implicit caching in place of explicit store: functions. Is that correct ? — Steve M.

2 1

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

BaseX-Talk