- BaseX-Talk - mailman.uni-konstanz.de

Feature request - file listing
by Hans-Juergen Rennau 12 Feb '22

12 Feb '22

Dear BaseX people, I have a feature request: a function which does the same as file:list, but returns the full paths. (In the explanation, I call it "file:list2") Rationale: (a) It is the full paths what we need in order to parse the files (doc(), json:doc(), csv:doc(), html:doc()). (b) The combination of file listing and parsing in a single expression is of matchless elegance and expressivenessJust think about how we can extend this into a single list+parse+navigate+report expression, like: file:list2($folder, true(), $fname) ! doc(.) ! //returnCode[. ne 0]/concat(base-uri(.), ': ', .) (Wow! Is there any language or tool competing with this compactness and readability?) Currently, if I don't overlook something, I have to do this: file:list($dir, false(), $fname) ! concat($dir, '/', .) ! doc(.) => count() I emphasize the psychological aspect. Every time I have the pleasure to give a workshop about XQuery, I show off how elegant it is to parse a whole file system tree into a forest of nodes in a single expression. And I am always embarrassed about this clumsy fumbling montage of a path. It's as if Mozart had fallen asleep while writing his xquery sonata, knocked over the inkwell and now there is this splash smeared out on the score. You would be my heroes if you can spare us that. Kind regards,Hans-Jürgen

6 9

my application's performance statistics
by Rob Stapper 11 Feb '22

11 Feb '22

Hi Christian, I did some performance statistics on my application and something caught my eye, which I hope you can shed some light on. I found that when running the application the distribution of the life-cycle fases: Parsing, Compiling, Evaluating and Printing, is: Parsing : 95%, Compiling : 4%, Evaluating : 0.4% and Printing : 0.03% . My development-paradigm is maintainability and not as much as performance optimization. So I expect is a lot of code-optimization during parsing and therefore a relatively high amount of time consumed for parsing. This is confirmed by the relatively low amount of time for the Compiling- and Evaluating-time. Code-optimization actually is a strong point of the parser. However I had expected/hoped that this code-optimization only occurs at the first run and that the optimized code is kept for following runs. This does not seem to be the case. First of all: am I correct in my analysis? Secondly, can you give me any advise on improving the parsing-time, not being giving up on my development-paradigm, based on these stats? Some more stats: The info-view gives for the compiling -fase a total of 2124 lines of which: - Inline …… 923 lines - Remove type check …. 710 lines - Simplify FLWOR …. 218 lines - Rewrite … 150 lines - Unroll …. 12 lines - Merge: concat ….. 11 lines - Open database … 3 lines - Swap operant … 1 lines - Simplify catch * …. 4 lines - Thnx in advance for your reply. Rob Stapper Sent from Mail for Windows -- This email has been checked for viruses by Avast antivirus software. https://www.avast.com/antivirus

2 1

file:descendants - returns relative paths
by Hans-Juergen Rennau 11 Feb '22

11 Feb '22

Dear BaseX people, functions file:children() file:descendants() should return the full paths of the resources found, but they do so only if the argument was in turn a full path. A bug? Example: file:children('.') =>PS C:\projects\xquery-introduction\tips> basex "file:children('.')" .\foo.xq .\bar.docx Kind regards,Hans-Jürgen

2 1

How to extract value from fn:analyze-string
by Ben Engbers 11 Feb '22

11 Feb '22

Hi, This query produces the following result: let $debates := collection("Parliament") for $debate-item in $debates let $item-file := $debate-item/officiele-publicatie//meta/@content let $debate-id := fn:analyze-string( $debate-item/officiele-publicatie//meta/@content, "(\d{8}-\d*)-(\d*)") return ($debate-id) => <analyze-string-result xmlns="http://www.w3.org/2005/xpath-functions"> <non-match>https://zoek.officielebekendmakingen.nl/h-tk-</non-match> <match> <group nr="1">20202021-102</group>-<group nr="2">1</group> </match> <non-match>/metadata.xml</non-match> </analyze-string-result> ... I am trying to extract the values from group 1 and 2 but this query returns 0 results: let $debates := collection("Parliament") for $debate-item in $debates let $item-file := $debate-item/officiele-publicatie//meta/@content let $debate-id := fn:analyze-string( $debate-item/officiele-publicatie//meta/@content, "(\d{8}-\d*)-(\d*)") let $debate-nr := $debate-id//match/group[@nr="1"]/text() let $item-nr := $debate-id//match/group[@nr="2"]/text() return ($debate-nr, $item-nr) My guess is that analyze-string inserts new elements in the query and that that is the reason why this does not work. How can I extract debate-nr and item-nr from $debate-id? Ben Engbers

4 4

BaseX Doing Lots of Work on Startup: How to Diagnose?
by Eliot Kimber 10 Feb '22

10 Feb '22

I now have my server loaded with lots of data and I’m seeing that when the server starts it is taking 100% of a processor for 10s of minutes or more. I expect some initial startup cost of course as BaseX prepares whatever it needs to satisfy queries but this feels like something is not right. I don’t see anything obvious in the logs but I suspect I’m not looking in the right place or don’t have appropriate logging options turned on. Any tips on what I can look for to see what might be causing this startup issue? Thanks, E. _____________________________________________ Eliot Kimber Sr Staff Content Engineer O: 512 554 9368 M: 512 554 9368 servicenow.com<https://www.servicenow.com> LinkedIn<https://www.linkedin.com/company/servicenow> | Twitter<https://twitter.com/servicenow> | YouTube<https://www.youtube.com/user/servicenowinc> | Facebook<https://www.facebook.com/servicenow>

3 7

Controlling order of attributes with updates
by Jonathan Robie 10 Feb '22

10 Feb '22

Yes, I know that the order of attributes is insignificant. Except to some human beings who complain to me about the order of elements. Is there any way for me to control the order of attributes in BaseX when I insert new attributes? Jonathan

3 5

BaseX and 'view'?
by Ben Engbers 09 Feb '22

09 Feb '22

Hi, I am writing a blog for R-bloggers with the aim of raising awareness of my RbaseX package. The latest version is super fast! Loading and saving 740 xml documents from the R environment took only 34 seconds! Before I can use those documents I will probably have to convert them using xsl. I was wondering if BaseX, like Oracle, also has a 'view' over the original data? I can then define multiple views that are better adapted to the intended use. ben

2 1

Managing/Debugging Server Load and Performance
by Eliot Kimber 08 Feb '22

08 Feb '22

I’m getting my validation dashboard server almost ready for use and will be making available to real users. I want to make sure that I’ve done everything I can to ensure maximum performance and stability of the server and I’m realizing I’m not entirely sure how to do that. This is running on a Centos virtual machine with 16GB of ram and 1TB of available disk space and 4 cores—not the beefiest but it’s what IT gave us (and we’re not sure how to ask for more on short notice). Here’s what I’ve done so far: * Using the JRE provided with Oxygen, allocated with 4GB (we are also using this server to run Oxygen via scripting and it needs 8GB to handle our insanely huge DITA maps) * Set parallel to 4 (to match the number of cores, but just guessing that this is a useful setting based on the docs) I have not modified any index settings but I’m also not doing anything other than querying on element types and attribute values, so from the docs it looks like the default indexes should be what I need. I’m seeing some apparent occasional slowness on pages that should not be slow (don’t reflect long-running queries or huge data volumes) but I’m not really sure how to diagnose it or even verify that I’ve succeeded in giving BaseX all the resources it needs. The same webapp performs well on my development mac, which makes me think the issue is one of resource allocation, so probably my user error setting up the server. Any guidance on how to ensure best server performance would be most helpful. Thanks, Eliot _____________________________________________ Eliot Kimber Sr Staff Content Engineer O: 512 554 9368 M: 512 554 9368 servicenow.com<https://www.servicenow.com> LinkedIn<https://www.linkedin.com/company/servicenow> | Twitter<https://twitter.com/servicenow> | YouTube<https://www.youtube.com/user/servicenowinc> | Facebook<https://www.facebook.com/servicenow>

4 12

StratML Query Service?
by Owen Ambur 07 Feb '22

07 Feb '22

I'm looking for a partner(s) to develop a StratML query service(s) for potential hosting at https://aboutthem.info/ The basic requirements are outlined at https://stratml.us/carmel/iso/SMLTASwStyle.xml#_15446932-208f-11e6-a80e-733… There are more than 5K files in the StratML collection <https://stratml.us/drybridge/index.htm> that can be used for demonstration/MVP <https://en.wikipedia.org/wiki/Minimum_viable_product> purposes and all of their URLs are listed in sitemap format at https://stratml.us/sitemap.xml Jorge Sanchez put a lot of effort into a prototype based on BaseX, which is briefly documented at https://stratml.us/#Vionta However, it appears he may have found it too difficult to accomplish and given up on the project. For additional background on the bigger picture, see https://connectedcommunity.net/ Please let me know if you or anyone you may know might be interested in partnering with me to develop and maintain such a service. My RMDs <https://www.irs.gov/retirement-plans/plan-participant-employee/retirement-t…> are kicking in this year and I can't think of a better way to spend some of them. Owen https://www.linkedin.com/in/owenambur/

1 0

Download As Binary for Docs in Databases?
by Eliot Kimber 07 Feb '22

07 Feb '22

I’d like to be able to create Zip files from large docs stored in a database (i.e., Oxygen validation reports, which can be 10s of MBs as raw XML). Looking at the Archive module it’s not immediately clear how to go about it (or how best to go about it)—it would make the most sense to construct the Zip as a stream and return the stream, as opposed to constructing the Zip in memory (or on disk) and then streaming it from there. Or am I overthinking it or missing some obvious solution? Thanks, E. _____________________________________________ Eliot Kimber Sr Staff Content Engineer O: 512 554 9368 M: 512 554 9368 servicenow.com<https://www.servicenow.com> LinkedIn<https://www.linkedin.com/company/servicenow> | Twitter<https://twitter.com/servicenow> | YouTube<https://www.youtube.com/user/servicenowinc> | Facebook<https://www.facebook.com/servicenow>

2 3