- BaseX-Talk - mailman.uni-konstanz.de

Running BaseX from the command line
by Philip Spaelti 07 Jan '20

07 Jan '20

I have made occasional tries at using BaseX for my project over the years. After a longer pause I am trying again. I downloaded the latest version (9.1.3). If I try to start BaseX from the command line it starts a version 7.9 and asks we if I want to update. Can someone please explain what is going on? Philip

2 3

Config error
by Philip Spaelti 07 Jan '20

07 Jan '20

When I try to run the basexhttp script I get the following error: BaseX 9.3.1 [HTTP Server] [main] INFO org.eclipse.jetty.util.log - Logging initialized @259ms to org.eclipse.jetty.util.log.Slf4jLog [main] WARN org.eclipse.jetty.xml.XmlConfiguration - Config error at <Call name="addConnector"><Arg> <New class="org.eclipse.jetty.server.nio.SelectChannelConnector"><Set name="host"><SystemProperty name="org.basex.serverhost" default="0.0.0.0"/></Set><Set name="port"><SystemProperty name="org.basex.serverport" default="8984"/></Set><Set name="maxIdleTime">60000</Set><Set name="reuseAddress">true</Set><Set name="Acceptors">2</Set></New> </Arg></Call> org.eclipse.jetty.server.nio.SelectChannelConnector

1 0

Client software and command scripts
by Ben Engbers 06 Jan '20

06 Jan '20

Hi Christian, I read in the documentation that a client should not only be able to execute commands but should also be able to execute command scripts. My question is if a command script (a file with extension '.bxs') is passed as a 'path' to the execute-command or is the client supposed to read the file, line by line, and then executing each line separately? Cheers, Ben

2 5

Happy New Year! -- A #DigitialHumanities/#MachineLearning #CitizenScientist needs you help... :-)
by Jim Salmons 05 Jan '20

05 Jan '20

Hello Christian & the BaseX Community, Happy New Year and Decade! I will get to my specific BaseX/XQuery question below, but first some long but mostly necessary context... I am a 69-year-old post-cancer #PayItForward indie #CitizenScientist working at the intersection of #DigitalHumanities and #AI/#MachineLearning. I am developing the #MAGAZINEgts ground-truth storage format for serial publications (primarily print magazines and newspapers). My format is based on a ontological stack of CIDOC-CRM, FRBRoo, and PRESSoo, and utilizes a metamodel subgraph design pattern to support integrated complex document structure and content depiction models. In addition, my fellow cancer-surviving wife and I are developing a reference implementation of the #MAGAZINEgts format for the 48-issue run of Softalk magazine published between 1980 and 1984 during the dawn of the Microcomputer and Digital Age. Post-cancer and with determination to reinvent ourselves as Citizen Scientists in the Digital Humanities, we funded digitization of our magazine collection into the Internet Archive as a gift to each other for our 25th wedding anniversary. Initially, I did all our exploratory #MAGAZINEgts prototyping development in Python using import/export of data via JSON format with one-off scripts to format batches of information into XML format to fold into the published #MAGAZINEgts file for the Softalk magazine reference implementation at the Archive. Then I found BaseX!!! :D Upon discovering BaseX, I began using it for direct #MAGAZINEgts XML access and development. The FactMiners Toolkit is, at this stage, the old pre-BaseX stuff plus the new features related to the generation of balanced training datasets for #MachineLearning model training. I am in the process now of revisiting my early development and refactoring it to incorporate BaseX direct editing. I recently completed the generation of a "seed"/master dataset for all the advertisements in Softalk magazine (https://github.com/SoftalkAppleProject/datasets_ml_all_ads_1M) This seed dataset can be used in conjunction with the PRESSoo Issuing Rules of the Advertising Model in the Metamodel partition of the #MAGAZINEgts file. This "seed" dataset and the metamodel can be used to generate any "slice and dice" dataset for #ML model training that is aware of the interrelationships of advertisements' size, shape, and allowable positions on variously formatted multi-column pages. (If this domain of research interests anyone, please do not hesitate to contact me. I can point you to our #DATeCH papers/posters and Neo4j #GraphGists, etc. for more information. You will also find relevant links to our research on the GitHub repo cited above.) Now on to my specific question/request-for-information... To generate #MachineLearning training datasets, I need to not only slice-and-dice the "actual" data of the advertisement dataset, I also need to incorporate page examples which DO NOT include the document structure being trained to be recognized. In other words, the "non-label" instances of a #ML training dataset are the members of the set-theoretic complement of pages that DO have advertisements. Fortunately and by design, the #MAGAZINEgts format incorporates all the information to do this dataset generation... the problem is that I have only ever known enough XML-related technologies to "git'er done." And even whatever I may have known is long-since gone following my cancer battle. Now, given the OCD-ness of my basic personality structure, I thought it would be best to develop a solution to present before asking for community help... and here I am. The digital collection of Softalk magazine is in the Internet Archive here: https://archive.org/details/softalkapple?sort=date The #MAGAZINEgts file for Softalk magazine is here: https://archive.org/download/softalkapple/softalkapple_publication.xml For simplicity's sake during my exploratory/learning experience, I import the softalkapple_publication.xml file into BaseX with the "strip namespaces" option checked and rename the imported database as 'MAGgts'. My problem is not that I cannot generate the non-label dataset, but rather that this query takes roughly around 30 seconds to execute!? Honestly, the fact that this query works is all I need. But given the world of modern near-instantaneous responses to virtually all data inquiries, I am wondering if there is a far better solution to my problem. (Also note, the somewhat strange format of the returned results -- 'NoAdPg' elements wrapped within a 'Result' element is for the convenience of bringing this data into Python to be parsed by the 'xmltodict' module.) If it turns out that this 30+-second run is, in fact, justified due to the unseen actual computations being done, no problem. An explanation of why that much processing power is needed is one possible response to this inquiry. On the other hand, there is a vast domain of XQuery know-how that I am blind to and would value any explanation and optimization that can be suggested to speed this operation up. Here is my current 30+-second query: ====== (: Round up a list by issue of the leaf images of Softalk magazine that DO NOT have an advertisement on them and that excludes Cover 1, inserts (e.g. 'blow-in' cards), and missing pages... :) (: The softalkapple_publication.xml file from the Internet Archive's Softalk collection is imported as 'MAGgts' into BaseX XML database, with stripped namespaces for interactive development/learning convenience. :) (: This NS declaration is not needed when namespaces are stripped. declare default element namespace "http://www.factminers.org/MAGAZINE/gts/2019-01-18"; :) declare variable $ad_index := doc('MAGgts')//AdIndex/*; (: Gather all issues in the Leaf2ppg map (page image to print page number map) :) declare variable $all_issues := doc('MAGgts')//Metadata//Leaf2ppg_map[1]/*; if (count($all_issues) > 0) then (<Result>{for $issue in $all_issues return (for $leaf_spec in $issue/* let $leaf_id := $leaf_spec/Issue_id let $leaf_pgnum := $leaf_spec/PageNum let $leaf_type := $leaf_spec/PgType where ($leaf_pgnum != 'Cover1' and $leaf_pgnum != 'Insert Stub' and $leaf_pgnum != 'Insert Content' and $leaf_type != 'MissingPgs') return (if (exists($ad_index[PageNum = $leaf_pgnum and Issue_id = $leaf_id])) (: We're not interested in pages w/ ads, only those WITHOUT ads :) then () else <NoAdPg> <LeafNum>{data($leaf_spec/@leafnum)}</LeafNum> {$leaf_spec/(Issue_id | PgType | PageNum) }</NoAdPg> ))}</Result>) else "Nothing found." ====== Again, Happy New Year/Decade. Any explanation or optimization greatly appreciated. Happy-Healthy Vibes from Colorado, -: Jim :-

2 2

Syntax to use profiling functions on updating functions
by France Baril 31 Dec '19

31 Dec '19

Hi, I can't seem to use the profiling functions on updating functions. I was wondering if it's a known limit or if I just need to use a different syntax in this context. For example in an updating function that returns 2 updates, I have something like this declare updating function > translation:import-and-process-returned-translations($package-number as > xs:string) { > .... some code .... > return ( > translation:save-space-issues($package-name, $space-issues) => > prof:time('Translate save space issues: '), > translation:save-source-mismatches($package-name, > $source-mismatches) => prof:time('Translate save source mismatches: '), > translation:save-zhtw-brackets-issues($menu-id, $package-name) => > prof:time('Translate check zhtw brackets, not saved?: ') > ) > }; This works with the prof:time, but if I try to profile, I get: [XUST0001] Expressions must all be updating or return an empty sequence. Thanks! P.S. By the way, I'm very excited about the improvements I am able to implement now that I can use the performance functions in this module. I'm one version late to say this, but awesome addition! -- France Baril Architecte documentaire / Documentation architect france.baril(a)architextus.com

2 2

Re: [basex-talk] HTTPServer + gzip compression
by Stefan Koch 30 Dec '19

30 Dec '19

3 2

Parsing MusicXML files
by Joe Curtis 29 Dec '19

29 Dec '19

I am trying to solve a problem transferring musical information between two high end scoring programs using the industry standard MusicXML language. Although the majority (say 95%) of information is transferred correctly the remainder does not because the originating program gives a general classification of <words> in the file whilst the receiving program has a more sophisticated breakdown of the information. Rather than classifying it incorrectly it doesn't translate the information at all. The information occurs randomly throughout the score rather than in a regular pattern. What I am trying to do is read the MusicXML file and list the <words> where they occur together with the part identification and measure (bar) number to enable me to add the information manually at a later stage. I have come across BaseX while researching how to parse the files but am struggling to master the logic of xquery to achieve my aim. The structure of the file at my interest level is:-  <measure number="27" width="224"> <direction> <direction-type> <words default-x="9" default-y="20" justify="left" valign="middle" font-family="Times New Roman" font-style="normal" font-size="5.9683" font-weight="normal">tutti</words> and my xquery entries are:- let $score := doc('score')/score-partwise(:/part/measure/direction/direction-type/words:) for $word in ($score//words/text()) let $p := $score/part/@id let $m := $score/part/measure/@number return $word, $p, $m If I return just $word I get the relevant <words>entries but if I add $p and $m I get an error message 'undeclared variable $p' I realise that this is more of an xquery question than BaseX but I thought I would come here first. Any help would be greatly appreciated. Joe Curtis .

2 1

Update to a single document changes @modified-date on all DB documents in db:list-details()
by France Baril 29 Dec '19

29 Dec '19

Hi, I'm running a few tests after upgrading a web application to 9.3.1. I have this issue where I am trying to get the documents modified since a certain date/time. However, even is a single document is modified db:list-details returns the same modified-date for all listed documents. To reproduce; 1. Run list-details() on a DB to see dates. 2. Thought WebDAV, modify one of the document in the DB 3. Run list-details() again on that DB. Expected result: the @modifed-date should be different on the document modified in #2 Actual result: all documents have a new @modified-date. -- France Baril Architecte documentaire / Documentation architect france.baril(a)architextus.com

2 2

Lost prof:time in terminal after upgrade to 9.3.1
by France Baril 28 Dec '19

28 Dec '19

Hi, After upgrading to 9.3.1, I no longer get my prof:time info in the terminal. DEBUG is set to true in .basex and I don't remember changing any option on in older installations to enable prof:time. Documentation of profiling module doesn't mention any required settings and I'm not seeing a note in the CHANGELOG either. What am I missing? -- France Baril Architecte documentaire / Documentation architect france.baril(a)architextus.com

2 1

Re: [basex-talk] HTTPServer + gzip compression
by Stefan Koch 23 Dec '19

23 Dec '19

1 0