December 2018 - BaseX-Talk - mailman.uni-konstanz.de

BaseX/GUI using all memory
by Rick Graham 18 Dec '18

18 Dec '18

Greetings, First of all, thank you very much for BaseX. It has made many of my assignments this semester doable, more enjoyable, and better. Using it, I could demonstrate database features at scale where only toy minimal examples were required. The source of most of my data has been NIST's National Vulnerability Database <https://nvd.nist.gov/vuln/data-feeds> and the Mitre curated Common Weakness Enumeration Lists <https://cwe.mitre.org/data/downloads.html>. Using BaseX, I found and reported to NIST some errors in the CVE database and the errors have since been fixed. I'm sure there are many more fixes and enhancements possible with the CVE database. I almost exclusively use BaseX/GUI (version 9.0.2, then 9.1, and now 9.1.1) on Fedora Linux. I do have some issues using BaseX/GUI and I am hoping that some improvements can be made. Eventually, BaseX/GUI uses all of it's allocated memory. Even after increasing the max memory to 3.5GB, eventually it is all used and BaseX/GUI essentially freezes. Any operation that freezes, executes quickly and completely after quitting/killing BaseX/GUI and then restarting. It seems that some memory just never gets freed as I develop different XQuery routines in the Editor, run them, save them to files, click in various places on the Map Visualization, run XQuery on the Input Bar, etc. Sometimes closing the database frees the memory, mostly it doesn't. Once I think memory was freed when I saved a file in the Editor. The Java error messages all seem to relate to running out of memory. Hitting the "GC" button never seems to help. I don't have a specific sequence of actions that eventually consumes all the memory. An example database that will demonstrate this memory consumption is composed of NVD-CVE-1.0-2018 <https://nvd.nist.gov/feeds/json/cve/1.0/nvdcve-1.0-2018.json.zip> and CWE Comprehensive View <https://cwe.mitre.org/data/xml/views/2000.xml.zip>. The database takes about 109 MB and has about 5.25 million nodes. Just viewing the Visualization Map, clicking around, and running/editing queries like this will eventually use all the memory. let $cwe := distinct-values > ( > for $c in //cve//problemtype__data//value > return > tokenize($c, "-")[last()] > ) > for $c in $cwe > where > empty(//Weakness_Catalog[contains(@Name, "CWE-2000")]//Weakness[@ID = $c]) > and > empty(//Weakness_Catalog[contains(@Name, > "CWE-2000")]//Related_Weakness[@CWE_ID = $c]) > order by number($c) > return > $c `java -version` reports: openjdk version "1.8.0_191" > OpenJDK Runtime Environment (build 1.8.0_191-b12) > OpenJDK 64-Bit Server VM (build 25.191-b12, mixed mode) I will gladly provide any additional info that may help to diagnose these symptoms, etc. Thanks and best regards. RG

2 3

BaseX 9.1.1
by Christian Grün 17 Dec '18

17 Dec '18

Hi all, we are glad to provide you with version 9.1.1 of BaseX! It’s actually more than just a maintenance release: XQuery: - Comprehensive rewritings of positional predicates and functions - Higher-order functions: Improved type inference - Improved rewriting of context-based and/or nested predicates Java Bindings: - Faster access to and evaluation of Java functions and variables - Improved pre-selection of function candidates at compile time - Better error messages (incl. function arity and similar names) DBA: - Settings: user-defined pattern for ignoring log entries - Login: pass on URL query strings Minor improvements: - Import: detect epub files as ZIP archives - Digest Authentication: No delay after first request - GUI, Preferences: user-defined choice of XML suffixes The new version is online: http://basex.org/ Looking forward to your feedback as usual. Have a nice time, Christian

4 3

eq vs =
by Leo Studer 13 Dec '18

13 Dec '18

Hello Could someone explain me why this woks //country[@population = max(//@population)]/name Whereas this does not //country[@population eq max(//@population)]/name Thanks in advance Leo

3 3

Command line options not working with update operation on input file
by Zimmel, Daniel 13 Dec '18

13 Dec '18

Hello, can anyone give me a hint on the behaviour of the command line options in BaseX 9.0.2? When I try an update operation on a file given with "-i", neither the option "-w" nor any serialisation option gets evaluated; while the update operation with "-u" is working correctly, my whitespace gets happily chopped. CCL: $ basex.bat -ifile.xml -sindent=no -u -w update.xquery XQ: for $c in doc(document-uri())//element return replace value of node $c with "TEST" A workaround is using fetch:xml() with document-uri(), which does leave any whitespace intact and does no indentation (no parameters necessary). declare variable $doc := document-uri(); for $c in fetch:xml($doc)//element return replace value of node $c with "TEST" Unfortunately my requirement is I cannot use specific BaseX-functions in this specific update script. My XML: <root> <vs> <kennung>Kennung</kennung> <absatz> <fett>eins</fett> </absatz> <absatz id="sgb04_k_para067_p67_a0001_a0001"> Test <fett>fett</fett> <kursiv>kursiv</kursiv> Text Text.</absatz> </vs> </root> Thanks, Daniel

2 14

Strange response from Docker BaseXDBA
by Huib Verweij 07 Dec '18

07 Dec '18

Hi, I have a strange case of receiving a “No function found that matches the request” message from BaseXDBA. I did a "docker build -t basexdba ." in de root of the source folder of BaseX as per the DBA example in http://docs.basex.org/wiki/Docker <http://docs.basex.org/wiki/Docker>. I got a basexdba:latest image and started it with “docker run -p 8984:8984 basexdba:latest”. According to the documentation I should get the dba page with: http://localhost:8984/dba <http://localhost:8984/dba> but the server returns "No function found that matches the request.". There’s also a recent question on Stackoverflow about this: https://stackoverflow.com/questions/53462688/docker-basex-dba/53543130#5354… <https://stackoverflow.com/questions/53462688/docker-basex-dba/53543130#5354…> Regards, Huib.

3 2

BaseX WebDAV Rejecting Large Files
by Kanda (US), David 04 Dec '18

04 Dec '18

Hello, We are attempting to use WebDAV to pump a large zip file to BaseX then have BaseX unzip and import the uncompressed xml file. WebDAV transfer works well with small files, (less than 1Mb) but it fails immediately when uploading a large zip or xml file (~50MB). Is there any sample code to demonstrate programmatically transferring a large file to BaseX over WebDAV? Cheers, David

2 3

BaseX and docker
by France Baril 02 Dec '18

02 Dec '18

Hi, For one of my projects, we want to set up BaseX as part of an online application. I was wondering if anyone has experience installing BaseX in a Docker setup and if so how things got split up. In our case, each client will have many database, we're afraid that if we use a single baseX installation, it will have too many attached dbs. So we were thinking that we could use 1 application but save1 data/repo per client in their own docker unit , then again, since .basex can only link to 1 repo/data, it doesn't seem possible. If you consider 1000 clients with an average of 30 (1 per language) database each, would you use a single installation and manage security at the db level with all dbs for all clients in the same setup or would you suggest 1 full instance (server + set of db) per client? Links to any case study would be useful too. -- France Baril Architecte documentaire / Documentation architect france.baril(a)architextus.com

3 3