- BaseX-Talk - mailman.uni-konstanz.de

Re: [basex-talk] BaseX-Talk Digest, Vol 134, Issue 3
by Kourouma Ismael 06 Feb '21

06 Feb '21

Hi, Please, how these queries should be helpful in my journey for learning BaseX ? Best regards, Ismael Send BaseX-Talk mailing list submissions to basex-talk(a)mailman.uni-konstanz.de To subscribe or unsubscribe via the World Wide Web, visit https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk or, via email, send a message with subject or body 'help' to basex-talk-request(a)mailman.uni-konstanz.de You can reach the person managing the list at basex-talk-owner(a)mailman.uni-konstanz.de When replying, please edit your Subject line so it is more specific than "Re: Contents of BaseX-Talk digest..." Today's Topics: 1. Re: Feature requests: Backups, Indexes, Execution Times (Christian Gr?n) ---------------------------------------------------------------------- Message: 1 Date: Sat, 6 Feb 2021 11:12:01 +0100 From: Christian Gr?n <christian.gruen(a)gmail.com> To: Graydon Saunders <graydonish(a)gmail.com> Cc: BIRKNER Michael <Michael.BIRKNER(a)akwien.at>, "basex-talk(a)mailman.uni-konstanz.de" <basex-talk(a)mailman.uni-konstanz.de> Subject: Re: [basex-talk] Feature requests: Backups, Indexes, Execution Times Message-ID: <CAP94bnN_4iLWvPG+S3C068jPx=0QXH_d1OtZrwJ+RLH-frAC6A(a)mail.gmail.com> Content-Type: text/plain; charset="UTF-8" Execution times will now be shown in the MM:SS.mm or HH:MM:SS.mm format, but only? ? in the header of the info view, and ? if the measured time exceeds 60 seconds. A new snapshot is available [1]. Have fun, Christian [1] https://files.basex.org/releases/latest/ On Thu, Feb 4, 2021 at 6:15 PM Graydon <graydonish(a)gmail.com> wrote: > > On Thu, Feb 04, 2021 at 05:21:24PM +0100, Christian Gr?n scripsit: > > > - *Human readable execution times in GUI*: Maybe a small change but > > > - at least in my case - it would make developing performant xQueries > > > much easier: Having the "Timing" section in the Info-View of the GUI > > > display human readable times. Right now, the values are displayed > > > only in milliseconds like 175713.28 ms. But an additional display in > > > a more human readable format, e. g. hh:mm:ss.ms would sometimes be > > > very useful. > > > > > Sounds reasonable and doable. I?ll think about it. > > Please leave the time formatting switchable! I'd strongly prefer > straight milliseconds to a human-readable format. > > -- > Graydon Saunders | graydonish(a)gmail.com > ??s ofer?ode, ?isses sw? m?g. > -- Deor ("That passed, so may this.") End of BaseX-Talk Digest, Vol 134, Issue 3 ******************************************

1 0

Re: [basex-talk] FLOATS
by Adam Law 04 Feb '21

04 Feb '21

Many thanks Christian - I don't know why I didn't think of decimal. I will review my xquery types and arrays. Thanks again - much appreciated :) Adam ! lobotomy() On Thu, Feb 4, 2021 at 11:57 PM Christian Grün <christian.gruen(a)gmail.com> wrote: > Hi Adam, > > It’s recommendable to use xs:decimal instead of xs:float. Here’s one > solution… > > declare function local:sumVec($seq) { > let $count := max(($seq ! string() ! > replace(., '[^_]', '') ! string-length())) + 1 > let $sums := ( > for $c in 1 to $count > return sum( > for $item in $seq > let $tokens := tokenize(string($item), '_') > return try { xs:decimal($tokens[$c]) } catch * { 0 } > ) > ) > return string-join($sums, '_') > }; > > …and another one (presumably more cryptic): > > declare function local:sumVec($seq) { > let $numbers := ( > for $item in $seq > return array { > for $token in tokenize(string($item), '_') > return try { xs:decimal($token) } catch * { 0 } > } > ) > return string-join( > for $c in 1 to max($numbers ! array:size(.)) > return sum($numbers[array:size(.) >= $c]?($c)) > , '_') > }; > > Hope this helps, > Christian > > > > On 2/4/21, Adam Law <adamjameslaw(a)gmail.com> wrote: > > (: > > Hello I have a hopefully not too silly question ... > > Is there an easy way to remove the exponent notation and return > > 30000201_30000000 > > without working out where the E is in the parts and padding zeros > > ====> <a> > > <d>3.0000201E7_3.0E7</d> > > </a> > > :) > > declare function local:sumVec($a,$opt){ > > let $a := <x>{for $w at $pos in $a > > return > > for $v at $pos in tokenize(xs:string($w),"_") > > return element {'a'} { (attribute {'pos'} {$pos}, try > > {xs:float($v)} catch * {0} )} > > }</x> > > let $b := string-join(for $w in distinct-values($a//@pos) > > return if ($opt=0) then sum($a/a[@pos=$w]) ,"_") > > return $b > > }; > > > > let $a := '' > > return <a> > > > > > <d>{local:sumVec(('10000001_10000000','10000000_10000000','10000000_10000000','100_',100),0)}</d> > > </a> > > >

1 0

FLOATS
by Adam Law 04 Feb '21

04 Feb '21

(: Hello I have a hopefully not too silly question ... Is there an easy way to remove the exponent notation and return 30000201_30000000 without working out where the E is in the parts and padding zeros ====> <a> <d>3.0000201E7_3.0E7</d> </a> :) declare function local:sumVec($a,$opt){ let $a := <x>{for $w at $pos in $a return for $v at $pos in tokenize(xs:string($w),"_") return element {'a'} { (attribute {'pos'} {$pos}, try {xs:float($v)} catch * {0} )} }</x> let $b := string-join(for $w in distinct-values($a//@pos) return if ($opt=0) then sum($a/a[@pos=$w]) ,"_") return $b }; let $a := '' return <a> <d>{local:sumVec(('10000001_10000000','10000000_10000000','10000000_10000000','100_',100),0)}</d> </a>

1 0

Mapping and Basex
by Ben Pracht 30 Jan '21

30 Jan '21

Hi, I've seen this geo module and wondered what this is used for. I'd like to see if there's a way of extracting addresses from a website and their corresponding school. I could be way off but I was hoping this would be a start. Thanks, Ben

1 0

Fwd: Securing All Endpoints
by Adam Law 27 Jan '21

27 Jan '21

Hi Christian, Thanks for the permissions pointer- it works well. As an aside I had to use let $vbFailure1 := ($perm?authorization != 'example' or exists( $perm?authorization )=false()) Rather than let $vbFailure1 := ($perm?authorization != 'example' ) To handle the case of no Authorisation. I am not sure why. I should be able to use request:cookie for ws (when I need to) which is great. I wasn't able to work out a catchall rest:path solution but it is not really important for me. Thanks again for the pointer. Adam On Mon, Jan 18, 2021 at 8:33 PM Christian Grün <christian.gruen(a)gmail.com> wrote: > You could have a look at the "Catch HTTP Errors" section in our > documentation. > > [1] https://docs.basex.org/wiki/RESTXQ#Error_Handling > > On Mon, Jan 18, 2021 at 1:30 PM Adam Law <adamjameslaw(a)gmail.com> wrote: > > > > Another thought - a 'catch all' path for fail2ban to catch attempted > accessess to undefined paths. > > I haven't tested this to see if there is such a thing in BaseX > > https://stackoverflow.com/questions/50282541/catch-all-url-in-golang > > > > On Mon, Jan 18, 2021 at 5:26 PM Christian Grün < > christian.gruen(a)gmail.com> wrote: > >> > >> Hi Adam, > >> > >> You could have a loot at our Permissions layer [1]. It can be used for > >> all kinds of checks, not only those specific to authentication. I > >> think we should generalize the concept in a future version of BaseX > >> and introduce additional post-hooks. > >> > >> Hope this helps? > >> Christian > >> > >> [1] https://docs.basex.org/wiki/Permissions > >> > >> > >> > >> On Mon, Jan 18, 2021 at 3:19 AM Adam Law <adamjameslaw(a)gmail.com> > wrote: > >> > > >> > Golang has great middleware functionality for endpoints that allows a > person to readily secure all of them. I was wondering there would be > anyway doing similar in BaseX for restxq & websockets. For example by > requiring all requests to have a particular query parameter (very basic > example). ... to protect say against browser / javascript port scanning. > >> > > >> > Eg > https://stackoverflow.com/questions/26204485/gorilla-mux-custom-middleware > and check that request contains a particular parameter (very basic security) > >> > > >> > It seems that I have to secure each endpoint individually. > >> > > >> > Adam >

2 4

Shrinking view in Manage Database Dialog
by Markus Schmid 27 Jan '21

27 Jan '21

Hallo in Basex v9.4.5 im Dialog Manage Database wird die linke View zusammengeschrumpft wenn die rechte Detail View wegen langer Pfad- oder Datenbanknamen viel Platz braucht. Die Views ändern die Breite während man die Einträge in der Liste der Datenbanken durchläuft und auswählt. Beides nervt leicht. Freundliche Grüße Markus S.

2 2

cross-platform file name parameters
by Graydon Saunders 27 Jan '21

27 Jan '21

Hello -- So I'm on a Linux box and my colleague is on a Windows system. We're both using a query that takes before and after versions of a file, converts each to a single string, and writes that back out as two additional files. (This exists to check if we still have all the words in the same order between before and after.) There must be a sensible way to parameterize the file names so they can be relative to the query on both platforms/easily loaded from a file/something that isn't hand-editing a variable (external or not), but I can't think of it. What's the sensible way to do this? Thanks! Graydon

3 2

[Configuration improvements] Separate client and server config, stop overwriting config files
by Volodymyr Prokopyuk 26 Jan '21

26 Jan '21

Hello the BaseX team, First of all I'm really impressed with the quality of the BaseX XML database. I'd particularly highlight the high compliance with the standards XQuery, XPath and the outstanding performance! Thank you very much for the great product! I'm new to the system and I've faced several configuration deficiencies while trying to configure the BaseX database to fit my needs. What follows is my use case (I hope it is a basic and classical one required by much wider audience), the configuration problems I've faced (I hope these can be easily solved by extending your current implementation) and my proposal to solve the above mentioned configuration problems (I hope this will further improve the quality of your product). *My use case (access the basexserver run as a systemd service via the basexclient)* - I've installed the basex package on Manjaro (Arch linux) into /usr/share/java/basex/ - I want to run the basexserver as a systemd service under the already created basex system user (see the attached basex.server unit file) - I want to store the XML databases and the BaseX logs under /var/lib/basex/data /var/lib/basex/log respectively - I want to manage the basexserver configuration (namely DBPATH, LOGPATH, SERVERPORT) separately from the basexclient configuration (namely HOST, PORT, USER, PASSWORD) in a separate files under /etc/basex and ~/ respectively. I do not want to get my configuration overwritten with unnecessary default options that are already in the system and so clutter my clean just-what-you-need configuration files - I want to manage the XML databases and execute queries against the server using the basexclient from my console by providing XQuery command and queries in a file *Configuration problems (mixed config for client and server in a single .basex file under hardly linked home directory)* - My .basex file mixes server and client configuration in one single file so I cannot separate the configuration into server-related at /etc/basex and client-retated at ~/ - My .basex file gets overwritten with unnecessary default options that clutter my minimal and clean configuration - When I start the basexclient a new .basex configuration file is created in the current working directory. I start the basexclient from many different direcotries, but I want to have the basexclient configuration only in my home ~/ directory - It seems that the location of the .basex file is hardly linked to the home directory forcing me to place data, log and config directories under a single home directory. I want to manage my data in /var/lib/basex/data, my logs in /var/lib/basex/log and my server and client configuration in /etc/basex and ~/ respectively - I cannot find any way (via environment variables or command line options) to instruct both the basexserver and the basexclient to look for the configuration file .basex *Proposal to improve configuration* - Do not overwrite the user-provided configuration files with unnecessary default options that are already in the system or at least use a different file (e. g. .basex-default) - Separate the server-related configuration from the client-related configuration into different files (e. g. .basexserver, .basexclient) - Do not mandate the location of configuration files to be under the home directory because there are different aspects of database management that are usually managed in different directories with different policies and quotas (e. g. data, log, client and server config) - Provide separate options for basexserver and basexclient to locate the corresponding configuration file (e. g. via environment variables BASEX_SERVER_CONFIG, BASEX_CLIENT_CONFIG or command line options -server-config, -client-config) - In order to introduce the proposed improvement you can keep your current implementation unchanged (following the open-closed principle) and just extend the implementation with the alternative locations for the separate configuration files via environment variables or command line options. The only change that is required is to stop overwriting the user-provided configuration files with the default options Can the above described use case be configured with the current implementation of the configuration management? Thank you in advance! I'm open to further discussion and collaboration mostly on the design side to get these improvements to work for all other use cases not covered here. Thank you, Vlad

2 1

BaseX python replace bug
by Daniel Kaplan 22 Jan '21

22 Jan '21

Hello, I'm experiencing some issue(s) when using BaseX... wanted to report it here, but not sure what information you need. I'm accessing baseX via python. Attached is some code that reproduces it for me. It doesn't seem to occur when I am not manually flushing, and only seems to occur with replace, but not add. How should I proceed? Cheers, Daniel PS: I am using a folder containing all the items in question.

3 5

Securing All Endpoints
by Adam Law 19 Jan '21

19 Jan '21

Golang has great middleware functionality for endpoints that allows a person to readily secure all of them. I was wondering there would be anyway doing similar in BaseX for restxq & websockets. For example by requiring all requests to have a particular query parameter (very basic example). ... to protect say against browser / javascript port scanning. Eg https://stackoverflow.com/questions/26204485/gorilla-mux-custom-middleware and check that request contains a particular parameter (very basic security) It seems that I have to secure each endpoint individually. Adam

2 6