Database specific xquery modules
I am looking for some ideas on some architecture surrounding xquery modules. Our situation is that we have multiple client applications in various languages that will connect to a database. We will have some reasonably complex xqueries to extract data in certain structures that will be used by each of these clients. So to avoid having to duplicate the query logic we would like the xqueries to reside "in" the database. Additionally, we would like to use the same BaseX installation for multiple databases, corresponding to multiple projects. The data in each of these databases may be against a different schema and our xqueries are essentially tied to the schema of the data. Thus, each database may need slightly different xqueries. The xquery library modules looks to be global for the BaseX instance, as does the loading of main modules off of disk. Is there any way to tie the query module or file to a specific database? We are trying to avoid being limited to 1 database per BaseX instance. An alternative would be to embed the xqueries within our client applications, but that involves duplication. Also, we are not sure if there are any performance difference with having the queries stored in the repo vs client side? For background, we are looking at migrating from oracle to BaseX. So, we are effectively looking for an equivalent of multiple clients connecting to a userspace and using stored procedures which are specific to that userspace. Any ideas on how to achieve this currently? Or is this something that BaseX could support? Cheers, Chris
Hi Chris, as you might already know, BaseX has built-in REST support [1] which you can use to query and update data via HTTP. You can also use the RESTXQ [2] functionality to define arbitrary server side XQueries which can also be requested by HTTP and are also suitable for web applications. Both methods are not restricted to a single database. Best regards, Markus [1] http://docs.basex.org/wiki/REST [2] http://docs.basex.org/wiki/RESTXQ Am 21.02.2020 um 02:05 schrieb DYER Chris:
I am looking for some ideas on some architecture surrounding xquery modules.
Our situation is that we have multiple client applications in various languages that will connect to a database.
We will have some reasonably complex xqueries to extract data in certain structures that will be used by each of these clients.
So to avoid having to duplicate the query logic we would like the xqueries to reside "in" the database.
Additionally, we would like to use the same BaseX installation for multiple databases, corresponding to multiple projects.
The data in each of these databases may be against a different schema and our xqueries are essentially tied to the schema of the data. Thus, each database may need slightly different xqueries.
The xquery library modules looks to be global for the BaseX instance, as does the loading of main modules off of disk.
Is there any way to tie the query module or file to a specific database?
We are trying to avoid being limited to 1 database per BaseX instance.
An alternative would be to embed the xqueries within our client applications, but that involves duplication. Also, we are not sure if there are any performance difference with having the queries stored in the repo vs client side?
For background, we are looking at migrating from oracle to BaseX. So, we are effectively looking for an equivalent of multiple clients connecting to a userspace and using stored procedures which are specific to that userspace.
Any ideas on how to achieve this currently? Or is this something that BaseX could support?
Cheers,
Chris
-- Markus Wittenberg Tel +49 (0)341 248 475 36 Mail wittenberg@axxepta.de ---- axxepta solutions GmbH Lehmgrubenweg 17, 88131 Lindau Amtsgericht Berlin HRB 97544B Geschäftsführer: Karsten Becke, Maximilian Gärber
Hi Markus, Our intent is to use REST and RESTXQ interfaces to interact with data on a single database using some server side XQueries. However, we may have 2 (or more) databases provided by some server, and associated xqueries are specific to the contents of each database (as the contents of each of these databases may be structured differently). We would like for a client to be able to open a single database and be able to run associated XQueries, without requiring client side knowledge of which server side xqueries are associated with the database it has opened. Does this help describe what we are trying to achieve? Cheers, Chris From: BaseX-Talk <basex-talk-bounces@mailman.uni-konstanz.de> On Behalf Of Markus Wittenberg Sent: Friday, 21 February 2020 12:08 PM To: basex-talk@mailman.uni-konstanz.de Subject: Re: [basex-talk] Database specific xquery modules Hi Chris, as you might already know, BaseX has built-in REST support [1] which you can use to query and update data via HTTP. You can also use the RESTXQ [2] functionality to define arbitrary server side XQueries which can also be requested by HTTP and are also suitable for web applications. Both methods are not restricted to a single database. Best regards, Markus [1] http://docs.basex.org/wiki/REST [2] http://docs.basex.org/wiki/RESTXQ Am 21.02.2020 um 02:05 schrieb DYER Chris: I am looking for some ideas on some architecture surrounding xquery modules. Our situation is that we have multiple client applications in various languages that will connect to a database. We will have some reasonably complex xqueries to extract data in certain structures that will be used by each of these clients. So to avoid having to duplicate the query logic we would like the xqueries to reside "in" the database. Additionally, we would like to use the same BaseX installation for multiple databases, corresponding to multiple projects. The data in each of these databases may be against a different schema and our xqueries are essentially tied to the schema of the data. Thus, each database may need slightly different xqueries. The xquery library modules looks to be global for the BaseX instance, as does the loading of main modules off of disk. Is there any way to tie the query module or file to a specific database? We are trying to avoid being limited to 1 database per BaseX instance. An alternative would be to embed the xqueries within our client applications, but that involves duplication. Also, we are not sure if there are any performance difference with having the queries stored in the repo vs client side? For background, we are looking at migrating from oracle to BaseX. So, we are effectively looking for an equivalent of multiple clients connecting to a userspace and using stored procedures which are specific to that userspace. Any ideas on how to achieve this currently? Or is this something that BaseX could support? Cheers, Chris -- Markus Wittenberg Tel +49 (0)341 248 475 36 Mail wittenberg@axxepta.de<mailto:wittenberg@axxepta.de> ---- axxepta solutions GmbH Lehmgrubenweg 17, 88131 Lindau Amtsgericht Berlin HRB 97544B Geschäftsführer: Karsten Becke, Maximilian Gärber
A RESTXQ definition could look something like this: |declare| | %rest:path||(||"/a/{$client}/{$path}"||)| |function| |yourmodule:test(||$client as xs:string||, ||$||path ||as xs:string||||) {| |let $db := switch ($client)| | case "clientA" return db:open("dbA")| ||| | | case "clientA" return db:open("dbA")||| | default return db:open("dbX")| | return yourmodule:do-something-with($db, $path) | |};| And the called function |do-something-with| could also depend on the client Am 21.02.2020 um 03:48 schrieb DYER Chris:
Hi Markus,
Our intent is to use REST and RESTXQ interfaces to interact with data on a /single/ database using some server side XQueries.
However, we may have 2 (or more) databases provided by some server, and associated xqueries are specific to the contents of each database (as the contents of each of these databases may be structured differently).
We would like for a client to be able to open a single database and be able to run associated XQueries, without requiring client side knowledge of which server side xqueries are associated with the database it has opened.
Does this help describe what we are trying to achieve?
Cheers, Chris
*From:*BaseX-Talk <basex-talk-bounces@mailman.uni-konstanz.de> *On Behalf Of *Markus Wittenberg *Sent:* Friday, 21 February 2020 12:08 PM *To:* basex-talk@mailman.uni-konstanz.de *Subject:* Re: [basex-talk] Database specific xquery modules
Hi Chris,
as you might already know, BaseX has built-in REST support [1] which you can use to query and update data via HTTP. You can also use the RESTXQ [2] functionality to define arbitrary server side XQueries which can also be requested by HTTP and are also suitable for web applications. Both methods are not restricted to a single database.
Best regards,
Markus
[1] http://docs.basex.org/wiki/REST
[2] http://docs.basex.org/wiki/RESTXQ
Am 21.02.2020 um 02:05 schrieb DYER Chris:
I am looking for some ideas on some architecture surrounding xquery modules.
Our situation is that we have multiple client applications in various languages that will connect to a database.
We will have some reasonably complex xqueries to extract data in certain structures that will be used by each of these clients.
So to avoid having to duplicate the query logic we would like the xqueries to reside "in" the database.
Additionally, we would like to use the same BaseX installation for multiple databases, corresponding to multiple projects.
The data in each of these databases may be against a different schema and our xqueries are essentially tied to the schema of the data. Thus, each database may need slightly different xqueries.
The xquery library modules looks to be global for the BaseX instance, as does the loading of main modules off of disk.
Is there any way to tie the query module or file to a specific database?
We are trying to avoid being limited to 1 database per BaseX instance.
An alternative would be to embed the xqueries within our client applications, but that involves duplication. Also, we are not sure if there are any performance difference with having the queries stored in the repo vs client side?
For background, we are looking at migrating from oracle to BaseX. So, we are effectively looking for an equivalent of multiple clients connecting to a userspace and using stored procedures which are specific to that userspace.
Any ideas on how to achieve this currently? Or is this something that BaseX could support?
Cheers,
Chris
-- Markus Wittenberg Tel +49 (0)341 248 475 36 Mailwittenberg@axxepta.de <mailto:wittenberg@axxepta.de> ---- axxepta solutions GmbH Lehmgrubenweg 17, 88131 Lindau Amtsgericht Berlin HRB 97544B Geschäftsführer: Karsten Becke, Maximilian Gärber
-- Markus Wittenberg Tel +49 (0)341 248 475 36 Mail wittenberg@axxepta.de ---- axxepta solutions GmbH Lehmgrubenweg 17, 88131 Lindau Amtsgericht Berlin HRB 97544B Geschäftsführer: Karsten Becke, Maximilian Gärber
Hi Markus, Not quite what I am aiming for. We do want a single BaseX Instance to provide access to multiple databases. Further to this though, a set of client applications will only connect to one database. We have multiple sets of client applications, with each set each connecting to one database. We do want to modularise the functions, however as an example we want to be able to have 2 versions of the same function, where 1 version works with the schema of the data in database 1, and the second version with a different schema in database 2. If I could explicitly link the function modules with a database, I could call the same function from both application sets, and use the context of the open database to call the correct version of the function. An example situation may help Say we have Database for "Black Books", which has book data in xml structures which conform to some version (1.1) of a schema. We want a query stored on the server that returns books on art Get-Books-About-Art We have a client application specifically for this database that calls this function. It will only need to query the data in this database. Now we add a second Database for "Goliath Books" The book data in this database uses a modified xml structure (version 2 of a schema) We have a client application for this database that also wants to get books about art. The query for this database is slightly different because the xml schema has changed. I would like to store both versions of Get-Books-About-Art in the database. Then for each client to connect to their associated database, and be able to execute the correct version of the xquery function. The idea is that each time I add a new database that has data in some format, I can install associated query modules that are restricted to the context of that database to avoid conflicts with existing databases. Maybe what I am looking for is effectively a database specific repo to install modules to, rather than a basex install repo. From: Markus Wittenberg <wittenberg@axxepta.de> Sent: Friday, 21 February 2020 2:15 PM To: DYER Chris <Chris.Dyer@sydac.com>; basex-talk@mailman.uni-konstanz.de Subject: Re: [basex-talk] Database specific xquery modules A RESTXQ definition could look something like this: declare %rest:path("/a/{$client}/{$path}") function yourmodule:test($client as xs:string, $path as xs:string) { let $db := switch ($client) case "clientA" return db:open("dbA") case "clientA" return db:open("dbA") default return db:open("dbX") return yourmodule:do-something-with($db, $path) }; And the called function do-something-with could also depend on the client Am 21.02.2020 um 03:48 schrieb DYER Chris: Hi Markus, Our intent is to use REST and RESTXQ interfaces to interact with data on a single database using some server side XQueries. However, we may have 2 (or more) databases provided by some server, and associated xqueries are specific to the contents of each database (as the contents of each of these databases may be structured differently). We would like for a client to be able to open a single database and be able to run associated XQueries, without requiring client side knowledge of which server side xqueries are associated with the database it has opened. Does this help describe what we are trying to achieve? Cheers, Chris From: BaseX-Talk <basex-talk-bounces@mailman.uni-konstanz.de><mailto:basex-talk-bounces@mailman.uni-konstanz.de> On Behalf Of Markus Wittenberg Sent: Friday, 21 February 2020 12:08 PM To: basex-talk@mailman.uni-konstanz.de<mailto:basex-talk@mailman.uni-konstanz.de> Subject: Re: [basex-talk] Database specific xquery modules Hi Chris, as you might already know, BaseX has built-in REST support [1] which you can use to query and update data via HTTP. You can also use the RESTXQ [2] functionality to define arbitrary server side XQueries which can also be requested by HTTP and are also suitable for web applications. Both methods are not restricted to a single database. Best regards, Markus [1] http://docs.basex.org/wiki/REST [2] http://docs.basex.org/wiki/RESTXQ Am 21.02.2020 um 02:05 schrieb DYER Chris: I am looking for some ideas on some architecture surrounding xquery modules. Our situation is that we have multiple client applications in various languages that will connect to a database. We will have some reasonably complex xqueries to extract data in certain structures that will be used by each of these clients. So to avoid having to duplicate the query logic we would like the xqueries to reside "in" the database. Additionally, we would like to use the same BaseX installation for multiple databases, corresponding to multiple projects. The data in each of these databases may be against a different schema and our xqueries are essentially tied to the schema of the data. Thus, each database may need slightly different xqueries. The xquery library modules looks to be global for the BaseX instance, as does the loading of main modules off of disk. Is there any way to tie the query module or file to a specific database? We are trying to avoid being limited to 1 database per BaseX instance. An alternative would be to embed the xqueries within our client applications, but that involves duplication. Also, we are not sure if there are any performance difference with having the queries stored in the repo vs client side? For background, we are looking at migrating from oracle to BaseX. So, we are effectively looking for an equivalent of multiple clients connecting to a userspace and using stored procedures which are specific to that userspace. Any ideas on how to achieve this currently? Or is this something that BaseX could support? Cheers, Chris -- Markus Wittenberg Tel +49 (0)341 248 475 36 Mail wittenberg@axxepta.de<mailto:wittenberg@axxepta.de> ---- axxepta solutions GmbH Lehmgrubenweg 17, 88131 Lindau Amtsgericht Berlin HRB 97544B Geschäftsführer: Karsten Becke, Maximilian Gärber -- Markus Wittenberg Tel +49 (0)341 248 475 36 Mail wittenberg@axxepta.de<mailto:wittenberg@axxepta.de> ---- axxepta solutions GmbH Lehmgrubenweg 17, 88131 Lindau Amtsgericht Berlin HRB 97544B Geschäftsführer: Karsten Becke, Maximilian Gärber
If you want to use RESTXQ you would have to put the modules, or least the functions defining the endpoints, in the webapp directory and not in the repo directory. And far as I know, BaseX doesn't support installation for modules in webapp as for in repo. So you have to explicitly change/add files in webapp directory. You could write a module for each database, defining its own endpoints, and maybe still reuse some code by importing a module with common functions in every specific module. Am 21.02.2020 um 07:11 schrieb DYER Chris:
Hi Markus,
Not quite what I am aiming for.
We /do/ want a single BaseX Instance to provide access to multiple databases.
Further to this though, a set of client applications will only connect to one database. We have multiple sets of client applications, with each set each connecting to one database.
We do want to modularise the functions, however as an example we want to be able to have 2 versions of the same function, where 1 version works with the schema of the data in database 1, and the second version with a different schema in database 2.
If I could explicitly link the function modules with a database, I could call the same function from both application sets, and use the context of the open database to call the correct version of the function.
**
An example situation may help
Say we have Database for “Black Books”, which has book data in xml structures which conform to some version (1.1) of a schema.
We want a query stored on the server that returns books on art
Get-Books-About-Art
We have a client application specifically for this database that calls this function.
It will only need to query the data in this database.
Now we add a second Database for "Goliath Books"
The book data in this database uses a modified xml structure (version 2 of a schema)
We have a client application for this database that also wants to get books about art.
The query for this database is slightly different because the xml schema has changed.
I would like to store both versions of Get-Books-About-Art in the database.
Then for each client to connect to their associated database, and be able to execute the correct version of the xquery function.
The idea is that each time I add a new database that has data in some format, I can install associated query modules that are restricted to the context of that database to avoid conflicts with existing databases.
Maybe what I am looking for is effectively a database specific repo to install modules to, rather than a basex install repo.
*From:*Markus Wittenberg <wittenberg@axxepta.de> *Sent:* Friday, 21 February 2020 2:15 PM *To:* DYER Chris <Chris.Dyer@sydac.com>; basex-talk@mailman.uni-konstanz.de *Subject:* Re: [basex-talk] Database specific xquery modules
A RESTXQ definition could look something like this:
|declare|
|%rest:path("/a/{$client}/{$path}")|
|function| |yourmodule:test($client as xs:string, $path as xs:string) {|
|let $db := switch ($client)|
| case "clientA" return db:open("dbA")|
| case "clientA" return db:open("dbA")|
| default return db:open("dbX")|
| return yourmodule:do-something-with($db, $path)|
|};|
And the called function |do-something-with| could also depend on the client
Am 21.02.2020 um 03:48 schrieb DYER Chris:
Hi Markus,
Our intent is to use REST and RESTXQ interfaces to interact with data on a /single/ database using some server side XQueries.
However, we may have 2 (or more) databases provided by some server, and associated xqueries are specific to the contents of each database (as the contents of each of these databases may be structured differently).
We would like for a client to be able to open a single database and be able to run associated XQueries, without requiring client side knowledge of which server side xqueries are associated with the database it has opened.
Does this help describe what we are trying to achieve?
Cheers, Chris
*From:*BaseX-Talk <basex-talk-bounces@mailman.uni-konstanz.de> <mailto:basex-talk-bounces@mailman.uni-konstanz.de> *On Behalf Of *Markus Wittenberg *Sent:* Friday, 21 February 2020 12:08 PM *To:* basex-talk@mailman.uni-konstanz.de <mailto:basex-talk@mailman.uni-konstanz.de> *Subject:* Re: [basex-talk] Database specific xquery modules
Hi Chris,
as you might already know, BaseX has built-in REST support [1] which you can use to query and update data via HTTP. You can also use the RESTXQ [2] functionality to define arbitrary server side XQueries which can also be requested by HTTP and are also suitable for web applications. Both methods are not restricted to a single database.
Best regards,
Markus
[1] http://docs.basex.org/wiki/REST
[2] http://docs.basex.org/wiki/RESTXQ
Am 21.02.2020 um 02:05 schrieb DYER Chris:
I am looking for some ideas on some architecture surrounding xquery modules.
Our situation is that we have multiple client applications in various languages that will connect to a database.
We will have some reasonably complex xqueries to extract data in certain structures that will be used by each of these clients.
So to avoid having to duplicate the query logic we would like the xqueries to reside "in" the database.
Additionally, we would like to use the same BaseX installation for multiple databases, corresponding to multiple projects.
The data in each of these databases may be against a different schema and our xqueries are essentially tied to the schema of the data. Thus, each database may need slightly different xqueries.
The xquery library modules looks to be global for the BaseX instance, as does the loading of main modules off of disk.
Is there any way to tie the query module or file to a specific database?
We are trying to avoid being limited to 1 database per BaseX instance.
An alternative would be to embed the xqueries within our client applications, but that involves duplication. Also, we are not sure if there are any performance difference with having the queries stored in the repo vs client side?
For background, we are looking at migrating from oracle to BaseX. So, we are effectively looking for an equivalent of multiple clients connecting to a userspace and using stored procedures which are specific to that userspace.
Any ideas on how to achieve this currently? Or is this something that BaseX could support?
Cheers,
Chris
--
Markus Wittenberg
Tel +49 (0)341 248 475 36
Mailwittenberg@axxepta.de <mailto:wittenberg@axxepta.de>
----
axxepta solutions GmbH
Lehmgrubenweg 17, 88131 Lindau
Amtsgericht Berlin HRB 97544B
Geschäftsführer: Karsten Becke, Maximilian Gärber
-- Markus Wittenberg Tel +49 (0)341 248 475 36 Mailwittenberg@axxepta.de <mailto:wittenberg@axxepta.de> ---- axxepta solutions GmbH Lehmgrubenweg 17, 88131 Lindau Amtsgericht Berlin HRB 97544B Geschäftsführer: Karsten Becke, Maximilian Gärber
-- Markus Wittenberg Tel +49 (0)341 248 475 36 Mail wittenberg@axxepta.de ---- axxepta solutions GmbH Lehmgrubenweg 17, 88131 Lindau Amtsgericht Berlin HRB 97544B Geschäftsführer: Karsten Becke, Maximilian Gärber
I’m not sure this is the best approach, but you seem determined to try to figure out how to turn XQuery into an Object-Oriented language, so, if you want to bind query functions to a particular database, you can put the functions in the database, and use xquery:eval() http://docs.basex.org/wiki/XQuery_Module <http://docs.basex.org/wiki/XQuery_Module> to call the function as a node() argument to eval. You would be using the context database as the “self” variable and xquery:eval() as dispatcher. If you don’t literally want to bind functions to DB, but instead dispatch by the schema of the data, you could use a typeswitch if the schema have different namespaces or root elements,. ( I’ve tried this to switch between processing TEIP4 and TEIP5, for example. ) — Steve M.
On Feb 21, 2020, at 1:41 AM, Markus Wittenberg <wittenberg@axxepta.de> wrote:
If you want to use RESTXQ you would have to put the modules, or least the functions defining the endpoints, in the webapp directory and not in the repo directory. And far as I know, BaseX doesn't support installation for modules in webapp as for in repo. So you have to explicitly change/add files in webapp directory.
You could write a module for each database, defining its own endpoints, and maybe still reuse some code by importing a module with common functions in every specific module.
Am 21.02.2020 um 07:11 schrieb DYER Chris:
Hi Markus, Not quite what I am aiming for.
We do want a single BaseX Instance to provide access to multiple databases.
Further to this though, a set of client applications will only connect to one database. We have multiple sets of client applications, with each set each connecting to one database.
We do want to modularise the functions, however as an example we want to be able to have 2 versions of the same function, where 1 version works with the schema of the data in database 1, and the second version with a different schema in database 2.
If I could explicitly link the function modules with a database, I could call the same function from both application sets, and use the context of the open database to call the correct version of the function.
An example situation may help Say we have Database for “Black Books”, which has book data in xml structures which conform to some version (1.1) of a schema. We want a query stored on the server that returns books on art Get-Books-About-Art We have a client application specifically for this database that calls this function. It will only need to query the data in this database.
Now we add a second Database for "Goliath Books" The book data in this database uses a modified xml structure (version 2 of a schema) We have a client application for this database that also wants to get books about art. The query for this database is slightly different because the xml schema has changed.
I would like to store both versions of Get-Books-About-Art in the database. Then for each client to connect to their associated database, and be able to execute the correct version of the xquery function.
The idea is that each time I add a new database that has data in some format, I can install associated query modules that are restricted to the context of that database to avoid conflicts with existing databases.
Maybe what I am looking for is effectively a database specific repo to install modules to, rather than a basex install repo.
From: Markus Wittenberg <wittenberg@axxepta.de> <mailto:wittenberg@axxepta.de> Sent: Friday, 21 February 2020 2:15 PM To: DYER Chris <Chris.Dyer@sydac.com> <mailto:Chris.Dyer@sydac.com>; basex-talk@mailman.uni-konstanz.de <mailto:basex-talk@mailman.uni-konstanz.de> Subject: Re: [basex-talk] Database specific xquery modules
A RESTXQ definition could look something like this:
declare %rest:path("/a/{$client}/{$path}") function yourmodule:test($client as xs:string, $path as xs:string) { let $db := switch ($client) case "clientA" return db:open("dbA") case "clientA" return db:open("dbA") default return db:open("dbX") return yourmodule:do-something-with($db, $path) }; And the called function do-something-with could also depend on the client
Am 21.02.2020 um 03:48 schrieb DYER Chris: Hi Markus, Our intent is to use REST and RESTXQ interfaces to interact with data on a single database using some server side XQueries.
However, we may have 2 (or more) databases provided by some server, and associated xqueries are specific to the contents of each database (as the contents of each of these databases may be structured differently).
We would like for a client to be able to open a single database and be able to run associated XQueries, without requiring client side knowledge of which server side xqueries are associated with the database it has opened.
Does this help describe what we are trying to achieve?
Cheers, Chris
From: BaseX-Talk <basex-talk-bounces@mailman.uni-konstanz.de> <mailto:basex-talk-bounces@mailman.uni-konstanz.de> On Behalf Of Markus Wittenberg Sent: Friday, 21 February 2020 12:08 PM To: basex-talk@mailman.uni-konstanz.de <mailto:basex-talk@mailman.uni-konstanz.de> Subject: Re: [basex-talk] Database specific xquery modules
Hi Chris,
as you might already know, BaseX has built-in REST support [1] which you can use to query and update data via HTTP. You can also use the RESTXQ [2] functionality to define arbitrary server side XQueries which can also be requested by HTTP and are also suitable for web applications. Both methods are not restricted to a single database.
Best regards,
Markus
[1] http://docs.basex.org/wiki/REST <http://docs.basex.org/wiki/REST> [2] http://docs.basex.org/wiki/RESTXQ <http://docs.basex.org/wiki/RESTXQ>
Am 21.02.2020 um 02:05 schrieb DYER Chris: I am looking for some ideas on some architecture surrounding xquery modules.
Our situation is that we have multiple client applications in various languages that will connect to a database. We will have some reasonably complex xqueries to extract data in certain structures that will be used by each of these clients. So to avoid having to duplicate the query logic we would like the xqueries to reside "in" the database.
Additionally, we would like to use the same BaseX installation for multiple databases, corresponding to multiple projects. The data in each of these databases may be against a different schema and our xqueries are essentially tied to the schema of the data. Thus, each database may need slightly different xqueries.
The xquery library modules looks to be global for the BaseX instance, as does the loading of main modules off of disk. Is there any way to tie the query module or file to a specific database?
We are trying to avoid being limited to 1 database per BaseX instance.
An alternative would be to embed the xqueries within our client applications, but that involves duplication. Also, we are not sure if there are any performance difference with having the queries stored in the repo vs client side?
For background, we are looking at migrating from oracle to BaseX. So, we are effectively looking for an equivalent of multiple clients connecting to a userspace and using stored procedures which are specific to that userspace.
Any ideas on how to achieve this currently? Or is this something that BaseX could support?
Cheers, Chris
-- Markus Wittenberg
Tel +49 (0)341 248 475 36 Mail wittenberg@axxepta.de <mailto:wittenberg@axxepta.de>
----
axxepta solutions GmbH Lehmgrubenweg 17, 88131 Lindau
Amtsgericht Berlin HRB 97544B Geschäftsführer: Karsten Becke, Maximilian Gärber -- Markus Wittenberg
Tel +49 (0)341 248 475 36 Mail wittenberg@axxepta.de <mailto:wittenberg@axxepta.de>
----
axxepta solutions GmbH Lehmgrubenweg 17, 88131 Lindau
Amtsgericht Berlin HRB 97544B Geschäftsführer: Karsten Becke, Maximilian Gärber -- Markus Wittenberg
Tel +49 (0)341 248 475 36 Mail wittenberg@axxepta.de <mailto:wittenberg@axxepta.de>
----
axxepta solutions GmbH Lehmgrubenweg 17, 88131 Lindau
Amtsgericht Berlin HRB 97544B Geschäftsführer: Karsten Becke, Maximilian Gärber
Hi Steve, I do not think what we are trying to achieve is necessarily OO related (although I can see why it may look that way). It may be more that we would be looking at another layer of encapsulation of some basex features. Your suggestion on storing the queries within the database and then evaluating them does sound like a possible option for us. It would be nice to achieve this scoping of the queries using explicitly registered function modules. This is all based on what we are currently achieving with our oracle database solution. We have a single oracle database server, and we have developers create new user spaces on that server which encapsulates a set of tables and associated stored procedures (functions) for working with those tables. Those functions are only visible within the scope of the user space It sounds like this sort of use case would generally be handled with separate basex installs/servers. So I am possibly looking at a feature request. Cheers, Chris From: Majewski, Steven Dennis (sdm7g) <sdm7g@virginia.edu> Sent: Saturday, 22 February 2020 12:37 AM To: DYER Chris <Chris.Dyer@sydac.com>; basex-talk@mailman.uni-konstanz.de Cc: Markus Wittenberg <wittenberg@axxepta.de> Subject: Re: [basex-talk] Database specific xquery modules I’m not sure this is the best approach, but you seem determined to try to figure out how to turn XQuery into an Object-Oriented language, so, if you want to bind query functions to a particular database, you can put the functions in the database, and use xquery:eval() http://docs.basex.org/wiki/XQuery_Module to call the function as a node() argument to eval. You would be using the context database as the “self” variable and xquery:eval() as dispatcher. If you don’t literally want to bind functions to DB, but instead dispatch by the schema of the data, you could use a typeswitch if the schema have different namespaces or root elements,. ( I’ve tried this to switch between processing TEIP4 and TEIP5, for example. ) — Steve M. On Feb 21, 2020, at 1:41 AM, Markus Wittenberg <wittenberg@axxepta.de<mailto:wittenberg@axxepta.de>> wrote: If you want to use RESTXQ you would have to put the modules, or least the functions defining the endpoints, in the webapp directory and not in the repo directory. And far as I know, BaseX doesn't support installation for modules in webapp as for in repo. So you have to explicitly change/add files in webapp directory. You could write a module for each database, defining its own endpoints, and maybe still reuse some code by importing a module with common functions in every specific module. Am 21.02.2020 um 07:11 schrieb DYER Chris: Hi Markus, Not quite what I am aiming for. We do want a single BaseX Instance to provide access to multiple databases. Further to this though, a set of client applications will only connect to one database. We have multiple sets of client applications, with each set each connecting to one database. We do want to modularise the functions, however as an example we want to be able to have 2 versions of the same function, where 1 version works with the schema of the data in database 1, and the second version with a different schema in database 2. If I could explicitly link the function modules with a database, I could call the same function from both application sets, and use the context of the open database to call the correct version of the function. An example situation may help Say we have Database for “Black Books”, which has book data in xml structures which conform to some version (1.1) of a schema. We want a query stored on the server that returns books on art Get-Books-About-Art We have a client application specifically for this database that calls this function. It will only need to query the data in this database. Now we add a second Database for "Goliath Books" The book data in this database uses a modified xml structure (version 2 of a schema) We have a client application for this database that also wants to get books about art. The query for this database is slightly different because the xml schema has changed. I would like to store both versions of Get-Books-About-Art in the database. Then for each client to connect to their associated database, and be able to execute the correct version of the xquery function. The idea is that each time I add a new database that has data in some format, I can install associated query modules that are restricted to the context of that database to avoid conflicts with existing databases. Maybe what I am looking for is effectively a database specific repo to install modules to, rather than a basex install repo. From: Markus Wittenberg <wittenberg@axxepta.de><mailto:wittenberg@axxepta.de> Sent: Friday, 21 February 2020 2:15 PM To: DYER Chris <Chris.Dyer@sydac.com><mailto:Chris.Dyer@sydac.com>; basex-talk@mailman.uni-konstanz.de<mailto:basex-talk@mailman.uni-konstanz.de> Subject: Re: [basex-talk] Database specific xquery modules A RESTXQ definition could look something like this: declare %rest:path("/a/{$client}/{$path}") function yourmodule:test($client as xs:string, $path as xs:string) { let $db := switch ($client) case "clientA" return db:open("dbA") case "clientA" return db:open("dbA") default return db:open("dbX") return yourmodule:do-something-with($db, $path) }; And the called function do-something-with could also depend on the client Am 21.02.2020 um 03:48 schrieb DYER Chris: Hi Markus, Our intent is to use REST and RESTXQ interfaces to interact with data on a single database using some server side XQueries. However, we may have 2 (or more) databases provided by some server, and associated xqueries are specific to the contents of each database (as the contents of each of these databases may be structured differently). We would like for a client to be able to open a single database and be able to run associated XQueries, without requiring client side knowledge of which server side xqueries are associated with the database it has opened. Does this help describe what we are trying to achieve? Cheers, Chris From: BaseX-Talk <basex-talk-bounces@mailman.uni-konstanz.de><mailto:basex-talk-bounces@mailman.uni-konstanz.de> On Behalf Of Markus Wittenberg Sent: Friday, 21 February 2020 12:08 PM To: basex-talk@mailman.uni-konstanz.de<mailto:basex-talk@mailman.uni-konstanz.de> Subject: Re: [basex-talk] Database specific xquery modules Hi Chris, as you might already know, BaseX has built-in REST support [1] which you can use to query and update data via HTTP. You can also use the RESTXQ [2] functionality to define arbitrary server side XQueries which can also be requested by HTTP and are also suitable for web applications. Both methods are not restricted to a single database. Best regards, Markus [1] http://docs.basex.org/wiki/REST [2] http://docs.basex.org/wiki/RESTXQ Am 21.02.2020 um 02:05 schrieb DYER Chris: I am looking for some ideas on some architecture surrounding xquery modules. Our situation is that we have multiple client applications in various languages that will connect to a database. We will have some reasonably complex xqueries to extract data in certain structures that will be used by each of these clients. So to avoid having to duplicate the query logic we would like the xqueries to reside "in" the database. Additionally, we would like to use the same BaseX installation for multiple databases, corresponding to multiple projects. The data in each of these databases may be against a different schema and our xqueries are essentially tied to the schema of the data. Thus, each database may need slightly different xqueries. The xquery library modules looks to be global for the BaseX instance, as does the loading of main modules off of disk. Is there any way to tie the query module or file to a specific database? We are trying to avoid being limited to 1 database per BaseX instance. An alternative would be to embed the xqueries within our client applications, but that involves duplication. Also, we are not sure if there are any performance difference with having the queries stored in the repo vs client side? For background, we are looking at migrating from oracle to BaseX. So, we are effectively looking for an equivalent of multiple clients connecting to a userspace and using stored procedures which are specific to that userspace. Any ideas on how to achieve this currently? Or is this something that BaseX could support? Cheers, Chris -- Markus Wittenberg Tel +49 (0)341 248 475 36 Mail wittenberg@axxepta.de<mailto:wittenberg@axxepta.de> ---- axxepta solutions GmbH Lehmgrubenweg 17, 88131 Lindau Amtsgericht Berlin HRB 97544B Geschäftsführer: Karsten Becke, Maximilian Gärber -- Markus Wittenberg Tel +49 (0)341 248 475 36 Mail wittenberg@axxepta.de<mailto:wittenberg@axxepta.de> ---- axxepta solutions GmbH Lehmgrubenweg 17, 88131 Lindau Amtsgericht Berlin HRB 97544B Geschäftsführer: Karsten Becke, Maximilian Gärber -- Markus Wittenberg Tel +49 (0)341 248 475 36 Mail wittenberg@axxepta.de<mailto:wittenberg@axxepta.de> ---- axxepta solutions GmbH Lehmgrubenweg 17, 88131 Lindau Amtsgericht Berlin HRB 97544B Geschäftsführer: Karsten Becke, Maximilian Gärber
Hi Chris,
Our situation is that we have multiple client applications in various languages that will connect to a database.
Are you already using some of the clients that are available for communication with the BaseX client/server architecture [1], or are (planning to) communicating with the HTTP server? In the first case, or if you use HTTP with the basic REST API, it’s recommendable to give your users access to this specific database [2]. You could then store all queries on the server, and launch them via the RUN command [3] or the equivalent REST 'run' operation [4]. In the second case, and if you decide to write a RESTXQ application, you will have full flexibility, as you control in your server-side entry point which database is to be opened and accessed [5]. As indicated by Steve, you can use xquery:eval [6] to address and evaluate queries that are either organized as static strings, stored in the file system or stored in a database (possibly a separate instance that contains only queries). Compared to most relational DBMS, BaseX is much more light-weight, and startup time is minimal. If you have a limited number of database that you want to provide, it’s perfectly ok to launch a number of BaseX instances. I remember a use case in which we had deployed 12 BaseX instances on a single server, mostly because we wanted to simulate horizontal scalability. Personally, I would recommend the RESTXQ approach, but it also depends on how much time you want to invest in writing more complex XQuery applications. Cheers, Christian [1] http://docs.basex.org/wiki/Clients [2] http://docs.basex.org/wiki/User_Management [3] http://docs.basex.org/wiki/Commands#RUN [4] http://docs.basex.org/wiki/REST#GET_Method [5] http://docs.basex.org/wiki/RESTXQ [6] http://docs.basex.org/wiki/XQuery_Module#xquery:eval On Mon, Feb 24, 2020 at 6:22 AM DYER Chris <Chris.Dyer@sydac.com> wrote:
Hi Steve,
I do not think what we are trying to achieve is necessarily OO related (although I can see why it may look that way).
It may be more that we would be looking at another layer of encapsulation of some basex features.
Your suggestion on storing the queries within the database and then evaluating them does sound like a possible option for us. It would be nice to achieve this scoping of the queries using explicitly registered function modules.
This is all based on what we are currently achieving with our oracle database solution.
We have a single oracle database server, and we have developers create new user spaces on that server which encapsulates a set of tables and associated stored procedures (functions) for working with those tables. Those functions are only visible within the scope of the user space
It sounds like this sort of use case would generally be handled with separate basex installs/servers.
So I am possibly looking at a feature request.
Cheers, Chris
From: Majewski, Steven Dennis (sdm7g) <sdm7g@virginia.edu> Sent: Saturday, 22 February 2020 12:37 AM To: DYER Chris <Chris.Dyer@sydac.com>; basex-talk@mailman.uni-konstanz.de Cc: Markus Wittenberg <wittenberg@axxepta.de> Subject: Re: [basex-talk] Database specific xquery modules
I’m not sure this is the best approach, but you seem determined to try to figure out how to turn XQuery into an Object-Oriented language, so, if you want to bind query functions to a particular database, you can put the functions in the database, and use xquery:eval()
http://docs.basex.org/wiki/XQuery_Module to call the function as a node() argument to eval.
You would be using the context database as the “self” variable and xquery:eval() as dispatcher.
If you don’t literally want to bind functions to DB, but instead dispatch by the schema of the data, you could use a typeswitch if the schema have different namespaces or root elements,.
( I’ve tried this to switch between processing TEIP4 and TEIP5, for example. )
— Steve M.
On Feb 21, 2020, at 1:41 AM, Markus Wittenberg <wittenberg@axxepta.de> wrote:
If you want to use RESTXQ you would have to put the modules, or least the functions defining the endpoints, in the webapp directory and not in the repo directory. And far as I know, BaseX doesn't support installation for modules in webapp as for in repo. So you have to explicitly change/add files in webapp directory.
You could write a module for each database, defining its own endpoints, and maybe still reuse some code by importing a module with common functions in every specific module.
Am 21.02.2020 um 07:11 schrieb DYER Chris:
Hi Markus,
Not quite what I am aiming for.
We do want a single BaseX Instance to provide access to multiple databases.
Further to this though, a set of client applications will only connect to one database. We have multiple sets of client applications, with each set each connecting to one database.
We do want to modularise the functions, however as an example we want to be able to have 2 versions of the same function, where 1 version works with the schema of the data in database 1, and the second version with a different schema in database 2.
If I could explicitly link the function modules with a database, I could call the same function from both application sets, and use the context of the open database to call the correct version of the function.
An example situation may help
Say we have Database for “Black Books”, which has book data in xml structures which conform to some version (1.1) of a schema.
We want a query stored on the server that returns books on art
Get-Books-About-Art
We have a client application specifically for this database that calls this function.
It will only need to query the data in this database.
Now we add a second Database for "Goliath Books"
The book data in this database uses a modified xml structure (version 2 of a schema)
We have a client application for this database that also wants to get books about art.
The query for this database is slightly different because the xml schema has changed.
I would like to store both versions of Get-Books-About-Art in the database.
Then for each client to connect to their associated database, and be able to execute the correct version of the xquery function.
The idea is that each time I add a new database that has data in some format, I can install associated query modules that are restricted to the context of that database to avoid conflicts with existing databases.
Maybe what I am looking for is effectively a database specific repo to install modules to, rather than a basex install repo.
From: Markus Wittenberg <wittenberg@axxepta.de> Sent: Friday, 21 February 2020 2:15 PM To: DYER Chris <Chris.Dyer@sydac.com>; basex-talk@mailman.uni-konstanz.de Subject: Re: [basex-talk] Database specific xquery modules
A RESTXQ definition could look something like this:
declare
%rest:path("/a/{$client}/{$path}")
function yourmodule:test($client as xs:string, $path as xs:string) {
let $db := switch ($client)
case "clientA" return db:open("dbA")
case "clientA" return db:open("dbA")
default return db:open("dbX")
return yourmodule:do-something-with($db, $path)
};
And the called function do-something-with could also depend on the client
Am 21.02.2020 um 03:48 schrieb DYER Chris:
Hi Markus,
Our intent is to use REST and RESTXQ interfaces to interact with data on a single database using some server side XQueries.
However, we may have 2 (or more) databases provided by some server, and associated xqueries are specific to the contents of each database (as the contents of each of these databases may be structured differently).
We would like for a client to be able to open a single database and be able to run associated XQueries, without requiring client side knowledge of which server side xqueries are associated with the database it has opened.
Does this help describe what we are trying to achieve?
Cheers, Chris
From: BaseX-Talk <basex-talk-bounces@mailman.uni-konstanz.de> On Behalf Of Markus Wittenberg Sent: Friday, 21 February 2020 12:08 PM To: basex-talk@mailman.uni-konstanz.de Subject: Re: [basex-talk] Database specific xquery modules
Hi Chris,
as you might already know, BaseX has built-in REST support [1] which you can use to query and update data via HTTP. You can also use the RESTXQ [2] functionality to define arbitrary server side XQueries which can also be requested by HTTP and are also suitable for web applications. Both methods are not restricted to a single database.
Best regards,
Markus
[1] http://docs.basex.org/wiki/REST
[2] http://docs.basex.org/wiki/RESTXQ
Am 21.02.2020 um 02:05 schrieb DYER Chris:
I am looking for some ideas on some architecture surrounding xquery modules.
Our situation is that we have multiple client applications in various languages that will connect to a database.
We will have some reasonably complex xqueries to extract data in certain structures that will be used by each of these clients.
So to avoid having to duplicate the query logic we would like the xqueries to reside "in" the database.
Additionally, we would like to use the same BaseX installation for multiple databases, corresponding to multiple projects.
The data in each of these databases may be against a different schema and our xqueries are essentially tied to the schema of the data. Thus, each database may need slightly different xqueries.
The xquery library modules looks to be global for the BaseX instance, as does the loading of main modules off of disk.
Is there any way to tie the query module or file to a specific database?
We are trying to avoid being limited to 1 database per BaseX instance.
An alternative would be to embed the xqueries within our client applications, but that involves duplication. Also, we are not sure if there are any performance difference with having the queries stored in the repo vs client side?
For background, we are looking at migrating from oracle to BaseX. So, we are effectively looking for an equivalent of multiple clients connecting to a userspace and using stored procedures which are specific to that userspace.
Any ideas on how to achieve this currently? Or is this something that BaseX could support?
Cheers,
Chris
--
Markus Wittenberg
Tel +49 (0)341 248 475 36
Mail wittenberg@axxepta.de
----
axxepta solutions GmbH
Lehmgrubenweg 17, 88131 Lindau
Amtsgericht Berlin HRB 97544B
Geschäftsführer: Karsten Becke, Maximilian Gärber
--
Markus Wittenberg
Tel +49 (0)341 248 475 36
Mail wittenberg@axxepta.de
----
axxepta solutions GmbH
Lehmgrubenweg 17, 88131 Lindau
Amtsgericht Berlin HRB 97544B
Geschäftsführer: Karsten Becke, Maximilian Gärber
--
Markus Wittenberg
Tel +49 (0)341 248 475 36
Mail wittenberg@axxepta.de
----
axxepta solutions GmbH
Lehmgrubenweg 17, 88131 Lindau
Amtsgericht Berlin HRB 97544B
Geschäftsführer: Karsten Becke, Maximilian Gärber
Hi Christian, As I understand Chris' issue, it is that he wants to be able to (for example) call the following two REST GETs ... localhost:8984/rest/BookstoreA?run=list_books.xq localhost:8984/rest/BookstoreB?run=list_books.xq ... and have it call two different XQueries - list_books.xq may need to be different for each database (perhaps due to a change in underlying data representation). Consequently, the desire is that XQueries can be associated/bound to a particular database, and not be global. This way the client does not have to worry about the databases' internal data representation or query implementation. Similarly, when creating the database it would only be necessary to import/add the desired XQueries, and not have to worry about what other databases and global XQueries already sit in the BaseX instance. It isn't clear to me how RESTXQ helps with this. There would still have to be a single RESTXQuery module (not one per database), just bound to a particular endpoint pattern. It would need to know how to work for all different databases, based on the database name. The database name could be obtained from the endpoint and used, but to have XQueries know specific database names seems undesirable, as they can't be deployed unmodified for different databases. The storing of XQueries within a database and using xquery:eval could work (where the XQueries are loaded from an XPath in the context), but it seems that this would change the syntax ... localhost:8984/rest/BookstoreA?run=execute_query.xq&query=list_books localhost:8984/rest/BookstoreB?run=execute_query.xq&query=list_books ... but this requires trampolining and forwarding of variables, options, etc ... and be more complex. Possible for simple cases, I guess, but it doesn't feel natural. The compartmentalisation of XQueries within the webapp folder could allow for this ... localhost:8984/rest/BookstoreA?run=BookstoreA/list_books.xq localhost:8984/rest/BookstoreB?run=BookstoreB/list_books.xq ... which is a bit unnatural and open to error for the client, but seems possible. It would be nice not to have to worry about directory structures on the server. I don't think that this would be an option if using RESTXQ though, because the path to the query is not relevant, just the bound endpoint. Or, we can just run different servers, and configure them accordingly ... localhost:8984/rest/BookstoreA?run=list_books.xq localhost:8985/rest/BookstoreB?run=list_books.xq However, I imagine a BaseX instance on a server that multiple projects all use as their development database. Each project would create its own database within that instance. It is much easier to CREATE DB than to have to get access to the server, set up another instance, configure ports etc. If every database was its own instance of BaseX I can imagine far more than 12 instances, or 12 concurrent HTTP servers running - our company's development database instance currently has 108 different databases in it. So, is it a lot of work to achieve something like the initial syntax above? localhost:8984/rest/BookstoreA?run=list_books.xq localhost:8984/rest/BookstoreB?run=list_books.xq Hopefully that makes sense. Thanks, Luke -----Original Message----- From: BaseX-Talk <basex-talk-bounces@mailman.uni-konstanz.de> On Behalf Of Christian Grün Sent: Tuesday, 25 February 2020 3:20 AM To: DYER Chris <Chris.Dyer@sydac.com> Cc: basex-talk@mailman.uni-konstanz.de Subject: Re: [basex-talk] Database specific xquery modules Hi Chris,
Our situation is that we have multiple client applications in various languages that will connect to a database.
Are you already using some of the clients that are available for communication with the BaseX client/server architecture [1], or are (planning to) communicating with the HTTP server? In the first case, or if you use HTTP with the basic REST API, it’s recommendable to give your users access to this specific database [2]. You could then store all queries on the server, and launch them via the RUN command [3] or the equivalent REST 'run' operation [4]. In the second case, and if you decide to write a RESTXQ application, you will have full flexibility, as you control in your server-side entry point which database is to be opened and accessed [5]. As indicated by Steve, you can use xquery:eval [6] to address and evaluate queries that are either organized as static strings, stored in the file system or stored in a database (possibly a separate instance that contains only queries). Compared to most relational DBMS, BaseX is much more light-weight, and startup time is minimal. If you have a limited number of database that you want to provide, it’s perfectly ok to launch a number of BaseX instances. I remember a use case in which we had deployed 12 BaseX instances on a single server, mostly because we wanted to simulate horizontal scalability. Personally, I would recommend the RESTXQ approach, but it also depends on how much time you want to invest in writing more complex XQuery applications. Cheers, Christian [1] http://docs.basex.org/wiki/Clients [2] http://docs.basex.org/wiki/User_Management [3] http://docs.basex.org/wiki/Commands#RUN [4] http://docs.basex.org/wiki/REST#GET_Method [5] http://docs.basex.org/wiki/RESTXQ [6] http://docs.basex.org/wiki/XQuery_Module#xquery:eval On Mon, Feb 24, 2020 at 6:22 AM DYER Chris <Chris.Dyer@sydac.com> wrote:
Hi Steve,
I do not think what we are trying to achieve is necessarily OO related (although I can see why it may look that way).
It may be more that we would be looking at another layer of encapsulation of some basex features.
Your suggestion on storing the queries within the database and then evaluating them does sound like a possible option for us. It would be nice to achieve this scoping of the queries using explicitly registered function modules.
This is all based on what we are currently achieving with our oracle database solution.
We have a single oracle database server, and we have developers create new user spaces on that server which encapsulates a set of tables and associated stored procedures (functions) for working with those tables. Those functions are only visible within the scope of the user space
It sounds like this sort of use case would generally be handled with separate basex installs/servers.
So I am possibly looking at a feature request.
Cheers, Chris
From: Majewski, Steven Dennis (sdm7g) <sdm7g@virginia.edu> Sent: Saturday, 22 February 2020 12:37 AM To: DYER Chris <Chris.Dyer@sydac.com>; basex-talk@mailman.uni-konstanz.de Cc: Markus Wittenberg <wittenberg@axxepta.de> Subject: Re: [basex-talk] Database specific xquery modules
I’m not sure this is the best approach, but you seem determined to try to figure out how to turn XQuery into an Object-Oriented language, so, if you want to bind query functions to a particular database, you can put the functions in the database, and use xquery:eval()
http://docs.basex.org/wiki/XQuery_Module to call the function as a node() argument to eval.
You would be using the context database as the “self” variable and xquery:eval() as dispatcher.
If you don’t literally want to bind functions to DB, but instead dispatch by the schema of the data, you could use a typeswitch if the schema have different namespaces or root elements,.
( I’ve tried this to switch between processing TEIP4 and TEIP5, for example. )
— Steve M.
On Feb 21, 2020, at 1:41 AM, Markus Wittenberg <wittenberg@axxepta.de> wrote:
If you want to use RESTXQ you would have to put the modules, or least the functions defining the endpoints, in the webapp directory and not in the repo directory. And far as I know, BaseX doesn't support installation for modules in webapp as for in repo. So you have to explicitly change/add files in webapp directory.
You could write a module for each database, defining its own endpoints, and maybe still reuse some code by importing a module with common functions in every specific module.
Am 21.02.2020 um 07:11 schrieb DYER Chris:
Hi Markus,
Not quite what I am aiming for.
We do want a single BaseX Instance to provide access to multiple databases.
Further to this though, a set of client applications will only connect to one database. We have multiple sets of client applications, with each set each connecting to one database.
We do want to modularise the functions, however as an example we want to be able to have 2 versions of the same function, where 1 version works with the schema of the data in database 1, and the second version with a different schema in database 2.
If I could explicitly link the function modules with a database, I could call the same function from both application sets, and use the context of the open database to call the correct version of the function.
An example situation may help
Say we have Database for “Black Books”, which has book data in xml structures which conform to some version (1.1) of a schema.
We want a query stored on the server that returns books on art
Get-Books-About-Art
We have a client application specifically for this database that calls this function.
It will only need to query the data in this database.
Now we add a second Database for "Goliath Books"
The book data in this database uses a modified xml structure (version 2 of a schema)
We have a client application for this database that also wants to get books about art.
The query for this database is slightly different because the xml schema has changed.
I would like to store both versions of Get-Books-About-Art in the database.
Then for each client to connect to their associated database, and be able to execute the correct version of the xquery function.
The idea is that each time I add a new database that has data in some format, I can install associated query modules that are restricted to the context of that database to avoid conflicts with existing databases.
Maybe what I am looking for is effectively a database specific repo to install modules to, rather than a basex install repo.
From: Markus Wittenberg <wittenberg@axxepta.de> Sent: Friday, 21 February 2020 2:15 PM To: DYER Chris <Chris.Dyer@sydac.com>; basex-talk@mailman.uni-konstanz.de Subject: Re: [basex-talk] Database specific xquery modules
A RESTXQ definition could look something like this:
declare
%rest:path("/a/{$client}/{$path}")
function yourmodule:test($client as xs:string, $path as xs:string) {
let $db := switch ($client)
case "clientA" return db:open("dbA")
case "clientA" return db:open("dbA")
default return db:open("dbX")
return yourmodule:do-something-with($db, $path)
};
And the called function do-something-with could also depend on the client
Am 21.02.2020 um 03:48 schrieb DYER Chris:
Hi Markus,
Our intent is to use REST and RESTXQ interfaces to interact with data on a single database using some server side XQueries.
However, we may have 2 (or more) databases provided by some server, and associated xqueries are specific to the contents of each database (as the contents of each of these databases may be structured differently).
We would like for a client to be able to open a single database and be able to run associated XQueries, without requiring client side knowledge of which server side xqueries are associated with the database it has opened.
Does this help describe what we are trying to achieve?
Cheers, Chris
From: BaseX-Talk <basex-talk-bounces@mailman.uni-konstanz.de> On Behalf Of Markus Wittenberg Sent: Friday, 21 February 2020 12:08 PM To: basex-talk@mailman.uni-konstanz.de Subject: Re: [basex-talk] Database specific xquery modules
Hi Chris,
as you might already know, BaseX has built-in REST support [1] which you can use to query and update data via HTTP. You can also use the RESTXQ [2] functionality to define arbitrary server side XQueries which can also be requested by HTTP and are also suitable for web applications. Both methods are not restricted to a single database.
Best regards,
Markus
[1] http://docs.basex.org/wiki/REST
[2] http://docs.basex.org/wiki/RESTXQ
Am 21.02.2020 um 02:05 schrieb DYER Chris:
I am looking for some ideas on some architecture surrounding xquery modules.
Our situation is that we have multiple client applications in various languages that will connect to a database.
We will have some reasonably complex xqueries to extract data in certain structures that will be used by each of these clients.
So to avoid having to duplicate the query logic we would like the xqueries to reside "in" the database.
Additionally, we would like to use the same BaseX installation for multiple databases, corresponding to multiple projects.
The data in each of these databases may be against a different schema and our xqueries are essentially tied to the schema of the data. Thus, each database may need slightly different xqueries.
The xquery library modules looks to be global for the BaseX instance, as does the loading of main modules off of disk.
Is there any way to tie the query module or file to a specific database?
We are trying to avoid being limited to 1 database per BaseX instance.
An alternative would be to embed the xqueries within our client applications, but that involves duplication. Also, we are not sure if there are any performance difference with having the queries stored in the repo vs client side?
For background, we are looking at migrating from oracle to BaseX. So, we are effectively looking for an equivalent of multiple clients connecting to a userspace and using stored procedures which are specific to that userspace.
Any ideas on how to achieve this currently? Or is this something that BaseX could support?
Cheers,
Chris
--
Markus Wittenberg
Tel +49 (0)341 248 475 36
Mail wittenberg@axxepta.de
----
axxepta solutions GmbH
Lehmgrubenweg 17, 88131 Lindau
Amtsgericht Berlin HRB 97544B
Geschäftsführer: Karsten Becke, Maximilian Gärber
--
Markus Wittenberg
Tel +49 (0)341 248 475 36
Mail wittenberg@axxepta.de
----
axxepta solutions GmbH
Lehmgrubenweg 17, 88131 Lindau
Amtsgericht Berlin HRB 97544B
Geschäftsführer: Karsten Becke, Maximilian Gärber
--
Markus Wittenberg
Tel +49 (0)341 248 475 36
Mail wittenberg@axxepta.de
----
axxepta solutions GmbH
Lehmgrubenweg 17, 88131 Lindau
Amtsgericht Berlin HRB 97544B
Geschäftsführer: Karsten Becke, Maximilian Gärber
Hi Luke,
It isn't clear to me how RESTXQ helps with this.
Here is a (very minimal) example how an XQuery endpoint could look like: declare %rest:path('{$db}/{$query}') function rest:execute( $db as xs:string, $query as xs:string ) { xquery:eval( xs:anyURI($db || '/' || $query), map { '': db:open($db) } ) }; One endpoint for all databases would suffice indeed. Extending the semantics of our REST API would be another option, but we may need to clarify various questions (will there be an extra server directory for each database? What happens if people want to access both global and database-specific queries? etc). Hope this helps, Christian
I was thinking that there would be a simpler way to bind function namespaces and databases, but that runs into the asymmetry that string in import and namespace declarations in the prolog must be literal strings and can’t be variables. ( I still tend to think from XSLT mental models instead of XQuery sometimes. ) And it seems that even the extended Q{} syntax requires a literal, so it’s not simple to make the function namespace variable. I can do something like this ( with modules for English, French, Italian added to my repo ): declare variable $it := 'http://example.org/modules/Italian' ; declare variable $fr := 'http://example.org/modules/French' ; declare variable $head := 'import module namespace lang = "' ; declare variable $tail := '"; ' ; for $lang in ( $it, $fr ) let $prolog := $head || $lang || $tail return xquery:eval( $prolog || 'lang:hello("Everybody!")' ) Returns => Ciao Everybody! Bonjour Everybody! In the XSLT 1 days, we used to get around the lack of dynamic eval by the functional programming technique of having one stylesheet produce another with the variables resolved. So another way of packaging this might be to create a RESTXQ endpoint that generates an XQuery module from a template and use that endpoint URL in an “import module … at” expression. But that location must also be a literal, so it’s just another variation of the above. I can also think of some tricks using higher level functions, but so far, nothing that makes it all simple. — Steve M.
On Feb 25, 2020, at 5:50 AM, Christian Grün <christian.gruen@gmail.com> wrote:
Hi Luke,
It isn't clear to me how RESTXQ helps with this.
Here is a (very minimal) example how an XQuery endpoint could look like:
declare %rest:path('{$db}/{$query}') function rest:execute( $db as xs:string, $query as xs:string ) { xquery:eval( xs:anyURI($db || '/' || $query), map { '': db:open($db) } ) };
One endpoint for all databases would suffice indeed.
Extending the semantics of our REST API would be another option, but we may need to clarify various questions (will there be an extra server directory for each database? What happens if people want to access both global and database-specific queries? etc).
Hope this helps, Christian
Also, question for the OP (Chris): It’s not completely clear what exactly you want to hide from whom. That is, if you only want to hide the implementation details from API clients, then RESTXQ and some conditional code is probably sufficient, however your comparison with local stored procedures in SQL make me think you mean that database clients my provide their own implementations of lower level functions, and you want higher level procedures to be able to call those functions without being concerned with the implementations. ( Which is what made me think of comparison with Object Oriented programming. ) — Steve M.
On Feb 25, 2020, at 11:17 AM, Majewski, Steven Dennis (sdm7g) <sdm7g@virginia.edu> wrote:
I was thinking that there would be a simpler way to bind function namespaces and databases, but that runs into the asymmetry that string in import and namespace declarations in the prolog must be literal strings and can’t be variables. ( I still tend to think from XSLT mental models instead of XQuery sometimes. ) And it seems that even the extended Q{} syntax requires a literal, so it’s not simple to make the function namespace variable.
I can do something like this ( with modules for English, French, Italian added to my repo ):
declare variable $it := 'http://example.org/modules/Italian <http://example.org/modules/Italian>' ; declare variable $fr := 'http://example.org/modules/French <http://example.org/modules/French>' ; declare variable $head := 'import module namespace lang = "' ; declare variable $tail := '"; ' ;
for $lang in ( $it, $fr ) let $prolog := $head || $lang || $tail return xquery:eval( $prolog || 'lang:hello("Everybody!")' )
Returns =>
Ciao Everybody! Bonjour Everybody!
In the XSLT 1 days, we used to get around the lack of dynamic eval by the functional programming technique of having one stylesheet produce another with the variables resolved. So another way of packaging this might be to create a RESTXQ endpoint that generates an XQuery module from a template and use that endpoint URL in an “import module … at” expression. But that location must also be a literal, so it’s just another variation of the above.
I can also think of some tricks using higher level functions, but so far, nothing that makes it all simple.
— Steve M.
On Feb 25, 2020, at 5:50 AM, Christian Grün <christian.gruen@gmail.com <mailto:christian.gruen@gmail.com>> wrote:
Hi Luke,
It isn't clear to me how RESTXQ helps with this.
Here is a (very minimal) example how an XQuery endpoint could look like:
declare %rest:path('{$db}/{$query}') function rest:execute( $db as xs:string, $query as xs:string ) { xquery:eval( xs:anyURI($db || '/' || $query), map { '': db:open($db) } ) };
One endpoint for all databases would suffice indeed.
Extending the semantics of our REST API would be another option, but we may need to clarify various questions (will there be an extra server directory for each database? What happens if people want to access both global and database-specific queries? etc).
Hope this helps, Christian
I will try to describe our situation a little bit more clearly. This is focused on our development environment, but similar situations occur in a production environment. We have multiple developers, each working on multiple projects. Each project will require a database. Each database will need to be accessed by multiple applications. Applications may be developed and executed by multiple developers on a netwok. We want to have a central server so that all developers are connecting to a single location. The content in each of these databases may be similar but with some structural (schema) differences. As such, any queries for interacting with the data may be implemented slightly differently. When a new project is started, we would have a developer create an associated database on the central server. We would like to create associated queries/functions to access the data for this new project. These would only be applicable to the database we just created, so we would not want them executed on another database. They also may conflict with similar functions for other project databases, so keeping them isolated is ideal. From the suggestions, it sounds like storing the queries as data within the database is the most likely solution in the current software. It does mean we lose some of the benefits of them being actual modules. As per Christian’s response, the software would need modifications to allow these isolated modules. I would expect some isolated directory structure per database would be necessary (if utilised). I would also expect this to be used in conjunction to the current global modules (probably less concerned about how the implementation would end up). Cheers, Chris From: Majewski, Steven Dennis (sdm7g) <sdm7g@virginia.edu> Sent: Wednesday, 26 February 2020 4:21 AM To: DYER Chris <Chris.Dyer@sydac.com> Cc: basex-talk@mailman.uni-konstanz.de Subject: Re: [basex-talk] Database specific xquery modules Also, question for the OP (Chris): It’s not completely clear what exactly you want to hide from whom. That is, if you only want to hide the implementation details from API clients, then RESTXQ and some conditional code is probably sufficient, however your comparison with local stored procedures in SQL make me think you mean that database clients my provide their own implementations of lower level functions, and you want higher level procedures to be able to call those functions without being concerned with the implementations. ( Which is what made me think of comparison with Object Oriented programming. ) — Steve M. On Feb 25, 2020, at 11:17 AM, Majewski, Steven Dennis (sdm7g) <sdm7g@virginia.edu<mailto:sdm7g@virginia.edu>> wrote: I was thinking that there would be a simpler way to bind function namespaces and databases, but that runs into the asymmetry that string in import and namespace declarations in the prolog must be literal strings and can’t be variables. ( I still tend to think from XSLT mental models instead of XQuery sometimes. ) And it seems that even the extended Q{} syntax requires a literal, so it’s not simple to make the function namespace variable. I can do something like this ( with modules for English, French, Italian added to my repo ): declare variable $it := 'http://example.org/modules/Italian' ; declare variable $fr := 'http://example.org/modules/French' ; declare variable $head := 'import module namespace lang = "' ; declare variable $tail := '"; ' ; for $lang in ( $it, $fr ) let $prolog := $head || $lang || $tail return xquery:eval( $prolog || 'lang:hello("Everybody!")' ) Returns => Ciao Everybody! Bonjour Everybody! In the XSLT 1 days, we used to get around the lack of dynamic eval by the functional programming technique of having one stylesheet produce another with the variables resolved. So another way of packaging this might be to create a RESTXQ endpoint that generates an XQuery module from a template and use that endpoint URL in an “import module … at” expression. But that location must also be a literal, so it’s just another variation of the above. I can also think of some tricks using higher level functions, but so far, nothing that makes it all simple. — Steve M. On Feb 25, 2020, at 5:50 AM, Christian Grün <christian.gruen@gmail.com<mailto:christian.gruen@gmail.com>> wrote: Hi Luke, It isn't clear to me how RESTXQ helps with this. Here is a (very minimal) example how an XQuery endpoint could look like: declare %rest:path('{$db}/{$query}') function rest:execute( $db as xs:string, $query as xs:string ) { xquery:eval( xs:anyURI($db || '/' || $query), map { '': db:open($db) } ) }; One endpoint for all databases would suffice indeed. Extending the semantics of our REST API would be another option, but we may need to clarify various questions (will there be an extra server directory for each database? What happens if people want to access both global and database-specific queries? etc). Hope this helps, Christian
Hi Chris, Just to be sure: Have you read my response to Luke, and have you thought about using RESTXQ and providing a single entry point (for the beginning) for queries that calls xquery:eval?
From the suggestions, it sounds like storing the queries as data within the database is the most likely solution in the current software.
You can also store your queries in the filesystem of your server. And you could add other endpoints to store and update your custom database queries. XQuery is much more than a simple query language; you can use it to develop complex web applications. Did you already have a look at the DBA code that’s part of our distributions? Best, Christian
Just to be sure: Have you read my response to Luke, and have you thought about using RESTXQ and providing a single entry point (for the beginning) for queries that calls xquery:eval?
Yes, I have seen the response. Using the single restxq entry point and xquery:eval is one of the more likely solutions for us.
You can also store your queries in the filesystem of your server. And you could add other endpoints to store and update your custom database queries. Not entirely sure what you are suggesting here.
I understand that there are multiple methods we could employ to direct our applications to execute the correct queries, with varying levels of effort and varying returns on cleanliness and ease of use. I would suggest that extending the current module repository system would be a more natural approach, and would bring in some concepts that have parallels in other database applications. Granted this might be a large amount of effort, so I would be expecting this to be a feature request. For now, we will likely go with whichever is the simplest for us to achieve Chris
Hi Chris,
I would suggest that extending the current module repository system would be a more natural approach, and would bring in some concepts that have parallels in other database applications.
It may turn out to be natural, if someone manages to specify and formalize a solution that fits to the architecture of BaseX. It could be long way, as I’m still struggling to understand what first steps might be required to achieve what you are looking for. Is your idea of an enhancement of the BaseX repository concept similar to what Luke proposed (i.e., having a REST call locate query scripts that belong to a specific database), or it is something on top, or bottom, of that? Would you propose to introduce new options that change the default behavior of BaseX; would you overwrite existing functionality or add new features on top of the existing functionality? What exactly would be the feature request you would ask for? It’s a challenge to equate BaseX with relational database systems, or concepts residing in that systems. With BaseX, you can address an arbitrary number of databases, local documents, remote sources in any other format, etc., in a single query. You can write full web applications with XQuery, which is something you wouldn’t usually dare with SQL and its extensions. Maybe it helps if you regard BaseX and XQuery as a framework that allows you to *also* retrieve databases. Cheers, Christian
So, would the convention of using the database name as the function namespace for queries to that database be sufficient ? Or is convention not a sufficient encapsulation ? ( i.e. do you need stronger enforcement, so that db1 users can’t possibly call db2:func() ? ) — Steve M.
On Feb 26, 2020, at 11:07 PM, DYER Chris <Chris.Dyer@sydac.com> wrote:
I will try to describe our situation a little bit more clearly. This is focused on our development environment, but similar situations occur in a production environment.
We have multiple developers, each working on multiple projects. Each project will require a database. Each database will need to be accessed by multiple applications. Applications may be developed and executed by multiple developers on a netwok. We want to have a central server so that all developers are connecting to a single location.
The content in each of these databases may be similar but with some structural (schema) differences. As such, any queries for interacting with the data may be implemented slightly differently.
When a new project is started, we would have a developer create an associated database on the central server. We would like to create associated queries/functions to access the data for this new project. These would only be applicable to the database we just created, so we would not want them executed on another database. They also may conflict with similar functions for other project databases, so keeping them isolated is ideal.
From the suggestions, it sounds like storing the queries as data within the database is the most likely solution in the current software. It does mean we lose some of the benefits of them being actual modules.
As per Christian’s response, the software would need modifications to allow these isolated modules. I would expect some isolated directory structure per database would be necessary (if utilised). I would also expect this to be used in conjunction to the current global modules (probably less concerned about how the implementation would end up).
Cheers, Chris
From: Majewski, Steven Dennis (sdm7g) <sdm7g@virginia.edu> Sent: Wednesday, 26 February 2020 4:21 AM To: DYER Chris <Chris.Dyer@sydac.com> Cc: basex-talk@mailman.uni-konstanz.de Subject: Re: [basex-talk] Database specific xquery modules
Also, question for the OP (Chris):
It’s not completely clear what exactly you want to hide from whom.
That is, if you only want to hide the implementation details from API clients, then RESTXQ and some conditional code is probably sufficient, however your comparison with local stored procedures in SQL make me think you mean that database clients my provide their own implementations of lower level functions, and you want higher level procedures to be able to call those functions without being concerned with the implementations. ( Which is what made me think of comparison with Object Oriented programming. )
— Steve M.
On Feb 25, 2020, at 11:17 AM, Majewski, Steven Dennis (sdm7g) <sdm7g@virginia.edu <mailto:sdm7g@virginia.edu>> wrote:
I was thinking that there would be a simpler way to bind function namespaces and databases, but that runs into the asymmetry that string in import and namespace declarations in the prolog must be literal strings and can’t be variables. ( I still tend to think from XSLT mental models instead of XQuery sometimes. ) And it seems that even the extended Q{} syntax requires a literal, so it’s not simple to make the function namespace variable.
I can do something like this ( with modules for English, French, Italian added to my repo ):
declare variable $it := 'http://example.org/modules/Italian <http://example.org/modules/Italian>' ; declare variable $fr := 'http://example.org/modules/French <http://example.org/modules/French>' ; declare variable $head := 'import module namespace lang = "' ; declare variable $tail := '"; ' ;
for $lang in ( $it, $fr ) let $prolog := $head || $lang || $tail return xquery:eval( $prolog || 'lang:hello("Everybody!")' )
Returns =>
Ciao Everybody! Bonjour Everybody!
In the XSLT 1 days, we used to get around the lack of dynamic eval by the functional programming technique of having one stylesheet produce another with the variables resolved. So another way of packaging this might be to create a RESTXQ endpoint that generates an XQuery module from a template and use that endpoint URL in an “import module … at” expression. But that location must also be a literal, so it’s just another variation of the above.
I can also think of some tricks using higher level functions, but so far, nothing that makes it all simple.
— Steve M.
On Feb 25, 2020, at 5:50 AM, Christian Grün <christian.gruen@gmail.com <mailto:christian.gruen@gmail.com>> wrote:
Hi Luke,
It isn't clear to me how RESTXQ helps with this.
Here is a (very minimal) example how an XQuery endpoint could look like:
declare %rest:path('{$db}/{$query}') function rest:execute( $db as xs:string, $query as xs:string ) { xquery:eval( xs:anyURI($db || '/' || $query), map { '': db:open($db) } ) };
One endpoint for all databases would suffice indeed.
Extending the semantics of our REST API would be another option, but we may need to clarify various questions (will there be an extra server directory for each database? What happens if people want to access both global and database-specific queries? etc).
Hope this helps, Christian
Also, if you DO go the route of storing procedures in the database, you might consider whether you could express those procedures as XSLT stylesheets that you could call from XQuery using xslt:transform() function. Since XSLT is in XML syntax, it works a bit smoother for that scenario. XQuery modules would need to be stored in the DB as RAW or text. XSLT modules are XML — accessing them via the DB would return an XML node that is already parsed. I saw a significant speedup pulling XSLT stylesheets from the DB as compared to using a filesystem URI. XQuery and xslt:transform work very well together: you can mix both methods in your code, as xslt:transform() is just another XQuery function that takes an input node or document and returns a result, usually as another XML node. The only downside is that your developers need to know both XQuery and XSLT. However, there is a lot of overlap between the two. Both use XPath and many of the same functions and the same data model, but a different syntax. — Steve M.
So, would the convention of using the database name as the function namespace for queries to that database be sufficient ? Or is convention not a sufficient encapsulation ? ( i.e. do you need stronger enforcement, so that db1 users can’t possibly call db2:func() ? )
A stronger enforcement on the encapsulation would be ideal. I would suggest a built in concept of database specific modules should give you this. Using Convention and the database name in the function namespace could potentially be sufficient in terms of referencing. It would likely be more convenient if this was not through namespacing though. The database our applications connect to is configurable at application start up time, so we would need to take some care in making sure the function namespaces are generated as well. We would also need to avoid conflicts in their installation location on disk.
if you DO go the route of storing procedures in the database, you might consider whether you could express those procedures as XSLT stylesheets that you could call from XQuery using xslt:transform() function. For sure, something to consider here. We would likely start with raw/text to see how we go and potentially look to formalise using xslt at a later stage.
Chris
OK thanks Markus, I was probably thinking more about this functionality for the library modules that get installed in the repo. I was not explicitly thinking about restxq modules/functions but being able to install these would also be useful. I think effectively what I am trying to achieve is to have some method for a remote application to create isolated databases and install some associated basex modules through a single existing server. Maybe the general idea is that you would do this by having multiple basex instances (and multiple servers). So perhaps I could raise a couple of feature requests: * Ability to install webapp modules through the server as per REPO INSTALL * This would give us the ability to create a new database and install and manage associated modules from a client without needing to understand the basex files structure * Ability to install repo and webapp modules for a specific database (eg DBREPO INSTALL) * This would give us the ability to install modules for a new database on an existing server, without needing to understand what else is already on the server for other databases. * Ability to install main modules through the server as per REPO INSTALL (for consistency) Is there a preferred method to raise these requests and potentially maintain reference to this discussion for context? In the meantime, we can look at managing these by * Explicitly copying files to a predefined web app directory (or use an admin account to get the web app directory) when installing rexq modules when a database is created. * Isolating the database specific modules in the webapp and repo directory through naming conventions or by creating completely separate basex instances and servers Cheers, Chris From: Markus Wittenberg <wittenberg@axxepta.de> Sent: Friday, 21 February 2020 5:11 PM To: DYER Chris <Chris.Dyer@sydac.com>; basex-talk@mailman.uni-konstanz.de Subject: Re: [basex-talk] Database specific xquery modules If you want to use RESTXQ you would have to put the modules, or least the functions defining the endpoints, in the webapp directory and not in the repo directory. And far as I know, BaseX doesn't support installation for modules in webapp as for in repo. So you have to explicitly change/add files in webapp directory. You could write a module for each database, defining its own endpoints, and maybe still reuse some code by importing a module with common functions in every specific module. Am 21.02.2020 um 07:11 schrieb DYER Chris: Hi Markus, Not quite what I am aiming for. We do want a single BaseX Instance to provide access to multiple databases. Further to this though, a set of client applications will only connect to one database. We have multiple sets of client applications, with each set each connecting to one database. We do want to modularise the functions, however as an example we want to be able to have 2 versions of the same function, where 1 version works with the schema of the data in database 1, and the second version with a different schema in database 2. If I could explicitly link the function modules with a database, I could call the same function from both application sets, and use the context of the open database to call the correct version of the function. An example situation may help Say we have Database for "Black Books", which has book data in xml structures which conform to some version (1.1) of a schema. We want a query stored on the server that returns books on art Get-Books-About-Art We have a client application specifically for this database that calls this function. It will only need to query the data in this database. Now we add a second Database for "Goliath Books" The book data in this database uses a modified xml structure (version 2 of a schema) We have a client application for this database that also wants to get books about art. The query for this database is slightly different because the xml schema has changed. I would like to store both versions of Get-Books-About-Art in the database. Then for each client to connect to their associated database, and be able to execute the correct version of the xquery function. The idea is that each time I add a new database that has data in some format, I can install associated query modules that are restricted to the context of that database to avoid conflicts with existing databases. Maybe what I am looking for is effectively a database specific repo to install modules to, rather than a basex install repo. From: Markus Wittenberg <wittenberg@axxepta.de><mailto:wittenberg@axxepta.de> Sent: Friday, 21 February 2020 2:15 PM To: DYER Chris <Chris.Dyer@sydac.com><mailto:Chris.Dyer@sydac.com>; basex-talk@mailman.uni-konstanz.de<mailto:basex-talk@mailman.uni-konstanz.de> Subject: Re: [basex-talk] Database specific xquery modules A RESTXQ definition could look something like this: declare %rest:path("/a/{$client}/{$path}") function yourmodule:test($client as xs:string, $path as xs:string) { let $db := switch ($client) case "clientA" return db:open("dbA") case "clientA" return db:open("dbA") default return db:open("dbX") return yourmodule:do-something-with($db, $path) }; And the called function do-something-with could also depend on the client Am 21.02.2020 um 03:48 schrieb DYER Chris: Hi Markus, Our intent is to use REST and RESTXQ interfaces to interact with data on a single database using some server side XQueries. However, we may have 2 (or more) databases provided by some server, and associated xqueries are specific to the contents of each database (as the contents of each of these databases may be structured differently). We would like for a client to be able to open a single database and be able to run associated XQueries, without requiring client side knowledge of which server side xqueries are associated with the database it has opened. Does this help describe what we are trying to achieve? Cheers, Chris From: BaseX-Talk <basex-talk-bounces@mailman.uni-konstanz.de><mailto:basex-talk-bounces@mailman.uni-konstanz.de> On Behalf Of Markus Wittenberg Sent: Friday, 21 February 2020 12:08 PM To: basex-talk@mailman.uni-konstanz.de<mailto:basex-talk@mailman.uni-konstanz.de> Subject: Re: [basex-talk] Database specific xquery modules Hi Chris, as you might already know, BaseX has built-in REST support [1] which you can use to query and update data via HTTP. You can also use the RESTXQ [2] functionality to define arbitrary server side XQueries which can also be requested by HTTP and are also suitable for web applications. Both methods are not restricted to a single database. Best regards, Markus [1] http://docs.basex.org/wiki/REST [2] http://docs.basex.org/wiki/RESTXQ Am 21.02.2020 um 02:05 schrieb DYER Chris: I am looking for some ideas on some architecture surrounding xquery modules. Our situation is that we have multiple client applications in various languages that will connect to a database. We will have some reasonably complex xqueries to extract data in certain structures that will be used by each of these clients. So to avoid having to duplicate the query logic we would like the xqueries to reside "in" the database. Additionally, we would like to use the same BaseX installation for multiple databases, corresponding to multiple projects. The data in each of these databases may be against a different schema and our xqueries are essentially tied to the schema of the data. Thus, each database may need slightly different xqueries. The xquery library modules looks to be global for the BaseX instance, as does the loading of main modules off of disk. Is there any way to tie the query module or file to a specific database? We are trying to avoid being limited to 1 database per BaseX instance. An alternative would be to embed the xqueries within our client applications, but that involves duplication. Also, we are not sure if there are any performance difference with having the queries stored in the repo vs client side? For background, we are looking at migrating from oracle to BaseX. So, we are effectively looking for an equivalent of multiple clients connecting to a userspace and using stored procedures which are specific to that userspace. Any ideas on how to achieve this currently? Or is this something that BaseX could support? Cheers, Chris -- Markus Wittenberg Tel +49 (0)341 248 475 36 Mail wittenberg@axxepta.de<mailto:wittenberg@axxepta.de> ---- axxepta solutions GmbH Lehmgrubenweg 17, 88131 Lindau Amtsgericht Berlin HRB 97544B Geschäftsführer: Karsten Becke, Maximilian Gärber -- Markus Wittenberg Tel +49 (0)341 248 475 36 Mail wittenberg@axxepta.de<mailto:wittenberg@axxepta.de> ---- axxepta solutions GmbH Lehmgrubenweg 17, 88131 Lindau Amtsgericht Berlin HRB 97544B Geschäftsführer: Karsten Becke, Maximilian Gärber -- Markus Wittenberg Tel +49 (0)341 248 475 36 Mail wittenberg@axxepta.de<mailto:wittenberg@axxepta.de> ---- axxepta solutions GmbH Lehmgrubenweg 17, 88131 Lindau Amtsgericht Berlin HRB 97544B Geschäftsführer: Karsten Becke, Maximilian Gärber
participants (5)
-
Christian Grün -
DYER Chris -
ERRINGTON Luke -
Majewski, Steven Dennis (sdm7g) -
Markus Wittenberg