Add metadata to a group of documents inside the database
Hello, I have two groups of xml to be included in the same database. Usually the same query will be performed on both of them, but I need to able to query only one group. The difference between the groups is known only to add the xml files, e.g. the origin of the files. This information is not found inside the files, and I prefer not modifying their content. I was hoping to be able to define groups, using subfolders and base-uri or multiple collections, but I know it's not possible using Basex. Maybe I can tag them? The group type is both a property and a possible filter in a query, so I need a good performance. Right now I see two alternatives: 1. Using two separate databases. Once I need to query all files, I'll make the same query on multiple databases... 2. One database, but a query to a new xml document, which includes list of node-ids per group type, will be used both for knowing the type and for querying a subset. Any ideas, please? -- With kind regards, Menashè
Hi Menashè, Just use different database paths for each group (e.g. '/path1/' and '/path2/'), and specify the sub path with db:open: db:open('db', '/path1')/... You can also store the documents in two separate databases and use a single XQuery expression to query all documents: for $db in ('db1', 'db2') return db:open($db)/... Hope this helps, Christian On Tue, May 12, 2015 at 2:14 PM, Menashè Eliezer <meliezer@ogs.trieste.it> wrote:
Hello, I have two groups of xml to be included in the same database. Usually the same query will be performed on both of them, but I need to able to query only one group. The difference between the groups is known only to add the xml files, e.g. the origin of the files. This information is not found inside the files, and I prefer not modifying their content.
I was hoping to be able to define groups, using subfolders and base-uri or multiple collections, but I know it's not possible using Basex. Maybe I can tag them? The group type is both a property and a possible filter in a query, so I need a good performance.
Right now I see two alternatives: 1. Using two separate databases. Once I need to query all files, I'll make the same query on multiple databases... 2. One database, but a query to a new xml document, which includes list of node-ids per group type, will be used both for knowing the type and for querying a subset.
Any ideas, please?
-- With kind regards, Menashè
Hi Christian, Thank you very much for your reply! As for path: A database is created from all files in a specific folder which has subfolders 'group1', 'group2' etc. and the path is derived from these subfolders of adding files to db? With kind regards, Menashè On 05/12/2015 02:19 PM, Christian Grün wrote:
Hi Menashè,
Just use different database paths for each group (e.g. '/path1/' and '/path2/'), and specify the sub path with db:open:
db:open('db', '/path1')/...
You can also store the documents in two separate databases and use a single XQuery expression to query all documents:
for $db in ('db1', 'db2') return db:open($db)/...
Hope this helps, Christian
On Tue, May 12, 2015 at 2:14 PM, Menashè Eliezer <meliezer@ogs.trieste.it> wrote:
Hello, I have two groups of xml to be included in the same database. Usually the same query will be performed on both of them, but I need to able to query only one group. The difference between the groups is known only to add the xml files, e.g. the origin of the files. This information is not found inside the files, and I prefer not modifying their content.
I was hoping to be able to define groups, using subfolders and base-uri or multiple collections, but I know it's not possible using Basex. Maybe I can tag them? The group type is both a property and a possible filter in a query, so I need a good performance.
Right now I see two alternatives: 1. Using two separate databases. Once I need to query all files, I'll make the same query on multiple databases... 2. One database, but a query to a new xml document, which includes list of node-ids per group type, will be used both for knowing the type and for querying a subset.
Any ideas, please?
-- With kind regards, Menashè
If you create a database with a single CREATE or db:create call, the resulting path structure will reflect the original directory structure. However, you can also add files and directories to specific target paths in a second steps, e.g. via db:add [1]. Our Wiki should contain all relevant information (see e.g. [2]). [1] http://docs.basex.org/wiki/Database_Module#db:add [2] http://docs.basex.org/wiki/Databases On Tue, May 12, 2015 at 2:35 PM, Menashè Eliezer <meliezer@ogs.trieste.it> wrote:
Hi Christian,
Thank you very much for your reply! As for path: A database is created from all files in a specific folder which has subfolders 'group1', 'group2' etc. and the path is derived from these subfolders of adding files to db?
With kind regards, Menashè
On 05/12/2015 02:19 PM, Christian Grün wrote:
Hi Menashè,
Just use different database paths for each group (e.g. '/path1/' and '/path2/'), and specify the sub path with db:open:
db:open('db', '/path1')/...
You can also store the documents in two separate databases and use a single XQuery expression to query all documents:
for $db in ('db1', 'db2') return db:open($db)/...
Hope this helps, Christian
On Tue, May 12, 2015 at 2:14 PM, Menashè Eliezer <meliezer@ogs.trieste.it> wrote:
Hello, I have two groups of xml to be included in the same database. Usually the same query will be performed on both of them, but I need to able to query only one group. The difference between the groups is known only to add the xml files, e.g. the origin of the files. This information is not found inside the files, and I prefer not modifying their content.
I was hoping to be able to define groups, using subfolders and base-uri or multiple collections, but I know it's not possible using Basex. Maybe I can tag them? The group type is both a property and a possible filter in a query, so I need a good performance.
Right now I see two alternatives: 1. Using two separate databases. Once I need to query all files, I'll make the same query on multiple databases... 2. One database, but a query to a new xml document, which includes list of node-ids per group type, will be used both for knowing the type and for querying a subset.
Any ideas, please?
-- With kind regards, Menashè
Menashè, In similar situations I've used a second database to store metadata at the same path as the document in the primary database. For example: db:open('database', '/path/a.xml') db:open('database_metadata', '/path/a.xml') Also, there is a feature request for adding properties as additional metadata for documents in the database: https://github.com/BaseXdb/basex/issues/988 If I may ask, what metadata information do you need to record about each document? Vincent -----Original Message----- From: basex-talk-bounces@mailman.uni-konstanz.de [mailto:basex-talk-bounces@mailman.uni-konstanz.de] On Behalf Of Christian Grün Sent: Tuesday, May 12, 2015 8:41 AM To: Menashè Eliezer Cc: BaseX Subject: Re: [basex-talk] Add metadata to a group of documents inside the database If you create a database with a single CREATE or db:create call, the resulting path structure will reflect the original directory structure. However, you can also add files and directories to specific target paths in a second steps, e.g. via db:add [1]. Our Wiki should contain all relevant information (see e.g. [2]). [1] http://docs.basex.org/wiki/Database_Module#db:add [2] http://docs.basex.org/wiki/Databases On Tue, May 12, 2015 at 2:35 PM, Menashè Eliezer <meliezer@ogs.trieste.it> wrote:
Hi Christian,
Thank you very much for your reply! As for path: A database is created from all files in a specific folder which has subfolders 'group1', 'group2' etc. and the path is derived from these subfolders of adding files to db?
With kind regards, Menashè
On 05/12/2015 02:19 PM, Christian Grün wrote:
Hi Menashè,
Just use different database paths for each group (e.g. '/path1/' and '/path2/'), and specify the sub path with db:open:
db:open('db', '/path1')/...
You can also store the documents in two separate databases and use a single XQuery expression to query all documents:
for $db in ('db1', 'db2') return db:open($db)/...
Hope this helps, Christian
On Tue, May 12, 2015 at 2:14 PM, Menashè Eliezer <meliezer@ogs.trieste.it> wrote:
Hello, I have two groups of xml to be included in the same database. Usually the same query will be performed on both of them, but I need to able to query only one group. The difference between the groups is known only to add the xml files, e.g. the origin of the files. This information is not found inside the files, and I prefer not modifying their content.
I was hoping to be able to define groups, using subfolders and base-uri or multiple collections, but I know it's not possible using Basex. Maybe I can tag them? The group type is both a property and a possible filter in a query, so I need a good performance.
Right now I see two alternatives: 1. Using two separate databases. Once I need to query all files, I'll make the same query on multiple databases... 2. One database, but a query to a new xml document, which includes list of node-ids per group type, will be used both for knowing the type and for querying a subset.
Any ideas, please?
-- With kind regards, Menashè
Vincent, Thank you for the creative solution. I've also commented the feature request as a reply to your question. With kind regards, Menashè On 05/12/2015 03:57 PM, Lizzi, Vincent wrote:
Menashè,
In similar situations I've used a second database to store metadata at the same path as the document in the primary database. For example:
db:open('database', '/path/a.xml') db:open('database_metadata', '/path/a.xml')
Also, there is a feature request for adding properties as additional metadata for documents in the database:
https://github.com/BaseXdb/basex/issues/988
If I may ask, what metadata information do you need to record about each document?
Vincent
-----Original Message----- From:basex-talk-bounces@mailman.uni-konstanz.de [mailto:basex-talk-bounces@mailman.uni-konstanz.de] On Behalf Of Christian Grün Sent: Tuesday, May 12, 2015 8:41 AM To: Menashè Eliezer Cc: BaseX Subject: Re: [basex-talk] Add metadata to a group of documents inside the database
If you create a database with a single CREATE or db:create call, the resulting path structure will reflect the original directory structure. However, you can also add files and directories to specific target paths in a second steps, e.g. via db:add [1]. Our Wiki should contain all relevant information (see e.g. [2]).
[1]http://docs.basex.org/wiki/Database_Module#db:add [2]http://docs.basex.org/wiki/Databases
On Tue, May 12, 2015 at 2:35 PM, Menashè Eliezer<meliezer@ogs.trieste.it> wrote:
Hi Christian,
Thank you very much for your reply! As for path: A database is created from all files in a specific folder which has subfolders 'group1', 'group2' etc. and the path is derived from these subfolders of adding files to db?
With kind regards, Menashè
On 05/12/2015 02:19 PM, Christian Grün wrote:
Hi Menashè,
Just use different database paths for each group (e.g. '/path1/' and '/path2/'), and specify the sub path with db:open:
db:open('db', '/path1')/...
You can also store the documents in two separate databases and use a single XQuery expression to query all documents:
for $db in ('db1', 'db2') return db:open($db)/...
Hope this helps, Christian
On Tue, May 12, 2015 at 2:14 PM, Menashè Eliezer <meliezer@ogs.trieste.it> wrote:
Hello, I have two groups of xml to be included in the same database. Usually the same query will be performed on both of them, but I need to able to query only one group. The difference between the groups is known only to add the xml files, e.g. the origin of the files. This information is not found inside the files, and I prefer not modifying their content.
I was hoping to be able to define groups, using subfolders and base-uri or multiple collections, but I know it's not possible using Basex. Maybe I can tag them? The group type is both a property and a possible filter in a query, so I need a good performance.
Right now I see two alternatives: 1. Using two separate databases. Once I need to query all files, I'll make the same query on multiple databases... 2. One database, but a query to a new xml document, which includes list of node-ids per group type, will be used both for knowing the type and for querying a subset.
Any ideas, please?
-- With kind regards, Menashè
participants (3)
-
Christian Grün -
Lizzi, Vincent -
Menashè Eliezer