Hi Martin (cc to the list),
- each XML node => 500 bytes
- total size of XML DB => 10million nodes(which will grow continuously as
new files are added).
So one node is one document, right? I’m just asking because, in the XML terminology, each XML document has nodes itself: element nodes, attributes nodes, text nodes, etc.
If you are working with millions of documents, it can be helpful to work with a "daily" database, which contains auxiliary references on documents that have been deleted or updated. If there is a time interval in which no people access your database (e.g. each night), you can merge these databases and recreate your index structures. If you query your database, you can a few more lines of XQuery: first check your daily database, and (if it doesn’t contain the required document) look up the document in your fully indexed database.
I have to do a update of the nodes as I need to replace them. I chose delete
- insert instead of replace
Please note that a replace operation can be much faster: If the structure of the updated document is similar to the old document, the document can be replace in-place.
Also, I am thinking of indexing on the node id, as that is like the primary key on which I base my operations, any suggestions might also be helpful,
Here it may also be interesting to know if you will only address documents or arbitrary nodes of your document. In the first case, addressing the document path via db:open should be sufficient [1]. In the latter case, you could possibly use the existing function db:node-id [2].
Hope this helps, Christian
[1] http://docs.basex.org/wiki/Databases [2] http://docs.basex.org/wiki/Database_Module#db:node-id
Thanks again, Regards Martin Lourduswamy
On Sun, May 20, 2018 at 9:28 AM, Christian Grün christian.gruen@gmail.com wrote:
Hi Martin,
I am new to BaseX, I would like ot speed up XQuery of insert and delete and replace through options.
Welcome to the list. As there are numerous ways to do updates in BaseX, feel free to give us more information on your insert and delete operations. Do you work with large single documents or many small documents? How large is your database?
While I query basex through perl, and try to connect through GUI, the perl connection aborts. Is there a parameters for parallel connections, please let me know.
I can’t tell why your perl connection is interrupted by opening the GUI (because they should be completely independent from each other). A step-by-step description on how you proceeded might be helpful.
Because there is no coupling between GUI and the client/server architecture, however, you should avoid running updates outside the GUI. Please check out [1] for more information.
Best, Christian
[1] http://docs.basex.org/wiki/Startup#Concurrent_Operations
Thanks, Regards Martin Lourduswamy
Hi, Can we delete a node with a node id got from a query like this
Query: db:node-id(db:text("martin1", "23b2513e6118bd32d53b128d19d339 0f6374abac")) I got a id 54210971 Can I use it to delete it form database, Thanks, Regards
On Sun, May 20, 2018 at 12:51 PM, Christian Grün christian.gruen@gmail.com wrote:
Hi Martin (cc to the list),
- each XML node => 500 bytes
- total size of XML DB => 10million nodes(which will grow continuously
as
new files are added).
So one node is one document, right? I’m just asking because, in the XML terminology, each XML document has nodes itself: element nodes, attributes nodes, text nodes, etc.
If you are working with millions of documents, it can be helpful to work with a "daily" database, which contains auxiliary references on documents that have been deleted or updated. If there is a time interval in which no people access your database (e.g. each night), you can merge these databases and recreate your index structures. If you query your database, you can a few more lines of XQuery: first check your daily database, and (if it doesn’t contain the required document) look up the document in your fully indexed database.
I have to do a update of the nodes as I need to replace them. I chose
delete
- insert instead of replace
Please note that a replace operation can be much faster: If the structure of the updated document is similar to the old document, the document can be replace in-place.
Also, I am thinking of indexing on the node id, as that is like the
primary
key on which I base my operations, any suggestions might also be helpful,
Here it may also be interesting to know if you will only address documents or arbitrary nodes of your document. In the first case, addressing the document path via db:open should be sufficient [1]. In the latter case, you could possibly use the existing function db:node-id [2].
Hope this helps, Christian
[1] http://docs.basex.org/wiki/Databases [2] http://docs.basex.org/wiki/Database_Module#db:node-id
Thanks again, Regards Martin Lourduswamy
On Sun, May 20, 2018 at 9:28 AM, Christian Grün <
christian.gruen@gmail.com>
wrote:
Hi Martin,
I am new to BaseX, I would like ot speed up XQuery of insert and
delete
and replace through options.
Welcome to the list. As there are numerous ways to do updates in BaseX, feel free to give us more information on your insert and delete operations. Do you work with large single documents or many small documents? How large is your database?
While I query basex through perl, and try to connect through GUI, the perl connection aborts. Is there a parameters for parallel connections, please let me know.
I can’t tell why your perl connection is interrupted by opening the GUI (because they should be completely independent from each other). A step-by-step description on how you proceeded might be helpful.
Because there is no coupling between GUI and the client/server architecture, however, you should avoid running updates outside the GUI. Please check out [1] for more information.
Best, Christian
[1] http://docs.basex.org/wiki/Startup#Concurrent_Operations
Thanks, Regards Martin Lourduswamy
Hi Martin,
You can delete this node with the db:open-id function:
delete node db:open-id('db', 54210971)
Please note the incremental indexing (the UPDINDEX option) should be enabled; otherwise, the lookup will be pretty slow.
Best, Christian
Martin Lourduswamy martin.louis@gmail.com schrieb am Sa., 26. Mai 2018, 15:50:
Hi, Can we delete a node with a node id got from a query like this
Query: db:node-id(db:text("martin1", "23b2513e6118bd32d53b128d19d3390f6374abac")) I got a id 54210971 Can I use it to delete it form database, Thanks, Regards
On Sun, May 20, 2018 at 12:51 PM, Christian Grün < christian.gruen@gmail.com> wrote:
Hi Martin (cc to the list),
- each XML node => 500 bytes
- total size of XML DB => 10million nodes(which will grow continuously
as
new files are added).
So one node is one document, right? I’m just asking because, in the XML terminology, each XML document has nodes itself: element nodes, attributes nodes, text nodes, etc.
If you are working with millions of documents, it can be helpful to work with a "daily" database, which contains auxiliary references on documents that have been deleted or updated. If there is a time interval in which no people access your database (e.g. each night), you can merge these databases and recreate your index structures. If you query your database, you can a few more lines of XQuery: first check your daily database, and (if it doesn’t contain the required document) look up the document in your fully indexed database.
I have to do a update of the nodes as I need to replace them. I chose
delete
- insert instead of replace
Please note that a replace operation can be much faster: If the structure of the updated document is similar to the old document, the document can be replace in-place.
Also, I am thinking of indexing on the node id, as that is like the
primary
key on which I base my operations, any suggestions might also be
helpful,
Here it may also be interesting to know if you will only address documents or arbitrary nodes of your document. In the first case, addressing the document path via db:open should be sufficient [1]. In the latter case, you could possibly use the existing function db:node-id [2].
Hope this helps, Christian
[1] http://docs.basex.org/wiki/Databases [2] http://docs.basex.org/wiki/Database_Module#db:node-id
Thanks again, Regards Martin Lourduswamy
On Sun, May 20, 2018 at 9:28 AM, Christian Grün <
christian.gruen@gmail.com>
wrote:
Hi Martin,
I am new to BaseX, I would like ot speed up XQuery of insert and
delete
and replace through options.
Welcome to the list. As there are numerous ways to do updates in BaseX, feel free to give us more information on your insert and delete operations. Do you work with large single documents or many small documents? How large is your database?
While I query basex through perl, and try to connect through GUI, the perl connection aborts. Is there a
parameters
for parallel connections, please let me know.
I can’t tell why your perl connection is interrupted by opening the GUI (because they should be completely independent from each other). A step-by-step description on how you proceeded might be helpful.
Because there is no coupling between GUI and the client/server architecture, however, you should avoid running updates outside the GUI. Please check out [1] for more information.
Best, Christian
[1] http://docs.basex.org/wiki/Startup#Concurrent_Operations
Thanks, Regards Martin Lourduswamy
basex-talk@mailman.uni-konstanz.de
-
Christian Grün -
Martin Lourduswamy