UPDINDEX on custom index structures not working... where is my missing link?
Hi, I'm trying to build something that is similar to custom indices but that actually returns resolved nodes (nodes created by manipulating and merging nodes from the original DB). 1- I created a script that creates new index DBs. UPDINDEX and AUTOOPTIMIZED are set on both the 'index' DB and the DB from which the new DBs were created 2- I can create a query on any of the new 'index' DBs to extract the nodes I want. 3- After I change the content in the source DB, the response shows that the nodes in the 'index' DBs are not updated. I get the old result. I feel that I am missing a link between the source and the index DBs to make the update mechanism work. Since I do not want to get the original node, but one that I create, I did not use db:node-id or db:open-id shown in the documentation example. Is that where the link is created? CODE THAT CREATES THE INDEX DBs let $langs := for $lang in ('en-us', 'es-es') (:db:list()[string-length(.) = 5][substring(., 3, 1) = '-']:) return $lang return for $lang in $langs let $index := <index>{ for $string-group in db:open('global-content')//*[name()='string-group'] let $prompt-this-lang := $string-group/child::*[@xml:lang=$lang][normalize-space(.)!=''] let $prompt-base := if (exists($prompt-this-lang)) then $prompt-this-lang else $string-group/child::*[@xml:lang='en-us'] let $prompts := for $prompt-match in db:open('global-content')/*[name()='prompt-refs']/*[name()='prompt'][@package=$string-group/data(@package)][@pkg-version=$string-group/data(@pkg-version)][@key=$string-group/data(@key)] return copy $copy := $prompt-base modify ( for $attr in $prompt-match/@*[name()!='package' and name()!='pkg-version' and name()!='key'] return insert node $attr into $copy ) return $copy return $prompts }</index> return db:create('index-prompt-' || $lang, $index, 'prompts-' || $lang || '.xml', map { 'updindex': true(), 'autooptimize': true() }) CODE THAT QUERIES THE 'INDEX' DBS (# db:enforceindex #) { for $prompt in db:attribute('index-prompt-en-us', 'setup_as_new', 'name')/.. return $prompt => prof:time() } -- France Baril Architecte documentaire / Documentation architect france.baril@architextus.com
Hi France, 1. If you use autooptimize, updindex won’t have any effect. What’s the background for the combination you have chosen? 2. How does your the section of your query looks like in which you are updating your index databases? Best, Christian France Baril <france.baril@architextus.com> schrieb am Fr., 20. Dez. 2019, 20:40:
Hi,
I'm trying to build something that is similar to custom indices but that actually returns resolved nodes (nodes created by manipulating and merging nodes from the original DB).
1- I created a script that creates new index DBs. UPDINDEX and AUTOOPTIMIZED are set on both the 'index' DB and the DB from which the new DBs were created 2- I can create a query on any of the new 'index' DBs to extract the nodes I want. 3- After I change the content in the source DB, the response shows that the nodes in the 'index' DBs are not updated. I get the old result.
I feel that I am missing a link between the source and the index DBs to make the update mechanism work. Since I do not want to get the original node, but one that I create, I did not use db:node-id or db:open-id shown in the documentation example. Is that where the link is created?
CODE THAT CREATES THE INDEX DBs
let $langs := for $lang in ('en-us', 'es-es') (:db:list()[string-length(.) = 5][substring(., 3, 1) = '-']:) return $lang return for $lang in $langs let $index := <index>{ for $string-group in db:open('global-content')//*[name()='string-group'] let $prompt-this-lang := $string-group/child::*[@xml:lang=$lang][normalize-space(.)!=''] let $prompt-base := if (exists($prompt-this-lang)) then $prompt-this-lang else $string-group/child::*[@xml:lang='en-us'] let $prompts := for $prompt-match in db:open('global-content')/*[name()='prompt-refs']/*[name()='prompt'][@package=$string-group/data(@package)][@pkg-version=$string-group/data(@pkg-version)][@key=$string-group/data(@key)] return copy $copy := $prompt-base modify ( for $attr in $prompt-match/@*[name()!='package' and name()!='pkg-version' and name()!='key'] return insert node $attr into $copy ) return $copy
return $prompts
}</index>
return db:create('index-prompt-' || $lang, $index, 'prompts-' || $lang || '.xml', map { 'updindex': true(), 'autooptimize': true() })
CODE THAT QUERIES THE 'INDEX' DBS
(# db:enforceindex #) { for $prompt in db:attribute('index-prompt-en-us', 'setup_as_new', 'name')/.. return $prompt => prof:time() }
-- France Baril Architecte documentaire / Documentation architect france.baril@architextus.com
Hi 1. I tried UPDINDEX and when it didn't work I figured I would need to AUTOOPTIMIZE, I just removed it, 2. I only have the create .xq, If I need to add code to update, I don't know where/how to connect it, which brings me back to my question, how do I connect my query so it will update my indices. That's where the missing link is. On Sat, Dec 21, 2019 at 9:19 AM Christian Grün <christian.gruen@gmail.com> wrote:
Hi France,
1. If you use autooptimize, updindex won’t have any effect. What’s the background for the combination you have chosen?
2. How does your the section of your query looks like in which you are updating your index databases?
Best, Christian
France Baril <france.baril@architextus.com> schrieb am Fr., 20. Dez. 2019, 20:40:
Hi,
I'm trying to build something that is similar to custom indices but that actually returns resolved nodes (nodes created by manipulating and merging nodes from the original DB).
1- I created a script that creates new index DBs. UPDINDEX and AUTOOPTIMIZED are set on both the 'index' DB and the DB from which the new DBs were created 2- I can create a query on any of the new 'index' DBs to extract the nodes I want. 3- After I change the content in the source DB, the response shows that the nodes in the 'index' DBs are not updated. I get the old result.
I feel that I am missing a link between the source and the index DBs to make the update mechanism work. Since I do not want to get the original node, but one that I create, I did not use db:node-id or db:open-id shown in the documentation example. Is that where the link is created?
CODE THAT CREATES THE INDEX DBs
let $langs := for $lang in ('en-us', 'es-es') (:db:list()[string-length(.) = 5][substring(., 3, 1) = '-']:) return $lang return for $lang in $langs let $index := <index>{ for $string-group in db:open('global-content')//*[name()='string-group'] let $prompt-this-lang := $string-group/child::*[@xml:lang=$lang][normalize-space(.)!=''] let $prompt-base := if (exists($prompt-this-lang)) then $prompt-this-lang else $string-group/child::*[@xml:lang='en-us'] let $prompts := for $prompt-match in db:open('global-content')/*[name()='prompt-refs']/*[name()='prompt'][@package=$string-group/data(@package)][@pkg-version=$string-group/data(@pkg-version)][@key=$string-group/data(@key)] return copy $copy := $prompt-base modify ( for $attr in $prompt-match/@*[name()!='package' and name()!='pkg-version' and name()!='key'] return insert node $attr into $copy ) return $copy
return $prompts
}</index>
return db:create('index-prompt-' || $lang, $index, 'prompts-' || $lang || '.xml', map { 'updindex': true(), 'autooptimize': true() })
CODE THAT QUERIES THE 'INDEX' DBS
(# db:enforceindex #) { for $prompt in db:attribute('index-prompt-en-us', 'setup_as_new', 'name')/.. return $prompt => prof:time() }
-- France Baril Architecte documentaire / Documentation architect france.baril@architextus.com
-- France Baril Architecte documentaire / Documentation architect france.baril@architextus.com
My question is related to this statement in the documentation: With some extra effort, and if UPDINDEX <http://docs.basex.org/wiki/Options#UPDINDEX> is enabled for both your original and your index database (see below), your index database will support updates as well (try it, it’s fun!). I think the question is what is the extra effort? What else do I need to implement for this to work? On Sat, Dec 21, 2019 at 11:17 AM France Baril <france.baril@architextus.com> wrote:
Hi
1. I tried UPDINDEX and when it didn't work I figured I would need to AUTOOPTIMIZE, I just removed it, 2. I only have the create .xq, If I need to add code to update, I don't know where/how to connect it, which brings me back to my question, how do I connect my query so it will update my indices. That's where the missing link is.
On Sat, Dec 21, 2019 at 9:19 AM Christian Grün <christian.gruen@gmail.com> wrote:
Hi France,
1. If you use autooptimize, updindex won’t have any effect. What’s the background for the combination you have chosen?
2. How does your the section of your query looks like in which you are updating your index databases?
Best, Christian
France Baril <france.baril@architextus.com> schrieb am Fr., 20. Dez. 2019, 20:40:
Hi,
I'm trying to build something that is similar to custom indices but that actually returns resolved nodes (nodes created by manipulating and merging nodes from the original DB).
1- I created a script that creates new index DBs. UPDINDEX and AUTOOPTIMIZED are set on both the 'index' DB and the DB from which the new DBs were created 2- I can create a query on any of the new 'index' DBs to extract the nodes I want. 3- After I change the content in the source DB, the response shows that the nodes in the 'index' DBs are not updated. I get the old result.
I feel that I am missing a link between the source and the index DBs to make the update mechanism work. Since I do not want to get the original node, but one that I create, I did not use db:node-id or db:open-id shown in the documentation example. Is that where the link is created?
CODE THAT CREATES THE INDEX DBs
let $langs := for $lang in ('en-us', 'es-es') (:db:list()[string-length(.) = 5][substring(., 3, 1) = '-']:) return $lang return for $lang in $langs let $index := <index>{ for $string-group in db:open('global-content')//*[name()='string-group'] let $prompt-this-lang := $string-group/child::*[@xml:lang=$lang][normalize-space(.)!=''] let $prompt-base := if (exists($prompt-this-lang)) then $prompt-this-lang else $string-group/child::*[@xml:lang='en-us'] let $prompts := for $prompt-match in db:open('global-content')/*[name()='prompt-refs']/*[name()='prompt'][@package=$string-group/data(@package)][@pkg-version=$string-group/data(@pkg-version)][@key=$string-group/data(@key)] return copy $copy := $prompt-base modify ( for $attr in $prompt-match/@*[name()!='package' and name()!='pkg-version' and name()!='key'] return insert node $attr into $copy ) return $copy
return $prompts
}</index>
return db:create('index-prompt-' || $lang, $index, 'prompts-' || $lang || '.xml', map { 'updindex': true(), 'autooptimize': true() })
CODE THAT QUERIES THE 'INDEX' DBS
(# db:enforceindex #) { for $prompt in db:attribute('index-prompt-en-us', 'setup_as_new', 'name')/.. return $prompt => prof:time() }
-- France Baril Architecte documentaire / Documentation architect france.baril@architextus.com
-- France Baril Architecte documentaire / Documentation architect france.baril@architextus.com
-- France Baril Architecte documentaire / Documentation architect france.baril@architextus.com
2. I only have the create .xq, If I need to add code to update, I don't know where/how to connect it, which brings me back to my question, how do I connect my query so it will update my indices. That's where the missing link is.
It’s not sufficient to connect your query, as it (as far as I can see) recreates the full index. Instead you’ll need to write additional code that updates the relevant parts of your custom index structure. Hope this helps Christian
I'm good with writing the code, but how do I make it run automatically on content update of the original source DB? What I am missing is where/how to connect that code so it runs on its own. On Sat, Dec 21, 2019 at 11:35 AM Christian Grün <christian.gruen@gmail.com> wrote:
2. I only have the create .xq, If I need to add code to update, I don't
know where/how to connect it, which brings me back to my question, how do I connect my query so it will update my indices. That's where the missing link is.
It’s not sufficient to connect your query, as it (as far as I can see) recreates the full index. Instead you’ll need to write additional code that updates the relevant parts of your custom index structure.
Hope this helps Christian
-- France Baril Architecte documentaire / Documentation architect france.baril@architextus.com
Your code needs to take care of both the actual update and the index update. As indicated in another thread on this list this or last week, the integration of triggers (with pre/post commit hooks) may allow you to separate these two tasks in a future version of BaseX. France Baril <france.baril@architextus.com> schrieb am Sa., 21. Dez. 2019, 11:53:
I'm good with writing the code, but how do I make it run automatically on content update of the original source DB? What I am missing is where/how to connect that code so it runs on its own.
On Sat, Dec 21, 2019 at 11:35 AM Christian Grün <christian.gruen@gmail.com> wrote:
2. I only have the create .xq, If I need to add code to update, I don't
know where/how to connect it, which brings me back to my question, how do I connect my query so it will update my indices. That's where the missing link is.
It’s not sufficient to connect your query, as it (as far as I can see) recreates the full index. Instead you’ll need to write additional code that updates the relevant parts of your custom index structure.
Hope this helps Christian
-- France Baril Architecte documentaire / Documentation architect france.baril@architextus.com
Ok, so I have to invent my own triggers until I can catch the index update on the source DB. I think I'll just put it on a timer, thanks. On Sat, Dec 21, 2019 at 12:07 PM Christian Grün <christian.gruen@gmail.com> wrote:
Your code needs to take care of both the actual update and the index update. As indicated in another thread on this list this or last week, the integration of triggers (with pre/post commit hooks) may allow you to separate these two tasks in a future version of BaseX.
France Baril <france.baril@architextus.com> schrieb am Sa., 21. Dez. 2019, 11:53:
I'm good with writing the code, but how do I make it run automatically on content update of the original source DB? What I am missing is where/how to connect that code so it runs on its own.
On Sat, Dec 21, 2019 at 11:35 AM Christian Grün < christian.gruen@gmail.com> wrote:
2. I only have the create .xq, If I need to add code to update, I don't
know where/how to connect it, which brings me back to my question, how do I connect my query so it will update my indices. That's where the missing link is.
It’s not sufficient to connect your query, as it (as far as I can see) recreates the full index. Instead you’ll need to write additional code that updates the relevant parts of your custom index structure.
Hope this helps Christian
-- France Baril Architecte documentaire / Documentation architect france.baril@architextus.com
-- France Baril Architecte documentaire / Documentation architect france.baril@architextus.com
participants (2)
-
Christian Grün -
France Baril