Hello,
something has gone awry in my database after a db:replace():
query: db:open('DB','/path/file.xml')/base-uri()
result: /DB/path/file.xml
This is as expected. The result is the replaced file. But when I
query: db:open('DB')/root[@id='id']/base-uri()
result is a sequence: /DB/path/file.xml /DB/path/file.xml
The first result is the file before the db:replace(), the second after.
Can somebody explain why db:open() with path parameter is returning one result, but without parameter two results? Database statistics say there are indeed two documents. But when I export the database, only one gets exported.
This happens with db:replace() on 9.2 from the DBA browser interface. When I use the same db:replace() XQuery on 9.2 with my local database, all is well (document gets replaced, no duplicates). Settings should be the default ones.
Thanks, Daniel
Hi Daniel,
seems hard to reproduce. Could you tell me what you did step by step? This is what I tried:
1. I created a DB database. 2. I added the document <root id='id'/> to the database path /path/file.xml. 3. I replaced this document with the same document via the DBA. 4. Your query db:open('DB')/root[@id='id']/base-uri() returned a single string.
Thanks in advance, Christian
PS: The form parameter handling in 9.2 contains a little bug (as a result, I couldn’t invoke the replace functionality with the current version). I just uploaded a new snapshot. Both the current and the snapshot version should behave identically in terms of adding and replacing documents.
On Fri, Apr 26, 2019 at 4:45 PM Zimmel, Daniel D.Zimmel@esvmedien.de wrote:
Hello,
something has gone awry in my database after a db:replace():
query: db:open('DB','/path/file.xml')/base-uri()
result: /DB/path/file.xml
This is as expected. The result is the replaced file. But when I
query: db:open('DB')/root[@id='id']/base-uri()
result is a sequence: /DB/path/file.xml /DB/path/file.xml
The first result is the file before the db:replace(), the second after.
Can somebody explain why db:open() with path parameter is returning one result, but without parameter two results? Database statistics say there are indeed two documents. But when I export the database, only one gets exported.
This happens with db:replace() on 9.2 from the DBA browser interface. When I use the same db:replace() XQuery on 9.2 with my local database, all is well (document gets replaced, no duplicates). Settings should be the default ones.
Thanks, Daniel
Oddly enough, I cannot reproduce by myself!
Still, when I browse the database, the /DB/path only exists once (as far as I can see from the sorting), but querying is giving me two documents, according to db:path() living in the same path. This should never happen, should it?
I will simply drop the database, but I wish I could understand what went wrong. I just did the same updating query and it is politely replacing the second file without adding another one (still, the initial file remains unreplaced).
Thanks, Daniel
-----Ursprüngliche Nachricht----- Von: Christian Grün [mailto:christian.gruen@gmail.com] Gesendet: Montag, 29. April 2019 11:08 An: Zimmel, Daniel Cc: BaseX Betreff: Re: [basex-talk] db:replace() in DBA resulting in duplicates
Hi Daniel,
seems hard to reproduce. Could you tell me what you did step by step? This is what I tried:
- I created a DB database.
- I added the document <root id='id'/> to the database path /path/file.xml.
- I replaced this document with the same document via the DBA.
- Your query db:open('DB')/root[@id='id']/base-uri() returned a single string.
Thanks in advance, Christian
PS: The form parameter handling in 9.2 contains a little bug (as a result, I couldn’t invoke the replace functionality with the current version). I just uploaded a new snapshot. Both the current and the snapshot version should behave identically in terms of adding and replacing documents.
On Fri, Apr 26, 2019 at 4:45 PM Zimmel, Daniel D.Zimmel@esvmedien.de wrote:
Hello,
something has gone awry in my database after a db:replace():
query: db:open('DB','/path/file.xml')/base-uri()
result: /DB/path/file.xml
This is as expected. The result is the replaced file. But when I
query: db:open('DB')/root[@id='id']/base-uri()
result is a sequence: /DB/path/file.xml /DB/path/file.xml
The first result is the file before the db:replace(), the second after.
Can somebody explain why db:open() with path parameter is returning one
result, but without parameter two results?
Database statistics say there are indeed two documents. But when I export the
database, only one gets exported.
This happens with db:replace() on 9.2 from the DBA browser interface. When I use the same db:replace() XQuery on 9.2 with my local database, all is
well (document gets replaced, no duplicates). Settings should be the default ones.
Thanks, Daniel
Still, when I browse the database, the /DB/path only exists once (as far as I can see from the sorting), but querying is giving me two documents, according to db:path() living in the same path. This should never happen, should it?
This is surprising indeed.
Right now, db:add is still faster than db:replace, because the check for existing database paths can be completely skipped. This allows high-performance database insertions.In a future version of BaseX, I would love to get rid of db:add and db:replace and replace these functions with a db:update function, which will enforce that each document path exists only once in the database. This would surely reduce patterns like the one you encountered.
If you still manage to get this reproduced, just come back to us.
Follow-up, some details:
* Create large (5GB) DB in BaseX GUI * Copy DB folder manually to another server * Optimize DB * The first db:replace() via RESTXQ creates a second file with the same base-uri() * The second db:replace() via RESTXQ replaces the earlier created second file (duplicate)
I cannot reproduce with a small newly created db. The initial database was created with BaseX 9.0.2, update operation runs with BaseX 9.2.1
Should I try to optimize the database with 9.1.2 before copying and try again? Perhaps this is a database-structure related issue?
-----Ursprüngliche Nachricht----- Von: Christian Grün [mailto:christian.gruen@gmail.com] Gesendet: Montag, 29. April 2019 16:00 An: Zimmel, Daniel Cc: BaseX Betreff: Re: [basex-talk] db:replace() in DBA resulting in duplicates
Still, when I browse the database, the /DB/path only exists once (as far as I can
see from the sorting), but querying is giving me two documents, according to db:path() living in the same path.
This should never happen, should it?
This is surprising indeed.
Right now, db:add is still faster than db:replace, because the check for existing database paths can be completely skipped. This allows high-performance database insertions.In a future version of BaseX, I would love to get rid of db:add and db:replace and replace these functions with a db:update function, which will enforce that each document path exists only once in the database. This would surely reduce patterns like the one you encountered.
If you still manage to get this reproduced, just come back to us.
- Optimize DB
- The first db:replace() via RESTXQ creates a second file with the same
base-uri()
- The second db:replace() via RESTXQ replaces the earlier created second
file (duplicate)
Do you perform the next steps in the database instance that has been copied to another server, or does the problem also happen on the original instance? In the first case, do both servers use the same operating system (either Linux or Windows)?
Should I try to optimize the database with 9.1.2 before copying and try
again?
Definitely worth trying. If you export your data and create a new database with 9.2.x, and if the problem persists, we could have a look at it.
this is a database-structure related issue?
-----Ursprüngliche Nachricht----- Von: Christian Grün [mailto:christian.gruen@gmail.com] Gesendet: Montag, 29. April 2019 16:00 An: Zimmel, Daniel Cc: BaseX Betreff: Re: [basex-talk] db:replace() in DBA resulting in duplicates
Still, when I browse the database, the /DB/path only exists once (as
far as I can
see from the sorting), but querying is giving me two documents,
according to
db:path() living in the same path.
This should never happen, should it?
This is surprising indeed.
Right now, db:add is still faster than db:replace, because the check for existing database paths can be completely skipped. This allows high-performance database insertions.In a future version of BaseX, I would love to get rid of db:add and db:replace and replace these functions with a db:update function, which will enforce that each document path exists only once in the database. This would surely reduce patterns like the one you encountered.
If you still manage to get this reproduced, just come back to us.
- Optimize DB
- The first db:replace() via RESTXQ creates a second file with the same base-uri()
- The second db:replace() via RESTXQ replaces the earlier created second file (duplicate)
Do you perform the next steps in the database instance that has been copied to another server, or does the problem also happen on the original instance? In the first case, do both servers use the same operating system (either Linux or Windows)?
* both; does not happen in the original instance, does only happen in the copied instance * cannot reproduce with small fresh database created with 9.2.1 * not the same OS; original was created in Win 7, copied to linux server
OK, with a removal of the target database folder, a database optimization on the source and target and a fresh copy I cannot reproduce. I will definitely write a note to myself to always optimize the database before/after moving them and be sure to remove any artifacts.
Daniel .
- not the same OS; original was created in Win 7, copied to linux server
So this might be related to OS-specific case sensitivity issues; I remember we had other users discussing this on the list. Maybe it would be consistent in the long term to treat all operating systems identically.
I’m glad I caught this thread. Replace was working for me in basexgui but not from command line. ( And I needed to generate replacement list from the shell using ‘find … -newer basex/data/collection’ )
I pulled the basex/lib/basex-api-9.2.1-SNAPSHOT.jar file out of the snapshot zip file and copied it to home-brew /usr/local/Cellar/basex/9.2/libexec/lib/basex-api-9.2.jar and replacement from the basex shell command appears to work now.
Previously, I was getting this error, trying several variations of absolute/relative filenames or file: URIs.
published$ basex -i published -c "replace jmu/vihart00338.xml /projects/vh-migrate/published/jmu/vihart00338.xml" Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk@mailman.uni-konstanz.de Version: BaseX 9.2 Java: Oracle Corporation, 1.8.0_201 OS: Mac OS X, x86_64 Stack Trace: java.lang.RuntimeException: published: lock file does not exist. at org.basex.util.Util.notExpected(Util.java:61) at org.basex.data.DiskData.finishUpdate(DiskData.java:246) at org.basex.core.cmd.ACreate.update(ACreate.java:97) at org.basex.core.cmd.Replace.run(Replace.java:55) at org.basex.core.Command.run(Command.java:257) at org.basex.core.Command.execute(Command.java:93) at org.basex.api.client.LocalSession.execute(LocalSession.java:132) at org.basex.api.client.Session.execute(Session.java:36) at org.basex.core.CLI.execute(CLI.java:92) at org.basex.core.CLI.execute(CLI.java:76) at org.basex.core.CLI.execute(CLI.java:63) at org.basex.BaseX.<init>(BaseX.java:80) at org.basex.BaseX.main(BaseX.java:42)
— Steve Majewski
On Apr 29, 2019, at 5:08 AM, Christian Grün christian.gruen@gmail.com wrote:
Hi Daniel,
seems hard to reproduce. Could you tell me what you did step by step? This is what I tried:
- I created a DB database.
- I added the document <root id='id'/> to the database path /path/file.xml.
- I replaced this document with the same document via the DBA.
- Your query db:open('DB')/root[@id='id']/base-uri() returned a single string.
Thanks in advance, Christian
PS: The form parameter handling in 9.2 contains a little bug (as a result, I couldn’t invoke the replace functionality with the current version). I just uploaded a new snapshot. Both the current and the snapshot version should behave identically in terms of adding and replacing documents.
On Fri, Apr 26, 2019 at 4:45 PM Zimmel, Daniel D.Zimmel@esvmedien.de wrote:
Hello,
something has gone awry in my database after a db:replace():
query: db:open('DB','/path/file.xml')/base-uri()
result: /DB/path/file.xml
This is as expected. The result is the replaced file. But when I
query: db:open('DB')/root[@id='id']/base-uri()
result is a sequence: /DB/path/file.xml /DB/path/file.xml
The first result is the file before the db:replace(), the second after.
Can somebody explain why db:open() with path parameter is returning one result, but without parameter two results? Database statistics say there are indeed two documents. But when I export the database, only one gets exported.
This happens with db:replace() on 9.2 from the DBA browser interface. When I use the same db:replace() XQuery on 9.2 with my local database, all is well (document gets replaced, no duplicates). Settings should be the default ones.
Thanks, Daniel
basex-talk@mailman.uni-konstanz.de
-
Christian Grün -
Majewski, Steven Dennis (sdm7g) -
Zimmel, Daniel