Pruned, optimized DB shows same document count, db size
Hello all, I've pruned about half the documents of a +-5GB database using XQuery delete. I then optimised the database using db:optimize. The database metadata still shows the original number of documents, and the overall db filesize remains roughly the same. I was sort of hoping this pruning would improve performance, but if there's any difference it's negligable. Am I missing something obvious? Thanks in advance, Constantine Hondros ________________________________ Elsevier B.V. Registered Office: Radarweg 29, 1043 NX Amsterdam, The Netherlands, Registration No. 33156677, Registered in The Netherlands.
Dear Constantine,
From what I know, optimize only rebuilt indexes and metadata. Did you try "optimize all" to compact db files ?
But documents' number should be updated. How do you get metadata information ? Best regards, Fabrice Etanchaud Questel/Orbit http://docs.basex.org/wiki/Database_Module#db:optimize De : basex-talk-bounces@mailman.uni-konstanz.de [mailto:basex-talk-bounces@mailman.uni-konstanz.de] De la part de Hondros, Constantine (ELS-AMS) Envoyé : lundi 12 janvier 2015 14:29 À : basex-talk@mailman.uni-konstanz.de Objet : [basex-talk] Pruned, optimized DB shows same document count, db size Hello all, I've pruned about half the documents of a +-5GB database using XQuery delete. I then optimised the database using db:optimize. The database metadata still shows the original number of documents, and the overall db filesize remains roughly the same. I was sort of hoping this pruning would improve performance, but if there's any difference it's negligable. Am I missing something obvious? Thanks in advance, Constantine Hondros ________________________________ Elsevier B.V. Registered Office: Radarweg 29, 1043 NX Amsterdam, The Netherlands, Registration No. 33156677, Registered in The Netherlands.
Aha, of course, the all boolean. I did not try that, and will. DB metadata I get via the GUI menu : Database -> Open and Manage Cheers, C. From: Fabrice Etanchaud [mailto:fetanchaud@questel.com] Sent: 12 January 2015 16:14 To: Hondros, Constantine (ELS-AMS); basex-talk@mailman.uni-konstanz.de Subject: RE: Pruned, optimized DB shows same document count, db size Dear Constantine,
From what I know, optimize only rebuilt indexes and metadata. Did you try "optimize all" to compact db files ?
But documents' number should be updated. How do you get metadata information ? Best regards, Fabrice Etanchaud Questel/Orbit http://docs.basex.org/wiki/Database_Module#db:optimize De : basex-talk-bounces@mailman.uni-konstanz.de<mailto:basex-talk-bounces@mailman.uni-konstanz.de> [mailto:basex-talk-bounces@mailman.uni-konstanz.de] De la part de Hondros, Constantine (ELS-AMS) Envoyé : lundi 12 janvier 2015 14:29 À : basex-talk@mailman.uni-konstanz.de<mailto:basex-talk@mailman.uni-konstanz.de> Objet : [basex-talk] Pruned, optimized DB shows same document count, db size Hello all, I've pruned about half the documents of a +-5GB database using XQuery delete. I then optimised the database using db:optimize. The database metadata still shows the original number of documents, and the overall db filesize remains roughly the same. I was sort of hoping this pruning would improve performance, but if there's any difference it's negligable. Am I missing something obvious? Thanks in advance, Constantine Hondros ________________________________ Elsevier B.V. Registered Office: Radarweg 29, 1043 NX Amsterdam, The Netherlands, Registration No. 33156677, Registered in The Netherlands. ________________________________ Elsevier B.V. Registered Office: Radarweg 29, 1043 NX Amsterdam, The Netherlands, Registration No. 33156677, Registered in The Netherlands.
Constantine, Did you mean db :delete or delete nodes ? I'm using 8.0 769b53b and after db:delete, even without optimize, I get correct documents' count in the GUI window. Best, Fabrice De : basex-talk-bounces@mailman.uni-konstanz.de [mailto:basex-talk-bounces@mailman.uni-konstanz.de] De la part de Hondros, Constantine (ELS-AMS) Envoyé : lundi 12 janvier 2015 16:32 À : basex-talk@mailman.uni-konstanz.de Objet : Re: [basex-talk] Pruned, optimized DB shows same document count, db size Aha, of course, the all boolean. I did not try that, and will. DB metadata I get via the GUI menu : Database -> Open and Manage Cheers, C. From: Fabrice Etanchaud [mailto:fetanchaud@questel.com] Sent: 12 January 2015 16:14 To: Hondros, Constantine (ELS-AMS); basex-talk@mailman.uni-konstanz.de<mailto:basex-talk@mailman.uni-konstanz.de> Subject: RE: Pruned, optimized DB shows same document count, db size Dear Constantine,
From what I know, optimize only rebuilt indexes and metadata. Did you try "optimize all" to compact db files ?
But documents' number should be updated. How do you get metadata information ? Best regards, Fabrice Etanchaud Questel/Orbit http://docs.basex.org/wiki/Database_Module#db:optimize De : basex-talk-bounces@mailman.uni-konstanz.de<mailto:basex-talk-bounces@mailman.uni-konstanz.de> [mailto:basex-talk-bounces@mailman.uni-konstanz.de] De la part de Hondros, Constantine (ELS-AMS) Envoyé : lundi 12 janvier 2015 14:29 À : basex-talk@mailman.uni-konstanz.de<mailto:basex-talk@mailman.uni-konstanz.de> Objet : [basex-talk] Pruned, optimized DB shows same document count, db size Hello all, I've pruned about half the documents of a +-5GB database using XQuery delete. I then optimised the database using db:optimize. The database metadata still shows the original number of documents, and the overall db filesize remains roughly the same. I was sort of hoping this pruning would improve performance, but if there's any difference it's negligable. Am I missing something obvious? Thanks in advance, Constantine Hondros ________________________________ Elsevier B.V. Registered Office: Radarweg 29, 1043 NX Amsterdam, The Netherlands, Registration No. 33156677, Registered in The Netherlands. ________________________________ Elsevier B.V. Registered Office: Radarweg 29, 1043 NX Amsterdam, The Netherlands, Registration No. 33156677, Registered in The Netherlands.
Hi Fabrice, I used the Xquery delete node command, rather than the (document-oriented) db:delete. I guess that explains the difference. C. From: Fabrice Etanchaud [mailto:fetanchaud@questel.com] Sent: 12 January 2015 17:01 To: Hondros, Constantine (ELS-AMS); basex-talk@mailman.uni-konstanz.de Subject: RE: [basex-talk] Pruned, optimized DB shows same document count, db size Constantine, Did you mean db :delete or delete nodes ? I'm using 8.0 769b53b and after db:delete, even without optimize, I get correct documents' count in the GUI window. Best, Fabrice De : basex-talk-bounces@mailman.uni-konstanz.de<mailto:basex-talk-bounces@mailman.uni-konstanz.de> [mailto:basex-talk-bounces@mailman.uni-konstanz.de] De la part de Hondros, Constantine (ELS-AMS) Envoyé : lundi 12 janvier 2015 16:32 À : basex-talk@mailman.uni-konstanz.de<mailto:basex-talk@mailman.uni-konstanz.de> Objet : Re: [basex-talk] Pruned, optimized DB shows same document count, db size Aha, of course, the all boolean. I did not try that, and will. DB metadata I get via the GUI menu : Database -> Open and Manage Cheers, C. From: Fabrice Etanchaud [mailto:fetanchaud@questel.com] Sent: 12 January 2015 16:14 To: Hondros, Constantine (ELS-AMS); basex-talk@mailman.uni-konstanz.de<mailto:basex-talk@mailman.uni-konstanz.de> Subject: RE: Pruned, optimized DB shows same document count, db size Dear Constantine,
From what I know, optimize only rebuilt indexes and metadata. Did you try "optimize all" to compact db files ?
But documents' number should be updated. How do you get metadata information ? Best regards, Fabrice Etanchaud Questel/Orbit http://docs.basex.org/wiki/Database_Module#db:optimize De : basex-talk-bounces@mailman.uni-konstanz.de<mailto:basex-talk-bounces@mailman.uni-konstanz.de> [mailto:basex-talk-bounces@mailman.uni-konstanz.de] De la part de Hondros, Constantine (ELS-AMS) Envoyé : lundi 12 janvier 2015 14:29 À : basex-talk@mailman.uni-konstanz.de<mailto:basex-talk@mailman.uni-konstanz.de> Objet : [basex-talk] Pruned, optimized DB shows same document count, db size Hello all, I've pruned about half the documents of a +-5GB database using XQuery delete. I then optimised the database using db:optimize. The database metadata still shows the original number of documents, and the overall db filesize remains roughly the same. I was sort of hoping this pruning would improve performance, but if there's any difference it's negligable. Am I missing something obvious? Thanks in advance, Constantine Hondros ________________________________ Elsevier B.V. Registered Office: Radarweg 29, 1043 NX Amsterdam, The Netherlands, Registration No. 33156677, Registered in The Netherlands. ________________________________ Elsevier B.V. Registered Office: Radarweg 29, 1043 NX Amsterdam, The Netherlands, Registration No. 33156677, Registered in The Netherlands. ________________________________ Elsevier B.V. Registered Office: Radarweg 29, 1043 NX Amsterdam, The Netherlands, Registration No. 33156677, Registered in The Netherlands.
Constantine, It seems Questel and Elsevier are competitors... But using the same great piece of software ! Best regards, Fabrice De : basex-talk-bounces@mailman.uni-konstanz.de [mailto:basex-talk-bounces@mailman.uni-konstanz.de] De la part de Hondros, Constantine (ELS-AMS) Envoyé : lundi 12 janvier 2015 17:10 À : basex-talk@mailman.uni-konstanz.de Objet : Re: [basex-talk] Pruned, optimized DB shows same document count, db size Hi Fabrice, I used the Xquery delete node command, rather than the (document-oriented) db:delete. I guess that explains the difference. C. From: Fabrice Etanchaud [mailto:fetanchaud@questel.com] Sent: 12 January 2015 17:01 To: Hondros, Constantine (ELS-AMS); basex-talk@mailman.uni-konstanz.de<mailto:basex-talk@mailman.uni-konstanz.de> Subject: RE: [basex-talk] Pruned, optimized DB shows same document count, db size Constantine, Did you mean db :delete or delete nodes ? I'm using 8.0 769b53b and after db:delete, even without optimize, I get correct documents' count in the GUI window. Best, Fabrice De : basex-talk-bounces@mailman.uni-konstanz.de<mailto:basex-talk-bounces@mailman.uni-konstanz.de> [mailto:basex-talk-bounces@mailman.uni-konstanz.de] De la part de Hondros, Constantine (ELS-AMS) Envoyé : lundi 12 janvier 2015 16:32 À : basex-talk@mailman.uni-konstanz.de<mailto:basex-talk@mailman.uni-konstanz.de> Objet : Re: [basex-talk] Pruned, optimized DB shows same document count, db size Aha, of course, the all boolean. I did not try that, and will. DB metadata I get via the GUI menu : Database -> Open and Manage Cheers, C. From: Fabrice Etanchaud [mailto:fetanchaud@questel.com] Sent: 12 January 2015 16:14 To: Hondros, Constantine (ELS-AMS); basex-talk@mailman.uni-konstanz.de<mailto:basex-talk@mailman.uni-konstanz.de> Subject: RE: Pruned, optimized DB shows same document count, db size Dear Constantine,
From what I know, optimize only rebuilt indexes and metadata. Did you try "optimize all" to compact db files ?
But documents' number should be updated. How do you get metadata information ? Best regards, Fabrice Etanchaud Questel/Orbit http://docs.basex.org/wiki/Database_Module#db:optimize De : basex-talk-bounces@mailman.uni-konstanz.de<mailto:basex-talk-bounces@mailman.uni-konstanz.de> [mailto:basex-talk-bounces@mailman.uni-konstanz.de] De la part de Hondros, Constantine (ELS-AMS) Envoyé : lundi 12 janvier 2015 14:29 À : basex-talk@mailman.uni-konstanz.de<mailto:basex-talk@mailman.uni-konstanz.de> Objet : [basex-talk] Pruned, optimized DB shows same document count, db size Hello all, I've pruned about half the documents of a +-5GB database using XQuery delete. I then optimised the database using db:optimize. The database metadata still shows the original number of documents, and the overall db filesize remains roughly the same. I was sort of hoping this pruning would improve performance, but if there's any difference it's negligable. Am I missing something obvious? Thanks in advance, Constantine Hondros ________________________________ Elsevier B.V. Registered Office: Radarweg 29, 1043 NX Amsterdam, The Netherlands, Registration No. 33156677, Registered in The Netherlands. ________________________________ Elsevier B.V. Registered Office: Radarweg 29, 1043 NX Amsterdam, The Netherlands, Registration No. 33156677, Registered in The Netherlands. ________________________________ Elsevier B.V. Registered Office: Radarweg 29, 1043 NX Amsterdam, The Netherlands, Registration No. 33156677, Registered in The Netherlands.
Yup - and for the record, BaseX simply rocks. Great work, guys! From: Fabrice Etanchaud [mailto:fetanchaud@questel.com] Sent: 12 January 2015 17:24 To: Hondros, Constantine (ELS-AMS) Cc: basex-talk@mailman.uni-konstanz.de Subject: RE: [basex-talk] Pruned, optimized DB shows same document count, db size Constantine, It seems Questel and Elsevier are competitors... But using the same great piece of software ! Best regards, Fabrice De : basex-talk-bounces@mailman.uni-konstanz.de<mailto:basex-talk-bounces@mailman.uni-konstanz.de> [mailto:basex-talk-bounces@mailman.uni-konstanz.de] De la part de Hondros, Constantine (ELS-AMS) Envoyé : lundi 12 janvier 2015 17:10 À : basex-talk@mailman.uni-konstanz.de<mailto:basex-talk@mailman.uni-konstanz.de> Objet : Re: [basex-talk] Pruned, optimized DB shows same document count, db size Hi Fabrice, I used the Xquery delete node command, rather than the (document-oriented) db:delete. I guess that explains the difference. C. From: Fabrice Etanchaud [mailto:fetanchaud@questel.com] Sent: 12 January 2015 17:01 To: Hondros, Constantine (ELS-AMS); basex-talk@mailman.uni-konstanz.de<mailto:basex-talk@mailman.uni-konstanz.de> Subject: RE: [basex-talk] Pruned, optimized DB shows same document count, db size Constantine, Did you mean db :delete or delete nodes ? I'm using 8.0 769b53b and after db:delete, even without optimize, I get correct documents' count in the GUI window. Best, Fabrice De : basex-talk-bounces@mailman.uni-konstanz.de<mailto:basex-talk-bounces@mailman.uni-konstanz.de> [mailto:basex-talk-bounces@mailman.uni-konstanz.de] De la part de Hondros, Constantine (ELS-AMS) Envoyé : lundi 12 janvier 2015 16:32 À : basex-talk@mailman.uni-konstanz.de<mailto:basex-talk@mailman.uni-konstanz.de> Objet : Re: [basex-talk] Pruned, optimized DB shows same document count, db size Aha, of course, the all boolean. I did not try that, and will. DB metadata I get via the GUI menu : Database -> Open and Manage Cheers, C. From: Fabrice Etanchaud [mailto:fetanchaud@questel.com] Sent: 12 January 2015 16:14 To: Hondros, Constantine (ELS-AMS); basex-talk@mailman.uni-konstanz.de<mailto:basex-talk@mailman.uni-konstanz.de> Subject: RE: Pruned, optimized DB shows same document count, db size Dear Constantine,
From what I know, optimize only rebuilt indexes and metadata. Did you try "optimize all" to compact db files ?
But documents' number should be updated. How do you get metadata information ? Best regards, Fabrice Etanchaud Questel/Orbit http://docs.basex.org/wiki/Database_Module#db:optimize De : basex-talk-bounces@mailman.uni-konstanz.de<mailto:basex-talk-bounces@mailman.uni-konstanz.de> [mailto:basex-talk-bounces@mailman.uni-konstanz.de] De la part de Hondros, Constantine (ELS-AMS) Envoyé : lundi 12 janvier 2015 14:29 À : basex-talk@mailman.uni-konstanz.de<mailto:basex-talk@mailman.uni-konstanz.de> Objet : [basex-talk] Pruned, optimized DB shows same document count, db size Hello all, I've pruned about half the documents of a +-5GB database using XQuery delete. I then optimised the database using db:optimize. The database metadata still shows the original number of documents, and the overall db filesize remains roughly the same. I was sort of hoping this pruning would improve performance, but if there's any difference it's negligable. Am I missing something obvious? Thanks in advance, Constantine Hondros ________________________________ Elsevier B.V. Registered Office: Radarweg 29, 1043 NX Amsterdam, The Netherlands, Registration No. 33156677, Registered in The Netherlands. ________________________________ Elsevier B.V. Registered Office: Radarweg 29, 1043 NX Amsterdam, The Netherlands, Registration No. 33156677, Registered in The Netherlands. ________________________________ Elsevier B.V. Registered Office: Radarweg 29, 1043 NX Amsterdam, The Netherlands, Registration No. 33156677, Registered in The Netherlands. ________________________________ Elsevier B.V. Registered Office: Radarweg 29, 1043 NX Amsterdam, The Netherlands, Registration No. 33156677, Registered in The Netherlands.
I vote that we have this statement ("BaseX simply rocks. Great work, guys!") eternalized in broad, golden letters in our office. Thanks for the nice feedback, it is appreciated :-) On 01/16/2015 09:45 AM, Hondros, Constantine (ELS-AMS) wrote:
Yup - and for the record, BaseX simply rocks. Great work, guys!
From: Fabrice Etanchaud [mailto:fetanchaud@questel.com] Sent: 12 January 2015 17:24 To: Hondros, Constantine (ELS-AMS) Cc: basex-talk@mailman.uni-konstanz.de Subject: RE: [basex-talk] Pruned, optimized DB shows same document count, db size
Constantine,
It seems Questel and Elsevier are competitors... But using the same great piece of software !
Best regards, Fabrice
De : basex-talk-bounces@mailman.uni-konstanz.de<mailto:basex-talk-bounces@mailman.uni-konstanz.de> [mailto:basex-talk-bounces@mailman.uni-konstanz.de] De la part de Hondros, Constantine (ELS-AMS) Envoyé : lundi 12 janvier 2015 17:10 À : basex-talk@mailman.uni-konstanz.de<mailto:basex-talk@mailman.uni-konstanz.de> Objet : Re: [basex-talk] Pruned, optimized DB shows same document count, db size
Hi Fabrice,
I used the Xquery delete node command, rather than the (document-oriented) db:delete.
I guess that explains the difference.
C.
From: Fabrice Etanchaud [mailto:fetanchaud@questel.com] Sent: 12 January 2015 17:01 To: Hondros, Constantine (ELS-AMS); basex-talk@mailman.uni-konstanz.de<mailto:basex-talk@mailman.uni-konstanz.de> Subject: RE: [basex-talk] Pruned, optimized DB shows same document count, db size
Constantine,
Did you mean db :delete or delete nodes ?
I'm using 8.0 769b53b and after db:delete, even without optimize, I get correct documents' count in the GUI window.
Best, Fabrice
De : basex-talk-bounces@mailman.uni-konstanz.de<mailto:basex-talk-bounces@mailman.uni-konstanz.de> [mailto:basex-talk-bounces@mailman.uni-konstanz.de] De la part de Hondros, Constantine (ELS-AMS) Envoyé : lundi 12 janvier 2015 16:32 À : basex-talk@mailman.uni-konstanz.de<mailto:basex-talk@mailman.uni-konstanz.de> Objet : Re: [basex-talk] Pruned, optimized DB shows same document count, db size
Aha, of course, the all boolean. I did not try that, and will.
DB metadata I get via the GUI menu : Database -> Open and Manage
Cheers, C.
From: Fabrice Etanchaud [mailto:fetanchaud@questel.com] Sent: 12 January 2015 16:14 To: Hondros, Constantine (ELS-AMS); basex-talk@mailman.uni-konstanz.de<mailto:basex-talk@mailman.uni-konstanz.de> Subject: RE: Pruned, optimized DB shows same document count, db size
Dear Constantine,
From what I know, optimize only rebuilt indexes and metadata. Did you try "optimize all" to compact db files ?
But documents' number should be updated. How do you get metadata information ?
Best regards,
Fabrice Etanchaud Questel/Orbit
http://docs.basex.org/wiki/Database_Module#db:optimize
De : basex-talk-bounces@mailman.uni-konstanz.de<mailto:basex-talk-bounces@mailman.uni-konstanz.de> [mailto:basex-talk-bounces@mailman.uni-konstanz.de] De la part de Hondros, Constantine (ELS-AMS) Envoyé : lundi 12 janvier 2015 14:29 À : basex-talk@mailman.uni-konstanz.de<mailto:basex-talk@mailman.uni-konstanz.de> Objet : [basex-talk] Pruned, optimized DB shows same document count, db size
Hello all,
I've pruned about half the documents of a +-5GB database using XQuery delete. I then optimised the database using db:optimize.
The database metadata still shows the original number of documents, and the overall db filesize remains roughly the same.
I was sort of hoping this pruning would improve performance, but if there's any difference it's negligable. Am I missing something obvious?
Thanks in advance,
Constantine Hondros
________________________________
Elsevier B.V. Registered Office: Radarweg 29, 1043 NX Amsterdam, The Netherlands, Registration No. 33156677, Registered in The Netherlands.
________________________________
Elsevier B.V. Registered Office: Radarweg 29, 1043 NX Amsterdam, The Netherlands, Registration No. 33156677, Registered in The Netherlands.
________________________________
Elsevier B.V. Registered Office: Radarweg 29, 1043 NX Amsterdam, The Netherlands, Registration No. 33156677, Registered in The Netherlands.
________________________________
Elsevier B.V. Registered Office: Radarweg 29, 1043 NX Amsterdam, The Netherlands, Registration No. 33156677, Registered in The Netherlands.
-- Dirk Kirsten, BaseX GmbH, http://basexgmbh.de |-- Firmensitz: Blarerstrasse 56, 78462 Konstanz |-- Registergericht Freiburg, HRB: 708285, Geschäftsführer: | Dr. Christian Grün, Dr. Alexander Holupirek, Michael Seiferle `-- Phone: 0049 7531 28 28 676, Fax: 0049 7531 20 05 22
participants (3)
-
Dirk Kirsten -
Fabrice Etanchaud -
Hondros, Constantine (ELS-AMS)