Hi Dirk, I didn't want to hijack the other thread, so I started a new one :) I've read https://github.com/BaseXdb/basex/issues/59 and I think it's great!. I have a slightly different request though that I was holding back because I didn't want to sound abusive, but now that you brought it up... I might be mistaken as I haven't really debugged the code, but I believe that indexing could maybe take so long (for our use case anyway) because it indexes all the values in the xml documents. So I wonder whether it would be beneficial for the indexing speed to be able to state which specific fields to index instead of all of them. This would allow for having fast regularly used queries, without impacting insertions that much. I can explain our use case quickly: we store XML files that are interchanged with third parties. We have lots of them all the time, so insertion speed is critical. Query speed on the other hand is not critical, though obviously having fast queries is a lot better than having slow ones. But it becomes more important as the queries lock the db for writing too, aside from feeling "if I just could index this one field", hehe. If we could just index the fields that we run most of our queries against, it would be perfect. Right now, I think we can get away with organizing resources inside directories and that would be enough as that seems to work perfectly (we'll perform a couple more tests with high volume), but I thought it would be really cool if we could have a similar insertion speed to what's achieved without indexing, by just indexing some fields (again, *guessing* that the cause of the decrease in speed is due to indexing all fields in the documents, which may or may not be the case). Thanks! Martín.
Haha, does the selective index feature just do what I spent 30 minutes writing about?Sorry, I think my reading abilities were impaired, or I was too impatient to read until the end of the issue. I'll give it a try when I have some moments, it looks pretty cool! Thanks, Martín.
From: ferrari_martin@hotmail.com To: basex-talk@mailman.uni-konstanz.de Subject: Selective indexes Date: Tue, 1 Sep 2015 01:37:31 +0000
Hi Dirk, I didn't want to hijack the other thread, so I started a new one :) I've read https://github.com/BaseXdb/basex/issues/59 and I think it's great!. I have a slightly different request though that I was holding back because I didn't want to sound abusive, but now that you brought it up... I might be mistaken as I haven't really debugged the code, but I believe that indexing could maybe take so long (for our use case anyway) because it indexes all the values in the xml documents. So I wonder whether it would be beneficial for the indexing speed to be able to state which specific fields to index instead of all of them. This would allow for having fast regularly used queries, without impacting insertions that much. I can explain our use case quickly: we store XML files that are interchanged with third parties. We have lots of them all the time, so insertion speed is critical. Query speed on the other hand is not critical, though obviously having fast queries is a lot better than having slow ones. But it becomes more important as the queries lock the db for writing too, aside from feeling "if I just could index this one field", hehe. If we could just index the fields that we run most of our queries against, it would be perfect. Right now, I think we can get away with organizing resources inside directories and that would be enough as that seems to work perfectly (we'll perform a couple more tests with high volume), but I thought it would be really cool if we could have a similar insertion speed to what's achieved without indexing, by just indexing some fields (again, *guessing* that the cause of the decrease in speed is due to indexing all fields in the documents, which may or may not be the case). Thanks! Martín.
Hello Martin,
I was glad to see your second mail, because that is what I wanted to write :) But if you miss some features or something doesn't work as expected don't hesitate to write.
Cheers Dirk
On 09/01/2015 03:55 AM, Martín Ferrari wrote:
Haha, does the selective index feature just do what I spent 30 minutes writing about? Sorry, I think my reading abilities were impaired, or I was too impatient to read until the end of the issue.
I'll give it a try when I have some moments, it looks pretty cool!
Thanks, Martín.
From: ferrari_martin@hotmail.com To: basex-talk@mailman.uni-konstanz.de Subject: Selective indexes Date: Tue, 1 Sep 2015 01:37:31 +0000
Hi Dirk, I didn't want to hijack the other thread, so I started a new one :)
I've read https://github.com/BaseXdb/basex/issues/59 and I thinkit's great!. I have a slightly different request though that I was holding back because I didn't want to sound abusive, but now that you brought it up... I might be mistaken as I haven't really debugged the code, but I believe that indexing could maybe take so long (for our use case anyway) because it indexes all the values in the xml documents. So I wonder whether it would be beneficial for the indexing speed to be able to state which specific fields to index instead of all of them. This would allow for having fast regularly used queries, without impacting insertions that much.
I can explain our use case quickly: we store XML files that areinterchanged with third parties. We have lots of them all the time, so insertion speed is critical. Query speed on the other hand is not critical, though obviously having fast queries is a lot better than having slow ones. But it becomes more important as the queries lock the db for writing too, aside from feeling "if I just could index this one field", hehe. If we could just index the fields that we run most of our queries against, it would be perfect. Right now, I think we can get away with organizing resources inside directories and that would be enough as that seems to work perfectly (we'll perform a couple more tests with high volume), but I thought it would be really cool if we could have a similar insertion speed to what's achieved without indexing, by just indexing some fields (again, *guessing* that the cause of the decrease in speed is due to indexing all fields in the documents, which may or may not be the case).
Thanks! Martín.
basex-talk@mailman.uni-konstanz.de
-
Dirk Kirsten -
Martín Ferrari