Dear Sateesh,
We have started a web application in that we have a requirement to create xml db's per user based means each user who logs in we need to create DB's on his xml files in the backend.
This should be no problem. New databases can easily be created and dropped via the existing APIs or even XQuery.
,for this we dont want to use the client-server architecture of the basex as we dont want that extra time to hit a http server request to basex server
Have you encountered the client/server architecture to be a serious bottleneck, or is this a theoretical assumption? If you want to distribute your data anyway, it could make sense to do some manual sharding and experiment with multiple BaseX instances on different servers. If you believe that this makes no sense in your scenario, you can surely use BaseX in an embedded way.
also note that volumes of the xml files are very huge they might go up to gb's also
Please note that the creation of a database with gigabytes of data might take some seconds or even minutes. Do you want to created these database every time a user logs in, or only once (the first time)?
and query output will also be in huge volume and also we must make sure that this huge volumes of query output will not eatup all the heap memory
If you use the APIs in the right way, all data will be streamed, so you can expect constant memory consumption (as long as you do not buffer the received results in your client).
as the number of parallel users using this site are very high
What does "very high" mean? Do you expect 1000 parallel requests per day, hour, minute, second (ms, ns, ...)? Will you have thousands or millions of users?
and we cannot afford to go out of memory.Also we would need to configure the DB paths as this are huge volumes we cannot store them in the default locations to.
One approach to distribute database directories is to create symlinks to other drives. If a new database is created, it will always be added to the default folder, but you could create empty databases in advance, which are then populated as soon as new users are registered.
Hope this helps, Christian