Hi James,
Finally some feedback:
Database paths are looked up differently on Windows/Mac and UNIX-based platforms: On the first ones, the lookup is case insensitive; on the latter ones, case matters. As a result, it may happen that path lookups will fail on UNIX/Linux systems (it shouldn’t happen in the other direction). Could you check if your document paths – which you e.g. check via db:list($db) – and your path strings are exactly matching?
The pth.basex file contains an index for documents paths. It is created when database path lookups are performed for the first time. The reason for the deferred index generation is that we have use cases in which millions of documents are stored in a database, and the document path is never requested.
The idp.basex file is created if incremental indexing (UPDINDEX) is enabled. "idp" stands for "id/pre mapping": It is used to quickly look up the pre value for a database node id.
Thanks for your offer to adopt this information in the Wiki. If my hints on what’s happening are too stingy, I’ll be glad to answer more questions.
Cheers, Christian
I’ve been debugging a strange issue on a system and I can’t quite understand what’s causing it. I create some databases on my local laptop (macOS). They provide master data for a larger system so I then copy them to a cloud server (Linode/Amazon). I recently switched updated some queries to use db:open with a path restriction to make them run faster. Everything worked on the laptop but nothing was returned on the server for the same paths. If I optimise(all) the database on the server it then works.
After the optimisation, on the server there is a file called pth.basex and a file called idp.basex. On the Mac there is NO file in the database folder called pth.basex or idp.basex. I’m not sure what pth.basex or idp.basex are as they’re not referenced on Storage Layout[1] This is copying the database files directly. I have had mixed results using backup zip files - sometimes it works and sometimes it fails.
Has anyone else observed this behaviour? What are the steps I should take to ensure reliable movement of databases? What are these mysterious files? I’m happy to update the Wiki with any new information . Many thanks, James