Use LRU cache for ESMReaders. When cache capacity is reached close least
recently used ESMReader. Remember the file name if a reader was open. Once the
reader requested again open the file if there is stored name for it. Put
released ESMReader to the back of the free items list. Close ESMReader's from
the front of the free items list.
Cached item can be used only by one client at the same time. If the same item is
requested twice exception is thrown. This should never happen in practice. If
this happens need to fix the client logic.
It's allowed to go over the capacity limit when requesting different readers.
Ideally this should never happen but there will be system error anyway
signalizing about too many open files. Need to fix client logic in this case.
All places that were using a vector of ESMReaders now using the cache. Cache is
local for each use case and there is no need for a thread safety.
Simultaneously writing to sqlite3 database is not possible. Process exclusively
locks the database for this. Another process will fail to perform any request
when database is locked. Alternatively it can wait. Handling this situation
properly requires complexity that is not really needed. Users are not expected
to run multiple openmw processes simultaneously using the same navmeshdb.
Before this change running multiple openmw processes using the same navmeshdb
can lead to a crash when first transaction fails to start because there is
exception thrown and not catched.
Remove use of explicit transactions from DbWorker. Handling all possible
transaction states due to different errors brings unnecessary complexity.
Initially they were introduced to increase time between flushes to disk. This
makes sense for navmeshtool because of massive number of writes but for the
engine this is not an issue.
Need to load the last present record from a sequence of loaded records. That
means reverse should be called before unique or unique should be applied for
a reversed range. Since unique keeps only the first element from a sub
sequence of equal elements.
Use forEachUnique with reversed range to avoid redundant container
modifications.
Use "pragma max_page_count" to define max allowed file size in combination with
"pragma page_size" based on a new setting "max navmeshdb file size".
* Stop navmeshtool on the first db error.
* Disable writes to db in the engine on first "database or disk is full"
SQLite3 error. There is no special error code for this error.
* Change default "write to navmeshdb" to true.
* Use time intervals for transaction duration instead of number of changes.
* Remove tiles outside processing range. Useful when new content profile map
has different bounds.
* Remove ignored tiles. For a case when content profile maps have intersection
but there is no more data for navmesh.
* Remove older tiles at the same worldspace position. If navmesh tile data has
changed with new content, the old ones unlikely to be used.
* Vacuum the database when there are modifications. SQLite leaves empty pages
in the file on database modification. Vacuum cleans up unused pages reducing
the file size.
If object is too big iteration over all tiles covering it can take too much
time. Limit bounds to a square around a player position to cover only tiles
that will be present in navmesh based on max tiles number option.
Each object is associated with a set of tiles its present in. Culling can
reduce this set but it has to be update when bounds change position. Do this
in TileCachedRecastMeshManager::setBounds updating the set and adding/removing
objects to the corresponding CachedRecastMeshManagers.
* Avoid inheritance.
* Define equality operators out of the class definition.
* Replace toString with toStringView where it doesn't make sense to create a string.
Choose a parent base on which node is used to iterate over children nodes.
This leads to duplicate handing of child nodes. A node will be handled so many
times how many parents it has.
For example:
p1 p2
\ /
c
Will be handled as:
p1 p2
| |
c c
If c has children they will be handled X times c is handled.
Perform all request to db in a single thread to avoid blocking navmesh
generator threads due to slow write operations.
Write to db navmesh for all changes except update as it done for memory cache.
Batch multiple db operations into a single transaction to speed up writing by
not executing fsync after each insert/update query. All reads are performed in
the same transaction so they see uncommited data.
Merge conflicts included:
* One setting being removed (branch had changed its type).
* One setting's description being changed (branch had changed its type).
* List of files in components/files was changed both upstream and on the
branch.
* Upstream had changed something in a file the branch deletes.
With this PR we refactor `StringUtils::replaceAll` to accept `string_view` as suggested in a code comment. In addition, while we are touching this rebuild happy file, we slim it down a bit by moving a few sparingly used functions elsewhere.
This PR aims to start addressing `ESM` design issues that have silenced errors we incorporated into groundcover `ESM` loading approaches.
- We move the resolution of `parentFileIndices` from `ESMStore` to `ESMReader` as suggested in a `TODO` comment.
- We improve a highly misleading comment which downplayed the significance of `parentFileIndices`.
- We document important preconditions.
- We move a user facing error message to the highest level and improve its context.
- We remove an inappropriate `setGlobalReaderList` method. We now pass this reader list into the method that requires it.
- We remove a thoroughly pointless optimisation of `Store<ESM::LandTexture>`'s construction that has unnecessarily depended on `getGlobalReaderList`.
There should be no functional changes for `master`, but this PR should remove an issue blocking PR #3208.
To construct serializer from given entities:
* Data source/destination - any value that has to be serialized/deserialized,
usually already existing type.
* Format - functional object to define high level serialization logic to
define specific format and data schema. Like order of fields, allocation.
* Visitor - functional object to define low level serialization logic to
operator on given data part.
* BinaryWriter - copies given value into provided buffer.
* BinaryReader - copies value into given destination from provided buffer.
* SizeAccumulator - calculates required buffer size for given data.
