You cannot select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
openmw/scripts/HOWTO-benchmark.md

113 lines
4.7 KiB
Markdown

OpenMW ships with its own benchmarking tool. This document describes how to collect performance data and vizualise it.
Collecting data
===============
The data collected is the same as displayed by repeatedly pressing F3 and/or F4 while in the game (correspondance is shown below).
We can select what should be collected by setting the `OPENMW_OSG_STATS_LIST` environment variables to a semi-colon separated list of metrics to collect. There can be any combination and order doesn't matter. Note that data collection can **significantly reduce** performance
| Metric to collect | Equivalent in-game profiler |
|-------------------|-------------------------------------------------------------------|
| `frame_rate` | Frame rate |
| `engine` | OpenMW specifics metric (white bars) |
| `event` | Event traversal bar |
| `update` | Update traversal bar |
| `rendering` | Draw bar |
| `gpu` | GPU bar |
| `times` | Alias for `event;update;rendering;engine;gpu`, that is all graphs |
| `cameraobjects` | Table shown when pressing F3 3 times |
| `viewerobjects` | Table shown when pressing F3 4 times |
| `resource` | Table shown when pressing F4 |
It is necessary to write the collected metrics to a file which needs to be defined by the `OPENMW_OSG_STATS_FILE` environment variable.
Example
-------
Posix shell
```sh
OPENMW_OSG_STATS_FILE=/tmp/stats OPENMW_OSG_STATS_LIST="resource;engine" /usr/local/bin/openmw
```
Windows PowerShell
```powershell
$env:OPENMW_OSG_STATS_FILE="c:\stats"
$env:OPENMW_OSG_STATS_LIST="resource;engine"
openmw
```
Analyzing results
=================
`scripts/osg_stats.py` is a python script that can perform statistical analysis and generate graphs from a trace. It doesn't need to be run on the same machine as the trace was taken.
`osg_stats.py --help` list the available options.
Examples
--------
### Print the available metrics in a given trace file
`osg_stats.py --print_keys /tmp/stats`
Sample output
>>>
gui_time_begin
gui_time_end
gui_time_taken
input_time_begin
input_time_end
input_time_taken
lua_time_begin
lua_time_end
lua_time_taken
mechanics_time_begin
mechanics_time_end
mechanics_time_taken
physics_time_begin
physics_time_end
physics_time_taken
physicsworker_time_begin
physicsworker_time_end
physicsworker_time_taken
script_time_begin
script_time_end
script_time_taken
sound_time_begin
sound_time_end
sound_time_taken
state_time_begin
state_time_end
state_time_taken
world_time_begin
world_time_end
world_time_taken
>>>
### Print a table with statistics data for some metrics
`osg_stats.py --stats physicsworker_time_taken --stats mechanics_time_taken --stats world_time_taken --stats physics_time_taken /tmp/stats`
Sample output
> | source | key | number | min | max | mean | median | stdev | q95 |
> |------------|--------------------------|--------|-----|----------|------------------------|----------|------------------------|----------------------|
> | /tmp/stats | physicsworker_time_taken | 56245 | 0.0 | 0.01526 | 0.0018826463330073784 | 7.7e-05 | 0.003210274653689913 | 0.009509799999999995 |
> | /tmp/stats | mechanics_time_taken | 56245 | 0.0 | 0.015808 | 0.0030202102942483776 | 0.002177 | 0.002565895193458489 | 0.008489799999999995 |
> | /tmp/stats | world_time_taken | 56245 | 0.0 | 0.003643 | 5.230777846919726e-05 | 5.1e-05 | 1.7322475294417906e-05 | 6.6e-05 |
> | /tmp/stats | physics_time_taken | 56245 | 0.0 | 0.004407 | 0.00019985403146946396 | 0.000129 | 0.0002106166003676915 | 0.000697 |
### Plot a timeserie of aforementioned metrics, ignoring first 1000 frames
`osg_stats.py --begin_frame 1000 --end_frame 1200 --timeseries physicsworker_time_taken --timeseries mechanics_time_taken --timeseries world_time_taken --timeseries physics_time_taken /tmp/stats`
### Plot time spent in physics and mechanics depending on number of actors in the scene
`osg_stats.py --plot 'Physics Actors' physicsworker_time_taken mean --plot 'Physics Actors' mechanics_time_taken mean --plot 'Physics Actors' physics_time_taken mean /tmp/stats`
### Plot timeserie from 2 traces
`osg_stats.py --timeseries 'Frame Duration' /tmp/shadowson /tmp/shadowsoff`