From c9279d413780096404f4785b99e6471c410c8e62 Mon Sep 17 00:00:00 2001 From: fredzio Date: Sat, 9 Oct 2021 17:54:51 +0200 Subject: [PATCH] Add a simple HOWTO for the builtin tracing functionality. --- scripts/HOWTO-benchmark.md | 112 +++++++++++++++++++++++++++++++++++++ 1 file changed, 112 insertions(+) create mode 100644 scripts/HOWTO-benchmark.md diff --git a/scripts/HOWTO-benchmark.md b/scripts/HOWTO-benchmark.md new file mode 100644 index 0000000000..9641bfcd66 --- /dev/null +++ b/scripts/HOWTO-benchmark.md @@ -0,0 +1,112 @@ +OpenMW ships with its own benchmarking tool. This document describes how to collect performance data and vizualise it. + +Collecting data +=============== + +The data collected is the same as displayed by repeatedly pressing F3 and/or F4 while in the game (correspondance is shown below). +We can select what should be collected by setting the `OPENMW_OSG_STATS_LIST` environment variables to a semi-colon separated list of metrics to collect. There can be any combination and order doesn't matter. Note that data collection can **significantly reduce** performance + +| Metric to collect | Equivalent in-game profiler | +|-------------------|-------------------------------------------------------------------| +| `frame_rate` | Frame rate | +| `engine` | OpenMW specifics metric (white bars) | +| `event` | Event traversal bar | +| `update` | Update traversal bar | +| `rendering` | Draw bar | +| `gpu` | GPU bar | +| `times` | Alias for `event;update;rendering;engine;gpu`, that is all graphs | +| `cameraobjects` | Table shown when pressing F3 3 times | +| `viewerobjects` | Table shown when pressing F3 4 times | +| `resource` | Table shown when pressing F4 | + +It is necessary to write the collected metrics to a file which needs to be defined by the `OPENMW_OSG_STATS_FILE` environment variable. + +Example +------- + +Posix shell +```sh +OPENMW_OSG_STATS_FILE=/tmp/stats OPENMW_OSG_STATS_LIST="resource;engine" /usr/local/bin/openmw +``` + +Windows PowerShell +```powershell +$env:OPENMW_OSG_STATS_FILE="c:\stats" +$env:OPENMW_OSG_STATS_LIST="resource;engine" +openmw +``` + + +Analyzing results +================= + +`scripts/osg_stats.py` is a python script that can perform statistical analysis and generate graphs from a trace. It doesn't need to be run on the same machine as the trace was taken. + +`osg_stats.py --help` list the available options. + +Examples +-------- + +### Print the available metrics in a given trace file + +`osg_stats.py --print_keys /tmp/stats` + +Sample output +>>> +gui_time_begin +gui_time_end +gui_time_taken +input_time_begin +input_time_end +input_time_taken +lua_time_begin +lua_time_end +lua_time_taken +mechanics_time_begin +mechanics_time_end +mechanics_time_taken +physics_time_begin +physics_time_end +physics_time_taken +physicsworker_time_begin +physicsworker_time_end +physicsworker_time_taken +script_time_begin +script_time_end +script_time_taken +sound_time_begin +sound_time_end +sound_time_taken +state_time_begin +state_time_end +state_time_taken +world_time_begin +world_time_end +world_time_taken +>>> + +### Print a table with statistics data for some metrics + +`osg_stats.py --stats physicsworker_time_taken --stats mechanics_time_taken --stats world_time_taken --stats physics_time_taken /tmp/stats` + +Sample output + +> | source | key | number | min | max | mean | median | stdev | q95 | +> |------------|--------------------------|--------|-----|----------|------------------------|----------|------------------------|----------------------| +> | /tmp/stats | physicsworker_time_taken | 56245 | 0.0 | 0.01526 | 0.0018826463330073784 | 7.7e-05 | 0.003210274653689913 | 0.009509799999999995 | +> | /tmp/stats | mechanics_time_taken | 56245 | 0.0 | 0.015808 | 0.0030202102942483776 | 0.002177 | 0.002565895193458489 | 0.008489799999999995 | +> | /tmp/stats | world_time_taken | 56245 | 0.0 | 0.003643 | 5.230777846919726e-05 | 5.1e-05 | 1.7322475294417906e-05 | 6.6e-05 | +> | /tmp/stats | physics_time_taken | 56245 | 0.0 | 0.004407 | 0.00019985403146946396 | 0.000129 | 0.0002106166003676915 | 0.000697 | + + +### Plot a timeserie of aforementioned metrics, ignoring first 1000 frames + +`osg_stats.py --begin_frame 1000 --end_frame 1200 --timeseries physicsworker_time_taken --timeseries mechanics_time_taken --timeseries world_time_taken --timeseries physics_time_taken /tmp/stats` + +### Plot time spent in physics and mechanics depending on number of actors in the scene + +`osg_stats.py --plot 'Physics Actors' physicsworker_time_taken mean --plot 'Physics Actors' mechanics_time_taken mean --plot 'Physics Actors' physics_time_taken mean /tmp/stats` + +### Plot timeserie from 2 traces + +`osg_stats.py --timeseries 'Frame Duration' /tmp/shadowson /tmp/shadowsoff`