mirror/openmw-tes3mp
1
0
Fork 1
mirror of https://github.com/TES3MP/openmw-tes3mp.git synced 2025-01-16 04:19:55 +00:00
Code Issues Releases Wiki Activity
openmw-tes3mp/apps/openmw/mwvr/openxrviewer.cpp

371 lines
14 KiB
C++
Raw Normal View History

First commit. Wrote an osgviewer, achieved display.
2020-01-09 23:10:09 +00:00
#include "openxrviewer.hpp"
#include "openxrmanagerimpl.hpp"
#include "Windows.h"
Split view into worldview and menu, for ease of layer management. Basic pose management.
2020-01-23 23:14:23 +00:00
#include "../mwrender/vismask.hpp"
Basic hand tracking.
2020-02-14 21:11:19 +00:00
#include "../mwmechanics/actorutil.hpp"
#include "../mwbase/world.hpp"
#include "../mwbase/environment.hpp"
#include "../mwworld/class.hpp"
#include "../mwworld/player.hpp"
#include "../mwworld/esmstore.hpp"
#include <components/esm/loadrace.hpp>
#include <osg/MatrixTransform>
First commit. Wrote an osgviewer, achieved display.
2020-01-09 23:10:09 +00:00
namespace MWVR
{
OpenXRViewer::OpenXRViewer(
osg::ref_ptr<OpenXRManager> XR,
osg::ref_ptr<osgViewer::Viewer> viewer,
float metersPerUnit)
Basic hand tracking.
2020-02-14 21:11:19 +00:00
: osg::Group()
, mXR(XR)
Split view into worldview and menu, for ease of layer management. Basic pose management.
2020-01-23 23:14:23 +00:00
, mRealizeOperation(new RealizeOperation(XR, this))
First commit. Wrote an osgviewer, achieved display.
2020-01-09 23:10:09 +00:00
, mViewer(viewer)
, mMetersPerUnit(metersPerUnit)
, mConfigured(false)
Split view into worldview and menu, for ease of layer management. Basic pose management.
2020-01-23 23:14:23 +00:00
, mCompositionLayerProjectionViews(2, {XR_TYPE_COMPOSITION_LAYER_PROJECTION_VIEW})
Predictions work perfectly with this. Committing that and clean up later.
2020-02-02 12:12:53 +00:00
, mXRSession(nullptr)
, mPreDraw(new PredrawCallback(this))
, mPostDraw(new PostdrawCallback(this))
First commit. Wrote an osgviewer, achieved display.
2020-01-09 23:10:09 +00:00
{
Split view into worldview and menu, for ease of layer management. Basic pose management.
2020-01-23 23:14:23 +00:00
mViewer->setRealizeOperation(mRealizeOperation);
Predictions work perfectly with this. Committing that and clean up later.
2020-02-02 12:12:53 +00:00
mCompositionLayerProjectionViews[0].pose.orientation.w = 1;
mCompositionLayerProjectionViews[1].pose.orientation.w = 1;
Basic hand tracking.
2020-02-14 21:11:19 +00:00
this->setName("OpenXRRoot");
this->setUserData(new TrackedNodeUpdateCallback(this));
mLeftHandTransform->setName("tracker l hand");
mLeftHandTransform->setUpdateCallback(new TrackedNodeUpdateCallback(this));
this->addChild(mLeftHandTransform);
mRightHandTransform->setName("tracker r hand");
mRightHandTransform->setUpdateCallback(new TrackedNodeUpdateCallback(this));
this->addChild(mRightHandTransform);
First commit. Wrote an osgviewer, achieved display.
2020-01-09 23:10:09 +00:00
}
OpenXRViewer::~OpenXRViewer(void)
{
}
void
OpenXRViewer::traverse(
osg::NodeVisitor& visitor)
{
Split view into worldview and menu, for ease of layer management. Basic pose management.
2020-01-23 23:14:23 +00:00
if (mRealizeOperation->realized())
osg::Group::traverse(visitor);
}
First commit. Wrote an osgviewer, achieved display.
2020-01-09 23:10:09 +00:00
Split view into worldview and menu, for ease of layer management. Basic pose management.
2020-01-23 23:14:23 +00:00
const XrCompositionLayerBaseHeader*
OpenXRViewer::layer()
{
Basic hand tracking.
2020-02-14 21:11:19 +00:00
// Does not check for visible, since this should always render
Split view into worldview and menu, for ease of layer management. Basic pose management.
2020-01-23 23:14:23 +00:00
return reinterpret_cast<XrCompositionLayerBaseHeader*>(mLayer.get());
First commit. Wrote an osgviewer, achieved display.
2020-01-09 23:10:09 +00:00
}
Split view into worldview and menu, for ease of layer management. Basic pose management.
2020-01-23 23:14:23 +00:00
void OpenXRViewer::traversals()
First commit. Wrote an osgviewer, achieved display.
2020-01-09 23:10:09 +00:00
{
Simplified render timing by separating rendering from the openxr swapchain, and instead blitting and submitting separately
2020-01-26 19:06:47 +00:00
mXR->handleEvents();
Split view into worldview and menu, for ease of layer management. Basic pose management.
2020-01-23 23:14:23 +00:00
mViewer->updateTraversal();
mViewer->renderingTraversals();
}
First commit. Wrote an osgviewer, achieved display.
2020-01-09 23:10:09 +00:00
Split view into worldview and menu, for ease of layer management. Basic pose management.
2020-01-23 23:14:23 +00:00
void OpenXRViewer::realize(osg::GraphicsContext* context)
{
std::unique_lock<std::mutex> lock(mMutex);
if (mConfigured)
return;
First commit. Wrote an osgviewer, achieved display.
2020-01-09 23:10:09 +00:00
Split view into worldview and menu, for ease of layer management. Basic pose management.
2020-01-23 23:14:23 +00:00
if (!context->isCurrent())
First commit. Wrote an osgviewer, achieved display.
2020-01-09 23:10:09 +00:00
if (!context->makeCurrent())
{
Split view into worldview and menu, for ease of layer management. Basic pose management.
2020-01-23 23:14:23 +00:00
throw std::logic_error("OpenXRViewer::configure() failed to make graphics context current.");
Improved frame pose prediction by barriering slave access as well. Slaves were updating on either the old frame timing or new by chance, causing jittery tracking. Accounted for morrowind's unit length (1 unit = 64 yards)
2020-01-25 13:27:13 +00:00
return;
First commit. Wrote an osgviewer, achieved display.
2020-01-09 23:10:09 +00:00
}
Predictions work perfectly with this. Committing that and clean up later.
2020-02-02 12:12:53 +00:00
auto mainCamera = mCameras["MainCamera"] = mViewer->getCamera();
mainCamera->setName("Main");
mainCamera->setInitialDrawCallback(new OpenXRView::InitialDrawCallback());
First commit. Wrote an osgviewer, achieved display.
2020-01-09 23:10:09 +00:00
Split view into worldview and menu, for ease of layer management. Basic pose management.
2020-01-23 23:14:23 +00:00
// Use the main camera to render any GUI to the OpenXR GUI quad's swapchain.
// (When swapping the window buffer we'll blit the mirror texture to it instead.)
Predictions work perfectly with this. Committing that and clean up later.
2020-02-02 12:12:53 +00:00
mainCamera->setCullMask(MWRender::Mask_GUI);
First commit. Wrote an osgviewer, achieved display.
2020-01-09 23:10:09 +00:00
Predictions work perfectly with this. Committing that and clean up later.
2020-02-02 12:12:53 +00:00
osg::Vec4 clearColor = mainCamera->getClearColor();
Split view into worldview and menu, for ease of layer management. Basic pose management.
2020-01-23 23:14:23 +00:00
Simplified render timing by separating rendering from the openxr swapchain, and instead blitting and submitting separately
2020-01-26 19:06:47 +00:00
if (!mXR->realized())
mXR->realize(context);
Predictions work perfectly with this. Committing that and clean up later.
2020-02-02 12:12:53 +00:00
OpenXRSwapchain::Config leftConfig;
leftConfig.width = mXR->impl().mConfigViews[(int)Chirality::LEFT_HAND].recommendedImageRectWidth;
leftConfig.height = mXR->impl().mConfigViews[(int)Chirality::LEFT_HAND].recommendedImageRectHeight;
leftConfig.samples = mXR->impl().mConfigViews[(int)Chirality::LEFT_HAND].recommendedSwapchainSampleCount;
OpenXRSwapchain::Config rightConfig;
rightConfig.width = mXR->impl().mConfigViews[(int)Chirality::RIGHT_HAND].recommendedImageRectWidth;
rightConfig.height = mXR->impl().mConfigViews[(int)Chirality::RIGHT_HAND].recommendedImageRectHeight;
rightConfig.samples = mXR->impl().mConfigViews[(int)Chirality::RIGHT_HAND].recommendedSwapchainSampleCount;
First commit. Wrote an osgviewer, achieved display.
2020-01-09 23:10:09 +00:00
Predictions work perfectly with this. Committing that and clean up later.
2020-02-02 12:12:53 +00:00
auto leftView = new OpenXRWorldView(mXR, "LeftEye", context->getState(), leftConfig, mMetersPerUnit);
auto rightView = new OpenXRWorldView(mXR, "RightEye", context->getState(), rightConfig, mMetersPerUnit);
First commit. Wrote an osgviewer, achieved display.
2020-01-09 23:10:09 +00:00
Predictions work perfectly with this. Committing that and clean up later.
2020-02-02 12:12:53 +00:00
mViews["LeftEye"] = leftView;
mViews["RightEye"] = rightView;
First commit. Wrote an osgviewer, achieved display.
2020-01-09 23:10:09 +00:00
Predictions work perfectly with this. Committing that and clean up later.
2020-02-02 12:12:53 +00:00
auto leftCamera = mCameras["LeftEye"] = leftView->createCamera(0, clearColor, context);
auto rightCamera = mCameras["RightEye"] = rightView->createCamera(1, clearColor, context);
leftCamera->setPreDrawCallback(mPreDraw);
rightCamera->setPreDrawCallback(mPreDraw);
leftCamera->setPostDrawCallback(mPostDraw);
rightCamera->setPostDrawCallback(mPostDraw);
// Stereo cameras should only draw the scene (AR layers should later add minimap, health, etc.)
leftCamera->setCullMask(~MWRender::Mask_GUI);
rightCamera->setCullMask(~MWRender::Mask_GUI);
First commit. Wrote an osgviewer, achieved display.
2020-01-09 23:10:09 +00:00
Predictions work perfectly with this. Committing that and clean up later.
2020-02-02 12:12:53 +00:00
leftCamera->setName("LeftEye");
rightCamera->setName("RightEye");
First commit. Wrote an osgviewer, achieved display.
2020-01-09 23:10:09 +00:00
Predictions work perfectly with this. Committing that and clean up later.
2020-02-02 12:12:53 +00:00
mViewer->addSlave(leftCamera, leftView->projectionMatrix(), leftView->viewMatrix(), true);
mViewer->addSlave(rightCamera, rightView->projectionMatrix(), rightView->viewMatrix(), true);
First commit. Wrote an osgviewer, achieved display.
2020-01-09 23:10:09 +00:00
mViewer->setLightingMode(osg::View::SKY_LIGHT);
mViewer->setReleaseContextAtEndOfFrameHint(false);
Predictions work perfectly with this. Committing that and clean up later.
2020-02-02 12:12:53 +00:00
mCompositionLayerProjectionViews[0].subImage = leftView->swapchain().subImage();
mCompositionLayerProjectionViews[1].subImage = rightView->swapchain().subImage();
Split view into worldview and menu, for ease of layer management. Basic pose management.
2020-01-23 23:14:23 +00:00
OpenXRSwapchain::Config config;
Predictions work perfectly with this. Committing that and clean up later.
2020-02-02 12:12:53 +00:00
config.width = mainCamera->getViewport()->width();
config.height = mainCamera->getViewport()->height();
Split view into worldview and menu, for ease of layer management. Basic pose management.
2020-01-23 23:14:23 +00:00
config.samples = 1;
// Mirror texture doesn't have to be an OpenXR swapchain.
// It's just convenient.
mMirrorTextureSwapchain.reset(new OpenXRSwapchain(mXR, context->getState(), config));
Predictions work perfectly with this. Committing that and clean up later.
2020-02-02 12:12:53 +00:00
auto menuView = new OpenXRMenu(mXR, config, context->getState(), "MainMenu", osg::Vec2(1.f, 1.f));
mViews["MenuView"] = menuView;
auto menuCamera = mCameras["MenuView"] = menuView->createCamera(2, clearColor, context);
menuCamera->setCullMask(MWRender::Mask_GUI);
menuCamera->setName("MenuView");
menuCamera->setPreDrawCallback(mPreDraw);
menuCamera->setPostDrawCallback(mPostDraw);
mViewer->addSlave(menuCamera, true);
mXRSession.reset(new OpenXRSession(mXR));
mViewer->getSlave(0)._updateSlaveCallback = new OpenXRWorldView::UpdateSlaveCallback(mXR, mXRSession.get(), leftView, context);
mViewer->getSlave(1)._updateSlaveCallback = new OpenXRWorldView::UpdateSlaveCallback(mXR, mXRSession.get(), rightView, context);
mainCamera->getGraphicsContext()->setSwapCallback(new OpenXRViewer::SwapBuffersCallback(this));
mainCamera->setGraphicsContext(nullptr);
mXRSession->setLayer(OpenXRLayerStack::WORLD_VIEW_LAYER, this);
mXRSession->setLayer(OpenXRLayerStack::MENU_VIEW_LAYER, dynamic_cast<OpenXRLayer*>(mViews["MenuView"].get()));
First commit. Wrote an osgviewer, achieved display.
2020-01-09 23:10:09 +00:00
mConfigured = true;
Predictions work perfectly with this. Committing that and clean up later.
2020-02-02 12:12:53 +00:00
Split view into worldview and menu, for ease of layer management. Basic pose management.
2020-01-23 23:14:23 +00:00
}
Basic hand tracking.
2020-02-14 21:11:19 +00:00
void OpenXRViewer::blitEyesToMirrorTexture(osg::GraphicsContext* gc, bool includeMenu)
Split view into worldview and menu, for ease of layer management. Basic pose management.
2020-01-23 23:14:23 +00:00
{
mMirrorTextureSwapchain->beginFrame(gc);
Basic hand tracking.
2020-02-14 21:11:19 +00:00
int mirror_width = 0;
if(includeMenu)
mirror_width = mMirrorTextureSwapchain->width() / 3;
else
mirror_width = mMirrorTextureSwapchain->width() / 2;
mViews["LeftEye"]->swapchain().renderBuffer()->blit(gc, 0, 0, mirror_width, mMirrorTextureSwapchain->height());
mViews["RightEye"]->swapchain().renderBuffer()->blit(gc, mirror_width, 0, 2 * mirror_width, mMirrorTextureSwapchain->height());
if(includeMenu)
mViews["MenuView"]->swapchain().renderBuffer()->blit(gc, 2 * mirror_width, 0, 3 * mirror_width, mMirrorTextureSwapchain->height());
mMirrorTextureSwapchain->endFrame(gc);
auto* state = gc->getState();
auto* gl = osg::GLExtensions::Get(state->getContextID(), false);
gl->glBindFramebuffer(GL_FRAMEBUFFER_EXT, 0);
mMirrorTextureSwapchain->renderBuffer()->blit(gc, 0, 0, mMirrorTextureSwapchain->width(), mMirrorTextureSwapchain->height());
Split view into worldview and menu, for ease of layer management. Basic pose management.
2020-01-23 23:14:23 +00:00
}
void
OpenXRViewer::SwapBuffersCallback::swapBuffersImplementation(
osg::GraphicsContext* gc)
{
Predictions work perfectly with this. Committing that and clean up later.
2020-02-02 12:12:53 +00:00
mViewer->mXRSession->swapBuffers(gc);
Split view into worldview and menu, for ease of layer management. Basic pose management.
2020-01-23 23:14:23 +00:00
}
void OpenXRViewer::swapBuffers(osg::GraphicsContext* gc)
{
Simplified render timing by separating rendering from the openxr swapchain, and instead blitting and submitting separately
2020-01-26 19:06:47 +00:00
Split view into worldview and menu, for ease of layer management. Basic pose management.
2020-01-23 23:14:23 +00:00
if (!mConfigured)
return;
Predictions work perfectly with this. Committing that and clean up later.
2020-02-02 12:12:53 +00:00
////// NEW SYSTEM
Timer timer("OpenXRViewer::SwapBuffers");
mViews["LeftEye"]->swapBuffers(gc);
mViews["RightEye"]->swapBuffers(gc);
timer.checkpoint("Views");
Basic hand tracking.
2020-02-14 21:11:19 +00:00
auto eyePoses = mXRSession->predictedPoses().eye;
Predictions work perfectly with this. Committing that and clean up later.
2020-02-02 12:12:53 +00:00
auto leftEyePose = toXR(mViews["LeftEye"]->predictedPose());
auto rightEyePose = toXR(mViews["RightEye"]->predictedPose());
mCompositionLayerProjectionViews[0].pose = leftEyePose;
mCompositionLayerProjectionViews[1].pose = rightEyePose;
timer.checkpoint("Poses");
// TODO: Keep track of these in the session too.
some cleanup
2020-02-02 16:37:43 +00:00
auto stageViews = mXR->impl().getPredictedViews(mXR->impl().frameState().predictedDisplayTime, TrackedSpace::STAGE);
Predictions work perfectly with this. Committing that and clean up later.
2020-02-02 12:12:53 +00:00
mCompositionLayerProjectionViews[0].fov = stageViews[0].fov;
mCompositionLayerProjectionViews[1].fov = stageViews[1].fov;
timer.checkpoint("Fovs");
Split view into worldview and menu, for ease of layer management. Basic pose management.
2020-01-23 23:14:23 +00:00
First commit. Wrote an osgviewer, achieved display.
2020-01-09 23:10:09 +00:00
Predictions work perfectly with this. Committing that and clean up later.
2020-02-02 12:12:53 +00:00
if (!mLayer)
{
mLayer.reset(new XrCompositionLayerProjection);
mLayer->type = XR_TYPE_COMPOSITION_LAYER_PROJECTION;
mLayer->space = mXR->impl().mReferenceSpaceStage;
mLayer->viewCount = 2;
mLayer->views = mCompositionLayerProjectionViews.data();
}
Basic hand tracking.
2020-02-14 21:11:19 +00:00
blitEyesToMirrorTexture(gc);
gc->swapBuffersImplementation();
Split view into worldview and menu, for ease of layer management. Basic pose management.
2020-01-23 23:14:23 +00:00
}
void
OpenXRViewer::RealizeOperation::operator()(
osg::GraphicsContext* gc)
{
OpenXRManager::RealizeOperation::operator()(gc);
mViewer->realize(gc);
}
bool
OpenXRViewer::RealizeOperation::realized()
{
return mViewer->realized();
First commit. Wrote an osgviewer, achieved display.
2020-01-09 23:10:09 +00:00
}
Predictions work perfectly with this. Committing that and clean up later.
2020-02-02 12:12:53 +00:00
void OpenXRViewer::preDrawCallback(osg::RenderInfo& info)
{
auto* camera = info.getCurrentCamera();
auto name = camera->getName();
auto& view = mViews[name];
if (name == "LeftEye")
{
Basic hand tracking.
2020-02-14 21:11:19 +00:00
if (mXR->sessionRunning())
{
mXR->beginFrame();
auto& poses = mXRSession->predictedPoses();
auto menuPose = poses.head[(int)TrackedSpace::STAGE];
mViews["MenuView"]->setPredictedPose(menuPose);
}
Predictions work perfectly with this. Committing that and clean up later.
2020-02-02 12:12:53 +00:00
}
view->prerenderCallback(info);
}
void OpenXRViewer::postDrawCallback(osg::RenderInfo& info)
{
auto* camera = info.getCurrentCamera();
auto name = camera->getName();
auto& view = mViews[name];
view->postrenderCallback(info);
// OSG will sometimes overwrite the predraw callback.
if (camera->getPreDrawCallback() != mPreDraw)
{
camera->setPreDrawCallback(mPreDraw);
Log(Debug::Warning) << ("osg overwrote predraw");
}
}
Basic hand tracking.
2020-02-14 21:11:19 +00:00
void OpenXRViewer::updateTransformNode(osg::Object* object, osg::Object* data)
{
auto* hand_transform = dynamic_cast<SceneUtil::PositionAttitudeTransform*>(object);
if (!hand_transform)
{
Log(Debug::Error) << "Update node was not PositionAttitudeTransform";
return;
}
auto& poses = mXRSession->predictedPoses();
auto handPosesView = poses.hands[(int)TrackedSpace::VIEW];
auto handPosesStage = poses.hands[(int)TrackedSpace::STAGE];
int chirality = (int)Chirality::LEFT_HAND;
if (hand_transform->getName() == "tracker r hand")
chirality = (int)Chirality::RIGHT_HAND;
MWVR::Pose hand = handPosesStage[chirality];
mXR->playerScale(hand);
auto orientation = hand.orientation;
auto position = hand.position;
position = position * mMetersPerUnit;
// Move OpenXR's poses into OpenMW's view by applying the inverse of the rotation of the view matrix.
// This works because OpenXR's conventions match opengl's clip space, thus the inverse of the view matrix converts an OpenXR pose to OpenMW's view space (including world rotation).
// For the hands we don't want the full camera view matrix, but the relative matrix from the player root. So i create a lookat matrix based on osg's conventions.
// TODO: The full camera view matrix could work if i change how animations are overriden.
osg::Matrix lookAt;
lookAt.makeLookAt(osg::Vec3(0, 0, 0), osg::Vec3(0, 1, 0), osg::Vec3(0, 0, 1));
lookAt = osg::Matrix::inverse(lookAt);
orientation = orientation * lookAt.getRotate();
//position = invViewMatrix.preMult(position);
position = lookAt.getRotate() * position;
// Morrowind's meshes do not point forward by default.
// Static since they do not need to be recomputed.
static float VRbias = osg::DegreesToRadians(-90.f);
static osg::Quat yaw(VRbias, osg::Vec3f(0, 1, 0));
static osg::Quat pitch(2.f * VRbias, osg::Vec3f(0, 0, 1));
static osg::Quat roll (-VRbias, osg::Vec3f(1, 0, 0));
orientation = pitch * yaw * orientation;
if (hand_transform->getName() == "tracker r hand")
orientation = roll * orientation;
else
orientation = roll.inverse() * orientation;
// Hand are by default not well-centered
// Note, these numbers are just a rough guess, but seem to work out well.
osg::Vec3 offcenter = osg::Vec3(-0.175, 0., .033);
if (hand_transform->getName() == "tracker r hand")
offcenter.z() *= -1.;
osg::Vec3 recenter = orientation * offcenter;
position = position + recenter * mMetersPerUnit;
hand_transform->setAttitude(orientation);
hand_transform->setPosition(position);
}
bool
OpenXRViewer::TrackedNodeUpdateCallback::run(
osg::Object* object,
osg::Object* data)
{
mViewer->updateTransformNode(object, data);
return traverse(object, data);
}
First commit. Wrote an osgviewer, achieved display.
2020-01-09 23:10:09 +00:00
}
Reference in a new issue Copy permalink