OpenSceneGraph Forum Forum Index OpenSceneGraph Forum
Official forum which mirrors the existent OSG mailing lists. Messages posted here are forwarded to the mailing list and vice versa.
 
   FAQFAQ    SearchSearch    MemberlistMemberlist    RulesRules    UsergroupsUsergroups    RegisterRegister 
 Mail2Forum SettingsMail2Forum Settings  ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 
   AlbumAlbum  OpenSceneGraph IRC ChatOpenSceneGraph IRC Chat   SmartFeedSmartFeed 

Loading problem, OpenVR integration


 
Post new topic   Reply to topic    OpenSceneGraph Forum Forum Index -> General
View previous topic :: View next topic  
Author Message
Oneiros
Newbie


Joined: 02 Nov 2016
Posts: 5

PostPosted: Wed Nov 02, 2016 4:59 pm    Post subject:
Loading problem, OpenVR integration
Reply with quote

Hi,

I'm working on a Virtual Reality project that involves the rendering of a OSG scene in an Oculus Rift CV1 device using the OpenVR library. All these technologies are pretty new to me so everything is as exciting as frustrating Smile
My code is based on the Chris Denham "osgopenvrviewer" project (you can find it on Github)

So, my starting point is a osgViewer::View that contains my immersive scene. I managed to render it in the headset display by doing this:

Code:
osg::ref_ptr<OpenVRViewer> vrViewer = new OpenVRViewer(myView, myDevice, myRealizeOperation);
osg::ref_ptr<osg::Node> sceneData = myView->getSceneData();
vrViewer->addChild(sceneData);
myView->setSceneData(vrViewer);


So far so good, the render quality and the stereo effect are perfect.
The problems start when I move my head to look around in the scene: it looks like the objects outside my first view frustum are not ready when the application starts, so I get serious frame drops when I see these objects for the first time. These frame drops are particulary annoying in the headset, where a frame drop freezes the view for a fraction of second multiple times, causing a very ugly effect.

This happens only the first time I look around: after a first "scan" of the scene, everything works smoothly and nice.

The system I'm working on started as a simple desktop application with a single view/camera and I couldn't reproduce these problems in the no-VR version. I can look around very fast using a gamepad without losing any frame.
So, I guess the problem is in the OpenVR integration.

Is there anyone who already had this kind of issues? Do you have an idea on how to manage pre-loading of the entire scene before the realtime simulation starts?

Thank you!

Cheers,
Lorenzo

P.S. I see that many people solved this problem using osgViewer::Renderer->setCompileOnNextDraw(true) but I think it is not enough in my case. I have two RTT slave cameras for VR rendering (left eye and right eye) and I saw that the "setSceneData" method internally calls "setCompileOnNextDraw" on every slave camera in the viewer.
Back to top
View user's profile Send private message
robertosfield
OSG Project Lead


Joined: 18 Mar 2009
Posts: 11275

PostPosted: Thu Nov 03, 2016 8:10 am    Post subject:
Loading problem, OpenVR integration
Reply with quote

Hi Lorenzo,

The performance constraints on immersive stereo display systems are
pretty well the tightest in graphics, even small glitchs can be lead
to unpleasant experience. Couple this with rendering more than twice
as much data (two eyes and distortion correction passes) means that
you don't have much wiggle room to make sure things run sweetly. Good
VR isn't easy so you'll need to be patient with learning the issues
and how to resolve them, as a dev new to the area you've jumped in the
deep end :-)

Is the scene static or is your application adding new scene graph
objects through the lifetime of the application?

It's only new objects that you need to compile to make sure that the
graphics objects are created and ready to download to the GPU?

The other issue might be that the graphic objects have are all
compiled already but not download to the GPU. Also if the GPU memory
is full the driver has to start moving graphics objects in/out of GPU
memory on the fly which can cause big frame drops.

What specifically you need to do to resolve the problem depends
entirely where the actual bottleneck is, we can't answer this without
knowing more about what is happening your scene graph, how much
headroom you have on the GPU to handle the extra memory usage,
bandwidth and compute work.

Robert.

On 2 November 2016 at 17:00, Lorenzo Valente <> wrote:
Quote:
Hi,

I'm working on a Virtual Reality project that involves the rendering of a OSG scene in an Oculus Rift CV1 device using the OpenVR library. All these technologies are pretty new to me so everything is as exciting as frustrating Smile
My code is based on the Chris Denham "osgopenvrviewer" project (you can find it on Github)

So, my starting point is a osgViewer::View that contains my immersive scene. I managed to render it in the headset display by doing this:


Code:
osg::ref_ptr<OpenVRViewer> vrViewer = new OpenVRViewer(myView, myDevice, myRealizeOperation);
osg::ref_ptr<osg::Node> sceneData = myView->getSceneData();
vrViewer->addChild(sceneData);
myView->setSceneData(vrViewer);



So far so good, the render quality and the stereo effect are perfect.
The problems start when I move my head to look around in the scene: it looks like the objects outside my first view frustum are not ready when the application starts, so I get serious frame drops when I see these objects for the first time. These frame drops are particulary annoying in the headset, where a frame drop freezes the view for a fraction of second multiple times, causing a very ugly effect.

This happens only the first time I look around: after a first "scan" of the scene, everything works smoothly and nice.

The system I'm working on started as a simple desktop application with a single view/camera and I couldn't reproduce these problems in the no-VR version. I can look around very fast using a gamepad without losing any frame.
So, I guess the problem is in the OpenVR integration.

Is there anyone who already had this kind of issues? Do you have an idea on how to manage pre-loading of the entire scene before the realtime simulation starts?

Thank you!

Cheers,
Lorenzo

P.S. I see that many people solved this problem using osgViewer::Renderer->setCompileOnNextDraw(true) but I think it is not enough in my case. I have two RTT slave cameras for VR rendering (left eye and right eye) and I saw that the "setSceneData" method internally calls "setCompileOnNextDraw" on every slave camera in the viewer.

------------------
Read this topic online here:
http://forum.openscenegraph.org/viewtopic.php?p=69205#69205








------------------
Post generated by Mail2Forum
Back to top
View user's profile Send private message
Oneiros
Newbie


Joined: 02 Nov 2016
Posts: 5

PostPosted: Thu Nov 03, 2016 8:49 am    Post subject:
Re: Loading problem, OpenVR integration
Reply with quote

Hi Robert! Thank you for your kind reply.

robertosfield wrote:
Is the scene static or is your application adding new scene graph
objects through the lifetime of the application?


The scene is completely static and motionless Smile

robertosfield wrote:
It's only new objects that you need to compile to make sure that the
graphics objects are created and ready to download to the GPU


Ok, perfect. I'm pretty sure that the compilation process is already performed on the entire scene by the system I'm working on.
Do you think that my problem could be caused by a missing compilation of objects? If you say that it might be the case, I will investigate more on this.

robertosfield wrote:
The other issue might be that the graphic objects have are all
compiled already but not download to the GPU. Also if the GPU memory
is full the driver has to start moving graphics objects in/out of GPU
memory on the fly which can cause big frame drops.


Do you think this might be the case? Is there a way in OSG to tell if an object is already in the GPU memory? And to push them there?
I remind you that once I scan the all scene with my vision the problem disappears and I can look around without any frame drop.
So I guess that the GPU memory is enough to handle the whole scene, I just need to force some sort of "pre-loading" of ALL objects.

robertosfield wrote:
What specifically you need to do to resolve the problem depends
entirely where the actual bottleneck is, we can't answer this without
knowing more about what is happening your scene graph, how much
headroom you have on the GPU to handle the extra memory usage,
bandwidth and compute work.


As I said, my conclusion is that I don't have any actual system bottleneck, I just have to get rid of this lazy loading process Smile
I just need to tell the GPU "hey, here's my scene, I know you can handle it all, just pre-process it in one shot and don't wait for me to look around!"
But I could be completely wrong and I'd love to know what you guys think about it. I want to 1) learn more about OSG and VR 2) solve this annoying problem.
Back to top
View user's profile Send private message
robertosfield
OSG Project Lead


Joined: 18 Mar 2009
Posts: 11275

PostPosted: Thu Nov 03, 2016 9:37 am    Post subject:
Loading problem, OpenVR integration
Reply with quote

Hi Lorenzo,

Quote:
From your description it's sounds like either objects aren't
pre-compiled or that they have been pre-compiled but not yet
downloaded by the GPU.

In OpenGL there is no way to formally force a GL object to be download
to the GPU, it's up to the drive to decide what it does when. You can
however leverage the fact that texture and vertex buffer objects will
be made resident on the GPU by rendering them and as long as there is
enough memory to keep them around they'll be left resident by the
driver for when you actually need them.

The osgUtil::GLObjectsVisitor just does compilation it doesn't do any
dummy rendering to force the download to the GPU. The
osgUtil::IncrementalCompileOperation does has an option for rendering
an dummy geometry to force the download of textures, but this is not
on by default, and only used during incremental compilation - i.e.
when using a paged database.

One possible route you could take would be to subclass from
GLObjectVisitor and add in a geometry in a similar way to what the
IncrementalCompileOpeation does and invoke this via a custom
RealizeOperation (see the osgvolume example to see how
RealizeOperation can be used.) You'd need to make sure that your scene
graph is assigned to the viewer before the windows are realized
though,

When rendering a force download dymmy geometry you'd want to do it in
a way the geometry rendering doesn't affect the framebuffer, either by
doing prior to a clear of the graphics window or by ensuring all
geometry is out of the view frustum or discarding the fragments.

Robert.


------------------
Post generated by Mail2Forum
Back to top
View user's profile Send private message
SMesserschmidt (Sebastian Messerschmidt)
Forum Moderator


Joined: 10 Sep 2013
Posts: 799

PostPosted: Thu Nov 03, 2016 10:20 am    Post subject:
Loading problem, OpenVR integration
Reply with quote

Hi,


Quote:
Hi Robert! Thank you for your kind reply.


robertosfield wrote:
Quote:
Is the scene static or is your application adding new scene graph
objects through the lifetime of the application?


The scene is completely static and motionless :)


robertosfield wrote:
Quote:
It's only new objects that you need to compile to make sure that the
graphics objects are created and ready to download to the GPU


Ok, perfect. I'm pretty sure that the compilation process is already performed on the entire scene by the system I'm working on.
Do you think that my problem could be caused by a missing compilation of objects? If you say that it might be the case, I will investigate more on this.
I struggled with some instance data not being sent to the GPU (2
gigabytes worth of data textures) resulting in stutter. I ended up using:
viewer->getDatabasePager()->setDoPreCompile(true);

which solved the problem for me, as I preload the scene anyways. So the
stutter is converted to a long pause before the first frame is displayed.

Maybe Robert can explain what is happening when using the pre-compiled
option is used. It helped with the textures for sure.

Cheers
Sebastian


------------------
Post generated by Mail2Forum
Back to top
View user's profile Send private message
robertosfield
OSG Project Lead


Joined: 18 Mar 2009
Posts: 11275

PostPosted: Thu Nov 03, 2016 11:08 am    Post subject:
Loading problem, OpenVR integration
Reply with quote

On 3 November 2016 at 10:19, Sebastian Messerschmidt
<> wrote:
Quote:
I struggled with some instance data not being sent to the GPU (2 gigabytes
worth of data textures) resulting in stutter. I ended up using:
viewer->getDatabasePager()->setDoPreCompile(true);

This only affects the handling of paged database, it has no effect on
static databases.

Quote:
Maybe Robert can explain what is happening when using the pre-compiled
option is used. It helped with the textures for sure.

I've already gone into enough detail for the topic in hand. The only
great detail I can provide on pre-compilation is the read the code and
spell it out line by line. OR... you could just open up an editor and
read the code yourself.

Robert.


------------------
Post generated by Mail2Forum
Back to top
View user's profile Send private message
Pelle Nordqvist
Guest





PostPosted: Thu Nov 03, 2016 12:49 pm    Post subject:
Loading problem, OpenVR integration
Reply with quote

Hi Lorenzo, maybe you want to check out this thread:

http://forum.openscenegraph.org/viewtopic.php?t=14222


For me it helped by simply 'parading' all objects in front of the camera
at the start of the simulation, this forces the push to GPU needed,
as long as the data fits on the card.
I don't think there is any other way to really force this push

for a static database.



Cheers,


/Pelle


On 3 November 2016 at 12:08, Robert Osfield < (
Only registered users can see emails on this board!
Get registred or enter the forums!
)> wrote:
Quote:
On 3 November 2016 at 10:19, Sebastian Messerschmidt
< (
Only registered users can see emails on this board!
Get registred or enter the forums!
)> wrote:
Quote:
I struggled with some instance data not being sent to the GPU (2 gigabytes
worth of data textures) resulting in stutter. I ended up using:
viewer->getDatabasePager()->setDoPreCompile(true);

This only affects the handling of paged database, it has no effect on
static databases.

Quote:
Maybe Robert can explain what is happening when using the pre-compiled
option is used. It helped with the textures for sure.

I've already gone into enough detail for the topic in hand. The only
great detail I can provide on pre-compilation is the read the code and
spell it out line by line.  OR... you could just open up an editor and
read the code yourself.

Robert.
_______________________________________________
osg-users mailing list
(
Only registered users can see emails on this board!
Get registred or enter the forums!
)
http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org




------------------
Post generated by Mail2Forum
Back to top
SMesserschmidt (Sebastian Messerschmidt)
Forum Moderator


Joined: 10 Sep 2013
Posts: 799

PostPosted: Thu Nov 03, 2016 1:08 pm    Post subject:
Loading problem, OpenVR integration
Reply with quote

Am 11/3/2016 um 12:08 PM schrieb Robert Osfield:
Quote:
On 3 November 2016 at 10:19, Sebastian Messerschmidt
<> wrote:
Quote:
I struggled with some instance data not being sent to the GPU (2 gigabytes
worth of data textures) resulting in stutter. I ended up using:
viewer->getDatabasePager()->setDoPreCompile(true);

This only affects the handling of paged database, it has no effect on
static databases.
You're right. Forgot to mention, that the instances are paged via PagedLOD.

Quote:

Quote:
Maybe Robert can explain what is happening when using the pre-compiled
option is used. It helped with the textures for sure.

I've already gone into enough detail for the topic in hand. The only
great detail I can provide on pre-compilation is the read the code and
spell it out line by line. OR... you could just open up an editor and
read the code yourself.

It wasn't meant as an offense or a request out of laziness. I simply
wanted to avoid to spread non-sense. I'm quite comfortable with reading
code, but that doesn't mean that there is a bigger picture one might miss.

Cheers
Sebastian



Quote:

Robert.




------------------
Post generated by Mail2Forum
Back to top
View user's profile Send private message
Oneiros
Newbie


Joined: 02 Nov 2016
Posts: 5

PostPosted: Fri Nov 04, 2016 8:49 am    Post subject:
Re: Loading problem, OpenVR integration
Reply with quote

Problem solved! Very Happy
As guessed by Robert, the GL objects were not compiled properly.

In the old desktop-version of the software the compilation of the scene was performed by a osgUtil::GLObjectsVisitor created and launched in a subclass of osg::Camera::DrawCallback, which was assigned to the main camera as a "preDrawCallback" (I don't know if this is the right approach but it works and I don't want to mess up with the old system Smile ).
It works fine but it relies on the renderer of a single main camera, the very camera that osgopenvrviewer deactivates! It means that in the VR environment that preDrawCallback was never called and the compilation was never performed.

The solution? I simply assigned the same callback also to every slave camera present in the view (in the osgopenvrviewer case these are two RTT cameras, one for the left eye and one for the right eye):
Code:
for (unsigned i = 0; i < view->getNumSlaves(); ++i)
{
   osgViewer::View::Slave& slave = view->getSlave(i);
   if (slave._camera.valid())
   {
         slave._camera->setPostDrawCallback(compilationCallback);
   }
}


I used "setPostDrawCallback" because a preDrawCallback was apparently already assigned (replacing it resulted in a full deactivation of the rendering, black screen).
Is this wrong on any level? It works like a charm, no more frame drops Very Happy

Thank you very much guys, your help pointed me to the right direction.
A special thanks to Robert!

Cheers!
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic   Reply to topic    OpenSceneGraph Forum Forum Index -> General All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You cannot download files in this forum

Similar Topics
Topic Author Forum Replies Posted
No new posts ImGui integration Flogo General [3rdparty] 3 Thu Nov 09, 2017 11:45 am View latest post
No new posts Osg 3.4.1 - Tutorial 12 Fails on Mac ... digitorus General 1 Sat Nov 04, 2017 12:32 pm View latest post
No new posts [CubeMap with mipmaps] About loading IBL mp3butcher General 3 Sat Nov 04, 2017 12:37 am View latest post
No new posts BVH and model axis centering: a model... Paradox Plugins [osgPlugins] 0 Mon Oct 16, 2017 4:27 pm View latest post
No new posts Problem With Osg Texture Colors! Bobykhani General [forum] 1 Tue Oct 10, 2017 8:20 am View latest post


Board Security Anti Bot Question MOD - phpBB MOD against Spam Bots
Powered by phpBB © 2001, 2005 phpBB Group
Protected by Anti-Spam ACP