OpenSceneGraph Forum Forum Index OpenSceneGraph Forum
Official forum which mirrors the existent OSG mailing lists. Messages posted here are forwarded to the mailing list and vice versa.
 
   FAQFAQ    SearchSearch    MemberlistMemberlist    RulesRules    UsergroupsUsergroups    RegisterRegister 
 Mail2Forum SettingsMail2Forum Settings  ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 
   AlbumAlbum  OpenSceneGraph IRC ChatOpenSceneGraph IRC Chat   SmartFeedSmartFeed 

Running multiple (independent) viewers in different threads


 
Post new topic   Reply to topic    OpenSceneGraph Forum Forum Index -> General
View previous topic :: View next topic  
Author Message
Aneoshun
Newbie


Joined: 22 May 2019
Posts: 1

PostPosted: Wed May 22, 2019 4:49 pm    Post subject:
Running multiple (independent) viewers in different threads
Reply with quote

Dear OSG Community,

I searched quite a lot through the source code and the beginner guide, but I was unable to find a way to make this work, so I am posting here hoping that you could help me.

I am developing a tool to run physical simulations (of robots) in parallel. This tool is used to run stochastic optimisation (like deep reinforcement learning) algorithms, and for this I need to be able to run several, but fully independent, simulations in parallel thanks to multithreading. This enables to collect data faster and for instance with a 32-core multithreaded CPU the speed-up factor is quite significant.

In each simulation, I have one camera that observes the environment, and which can be used to change the behaviour of the robot running in this simulation. The camera is currently implemented as a pbuffer following closely the osgscreencapture example. The actual code of the camera can be found on github, but apprently I am not allowed to post URLs at the moment.
The multithreading is managed by TBB (an Intel library that I have been using for 6 years now).

If I run my code in a "sequential mode", i.e., all the simulations running one after the other, everything is going well. However, if I run it in parallel, then I have some unstable behaviours. For instance, I can have a segmentation fault happening after a random number of simulations or suddenly after again a random number of simulations, the pbuffer cannot be created anymore (OSG displays "Error: Unable to create pbuffer."). It seems to me that "::osg::ref_ptr<::osg::GraphicsContext> pbuffer = ::osg::GraphicsContext::createGraphicsContext(traits.get()); "
does not work in this setup.

If I remove the cameras (and all OSG components) from my code, it is working well in parallel, but of course I would like to have cameras in my simulations.

Ideally, I would like to run OSG in a single-thread mode and let TBB distribute the different instances of the simulator and OSG in different threads, but even with "setThreadingModel(osgViewer::ViewerBase::ThreadingModel::SingleThreaded);" it does not work.
I tried different threading modes, with the same result.

So here is my question:
What would be the most appropriate way to run several independent viewers and pbuffers, each in a different thread?
The rendered scenes are really simple (no texture, just several primitive shapes) and I am only rendering a couple of frames in each simulation, so I don't need a complex threading model for OSG.


Some information about my system:
- Ubuntu 18.04
- OSG 3.6.4

Thank you in advance for your help.

Best regards,

Antoine
Back to top
View user's profile Send private message
GarfieldPig
Appreciator


Joined: 26 Oct 2018
Posts: 129

PostPosted: Fri May 24, 2019 10:40 am    Post subject:
Running multiple (independent) viewers in differentthreads
Reply with quote

Hi Cully,
It seems like the problem that split window or frame buffer to different part.
There are two way to do this in osg.
1.add slave camera to osgViewer::view,each camera has different graphiccontext.
2.add osgViewer::view to osgViewer::compositeViewer,and one osgViewer::view correspond one camera,and has the same graphiccontext.


method 1 prefer the situation that each camera has different view or proj matrix but has same node tree.like Virtual Reality rendering,or cad design.
method 2 prefer each view(camera) has independent node tree.Like multi-doc edit in Word.


And the thread problem,I think you'd better do update in osg update thread,like add updateCallback to node.By this way,your thread will not have sync problem with osg render thread.


To be honest,I don't pretty understand your problem.
you'd better simplify your code to a problem example.


Qi Yu.
------------------ Original ------------------
From: "Antoine Cully"<>;
Date: Thu, May 23, 2019 00:49 AM
To: "osg-users"<>;

Subject: Running multiple (independent) viewers in differentthreads



Dear OSG Community,

I searched quite a lot through the source code and the beginner guide, but I was unable to find a way to make this work, so I am posting here hoping that you could help me.

I am developing a tool to run physical simulations (of robots) in parallel. This tool is used to run stochastic optimisation (like deep reinforcement learning) algorithms, and for this I need to be able to run several, but fully independent, simulations in parallel thanks to multithreading. This enables to collect data faster and for instance with a 32-core multithreaded CPU the speed-up factor is quite significant.

In each simulation, I have one camera that observes the environment, and which can be used to change the behaviour of the robot running in this simulation. The camera is currently implemented as a pbuffer following closely the osgscreencapture example. The actual code of the camera can be found on github, but apprently I am not allowed to post URLs at the moment.
The multithreading is managed by TBB (an Intel library that I have been using for 6 years now).

If I run my code in a "sequential mode", i.e., all the simulations running one after the other, everything is going well. However, if I run it in parallel, then I have some unstable behaviours. For instance, I can have a segmentation fault happening after a random number of simulations or suddenly after again a random number of simulations, the pbuffer cannot be created anymore (OSG displays "Error: Unable to create pbuffer."). It seems to me that "::osg::ref_ptr<::osg::GraphicsContext> pbuffer = ::osg::GraphicsContext::createGraphicsContext(traits.get()); "
does not work in this setup.

If I remove the cameras (and all OSG components) from my code, it is working well in parallel, but of course I would like to have cameras in my simulations.

Ideally, I would like to run OSG in a single-thread mode and let TBB distribute the different instances of the simulator and OSG in different threads, but even with "setThreadingModel(osgViewer::ViewerBase::ThreadingModel::SingleThreaded);" it does not work.
I tried different threading modes, with the same result.

So here is my question:
What would be the most appropriate way to run several independent viewers and pbuffers, each in a different thread?
The rendered scenes are really simple (no texture, just several primitive shapes) and I am only rendering a couple of frames in each simulation, so I don't need a complex threading model for OSG.


Some information about my system:
- Ubuntu 18.04
- OSG 3.6.4

Thank you in advance for your help.

Best regards,

Antoine

------------------
Read this topic online here:
http://forum.openscenegraph.org/viewtopic.php?p=76134#76134







------------------
Post generated by Mail2Forum
Back to top
View user's profile Send private message
robertosfield
OSG Project Lead


Joined: 18 Mar 2009
Posts: 12324

PostPosted: Fri May 24, 2019 1:21 pm    Post subject:
Running multiple (independent) viewers in different threads
Reply with quote

Hi Antoine,

It's not possible to know what is causing the issues with running your
setup from your description, stack trace of the seg fault is the bare
minimum for starting to guess at what might be the issue. It could be
an OSG bug as you are not using the OSG how it's intended or usually
used, or it could be an error in your application, or in the driver.

As a general note, graphics hardware is geared up for massive
parallelism at the vertex and fragment level but, in particular
OpenGL, is less engineered for multiprocessing at the higher level.
Vulkan is better in this respect but still having many graphics
context and associated threads running is going to contend the memory
and buses.

Quote:
From your description it may be most efficient to have multiple
threads handling the sim side and have it post requests into a thread
safe queue that is read from the graphics thread and have that
graphics thread deal with all the work in series and send it into the
OpenGL FIFO and when the results are provided then signal the original
requests for rendering. This scheme could keep the OSG/OpenGL side
down to using one graphics context and one graphics thread so this
side most efficiently.

A Vulkan based approach could move more of the work to the original
threads requesting the work as they can build their own
VkCommandBuffers that get submitted into a graphics or compute queue
for dispatch to the processing on graphics/compute on the GPU.

While in theory the VSG work could do the above I haven't attempted
this type of work before and have built into any architecture for it
specifically, longer term this is something it'll support. The VSG is
really just in a alpha state at this point and under heavy development
so I wouldn't yet recommend general adoption, it might be worth
thinking about though if rendering you want to do is quite simple but
very specialized.

If you want to continue using the OSG then I'd recommend the queue
approach, with your creating a high level architecture that abstracts
away the work request from the OSG backend that deals with the work.

Cheers,
Robert.

On Fri, 24 May 2019 at 12:17, Antoine Cully <> wrote:
Quote:

Dear OSG Community,

I searched quite a lot through the source code and the beginner guide, but I was unable to find a way to make this work, so I am posting here hoping that you could help me.

I am developing a tool to run physical simulations (of robots) in parallel. This tool is used to run stochastic optimisation (like deep reinforcement learning) algorithms, and for this I need to be able to run several, but fully independent, simulations in parallel thanks to multithreading. This enables to collect data faster and for instance with a 32-core multithreaded CPU the speed-up factor is quite significant.

In each simulation, I have one camera that observes the environment, and which can be used to change the behaviour of the robot running in this simulation. The camera is currently implemented as a pbuffer following closely the osgscreencapture example. The actual code of the camera can be found on github, but apprently I am not allowed to post URLs at the moment.
The multithreading is managed by TBB (an Intel library that I have been using for 6 years now).

If I run my code in a "sequential mode", i.e., all the simulations running one after the other, everything is going well. However, if I run it in parallel, then I have some unstable behaviours. For instance, I can have a segmentation fault happening after a random number of simulations or suddenly after again a random number of simulations, the pbuffer cannot be created anymore (OSG displays "Error: Unable to create pbuffer."). It seems to me that "::osg::ref_ptr<::osg::GraphicsContext> pbuffer = ::osg::GraphicsContext::createGraphicsContext(traits.get()); "
does not work in this setup.

If I remove the cameras (and all OSG components) from my code, it is working well in parallel, but of course I would like to have cameras in my simulations.

Ideally, I would like to run OSG in a single-thread mode and let TBB distribute the different instances of the simulator and OSG in different threads, but even with "setThreadingModel(osgViewer::ViewerBase::ThreadingModel::SingleThreaded);" it does not work.
I tried different threading modes, with the same result.

So here is my question:
What would be the most appropriate way to run several independent viewers and pbuffers, each in a different thread?
The rendered scenes are really simple (no texture, just several primitive shapes) and I am only rendering a couple of frames in each simulation, so I don't need a complex threading model for OSG.


Some information about my system:
- Ubuntu 18.04
- OSG 3.6.4

Thank you in advance for your help.

Best regards,

Antoine

------------------
Read this topic online here:
http://forum.openscenegraph.org/viewtopic.php?p=76134#76134








------------------
Post generated by Mail2Forum
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic   Reply to topic    OpenSceneGraph Forum Forum Index -> General All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You cannot download files in this forum

Similar Topics
Topic Author Forum Replies Posted
No new posts Order Independent Transparency and os... beingbad@163.com General 3 Fri Jun 14, 2019 7:56 am View latest post
No new posts Add multiple OverlayNode. osgsim General 0 Sun Jun 09, 2019 2:30 am View latest post
No new posts osgText crash with multiple Viewers ravidavi General 3 Tue Dec 18, 2018 8:50 am View latest post
No new posts Render multiple PRE_RENDER cameras to... ivar General 3 Tue Nov 13, 2018 3:41 pm View latest post
No new posts Rendering multiple screens njr3 General 1 Wed Oct 31, 2018 8:46 pm View latest post


Board Security Anti Bot Question MOD - phpBB MOD against Spam Bots
Powered by phpBB © 2001, 2005 phpBB Group
Protected by Anti-Spam ACP