OpenSceneGraph Forum Forum Index OpenSceneGraph Forum
Official forum which mirrors the existent OSG mailing lists. Messages posted here are forwarded to the mailing list and vice versa.
 
   FAQFAQ    SearchSearch    MemberlistMemberlist    RulesRules    UsergroupsUsergroups    RegisterRegister 
 Mail2Forum SettingsMail2Forum Settings  ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 
   AlbumAlbum  OpenSceneGraph IRC ChatOpenSceneGraph IRC Chat   SmartFeedSmartFeed 

Multi-threaded usage of textures


 
Post new topic   Reply to topic    OpenSceneGraph Forum Forum Index -> Submission
View previous topic :: View next topic  
Author Message
anishuz (Anish Thomas)
User


Joined: 24 Jul 2012
Posts: 25

PostPosted: Tue Oct 17, 2017 8:38 am    Post subject:
Multi-threaded usage of textures
Reply with quote

Hi,

While modifying texture-atlases in DBPager threads we noticed an issue of thread-synchronization.

Code like the snippet below (From Texture2D.cpp):

else if (_image.valid() && getModifiedCount(contextID) != _image->getModifiedCount())
{
applyTexImage2D_subload(state,GL_TEXTURE_2D,_image.get(),
_textureWidth, _textureHeight, _internalFormat, _numMipmapLevels);

// update the modified tag to show that it is up to date.
getModifiedCount(contextID) = _image->getModifiedCount();

}

If another thread (like a DBPager thread) modifies the image of a texture (in this case a texture-atlas which gets more and more images added to it while it is also being applied on geometry), the modified count changes made by one or more dirty() calls can get missed.

If the image got dirtied before the RenderThread tried to apply this texture, code-execution enters the if() described above. If the texture upload takes too long and the image is dirtied again, the effect of the 2nd dirty can be missed when the modified count of the image is put into the texture.

A simple solution would be to use a local variable to store the image's modified-count, and apply it to the texture. The overhead of a possible redundant texture-upload on the next apply would be down-side. But texture-atlases would support multi-threaded updates better.

Thoughts?

Thank you!

Cheers,
Anish
Back to top
View user's profile Send private message
robertosfield
OSG Project Lead


Joined: 18 Mar 2009
Posts: 11910

PostPosted: Tue Oct 17, 2017 8:52 am    Post subject:
Multi-threaded usage of textures
Reply with quote

Hi Anish,



I think you are correct, the way to handle concurrrent reading/updates of an osg::Image is to take a local copy of the osg::Image::getModifedCount() prior to reading from the image and then use this value.  This would still have the danger of the image data being updated at the same time as it's being read but at least a texture update wouldn't be missed.


Robert.



On 17 October 2017 at 09:38, Anish Thomas < (
Only registered users can see emails on this board!
Get registred or enter the forums!
)> wrote:
Quote:
Hi,

While modifying texture-atlases in DBPager threads we noticed an issue of thread-synchronization.

Code like the snippet below (From Texture2D.cpp):

else if (_image.valid() && getModifiedCount(contextID) != _image->getModifiedCount())
{
            applyTexImage2D_subload(state,GL_TEXTURE_2D,_image.get(),
                                    _textureWidth, _textureHeight, _internalFormat, _numMipmapLevels);

            // update the modified tag to show that it is up to date.
            getModifiedCount(contextID) = _image->getModifiedCount();

}

If another thread (like a DBPager thread) modifies the image of a texture (in this case a texture-atlas which gets more and more images added to it while it is also being applied on geometry), the modified count changes made by one or more dirty() calls can get missed.

If the image got dirtied before the RenderThread tried to apply this texture, code-execution enters the if() described above. If the texture upload takes too long and the image is dirtied again, the effect of the 2nd dirty can be missed when the modified count of the image is put into the texture.

A simple solution would be to use a local variable to store the image's modified-count, and apply it to the texture. The overhead of a possible redundant texture-upload on the next apply would be down-side. But texture-atlases would support multi-threaded updates better.

Thoughts?

Thank you!

Cheers,
Anish

------------------
Read this topic online here:
http://forum.openscenegraph.org/viewtopic.php?p=72188#72188





_______________________________________________
osg-submissions mailing list
(
Only registered users can see emails on this board!
Get registred or enter the forums!
)
http://lists.openscenegraph.org/listinfo.cgi/osg-submissions-openscenegraph.org


------------------
Post generated by Mail2Forum
Back to top
View user's profile Send private message
anishuz (Anish Thomas)
User


Joined: 24 Jul 2012
Posts: 25

PostPosted: Tue Oct 17, 2017 9:01 am    Post subject:
Re: Multi-threaded usage of textures
Reply with quote

Hi Robert,

I can submit a fix then.
Thanks for the quick reply.

Cheers,
Anish

robertosfield wrote:
Hi Anish,



I think you are correct, the way to handle concurrrent reading/updates of an osg::Image is to take a local copy of the osg::Image::getModifedCount() prior to reading from the image and then use this value.  This would still have the danger of the image data being updated at the same time as it's being read but at least a texture update wouldn't be missed.


Robert.



On 17 October 2017 at 09:38, Anish Thomas < (
Only registered users can see emails on this board!
Get registred or enter the forums!
)> wrote:
Quote:
Hi,

While modifying texture-atlases in DBPager threads we noticed an issue of thread-synchronization.

Code like the snippet below (From Texture2D.cpp):

else if (_image.valid() && getModifiedCount(contextID) != _image->getModifiedCount())
{
            applyTexImage2D_subload(state,GL_TEXTURE_2D,_image.get(),
                                    _textureWidth, _textureHeight, _internalFormat, _numMipmapLevels);

            // update the modified tag to show that it is up to date.
            getModifiedCount(contextID) = _image->getModifiedCount();

}

If another thread (like a DBPager thread) modifies the image of a texture (in this case a texture-atlas which gets more and more images added to it while it is also being applied on geometry), the modified count changes made by one or more dirty() calls can get missed.

If the image got dirtied before the RenderThread tried to apply this texture, code-execution enters the if() described above. If the texture upload takes too long and the image is dirtied again, the effect of the 2nd dirty can be missed when the modified count of the image is put into the texture.

A simple solution would be to use a local variable to store the image's modified-count, and apply it to the texture. The overhead of a possible redundant texture-upload on the next apply would be down-side. But texture-atlases would support multi-threaded updates better.

Thoughts?

Thank you!

Cheers,
Anish

------------------
Read this topic online here:
http://forum.openscenegraph.org/viewtopic.php?p=72188#72188





_______________________________________________
osg-submissions mailing list
(
Only registered users can see emails on this board!
Get registred or enter the forums!
)
http://lists.openscenegraph.org/listinfo.cgi/osg-submissions-openscenegraph.org


------------------
Post generated by Mail2Forum
Back to top
View user's profile Send private message
Eric Sokolowsky
Guest





PostPosted: Tue Oct 17, 2017 11:30 am    Post subject:
Multi-threaded usage of textures
Reply with quote

This could apply to a problem I have been having in my own application. Please post your fix once you have it.

On Oct 17, 2017 5:01 AM, "Anish Thomas" < (
Only registered users can see emails on this board!
Get registred or enter the forums!
)> wrote:
Quote:
Hi Robert,

I can submit a fix then.
Thanks for the quick reply.

Cheers,
Anish


robertosfield wrote:
Quote:
Hi Anish,



I think you are correct, the way to handle concurrrent reading/updates of an osg::Image is to take a local copy of the osg::Image::getModifedCount() prior to reading from the image and then use this value.  This would still have the danger of the image data being updated at the same time as it's being read but at least a texture update wouldn't be missed.


Robert.



On 17 October 2017 at 09:38, Anish Thomas < ()> wrote:

Quote:
Hi,

While modifying texture-atlases in DBPager threads we noticed an issue of thread-synchronization.

Code like the snippet below (From Texture2D.cpp):

else if (_image.valid() && getModifiedCount(contextID) != _image->getModifiedCount())
{
            applyTexImage2D_subload(state,GL_TEXTURE_2D,_image.get(),
                                    _textureWidth, _textureHeight, _internalFormat, _numMipmapLevels);

            // update the modified tag to show that it is up to date.
            getModifiedCount(contextID) = _image->getModifiedCount();

}

If another thread (like a DBPager thread) modifies the image of a texture (in this case a texture-atlas which gets more and more images added to it while it is also being applied on geometry), the modified count changes made by one or more dirty() calls can get missed.

If the image got dirtied before the RenderThread tried to apply this texture, code-execution enters the if() described above. If the texture upload takes too long and the image is dirtied again, the effect of the 2nd dirty can be missed when the modified count of the image is put into the texture.

A simple solution would be to use a local variable to store the image's modified-count, and apply it to the texture. The overhead of a possible redundant texture-upload on the next apply would be down-side. But texture-atlases would support multi-threaded updates better.

Thoughts?

Thank you!

Cheers,
Anish

------------------
Read this topic online here:
http://forum.openscenegraph.org/viewtopic.php?p=72188#72188 (http://forum.openscenegraph.org/viewtopic.php?p=72188#72188)





_______________________________________________
osg-submissions mailing list
  ()
http://lists.openscenegraph.org/listinfo.cgi/osg-submissions-openscenegraph.org (http://lists.openscenegraph.org/listinfo.cgi/osg-submissions-openscenegraph.org)



  ------------------
Post generated by Mail2Forum


------------------
Read this topic online here:
http://forum.openscenegraph.org/viewtopic.php?p=72191#72191





_______________________________________________
osg-submissions mailing list
(
Only registered users can see emails on this board!
Get registred or enter the forums!
)
http://lists.openscenegraph.org/listinfo.cgi/osg-submissions-openscenegraph.org


------------------
Post generated by Mail2Forum
Back to top
anishuz (Anish Thomas)
User


Joined: 24 Jul 2012
Posts: 25

PostPosted: Thu Oct 19, 2017 5:31 am    Post subject:
Fixes based on the 3.2 branch
Reply with quote

Hi,

I've attached the original and the fixed versions of the texture-handling files based on the head of the 3.2 branch.

Thank you!

Cheers,
Anish
Back to top
View user's profile Send private message
robertosfield
OSG Project Lead


Joined: 18 Mar 2009
Posts: 11910

PostPosted: Thu Oct 19, 2017 8:00 pm    Post subject:
Multi-threaded usage of textures
Reply with quote

H Anish,


Thanks for the fixes, I've merged them with the OpenSceneGraph-3.2 branch and then applied similar changes to OpenSceneGraph-3.4 branch and master, I couldn't apply the changes directly to these as the code base has changed a little too much to make a direct merge easy.


Could you test out which versions you can and let me know if things are now working.


Thanks,

Robert.


On 19 October 2017 at 06:31, Anish Thomas < (
Only registered users can see emails on this board!
Get registred or enter the forums!
)> wrote:
Quote:
Hi,

I've attached the original and the fixed versions of the texture-handling files based on the head of the 3.2 branch.

Thank you!

Cheers,
Anish

------------------
Read this topic online here:
http://forum.openscenegraph.org/viewtopic.php?p=72199#72199




_______________________________________________
osg-submissions mailing list
(
Only registered users can see emails on this board!
Get registred or enter the forums!
)
http://lists.openscenegraph.org/listinfo.cgi/osg-submissions-openscenegraph.org



------------------
Post generated by Mail2Forum
Back to top
View user's profile Send private message
Ulrich Hertlein
Guest





PostPosted: Thu Oct 26, 2017 6:50 pm    Post subject:
Multi-threaded usage of textures
Reply with quote

Hi Anish,

Looking at the fix I don't think this works in all cases, it only makes it less likely to
occur.

The check and the set are still done non-atomically, so another thread could be scheduled
inbetween and one update would be lost.

My preferred solution would be to use std::atomic with exchange for the modifiedCount, but
I don't know if that's available across all compilers that OSG supports. It's been in
since C++11 though so I would hope it's safe to use.

Cheers,
/ulrich

On 19/10/17 07:31, Anish Thomas wrote:
Quote:
Hi,

I've attached the original and the fixed versions of the texture-handling files based on the head of the 3.2 branch.

Thank you!

Cheers,
Anish




------------------
Post generated by Mail2Forum
Back to top
anishuz (Anish Thomas)
User


Joined: 24 Jul 2012
Posts: 25

PostPosted: Fri Oct 27, 2017 4:56 am    Post subject:
Re: Multi-threaded usage of textures
Reply with quote

Hi Ulrich,

The updating of a texture's images happen before the image's modified-count is updated by the dirtying thread. This means that even if a third thread updates the image in the mean-time, this new update doesn't get missed from being uploaded to VRAM as the texture-upload happens *after* the modified count is read into the texture from its image. Therefore the updated texture in VRAM should reflect the 2nd update as well even if the modified-count is only from the 1st update. The only drawback I see is that a redundant texture upload could occur in a future texture apply().

Of course, since textures were not meant for multi-threaded updates, the application I'm working on uses a central texture container which prevents more than one DBPager thread from updating the same texture simultaneously. The fix was only to also synchronize the RenderThread too.

Cheers,
Anish

Ulrich Hertlein wrote:
Hi Anish,

Looking at the fix I don't think this works in all cases, it only makes it less likely to
occur.

The check and the set are still done non-atomically, so another thread could be scheduled
inbetween and one update would be lost.

My preferred solution would be to use std::atomic with exchange for the modifiedCount, but
I don't know if that's available across all compilers that OSG supports. It's been in
since C++11 though so I would hope it's safe to use.

Cheers,
/ulrich

On 19/10/17 07:31, Anish Thomas wrote:
Quote:
Hi,

I've attached the original and the fixed versions of the texture-handling files based on the head of the 3.2 branch.

Thank you!

Cheers,
Anish




------------------
Post generated by Mail2Forum
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic   Reply to topic    OpenSceneGraph Forum Forum Index -> Submission All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You cannot download files in this forum

Similar Topics
Topic Author Forum Replies Posted
No new posts Synchronizing with textures uploads. Tini General 11 Wed Mar 07, 2018 9:03 pm View latest post
No new posts Multiple video textures using ffmpeg ... mmaurus Plugins [osgPlugins] 6 Thu Oct 05, 2017 10:02 am View latest post
No new posts How to pre-compile shaders before act... kornerr General 1 Thu Sep 28, 2017 8:42 pm View latest post
No new posts Is it possible to clear just certain ... amudhan79 General 3 Mon Sep 11, 2017 7:48 pm View latest post
No new posts Multi layer effect rendering problem ... not3not4 General 0 Wed Jul 19, 2017 9:44 am View latest post


Board Security Anti Bot Question MOD - phpBB MOD against Spam Bots
Powered by phpBB © 2001, 2005 phpBB Group
Protected by Anti-Spam ACP