OpenSceneGraph Forum Forum Index OpenSceneGraph Forum
Official forum which mirrors the existent OSG mailing lists. Messages posted here are forwarded to the mailing list and vice versa.
 
   FAQFAQ    SearchSearch    MemberlistMemberlist    RulesRules    UsergroupsUsergroups    RegisterRegister 
 Mail2Forum SettingsMail2Forum Settings  ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 
   AlbumAlbum  OpenSceneGraph IRC ChatOpenSceneGraph IRC Chat   SmartFeedSmartFeed 

Unbounded growth in OpenFlight ReaderWriter


 
Post new topic   Reply to topic    OpenSceneGraph Forum Forum Index -> General
View previous topic :: View next topic  
Author Message
Daniel Emminizer, Code...
Guest





PostPosted: Tue Jul 31, 2018 9:55 am    Post subject:
Unbounded growth in OpenFlight ReaderWriter
Reply with quote

Hello All,

I've found what appears to be unbounded growth in the OpenFlight loader. I have a work-around but wanted to run it past the community since there are many here who are more familiar with the format and the loader than me.

I'm seeing uncontrolled memory growth when using the std::istream version of readNode. It appears to populate the local cache with new images, but never clears out the local cache. The filename-based version of readNode() does clear out the cache before it returns.

Does it make sense to add in:

flt::Registry::instance()->clearLocalCache();

... before the return of the ReadResult at the bottom of this method at https://github.com/openscenegraph/OpenSceneGraph/blob/master/src/osgPlugins/OpenFlight/ReaderWriterFLT.cpp#L533 ?


If that's not a good path forward, do you have any suggestions on how to clear the cache on demand?

Thanks,

- Dan



------------------
Post generated by Mail2Forum
Back to top
robertosfield
OSG Project Lead


Joined: 18 Mar 2009
Posts: 12322

PostPosted: Tue Jul 31, 2018 10:57 am    Post subject:
Unbounded growth in OpenFlight ReaderWriter
Reply with quote

Hi Dan,

I'm not at a point where I can easily switch over to thinking about
OSG code written by others so rather than jump it right now I'll ask a
simple question that might help point you in a direction worthy of
investigation.

FYI, I'm not the author of the OpenFlight plugin so am in the same
boat as yourself when it comes to working out what it does, what is
intended, what might be a bug etc. Each time I'm called in to fix
something in this plugin I learn a bit more... but still lots more to
learn.

I don't personally recall working with a flt::Registry::instance()
local cache so it's a new bit of OpenFlight plugin to me. I'm a bit
surprised it's there as osgDB has it's own object cache, so I'd wonder
what the motivation was behind the local cache and whether it's really
necessary. Could it be rewritten to use the osgDB's object cache?

Robert.







On Tue, 31 Jul 2018 at 11:19, Daniel Emminizer, Code 5773
<> wrote:
Quote:

Hello All,

I've found what appears to be unbounded growth in the OpenFlight loader. I have a work-around but wanted to run it past the community since there are many here who are more familiar with the format and the loader than me.

I'm seeing uncontrolled memory growth when using the std::istream version of readNode. It appears to populate the local cache with new images, but never clears out the local cache. The filename-based version of readNode() does clear out the cache before it returns.

Does it make sense to add in:

flt::Registry::instance()->clearLocalCache();

... before the return of the ReadResult at the bottom of this method at https://github.com/openscenegraph/OpenSceneGraph/blob/master/src/osgPlugins/OpenFlight/ReaderWriterFLT.cpp#L533 ?


If that's not a good path forward, do you have any suggestions on how to clear the cache on demand?

Thanks,

- Dan




------------------
Post generated by Mail2Forum
Back to top
View user's profile Send private message
Chris Hanson
Guest





PostPosted: Tue Jul 31, 2018 2:40 pm    Post subject:
Unbounded growth in OpenFlight ReaderWriter
Reply with quote

I think Brede Johansson wrote that loader once upon a time. I'm hoping to connect with him in Oslo this week. Maybe I'll ask him about this if I do.

On Tue, Jul 31, 2018 at 11:54 AM Daniel Emminizer, Code 5773 < (
Only registered users can see emails on this board!
Get registred or enter the forums!
)> wrote:

Quote:
Hello All,

I've found what appears to be unbounded growth in the OpenFlight loader.  I have a work-around but wanted to run it past the community since there are many here who are more familiar with the format and the loader than me.

I'm seeing uncontrolled memory growth when using the std::istream version of readNode.  It appears to populate the local cache with new images, but never clears out the local cache.  The filename-based version of readNode() does clear out the cache before it returns.

Does it make sense to add in:

        flt::Registry::instance()->clearLocalCache();

... before the return of the ReadResult at the bottom of this method at https://github.com/openscenegraph/OpenSceneGraph/blob/master/src/osgPlugins/OpenFlight/ReaderWriterFLT.cpp#L533 ?


If that's not a good path forward, do you have any suggestions on how to clear the cache on demand?

Thanks,

 - Dan

_______________________________________________
osg-users mailing list
(
Only registered users can see emails on this board!
Get registred or enter the forums!
)
http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org



--
Chris 'Xenon' Hanson, omo sanza lettere. http://www.alphapixel.com/
Training • Consulting • Contracting
3D • Scene Graphs (Open Scene Graph/OSG) • OpenGL 2 • OpenGL 3 • OpenGL 4 • GLSL • OpenGL ES 1 • OpenGL ES 2 • OpenCL
Legal/IP • Forensics • Imaging • UAVs • GIS • GPS • osgEarth • Terrain • Telemetry • Cryptography • LIDAR • Embedded • Mobile • iPhone/iPad/iOS • Android
@alphapixel facebook.com/alphapixel (775) 623-PIXL [7495]

------------------
Post generated by Mail2Forum
Back to top
Daniel Emminizer, Code...
Guest





PostPosted: Tue Jul 31, 2018 3:49 pm    Post subject:
Unbounded growth in OpenFlight ReaderWriter
Reply with quote

Hi Chris,

I think that would be good. I have some modifications following Robert’s advice and after more testing I will be posting a PR. Yes, all of the object caching in OpenFlight plugin is the original code from Brede. It all dates back to 2006.

I was able to swap to use osgDB::Registry’s object cache with no complications on my end. I also got to learn more about the object caching system and how it clears out old values. Thanks Robert for the suggestion to look in this direction.

I ended up posting a PR at https://github.com/openscenegraph/OpenSceneGraph/pull/583 - It’s low priority since it’s a 12 year old issue, and because it takes a lot of work to get large enough images to crash an app from the problem on modern systems.

Thanks,

- Dan




From: osg-users [mailto:] On Behalf Of Chris Hanson
Sent: Tuesday, July 31, 2018 10:40 AM
To: OpenSceneGraph Users
Subject: Re: Unbounded growth in OpenFlight ReaderWriter



I think Brede Johansson wrote that loader once upon a time. I'm hoping to connect with him in Oslo this week. Maybe I'll ask him about this if I do.


On Tue, Jul 31, 2018 at 11:54 AM Daniel Emminizer, Code 5773 < (
Only registered users can see emails on this board!
Get registred or enter the forums!
)> wrote:
Quote:

Hello All,

I've found what appears to be unbounded growth in the OpenFlight loader. I have a work-around but wanted to run it past the community since there are many here who are more familiar with the format and the loader than me.

I'm seeing uncontrolled memory growth when using the std::istream version of readNode. It appears to populate the local cache with new images, but never clears out the local cache. The filename-based version of readNode() does clear out the cache before it returns.

Does it make sense to add in:

flt::Registry::instance()->clearLocalCache();

... before the return of the ReadResult at the bottom of this method at https://github.com/openscenegraph/OpenSceneGraph/blob/master/src/osgPlugins/OpenFlight/ReaderWriterFLT.cpp#L533 ?


If that's not a good path forward, do you have any suggestions on how to clear the cache on demand?

Thanks,

- Dan

_______________________________________________
osg-users mailing list
(
Only registered users can see emails on this board!
Get registred or enter the forums!
)
http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org





--
Chris 'Xenon' Hanson, omo sanza lettere. http://www.alphapixel.com/
Training • Consulting • Contracting
3D • Scene Graphs (Open Scene Graph/OSG) • OpenGL 2 • OpenGL 3 • OpenGL 4 • GLSL • OpenGL ES 1 • OpenGL ES 2 • OpenCL
Legal/IP • Forensics • Imaging • UAVs • GIS • GPS • osgEarth • Terrain • Telemetry • Cryptography • LIDAR • Embedded • Mobile • iPhone/iPad/iOS • Android
@alphapixel facebook.com/alphapixel (775) 623-PIXL [7495]

------------------
Post generated by Mail2Forum
Back to top
robertosfield
OSG Project Lead


Joined: 18 Mar 2009
Posts: 12322

PostPosted: Tue Jul 31, 2018 4:01 pm    Post subject:
Unbounded growth in OpenFlight ReaderWriter
Reply with quote

Hi Dan,

Changes looks good to me, just merged. Nice to see a bug fix/feature
improvement that removes three times as much code as it adds! :-)

Cheers,
Robert.
On Tue, 31 Jul 2018 at 16:48, Daniel Emminizer, Code 5773
<> wrote:
Quote:

Hi Chris,



I think that would be good. I have some modifications following Robert’s advice and after more testing I will be posting a PR. Yes, all of the object caching in OpenFlight plugin is the original code from Brede. It all dates back to 2006.



I was able to swap to use osgDB::Registry’s object cache with no complications on my end. I also got to learn more about the object caching system and how it clears out old values. Thanks Robert for the suggestion to look in this direction.



I ended up posting a PR at https://github.com/openscenegraph/OpenSceneGraph/pull/583 - It’s low priority since it’s a 12 year old issue, and because it takes a lot of work to get large enough images to crash an app from the problem on modern systems.



Thanks,



- Dan







From: osg-users [mailto:] On Behalf Of Chris Hanson
Sent: Tuesday, July 31, 2018 10:40 AM
To: OpenSceneGraph Users
Subject: Re: Unbounded growth in OpenFlight ReaderWriter



I think Brede Johansson wrote that loader once upon a time. I'm hoping to connect with him in Oslo this week. Maybe I'll ask him about this if I do.



On Tue, Jul 31, 2018 at 11:54 AM Daniel Emminizer, Code 5773 <> wrote:

Hello All,

I've found what appears to be unbounded growth in the OpenFlight loader. I have a work-around but wanted to run it past the community since there are many here who are more familiar with the format and the loader than me.

I'm seeing uncontrolled memory growth when using the std::istream version of readNode. It appears to populate the local cache with new images, but never clears out the local cache. The filename-based version of readNode() does clear out the cache before it returns.

Does it make sense to add in:

flt::Registry::instance()->clearLocalCache();

... before the return of the ReadResult at the bottom of this method at https://github.com/openscenegraph/OpenSceneGraph/blob/master/src/osgPlugins/OpenFlight/ReaderWriterFLT.cpp#L533 ?


If that's not a good path forward, do you have any suggestions on how to clear the cache on demand?

Thanks,

- Dan






--

Chris 'Xenon' Hanson, omo sanza lettere. http://www.alphapixel.com/

Training • Consulting • Contracting

3D • Scene Graphs (Open Scene Graph/OSG) • OpenGL 2 • OpenGL 3 • OpenGL 4 • GLSL • OpenGL ES 1 • OpenGL ES 2 • OpenCL

Legal/IP • Forensics • Imaging • UAVs • GIS • GPS • osgEarth • Terrain • Telemetry • Cryptography • LIDAR • Embedded • Mobile • iPhone/iPad/iOS • Android

@alphapixel facebook.com/alphapixel (775) 623-PIXL [7495]




------------------
Post generated by Mail2Forum
Back to top
View user's profile Send private message
robertosfield
OSG Project Lead


Joined: 18 Mar 2009
Posts: 12322

PostPosted: Tue Jul 31, 2018 4:05 pm    Post subject:
Unbounded growth in OpenFlight ReaderWriter
Reply with quote

On Tue, 31 Jul 2018 at 15:40, Chris Hanson <> wrote:
Quote:
I think Brede Johansson wrote that loader once upon a time. I'm hoping to connect with him in Oslo this week. Maybe I'll ask him about this if I do.

Brede's original OpenFlight plugin back in fall of 2000 was our first
major community submission and probably one of the most important in
the history of the OpenSceneGraph project.

Being able to load OpenFlight models made it possible for the vis-sim
market to quickly evaluate the OSG and solidify it as a serious
contender. Brede's work and his companies willingness to donate it to
the open source community is something we should all celebrate :-)

Robert.


------------------
Post generated by Mail2Forum
Back to top
View user's profile Send private message
dglenn (David Glenn)
Appreciator


Joined: 07 Jan 2009
Posts: 211
Location: Burbank, Calif. U.S.A.

PostPosted: Tue Jul 31, 2018 8:16 pm    Post subject:
Re: Unbounded growth in OpenFlight ReaderWriter
Reply with quote

robertosfield wrote:
On Tue, 31 Jul 2018 at 15:40, Chris Hanson <> wrote:
Quote:
I think Brede Johansson wrote that loader once upon a time. I'm hoping to connect with him in Oslo this week. Maybe I'll ask him about this if I do.

Brede's original OpenFlight plugin back in fall of 2000 was our first
major community submission and probably one of the most important in
the history of the OpenSceneGraph project.

Being able to load OpenFlight models made it possible for the vis-sim
market to quickly evaluate the OSG and solidify it as a serious
contender. Brede's work and his companies willingness to donate it to
the open source community is something we should all celebrate Smile

Robert.


------------------
Post generated by Mail2Forum


Your not going to get any argument from me on that one Robert! I have leaned so much about and from the open flight format by creating models and useing them in my OSG projects. Hands down, it's still my faveret format for Sim models - dispight me being pushed into using FBX and OBJ formats by my handlers. Cheers to Brede Johansson!

_________________
David Glenn
---------------
D Glenn 3D Computer Graphics Entertainment.
www.dglenn.com
Back to top
View user's profile Send private message Visit poster's website
Display posts from previous:   
Post new topic   Reply to topic    OpenSceneGraph Forum Forum Index -> General All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You cannot download files in this forum

Similar Topics
Topic Author Forum Replies Posted
No new posts Returning FILE_REQUESTED from a Reade... Aaron Andersen General 1 Sun Dec 16, 2018 2:25 am View latest post
No new posts Exporting OpenFlight Models Daniel Emminizer, Code... General 2 Wed Jun 27, 2018 5:02 pm View latest post
No new posts Slow optimization and OpenFlight Andreas Ekstrand General 14 Fri Nov 17, 2017 4:16 pm View latest post
No new posts Deferring loading in an DB ReaderWrit... James Turner General 1 Fri Nov 18, 2016 8:03 pm View latest post
No new posts ReaderWriter result checking anishuz Submission 1 Fri Aug 26, 2016 6:25 am View latest post


Board Security Anti Bot Question MOD - phpBB MOD against Spam Bots
Powered by phpBB © 2001, 2005 phpBB Group
Protected by Anti-Spam ACP