OpenSceneGraph Forum Forum Index OpenSceneGraph Forum
Official forum which mirrors the existent OSG mailing lists. Messages posted here are forwarded to the mailing list and vice versa.
 
   FAQFAQ    SearchSearch    MemberlistMemberlist    RulesRules    UsergroupsUsergroups    RegisterRegister 
 Mail2Forum SettingsMail2Forum Settings  ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 
   AlbumAlbum  OpenSceneGraph IRC ChatOpenSceneGraph IRC Chat   SmartFeedSmartFeed 

Dynamical loading of resources


 
Post new topic   Reply to topic    OpenSceneGraph Forum Forum Index -> General
View previous topic :: View next topic  
Author Message
JeyD
Newbie


Joined: 14 Aug 2017
Posts: 4

PostPosted: Mon Aug 14, 2017 8:15 am    Post subject:
Dynamical loading of resources
Reply with quote

Hi,

I was tasked with writing a visualizer for large 3D areas, the kind of areas that may not fit into RAM, either because they are too large, or because the RAM on the machine is not large enough Wink.

I am new to OSG, this is my first post to the forum (hello, guys Smile!). I was wondering how to perform out-of core handling of the 3D resources I intent to render. Its not possible for me to load everything and then start rendering, i need to constantly load new parts of the environment and unload geometry that is out of view.

So I would like to know, is there any mechanism in OSG (or OSG Earth) which already supports this? Or would I have to implement it myself? Either way, I would be very happy if one of you could give me a rough explanation of how to approach this problem with OSG.

Thank you, I appreaciate yout help!

Cheers,
Jey
Back to top
View user's profile Send private message
SMesserschmidt (Sebastian Messerschmidt)
Forum Moderator


Joined: 10 Sep 2013
Posts: 796

PostPosted: Mon Aug 14, 2017 2:47 pm    Post subject:
Dynamical loading of resources
Reply with quote

Hi Joachim,
Quote:
Hi,

I was tasked with writing a visualizer for large 3D areas, the kind of areas that may not fit into RAM, either because they are too large, or because the RAM on the machine is not large enough Wink.

I am new to OSG, this is my first post to the forum (hello, guys Smile!). I was wondering how to perform out-of core handling of the 3D resources I intent to render. Its not possible for me to load everything and then start rendering, i need to constantly load new parts of the environment and unload geometry that is out of view.
That is where PagedLOD comes into play. It allows for background loading
of tiles.
Quote:

So I would like to know, is there any mechanism in OSG (or OSG Earth) which already supports this? Or would I have to implement it myself? Either way, I would be very happy if one of you could give me a rough explanation of how to approach this problem with OSG.

See the osgPagedLod example for reference.
If you need a resolution-pyramid approach, VPB(VirtualPlanetBuilder) can
generate multiscale data for paging from various sources.
Sorry for just giving you pointers, but a short answer might be better
than none Wink
Quote:

Thank you, I appreaciate yout help!

Cheers,
Jey
Cheers
Sebastian
Quote:
------------------
Read this topic online here:
http://forum.openscenegraph.org/viewtopic.php?p=71409#71409









------------------
Post generated by Mail2Forum
Back to top
View user's profile Send private message
robertosfield
OSG Project Lead


Joined: 18 Mar 2009
Posts: 11238

PostPosted: Mon Aug 14, 2017 6:42 pm    Post subject:
Dynamical loading of resources
Reply with quote

Hi Jey,

The OSG is designed to fully support paging of databases.  There are many discussions about paging here on the osg-users mailing list/forum so have a look through the archives for discussions about osgDB::DatabasePager (the class that manages the loading + unloading of data) and osg::PagedLOD (the level of detail node that provides the hooks for external resources.)


The osgViewer::Viewer and CompositeViewer class have builtin support for the DatabasePager so you don't need to explicitly do anything to leverage the database paging - you just stick the appropriate PagedLOD's in your database and everything will happen automatically for you. 



osgEarth is built intop of PageLOD so leverages all the core OSG's osgDB::DatabasePager support.  
The VirtualPlanetBuilder tool provides another way for creating native OSG paged databases from source geospatial imagery and DEMs.


Have a look at the archives though, there is wealth of discussion about paging.


Robert.

------------------
Post generated by Mail2Forum
Back to top
View user's profile Send private message
JeyD
Newbie


Joined: 14 Aug 2017
Posts: 4

PostPosted: Tue Aug 15, 2017 8:19 am    Post subject:
Reply with quote

Hello Sebastian, hello Robert,

thank you for your quick reply. Paging.. I was thinking about a lot of search terms, but not this one. In retrospect it seems obvious Smile.

Pointers in direction of osg::PagedLOD and osgDB::DatabasePager were all I was looking for, thanks. I did some reading on the topic, but there is one thing I still might need your expertise. Lets say I want to use a data structure such as point clouds, which are not supported out of the box and I want to load them from a database with a non-standardised interface.

In this case, I would implement the visualization within a osg::PagedLOD. I would also write my own osgDB::DatabasePager and register it at the visualizer. Is that correct? And that's also how it would work within the context of OSG Earth, or are there additional layers around the problem?

Thanks and best regards,
Joachim
Back to top
View user's profile Send private message
robertosfield
OSG Project Lead


Joined: 18 Mar 2009
Posts: 11238

PostPosted: Tue Aug 15, 2017 8:28 am    Post subject:
Dynamical loading of resources
Reply with quote

Hi

On 15 August 2017 at 09:20, Joachim Gehrung < (
Only registered users can see emails on this board!
Get registred or enter the forums!
)> wrote:
Quote:
In this case, I would implement the visualization within a osg::PagedLOD. I would also write my own osgDB::DatabasePager and register it at the visualizer. Is that correct? And that's also how it would work within the context of OSG Earth, or are there additional layers around the problem?


You shouldn't need to touch DatabasePager, it'll support any type of paged database.


What you need to do is create a paged databases that contains PagedLOD, as you say the existing tools like VirtualPlanentBuilder (VPB) and osgEarth aren't written for point clouds so you'll need to implement your own scheme for this.  Once you have built the database it should work with any OSG application.


How best to go about creating the paged database will depend upon the type of point cloud you have.  If you have a terrain database then the elevation will be generally much smaller than the horizontal extents then a quad tree hierarchy would be most appropriate (this is what VPB and osgEarth use), if you have an elevation of similar size to your horizontal extents then an octree hierarchy would be most appropriate.


Another factor is just how big your dataset is and whether it might be possible to create most of the final scene graph on the fly like osgEarth does or whether you want to pre-process the whole dataset like VPB does.



Robert.

------------------
Post generated by Mail2Forum
Back to top
View user's profile Send private message
JeyD
Newbie


Joined: 14 Aug 2017
Posts: 4

PostPosted: Tue Aug 15, 2017 1:21 pm    Post subject:
Reply with quote

Hi,

ok, thank you. I think I'm starting to get the picture Smile. When you talk about a database, you mean just a bunch of files on the filesystem, not some kind of SQL database, right?

I'm afraid I have to implement against our own backend. So let my guess, I would have to implement a subclass of the osgDB::DatabasePager to handle that?

Cheers,
Joachim
Back to top
View user's profile Send private message
robertosfield
OSG Project Lead


Joined: 18 Mar 2009
Posts: 11238

PostPosted: Tue Aug 15, 2017 2:25 pm    Post subject:
Dynamical loading of resources
Reply with quote

Hi Joachim,

On 15 August 2017 at 14:21, Joachim Gehrung < (
Only registered users can see emails on this board!
Get registred or enter the forums!
)> wrote:
Quote:
ok, thank you. I think I'm starting to get the picture Smile. When you talk about a database, you mean just a bunch of files on the filesystem, not some kind of SQL database, right?


It could be either...

For VPB generated paged database what you have is set of files, with one file as the topmost and lowest level of detail that your application loads and then this topmost file contains a PageLOD that references a file contains the next level details and so on.  With a quad tree you get four more files for each level, for an octree you get eight more files.


For osgEarth it dynamically creates the PagedLOD subgraphs in that extract image and elevation data from online or local raw data sources.


Other users have written custom osgPlugins that use tile references encoded into the PagedLOD filename strings that the custom plugin uses to read from an external source. some have even done this extracting the data from a SQL database.

 
Quote:
I'm afraid I have to implement against our own backend. So let my guess, I would have to implement a subclass of the osgDB::DatabasePager to handle that?

 
Um... how many times do I have to say you don't need to subclass from DatabasePager.


Repeat after me:
    I don't need to subclass from DatabasePager.
    I don't need to subclass from DatabasePager.
    I don't need to subclass from DatabasePager.

Smile


All you need to do create a paged database using standard osg::PagedLOD/osg::Geometry Nodes, you can do this offline like VPB or on the fly like osgEarth.  I suspect a custom plugin might be the way from for yourself, but this is just a guess, you haven't said anything how you data is stored and should be accessed.  Please share info on how your data stored and what it represents, this will help guide you in the right direction.



Robert.

------------------
Post generated by Mail2Forum
Back to top
View user's profile Send private message
JeyD
Newbie


Joined: 14 Aug 2017
Posts: 4

PostPosted: Tue Aug 22, 2017 1:05 pm    Post subject:
Reply with quote

Hi,

sorry for my late reply. One of my requirements is that the backend should be flexible... this was why is was so attached to the DatabasePager when I first heared about it Wink. To me it sounded like the part one needs to exchange to switch backends. Well, if it isn't the DatabasePager, what is it then? As far as I were able to follow your suggestions, I may need different PagedLODs or some kind of plugin you mentioned, but I don't know about already, correct Smile?

A possible "backend" is of course the file system, but it should also be possible to handle a server with a proprietary protocol.

Thank you!

Cheers,
Joachim
Back to top
View user's profile Send private message
robertosfield
OSG Project Lead


Joined: 18 Mar 2009
Posts: 11238

PostPosted: Tue Aug 22, 2017 1:28 pm    Post subject:
Dynamical loading of resources
Reply with quote

Hi Joachim,

On 22 August 2017 at 14:05, Joachim Gehrung < (
Only registered users can see emails on this board!
Get registred or enter the forums!
)> wrote:
Quote:
sorry for my late reply. One of my requirements is that the backend should be flexible... this was why is was so attached to the DatabasePager when I first heared about it Wink. To me it sounded like the part one needs to exchange to switch backends. Well, if it isn't the DatabasePager, what is it then? As far as I were able to follow your suggestions, I may need different PagedLODs or some kind of plugin you mentioned, but I don't know about already, correct Smile?


I think you mis-understood the role of the DatabasePager - it's the tool that the manages the theading or loading, compile, merging and expiring subgraphs, it doesn't do any loading itself, it defers all the loading to the standard osgDB file loading mechniams managed by the osgDB::Registry.



To do different backends all you need is different plugins that do the loading for you.

 

Quote:
A possible "backend" is of course the file system, but it should also be possible to handle a server with a proprietary protocol.


Have a look at the curl plugin that provides the OSG's support for http/https loading of data.  This takes filenames as input and opens up a connection using libcurl and then uses other OSG plugins to load the actual data,


The curl plugin doesn't actually know anything about the DatabasePager and the DatabasePager knows nothing about the curl plugin as the two are completely independent yet work really well together.



Robert,

 

------------------
Post generated by Mail2Forum
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic   Reply to topic    OpenSceneGraph Forum Forum Index -> General All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You cannot download files in this forum

Similar Topics
Topic Author Forum Replies Posted
No new posts [CubeMap with mipmaps] About loading IBL mp3butcher General 3 Sat Nov 04, 2017 12:37 am View latest post
No new posts Regarding Slowing loading of building... Rambabu General 2 Fri Apr 21, 2017 7:59 am View latest post
No new posts How to improve frame rate when loadin... vinothrajendran4 General 9 Wed Feb 22, 2017 2:01 pm View latest post
No new posts Deferring loading in an DB ReaderWrit... James Turner General 1 Fri Nov 18, 2016 8:03 pm View latest post
No new posts Problems with .ive loading Tiniriel Plugins [osgPlugins] 1 Wed Nov 09, 2016 9:55 am View latest post


Board Security Anti Bot Question MOD - phpBB MOD against Spam Bots
Powered by phpBB © 2001, 2005 phpBB Group
Protected by Anti-Spam ACP