OpenSceneGraph Forum Forum Index OpenSceneGraph Forum
Official forum which mirrors the existent OSG mailing lists. Messages posted here are forwarded to the mailing list and vice versa.
 
   FAQFAQ    SearchSearch    MemberlistMemberlist    RulesRules    UsergroupsUsergroups    RegisterRegister 
 Mail2Forum SettingsMail2Forum Settings  ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 
   AlbumAlbum  OpenSceneGraph IRC ChatOpenSceneGraph IRC Chat   SmartFeedSmartFeed 

Model(s) load ~10 times longer in 3.6.x than 3.4.x series in osgviewer (and my in-house software)

Goto page 1, 2  Next
 
Post new topic   Reply to topic    OpenSceneGraph Forum Forum Index -> General
View previous topic :: View next topic  
Author Message
davisjamesf
Newbie


Joined: 23 Feb 2009
Posts: 13

PostPosted: Wed Jul 04, 2018 7:18 pm    Post subject:
Model(s) load ~10 times longer in 3.6.x than 3.4.x series in osgviewer (and my in-house software)
Reply with quote

Hi,

The models I build and use are mainly jpeg UV textured obj files that have been osgconv'd into an ive. I have built OpenSceneGraph versions 3.6.0, 1 and 2, and they all display this issue on BOTH windows (10) and Linux (CentOS 7). Note that I also built the OSG 3.4.1

The issue is that I can open a model in osgviewer in 3.4.1 and the model loads in the window very, very fast. When I run osgviewer in 3.6.x, it is taking ~10 times longer to load the exact same model (somtimes 15+ seconds).

This is a major issue in the program I build that loads many, many different models.

Thanks for any help!
James
Back to top
View user's profile Send private message
L. Voerman
Guest





PostPosted: Thu Jul 05, 2018 7:29 am    Post subject:
Model(s) load ~10 times longer in 3.6.x than 3.4.x series in osgviewer (and my in-house software)
Reply with quote

Hi James,
Just a guess,could the 3.4.1 version be using jpeg-turbo - while the other version uses libjpeg? 
Laurens.


On Wed, Jul 4, 2018 at 9:18 PM James Davis < (
Only registered users can see emails on this board!
Get registred or enter the forums!
)> wrote:

Quote:
Hi,

The models I build and use are mainly jpeg UV textured obj files that have been osgconv'd into  an ive.   I have built OpenSceneGraph versions 3.6.0, 1 and 2, and they all display this issue on BOTH windows (10) and Linux (CentOS 7).  Note that I also built the OSG 3.4.1

The issue is that I can open a model in osgviewer in 3.4.1 and the model loads in the window very, very fast.   When I run osgviewer in 3.6.x, it is taking ~10 times longer to load the exact same model (somtimes 15+ seconds).

This is a major issue in the program I build that loads many, many different models.

Thanks for any help!
James

------------------
Read this topic online here:
http://forum.openscenegraph.org/viewtopic.php?p=74260#74260





_______________________________________________
osg-users mailing list
(
Only registered users can see emails on this board!
Get registred or enter the forums!
)
http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org


------------------
Post generated by Mail2Forum
Back to top
davisjamesf
Newbie


Joined: 23 Feb 2009
Posts: 13

PostPosted: Thu Jul 05, 2018 1:06 pm    Post subject:
Model(s) load ~10 times longer in 3.6.x than 3.4.x series in osgviewer (and my in-house software)
Reply with quote

Good guess, however, I'm linking both 3.4.1 and 3.6.x to the same 3rdParty libs.
Back to top
View user's profile Send private message
davisjamesf
Newbie


Joined: 23 Feb 2009
Posts: 13

PostPosted: Thu Jul 05, 2018 3:59 pm    Post subject:
Reply with quote

The image file the .obj and .ive converted uses is a .png

I did verify and both 3.4.1 and 3.6.x are linking to the x64\lib\libpng16.lib. But, perhaps this is a clue?

I'm doing some further investigating on a test model.
Back to top
View user's profile Send private message
davisjamesf
Newbie


Joined: 23 Feb 2009
Posts: 13

PostPosted: Thu Jul 05, 2018 6:49 pm    Post subject:
Reply with quote

Ok...did a little testing and found that loading speed does NOT depend on image textures.

I created a test plane and subdivided it to make many faces and extruded every other one to give it a little depth. The face count of my plane is ~43k faces. I UV textured the sample plane with a jpeg, one with a png, and one with no image. I converted each to a .obj. The no-image had about the same poor performance in loading as with the jpeg or png. Note that loading time doesn't matter if I convert to a osg or ive first and then load.

So, for some reason, OSG 3.6.x is taking a lot longer loading a higher faced/vert count .obj (or osg or ive) object than 3.4.1. This is a show stopper for my customer, and I'll have to stick with 3.4.1 until something is figured out.

Suggestions? Thoughts? Are there any optimization switches I've missed that was not default before building the 3.6.x series?
Back to top
View user's profile Send private message
L. Voerman
Guest





PostPosted: Fri Jul 06, 2018 7:47 am    Post subject:
Model(s) load ~10 times longer in 3.6.x than 3.4.x series in osgviewer (and my in-house software)
Reply with quote

Hi James,I tried to replicate your problem, but fail to do so.
I tried this on a 110 MB .ive file (~3M vertices)


set OSG_OPIMIZER=OFF

SET OSG_NOTIFY_LEVEL=INFO

osgconv "testfile.ive" dummy.jpg | grep "Time to load"


results with osg 3.4.2: 95, 95, 92, 93 ms (win10 / visual studio 2015 Update 3)
results with osg 3.6.2: 92, 92, 93, 93 ms  (win10 / visual studio 2017 15.7.4)


Can you see if your problem can be measured this way?
Regards, Laurens.


On Thu, Jul 5, 2018 at 8:49 PM James Davis < (
Only registered users can see emails on this board!
Get registred or enter the forums!
)> wrote:

Quote:
Ok...did a little testing and found that loading speed does NOT depend on image textures.   

I created a test plane and subdivided it to make many faces and extruded every other one to give it a little depth.  The face count of my plane is ~43k faces.  I UV textured the sample plane with a jpeg, one with a png, and one with no image.   I converted each to a .obj.  The no-image had about the same poor performance in loading as with the jpeg or png.   Note that loading time doesn't matter if I convert to a osg or ive first and then load.

So, for some reason, OSG 3.6.x is taking a lot longer loading a higher faced/vert count .obj (or osg or ive) object than 3.4.1.  This is a show stopper for my customer, and I'll have to stick with 3.4.1 until something is figured out.

Suggestions? Thoughts? Are there any optimization switches I've missed that was not default before building the 3.6.x series?

------------------
Read this topic online here:
http://forum.openscenegraph.org/viewtopic.php?p=74266#74266





_______________________________________________
osg-users mailing list
(
Only registered users can see emails on this board!
Get registred or enter the forums!
)
http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org


------------------
Post generated by Mail2Forum
Back to top
davisjamesf
Newbie


Joined: 23 Feb 2009
Posts: 13

PostPosted: Fri Jul 06, 2018 8:23 pm    Post subject:
Reply with quote

Thanks for insight Laurens.

I'm adding a test .obj file I'm using for osgconv testing. It contains NO image, so we can take any png or jpeg plugins out of the equation.

For this file, producing a .ive:

3.4.1 took 4941.62 ms (squaretest_341.ive)
3.6.2 took 274154 ms (4.5 minutes!). (squaretest_361.ive)

I've compiled both osg versions with VS2017 (note this occurs on Linux too)

Here is something to note:

Opening the squaretest_341.ive in the 3.4.1 osgviewer opens instantly. Opening it in the 3.6.2 osgviewer takes as long as osgconv took.

Opening the squaretest_362.ive in 3.6.2 osgviewer opens instantly (as it does also in 3.4.1 osgviewer).

So, at least a workaround may be to take all my original .obj files and convert to .ive using the 3.6.2 osgconvert(er).
Back to top
View user's profile Send private message
robertosfield
OSG Project Lead


Joined: 18 Mar 2009
Posts: 12093

PostPosted: Mon Jul 09, 2018 9:16 am    Post subject:
Model(s) load ~10 times longer in 3.6.x than 3.4.x series in osgviewer (and my in-house software)
Reply with quote

Hi James,

The file was too big to distribute on the osg-users mailing list that
mirrors the forum so I pulled down the data from the forum and can see
the loading performance issue when using OpenSceneGraph-3.6.2:

$ time osgconv SquareTest.obj test.osgb
Data written to 'test.osgb'.

real 5m2.443s
user 5m2.352s
sys 0m0.068s

The resulting test.osgb loads quickly and runs fine.

I will do a build of the OSG-3.4 branch and see what happens.

Robert.


------------------
Post generated by Mail2Forum
Back to top
View user's profile Send private message
robertosfield
OSG Project Lead


Joined: 18 Mar 2009
Posts: 12093

PostPosted: Mon Jul 09, 2018 9:31 am    Post subject:
Model(s) load ~10 times longer in 3.6.x than 3.4.x series in osgviewer (and my in-house software)
Reply with quote

Now tested the model with 3.4 - much faster:

$ time osgconv SquareTest.obj test_34.osgb
Data written to 'test_34.osgb'.

real 0m3.842s
user 0m3.112s
sys 0m0.728s

I've looked at the diffs between 3.4 and 3.6 for the obj plugin, the
only difference that stands out as a possible cause is the use of mesh
optimzer instead of the tri strip visitor. Mesh optimizer works much
better for modern graphics cards, I wouldn't have expected such a slow
down though, there must be something in the presentation of this
particular dataset that is cause issues.

Robert.


------------------
Post generated by Mail2Forum
Back to top
View user's profile Send private message
robertosfield
OSG Project Lead


Joined: 18 Mar 2009
Posts: 12093

PostPosted: Mon Jul 09, 2018 10:23 am    Post subject:
Model(s) load ~10 times longer in 3.6.x than 3.4.x series in osgviewer (and my in-house software)
Reply with quote

I have established it's the osgUtil::optimizeMesh(geometry); call in
the 3.6 obj plugin that is causing the slow down in .obj loading.

When writing the model out to .osgt without the optimization you can
see the likely reason why this dataset might be triggering issues:

PrimitiveSetList 265486 {
osg::DrawArrays {
UniqueID 6
Mode TRIANGLE_FAN
Count 4
}
osg::DrawArrays {
UniqueID 7
Mode TRIANGLE_FAN
First 4
Count 4
}
..

So we have 265 thousand triangle fan DrawArrays that loader has
created. That's about as inefficient as you can get as it not only is
using lots separate primitives but also implies that non of the vertex
data is being shared either.

I'm not the author of the obj plugin so can't immediately pinpoint
what bit of code is responsible for creating this poorly structured
dataset. The .obj format presents the individual primitives one by
one so is the root cause of this, but the loader should probably not
be happing this verbatim.

At this point I don't have a clear suggestion of solution, I'm
inclined to think that the plugin shouldn't load the data verbatim as
it's doing right now, then trying to fix the problems this causes
after the fact.

Robert.


------------------
Post generated by Mail2Forum
Back to top
View user's profile Send private message
robertosfield
OSG Project Lead


Joined: 18 Mar 2009
Posts: 12093

PostPosted: Mon Jul 09, 2018 11:12 am    Post subject:
Model(s) load ~10 times longer in 3.6.x than 3.4.x series in osgviewer (and my in-house software)
Reply with quote

I have other tasks to get on with today so I won't keep pursing this
loading speed regression right away, if others want to look at before
I come back to it then the way I think it should be tackled is be
changing the loader to so it creates a small set of primitives using
GL_TRIANGLES rather than many separate DrawArrays GL_TRIANGLE_FAN.

Robert.


------------------
Post generated by Mail2Forum
Back to top
View user's profile Send private message
davisjamesf
Newbie


Joined: 23 Feb 2009
Posts: 13

PostPosted: Thu Jul 19, 2018 3:10 pm    Post subject:
Reply with quote

I wanted to see if anyone had a chance to come up with some test code I might could try out. Sorry if I sound like I'm nagging. Confused
Back to top
View user's profile Send private message
robertosfield
OSG Project Lead


Joined: 18 Mar 2009
Posts: 12093

PostPosted: Thu Jul 19, 2018 6:10 pm    Post subject:
Model(s) load ~10 times longer in 3.6.x than 3.4.x series in osgviewer (and my in-house software)
Reply with quote

On Thu, 19 Jul 2018 at 16:19, James Davis <> wrote:
Quote:
I wanted to see if anyone had a chance to come up with some test code I might could try out. Sorry if I sound like I'm nagging. Confused

I haven't personally looked at the code since I last replied. Perhaps
others have.

You have full access to the source code so no need to wait on others,
you can dive in a make ammendments, if they are sound then I'd be
happy to review and merge them in appropriate.

As a general note, till the end of August my main focus is on the
Vulkan Exploration Phase, I did a major push on OSG bug fixes for
3.6.2 and for the time being will be taking a back seat on non
critical OSG bugs.

Cheers,
Robert.


------------------
Post generated by Mail2Forum
Back to top
View user's profile Send private message
davisjamesf
Newbie


Joined: 23 Feb 2009
Posts: 13

PostPosted: Thu Jul 19, 2018 9:49 pm    Post subject:
Reply with quote

After changing two files, I have ive files loading very fast AND osgconv converting obj files to osg and ive files very fast.

You'll have to forgive my lack of understanding a lot of the internals of OSG. The code I changed was due to noticing where bottle necks (for loops) were occurring when loading .ive files in my program. Once I changed the code and created new libraries, I tested them on the osgconv with speed success.

The code I changed was NOT in the obj plugin but in the osg library itself. I changed a few lines to previous changes compared to 3.4.1. Note that I did attempt at trying to use the 3.4.1 obj plugin, but quickly found it was not the problem.

The two files I changed were Geometry.cpp and Drawable.cpp, and I'll need some help understanding why my changes really made a difference.

When I would 'break' in my debugger when loading an ive file, most of the time the process ended up on a for loop in Geometry.cpp (line 582) in the getOrCreateElementBufferObject method. The callstack was Geometry::read, Geometry::addPrimitiveSet, Geometry::addElementBufferObjectRequired.

I decided to make the change line 28 to #if 1 and add back all the if conditions of _useVertexBufferObjects. This fixed the load .ive issue but still did not fix the osgconv. I had to change Drawable.cpp line 229 to #if 1 to make _userVertecBufferObjects = false.

Can someone take a look at the impact _useVertexBufferObjects has on certain files being loaded. It seems a check needs to be before the for loop of traversing through the drawElementList if it really doesn't need to be called.

Thanks!
Back to top
View user's profile Send private message
robertosfield
OSG Project Lead


Joined: 18 Mar 2009
Posts: 12093

PostPosted: Fri Jul 20, 2018 7:27 am    Post subject:
Model(s) load ~10 times longer in 3.6.x than 3.4.x series in osgviewer (and my in-house software)
Reply with quote

Hi James,

The checks are needed in Geometry,cpp and Drawable.cpp, if you've
changed these you will likely be breaking something else. You might
not be see any issues yet, and might not ever, but as I don't know
exactly what you've changed I can't say. All I can say it's not
really fixing the problem, it's fixing a symptom and likely to have
un-intended consequences.

As I said previous the problem is the obj plugin loading meshes with a
single triangle per PrimitiveSet. This is the absolutely worst/least
efficient way to create a mesh in the OSG and OpenGL. If you want an
example of how not to do things this would be a classic example.
Expecting the OSG to patch up a bad scene graph after it's been
creating is only hiding where the problem is.

Trust me, the problem is the obj plugin, this needs fixing.

Robert.


On Thu, 19 Jul 2018 at 22:48, James Davis <> wrote:
Quote:

After changing two files, I have ive files loading very fast AND osgconv converting obj files to osg and ive files very fast.

You'll have to forgive my lack of understanding a lot of the internals of OSG. The code I changed was due to noticing where bottle necks (for loops) were occurring when loading .ive files in my program. Once I changed the code and created new libraries, I tested them on the osgconv with speed success.

The code I changed was NOT in the obj plugin but in the osg library itself. I changed a few lines to previous changes compared to 3.4.1. Note that I did attempt at trying to use the 3.4.1 obj plugin, but quickly found it was not the problem.

The two files I changed were Geometry.cpp and Drawable.cpp, and I'll need some help understanding why my changes really made a difference.

When I would 'break' in my debugger when loading an ive file, most of the time the process ended up on a for loop in Geometry.cpp (line 582) in the getOrCreateElementBufferObject method. The callstack was Geometry::read, Geometry::addPrimitiveSet, Geometry::addElementBufferObjectRequired.

I decided to make the change line 28 to #if 1 and add back all the if conditions of _useVertexBufferObjects. This fixed the load .ive issue but still did not fix the osgconv. I had to change Drawable.cpp line 229 to #if 1 to make _userVertecBufferObjects = false.

Can someone take a look at the impact _useVertexBufferObjects has on certain files being loaded. It seems a check needs to be before the for loop of traversing through the drawElementList if it really doesn't need to be called.

Thanks!

------------------
Read this topic online here:
http://forum.openscenegraph.org/viewtopic.php?p=74366#74366








------------------
Post generated by Mail2Forum
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic   Reply to topic    OpenSceneGraph Forum Forum Index -> General All times are GMT
Goto page 1, 2  Next
Page 1 of 2

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You cannot download files in this forum

Similar Topics
Topic Author Forum Replies Posted
No new posts Model matrix kullboys General 0 Tue Aug 21, 2018 7:10 am View latest post
No new posts Load an obj file and mapping a given ... aaa3d General 3 Mon Aug 06, 2018 2:27 pm View latest post
No new posts missing color for simple model with G... aroth General 1 Fri Jul 27, 2018 1:55 pm View latest post
No new posts Load OSG trimesh to ODE: Read Vertice... TKoller General 2 Fri Jul 20, 2018 3:06 pm View latest post
No new posts Model flickering on osgViewer ceranco General 3 Thu Jun 21, 2018 10:02 am View latest post


Board Security Anti Bot Question MOD - phpBB MOD against Spam Bots
Powered by phpBB © 2001, 2005 phpBB Group
Protected by Anti-Spam ACP