Shader Engine

[admin]

This is the thread for asking more details about the Shader Engine proposal.

[GorDev]

Hi,
I've got some questions about engine development:

• *Should I think about compatibility with previous version of graphics engine?
  *What about engine integration process with Gephi? Is it part of GSOC?
  *What about user defined models loading(even primitive)? Should I think about it?
  *How many computers would be used for rendering(very important for architecture)?
  *Can I use only a fast subset of JOGL(important for performance)?
  *I'd like to propose fully shader-oriented architecture with configurable pipeline. Would it be alright?

[mbastian]

*Should I think about compatibility with previous version of graphics engine?

Not really, current graphics engine is not modular, don't have proper API. The idea is to restart from scratch, but with my help for design and architecture.

*What about engine integration process with Gephi? Is it part of GSOC?

Not completely, we would miss time for doing this. But a minimal integration has to be done, at least for testing purpose.

*What about user defined models loading(even primitive)? Should I think about it?

No, models are only disk and rectangles. Spheres and cuboid for 3d.

*How many computers would be used for rendering(very important for architecture)?

Hum I think you are wrong here; the idea is to make a real-time engine, not a rendering engine. Like a game. Features would be something like we have now (displaying graphs, playing with it with mouse...) but with using GLSL.

*Can I use only a fast subset of JOGL(important for performance)?

Could you detail this, what's your idea?

*I'd like to propose fully shader-oriented architecture with configurable pipeline. Would it be alright?

I guess so, but could you please expose what consequences has this architecture on the system, in term of updating nodes position for instance?

[GorDev]

*Can I use only a fast subset of JOGL(important for performance)?

Could you detail this, what's your idea?

I'd like to test JOGL function performance and use only fastest of them(these can take some time).

*I'd like to propose fully shader-oriented architecture with configurable pipeline. Would it be alright?

I guess so, but could you please expose what consequences has this architecture on the system, in term of updating nodes position for instance?

Sorry, I mean object-oriented. The engine's objects would be scene graph nodes and resources.

[mbastian]

JOGL is efficient and almost as fast as a C++ GLUT rendering. It just binds OpenGL functions to native.

And JOGL is also easy to learn, it's basically the same as you have seen in other languages bindings. That wouldn't be a problem. Students without JOGL experience can apply.

[darkhus]

Hi,
I understand that gephi would have best possible performance, and possibility to perform as much nodes and edges as possible. My ideas are focused on performance and also well looking 3D world.
-it would be full 3D world, so objects like nodes-spheres should be also 3D; my idea is to use flat object always faced to observer, each such flat object would be build as triangle and mapped by texture with alpha channel, so on that texture simply would be disk; this will highly reduce number of vertices to process. Light will give better 3D looks to flat objects, but even without light it would looks like delighted if texture would be delighted sphere. This way each node would be a three-vertices textured object. I think that such disks always faced to observer would looks exactly the same as full 3d spheres (difference is that using disk, light would be always from same direction according to observer, but using spheres it would change; but this would be an result of not to using light computation during rendering;)
-curved edges: My idea is to build edges in the same way as nodes, as flat primitive (in this case rectangle) covered by texture (also with alpha channel and shape on it). If edges could be only straight lines then even better because there could be only 2 verts per line.
Used texture would be a pattern for edges and also for nodes.
It would be like that if object could be flat, but with cubes flat idea is no more valid
I hope that I get idea if shader engine correctly.

[apatriarca]

I followed a couple of tutorial and read some documentation of JOGL and I think I shouldn't have problems with it. Indeed, it is just a native bindings as you said. I also navigated through the Gephi code to get a basic understanding on how your current engine works and interact with the rest of the system. But some questions still remains.

What is the target OpenGL version? Shaders have a long story in OpenGL and it would be useful to understand what kind of features can be used in the design of this engine. I would probably try to only use features that aren't deprecated in 3.x and 4.x versions of OpenGL to make the engine more future-proof, but I would probably target platforms supporting the OpenGL 2 version of the API.

What is the exact set of features the engine should implement? The current engine has a lot of features and I personally think displaying the nodes and the edges is probably the easier part in developing this engine. The engine should at least efficiently manage the nodes and support several different views of the graph. If I have understood it corrently, in the current engine the nodes are inserted in an octree and each octant is rendered separately. I'm not sure if it is the best way to do it. Indeed, the possible views of the graph are quite general and aren't always linked to the position of the nodes. Do you ever took some statistics on the engine behaviours?

What kind of camera should the engine support? Using Gephi it looks like you are only using an orthographic projection and all the nodes are in the same plane. Is it wrong? If the nodes are really only positioned in 2D, why do you use an octree instead of a quadtree?

Do the engine have to implement every model you listed? I plan to use impostors as in QuteMol. Squares, circles and spheres can be easily and efficiently implemented in this model but I'm not sure if cubes really make sense. In fact, an axis aligned cube is just a square in orthographic projection. I also think the current engine don't support cubes.

[mbastian]

@darkhus Yes you get the idea of reducing mesh, for sure. And yes, what we can do with texture and light is to have a 3d-looking sphere with just a triangle. For edges, having curved edges may burn performances, but I'm open to innovative solutions. However it's not a priority.

What is the target OpenGL version?

Target is OpenGL 2.0, or 2.1if we don't have a choice. One year ago I made some tests about Pseudo-instancing technique. It was good but not revolutionnary, but it had the advantage that you make the same design as hardware that support geometric shaders. I don't know what do you think about that.

What is the exact set of features the engine should implement?

Before answering, let me give more details about the current engine. Elements are loaded in an octree, the display loop only shows elements which are in visible octants. Edges elements are both in source and target octant. Picking is used to only test selection in the octant the mouse is over. The most tricky is the nodes position update. Nodes has a X, Y, Z position that the engine continuously reads at each frame. This position is supposed to be changed all the time, it's the case when we do layout. When the setX(), setY() or setZ() function is called in NodeData, the octant where the element belongs is notified an element has changed his position. At next display, visible octants update flag are checked and if true, all nodes position in the octant are checked. If the node is out of his octant, it's removed and added to the octree again. The fact at each display all elements could have changed their position is a point that makes design difficult. In games, for instance in strategy game the map is loaded and stays, only few elements are not static. That's why the octree I'm using is static and has a fixed depth. From one frame to another, the dense part could become the sparse part, so it doesn't make sense to optimize accordingly to elements density.

I'm not sure I understand you when you says "the possible views of the graph are quite general and aren't always linked to the position of the nodes"

Yes, displaying elements is one thing, but putting interaction is another one. As I developed everything in a active loop, without VBO for compatibility reasons I have difficulties to imagine what performance problems it has, for updating elements sizes, colors, labels...

What kind of camera should the engine support?

No it's real projection 3D. Actually, in options if you pick "Sphere 3D" model and use mouse middle button you can rotate on Y axis. Some people use 3d graphs, for instance to put time dimension on the third dimension. However most researchers says 3d is confusing users and 2D is preferred for network visualization. I also think so. Therefore a 3D camera could be considered, but it's not a top priority.

Do the engine have to implement every model you listed?

Yes cube is hard to do with impostors, the engine in 0.6 used to support cubes, but I didn't put it back in 0.7. It's not so important, Squares, circles and spheres is the essential.

[apatriarca]

Target is OpenGL 2.0, or 2.1if we don't have a choice. One year ago I made some tests about Pseudo-instancing technique. It was good but not revolutionnary, but it had the advantage that you make the same design as hardware that support geometric shaders. I don't know what do you think about that.

OpenGL 2.0 or 2.1 are fine for me. I haven't made a lot of experiments with pseudo-instancing but I think it is probably more useful with bigger meshes (like cubes). But it is surely something to experiment with. Geometry shaders can also be tested for some "advanced configuration", but I think it is probably better to stay focused on just one configuration at the beginning. I would start with something really basic using VBO and then test several possible optimization if it will be the bottleneck.

What is the exact set of features the engine should implement?

Before answering, let me give more details about the current engine. Elements are loaded in an octree, the display loop only shows elements which are in visible octants. Edges elements are both in source and target octant. Picking is used to only test selection in the octant the mouse is over. The most tricky is the nodes position update. Nodes has a X, Y, Z position that the engine continuously reads at each frame. This position is supposed to be changed all the time, it's the case when we do layout. When the setX(), setY() or setZ() function is called in NodeData, the octant where the element belongs is notified an element has changed his position. At next display, visible octants update flag are checked and if true, all nodes position in the octant are checked. If the node is out of his octant, it's removed and added to the octree again. The fact at each display all elements could have changed their position is a point that makes design difficult. In games, for instance in strategy game the map is loaded and stays, only few elements are not static. That's why the octree I'm using is static and has a fixed depth. From one frame to another, the dense part could become the sparse part, so it doesn't make sense to optimize accordingly to elements density.

I'm not sure I understand you when you says "the possible views of the graph are quite general and aren't always linked to the position of the nodes"

Yes, displaying elements is one thing, but putting interaction is another one. As I developed everything in a active loop, without VBO for compatibility reasons I have difficulties to imagine what performance problems it has, for updating elements sizes, colors, labels...

I would probably batch the updates together and then only modify "visible buffers" (VBO who contains visible nodes..). Another possible optimization can be to have different framerate for the updates and the rendering (using interpolation to fake a smoother update). Node update can be very tricky. Indeed, the most impressive particle system demos use the GPU to do that.

I think an octree isn't the best data structure for this kind of application. In real-time raytracers are quite populars BVHs (http://en.wikipedia.org/wiki/Bounding_volume_hierarchy) and I think it could be a good data structure to evaluate.

It is very late, so I will postpone to tomorrow a more in depth description of the engine I have in mind.

[apatriarca]

This is a rough description of my proposal to discuss it.

Node rendering
The nodes will be rendered as impostors: each node will be rendered as a view aligned quad and its appearance will be defined procedurally in a fragment shader. The 3 main node shapes (circle, square and sphere) can be easily and efficiently created in this way. Using this technique, it is very easy to add anti-aliasing (adding "soft edges" to the basic shapes) and effects like borders. Setting the depth of the fragments procedurally, it is also possible to correctly draw overlapping nodes. Another advantage of the procedural generation of the shapes is the ability to apply textures to the shapes.

Edge rendering
Edges will be rendered using quads. I think curved edges would be a cool feature, but they add a lot of additional complexity in both the renderer and layouts. They also can't be efficiently rendered. Therefore, I think I will only support straight edges.

Ambient occlusion
AO can really increase the perception of the shape of a 3D object. There are several existing algorithms and they all have their advantages and disadvantages. I don't know what is the best algorithm in this case and several tests should be done. I don't consider it as a priority in this project though.

Post processing effects
An halo implementation is described in the papers listed in the QuteMol project page. I think I will at least implement this and, probably, a "depth of field" effect (I simply want to blur non selected nodes).

Engine design
As I have described in the questionnaire, I plan to separate the engine design in two layers. The first layer, the core renderer, will be only responsible to render a set of nodes and edges passed to it. Indeed, the core renderer will be only able to create new node and edge buffers (that should be filled by the other layer) and shaders. It will have then the ability to render a buffer using a particular effect. The other layer has the responsability to communicate with the rest of the application and manage all the nodes. I would not use the octree of the original renderer. I prefer to evaluate other data structure to efficiently update the nodes. I think this layer will be the most harder to implement in an efficient way.

What do you think about it? Do you have some questions?