Multiple Render Targets (MRT) and OSG

Question

Folks,

I have studied about FBO, RTT and MRT to include this feature in my application, however I faced with some problems/doubts I did not find answers/tips during my search. Follows below the description of my scenario. I´ll be grateful if anyone can help me.

What I want to do?

• Attach two render textures (for color and depth buffers) to the same camera;
• Display only the color buffer in the post render camera;
• Read images from depth and color buffer in a final draw callback;
• Write collected float images in disk.

What have I got so far?

• Allow rendering for color or depth buffers separately, but not both on the same camera;
• Display the color buffer in the post render camera;
• Read color or depth buffer in the final draw callback;
• Write collected image (color or depth) in disk - only images as GL_UNSIGNED_BYTE. The following error is presented:

Error writing file ./Test-depth.png: Warning: Error in writing to "./Test-depth.png".

What are the doubts? (help!)

• How can I properly render both textures (color and depth buffer) in the same camera?
• How can I properly read both depth and color buffers in the final draw callback?
• During image writing in disk, why the error is presented only for images as GL_FLOAT, not for GL_UNSIGNED_BYTE?
• Is the render texture attached to an osg::Geode mandatory or optional in this process? Do I need to create two osg::Geode (one for each buffers), or only one osg::Geode for both?

Please take a look in my current source code (what am I doing wrong here?):

// OSG includes
#include <osgDB/ReadFile>
#include <osgDB/WriteFile>
#include <osgViewer/Viewer>
#include <osg/Camera>
#include <osg/Geode>
#include <osg/Geometry>
#include <osg/Texture2D>

struct SnapImage : public osg::Camera::DrawCallback {
    SnapImage(osg::GraphicsContext* gc) {
        _image = new osg::Image;
        _depth = new osg::Image;
        if (gc->getTraits()) {
            int width = gc->getTraits()->width;
            int height = gc->getTraits()->height;
            _image->allocateImage(width, height, 1, GL_RGBA, GL_FLOAT);
            _depth->allocateImage(width, height, 1, GL_DEPTH_COMPONENT, GL_FLOAT);
        }
    }

    virtual void operator () (osg::RenderInfo& renderInfo) const {
        osg::Camera* camera = renderInfo.getCurrentCamera();
        osg::GraphicsContext* gc = camera->getGraphicsContext();
        if (gc->getTraits() && _image.valid()) {
            int width = gc->getTraits()->width;
            int height = gc->getTraits()->height;
           _image->readPixels(0, 0, width, height, GL_RGBA, GL_FLOAT);
           _depth->readPixels(0, 0, width, height, GL_DEPTH_COMPONENT, GL_FLOAT);
            osgDB::writeImageFile(*_image,  "./Test-color.png");
            osgDB::writeImageFile(*_depth,  "./Test-depth.png");
        }
    }

    osg::ref_ptr<osg::Image> _image;
    osg::ref_ptr<osg::Image> _depth;
};

osg::Camera* setupMRTCamera( osg::ref_ptr<osg::Camera> camera, std::vector<osg::Texture2D*>& attachedTextures, int w, int h ) {
    camera->setClearColor( osg::Vec4() );
    camera->setClearMask( GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT );
    camera->setRenderTargetImplementation( osg::Camera::FRAME_BUFFER_OBJECT );
    camera->setRenderOrder( osg::Camera::PRE_RENDER );
    camera->setViewport( 0, 0, w, h );

    osg::Texture2D* tex = new osg::Texture2D;
    tex->setTextureSize( w, h );
    tex->setSourceType( GL_FLOAT );
    tex->setSourceFormat( GL_RGBA );
    tex->setInternalFormat( GL_RGBA32F_ARB );
    tex->setResizeNonPowerOfTwoHint( false );
    tex->setFilter( osg::Texture2D::MIN_FILTER, osg::Texture2D::LINEAR );
    tex->setFilter( osg::Texture2D::MAG_FILTER, osg::Texture2D::LINEAR );
    attachedTextures.push_back( tex );
    camera->attach( osg::Camera::COLOR_BUFFER, tex );

    tex = new osg::Texture2D;
    tex->setTextureSize( w, h );
    tex->setSourceType( GL_FLOAT );
    tex->setSourceFormat( GL_DEPTH_COMPONENT );
    tex->setInternalFormat( GL_DEPTH_COMPONENT32 );
    tex->setResizeNonPowerOfTwoHint( false );
    attachedTextures.push_back( tex );
    camera->attach( osg::Camera::DEPTH_BUFFER, tex );
    return camera.release();
}


int main() {
    osg::ref_ptr< osg::Group > root( new osg::Group );
    root->addChild( osgDB::readNodeFile( "cow.osg" ) );
    unsigned int winW = 800;
    unsigned int winH = 600;

    osgViewer::Viewer viewer;
    viewer.setUpViewInWindow( 0, 0, winW, winH );
    viewer.setSceneData( root.get() );
    viewer.realize();

    // setup MRT camera
    std::vector<osg::Texture2D*> attachedTextures;
    osg::Camera* mrtCamera ( viewer.getCamera() );
    setupMRTCamera( mrtCamera, attachedTextures, winW, winH );

    // set RTT textures to quad
    osg::Geode* geode( new osg::Geode );
    geode->addDrawable( osg::createTexturedQuadGeometry(
        osg::Vec3(-1,-1,0), osg::Vec3(2.0,0.0,0.0), osg::Vec3(0.0,2.0,0.0)) );
    geode->getOrCreateStateSet()->setTextureAttributeAndModes( 0, attachedTextures[0] );
    geode->getOrCreateStateSet()->setMode( GL_LIGHTING, osg::StateAttribute::OFF );
    geode->getOrCreateStateSet()->setMode( GL_DEPTH_TEST, osg::StateAttribute::OFF );

    // configure postRenderCamera to draw fullscreen textured quad
    osg::Camera* postRenderCamera( new osg::Camera );
    postRenderCamera->setClearMask( 0 );
    postRenderCamera->setRenderTargetImplementation( osg::Camera::FRAME_BUFFER, osg::Camera::FRAME_BUFFER );
    postRenderCamera->setReferenceFrame( osg::Camera::ABSOLUTE_RF );
    postRenderCamera->setRenderOrder( osg::Camera::POST_RENDER );
    postRenderCamera->setViewMatrix( osg::Matrixd::identity() );
    postRenderCamera->setProjectionMatrix( osg::Matrixd::identity() );
    postRenderCamera->addChild( geode );
    root->addChild(postRenderCamera);

    // setup the callback
    SnapImage* finalDrawCallback = new SnapImage(viewer.getCamera()->getGraphicsContext());
    mrtCamera->setFinalDrawCallback(finalDrawCallback);

    return (viewer.run());
}

Thanks in advance,

Rômulo Cerqueira