How to access the original ARKit camera image in a custom shader or SCNTechnique?

133 views Asked by At

ARKit materials have a .blendMode property that allows you to overlay the color of your rendered object on top of the video image. This works great, however, I need to use a different blend mode to apply the color to the video image. I have tried creating a custom SCNTechnique that applies my custom blend mode, and the shader works, but in order to blend the render objects color with the original ARKit video feed, I need access to the image before rendering. I have this working with the mask created from people occlusion in Metal rendering, but not with my own geometry. Additionally, I'd prefer to keep things in ARKit with the normal SceneKit rendering. Does anyone know how I can access the original ARKit video images in a custom SCNTechnique shader within the bounds of SceneKit? How I imagine this working is as part of the SCNTechnique Passes, I render the original video to a texture or pass it in somehow, then render the geometry to a different texture, then on a third pass, use both of those textures to compute my final image.

0

There are 0 answers