Combining AMI.js with XTK's volume rendering

Question

I am currently working on my Bachelor thesis about displaying 3D medical data in the browser. Therefore I am writing an application that can display DICOM images and meshes.

As far as I can see, the volume rendering of DICOMs is working better with XTK than with ami.js at the moment. The 3D rendering in ami.js of the DICOMs turned out to be not very useful (only max. 12 fps). I tried to render the DICOMs just like in Lesson 6 Volume Rendering example.

So my question is:
How can I combine the two frameworks, so that I can use the 3D volume rendering of XTK for the DICOMs, but all the other functionality from ami.js?
Or how do I have to use the volume rendering of ami.js, that it will turn out ok?

Answer 1

Edit: Improvement suggested in (2) has been implemented and is live at https://fnndsc.github.io/ami/#vr_singlepass

AMI and XTK use different techniques for volume rendering:

AMI supports:

• Raycasting
• MIP

XTK supports:

• Texture-based

I would recommend to only use 1 framework, depending on your needs.

Have you seen the Volume Rendering for AMI: https://fnndsc.github.io/ami/#vr_singlepass

Many techniques can be used to improve performance:

• (1) Only-render scene when the camera moved
• (2) Decrease pixel ratio when interacting with the scene

The AMI VR example implements (1).

To implement (2):

Initialize THREEJS renderer with setPixelRatio

threeD = document.getElementById('r3d');
renderer = new THREE.WebGLRenderer({
  alpha: true,
});
renderer.setPixelRatio(window.devicePixelRatio);
renderer.setSize(threeD.offsetWidth, threeD.offsetHeight);

Edit onStart() in one place:

// vrHelper.uniforms.uSteps.value = Math.floor(myStack.steps / 2);
// vrHelper.interpolation = 0;
renderer.setPixelRatio(.1 * window.devicePixelRatio);
renderer.setSize(threeD.offsetWidth, threeD.offsetHeight);

Edit onEnd() in 1 place:

// vrHelper.uniforms.uSteps.value = myStack.steps;
// vrHelper.interpolation = myStack.interpolation;
renderer.setPixelRatio(window.devicePixelRatio);
renderer.setSize(threeD.offsetWidth, threeD.offsetHeight);

Edit onWheel() in 2 places:

// vrHelper.uniforms.uSteps.value = Math.floor(myStack.steps / 2);
// vrHelper.interpolation = 0;
renderer.setPixelRatio(.1 * window.devicePixelRatio);
renderer.setSize(threeD.offsetWidth, threeD.offsetHeight);

and

wheelTO = setTimeout(function() {
// vrHelper.uniforms.uSteps.value = myStack.steps;
// vrHelper.interpolation = myStack.interpolation;
renderer.setPixelRatio(window.devicePixelRatio);
renderer.setSize(threeD.offsetWidth, threeD.offsetHeight);

HTH,