I have a list of 3d boxes determined by 8 Corner points (but they are assumed to be rectangular boxes).
What is an efficient algorithm to calculate which boxes overlap each other from the perspective of a certain position and get the info which is nearer (and at best with some overlap coefficient)?
The objects nearly sitting on the same ground plane - that may be a strong help for optimization.
The most naive approach may be to do intersection tests of each object corner with all planes of the other objects. this of course may be quite inefficient. May be optimized by first calculate the visible faces of each object. This approach also has the problem that smaller nearer object may completely inside the projection of a larger one more far away.
Another approach that came to my mind is to calculate all angles in the groundplane to all corners and sort them and then check if interleaved object corners exist. if yes, then these objects require additional (above mentioned intersection) checks.
This is a common subject in computer-graphics world. It's called "Occlusion Query"
The general approach, for all kinds of objects (not only boxes), is fully draw (pixel by pixel) them all into a 3D box (called NDC system). Only draw the nearest pixel to the camera (or "point of view"). You can store also the calculated "z" coord (in & out of screen) assigning an index, so as to get the proper object with such "z".
This is a intense work which is appropriate for a GPU.
If all objects are boxes then things gets a bit easier for a CPU calculation, you can work with corners and dismiss a box if its eight corners are hidden (far, bigger z) by another box.
But if boxes can intersect due to, for example, they are not axis-aligned, then, again, the work is better done in the GPU.
If you go with CPU solution, I recommend learning about matrix-transformations and the usual "model, view, projection" matrices required to draw onto a 2D plane.