Why use frustum in OpenGL on top of simple projection?

Question

Let's consider, for simplicity, a 3D scene where, after applying model and view matrices, all vertices are inside a cube [-1,-1,-1]...[1,1,1].

Based on the geometry set up shown on the image below, one can come up with a formula for projected coordinates (X_s, Y_s) of a point in space (X,Y,Z) onto the plane z=D:

from X_s / D = X / Z, it follows that

X_s = X / (Z/D), and similarly Y_s = Y / (Z/D).

From here we can construct a homogeneous coordinate (x, y, z, w) = (X, Y, Z, Z/D) that, when transformed back to Euclidean space, will give (X_s, Y_s, D).

The projection matrix P in this case is simply:

1, 0, 0,   0
0, 1, 0,   0
0, 0, 1,   0
0, 0, 1/D, 0

and P * (X, Y, Z, 1) = (x, y, z, w)

To keep z-test working in, say, fragment shader, one can restore z-information by specifying:

gl_FragDepth = gl_FragCoord.z / gl_FragCoord.w;

The code above and matrix P seem to be enough to have all the projective geometry working.

One could also use a scaling matrix along with P to account for the viewport aspect ratio and to scale the original scene to a [-1,-1,-1]...[1,1,1] space if necessary.

So why use frustum?

The deformation of frustum to the clip space is quite complex and is not needed to create perspective.

Is it only to simulate a physical camera that has a viewing cone or are there any other reasons?