How does ndarray.__new__ know from where it is being called?

Question

In the numpy documentation of subclassing ndarray here. It is said that

ndarray.__new__, passes __array_finalize__ the new object, of our own class (self) as well as the object from which the view has been taken (obj).

I understand ndarray.__new__ can create the new object self and pass that to __array_finalize__, but how does it get the obj from where it is called? From the signature of ndarray.__new__, looks like obj does not get passed from the calling function. So does ndarray.__new__ use some deep magic (probably at the C level) to determine where it is called (i.e. constructor call, view casting or new from template)?

Answer 1

TL:DR: it's the buffer argument to np.ndarray.__new__(...). The numpy docs on subclassing discuss creating a new object, a view, and a new-from-template. They show you how to create a new object of your subclass from scratch MySubClass(...) and a view object of your subclass arr.view(MySubClass), but they don't show you how to create a new-from-template of your subclass, other than mentioning that it happens in slicing.

You can see this by:

import numpy as np

arr = 2 * np.ones(3)
np.ndarray.__new__(np.ndarray, shape=(1,), dtype=float, buffer=arr)

gives array([2.]). Obviously, the object template is passed as a buffer, so things like striding, order, shape, dtype affect how the data in arr is interpreted.

As @tim-roberts pointed out, the InfoArray example is helpful, and in fact contains the complete call signature. I couldn't find it myself, possibly as numpy._core.__init__.pyi is deliberately left empty. Nevertheless, the type checker did find the full signature:

(cls: type[_ArraySelf@__new__], shape: _ShapeLike,
dtype: DTypeLike = ..., buffer: _SupportsBuffer | None = ...,
offset: SupportsIndex = ..., strides: _ShapeLike | None = ...,
order: _OrderKACF = ...) -> _ArraySelf@__new__

My use case, since Tim asked: My subclass carries around properties that refer to axes definitions (e.g. arr.ax_time) in order to make some math clearer (e.g. differentiate(arr, axis=arr.ax_time)). That meant that ndarray.view(MyClass) doesn't make a whole lot of sense, so I wanted to modify the example test (test_ufunc_override_with_super) that used standard views to use new-from-template instead. I know I can test this via slicing (I modify __getitem__), but I wasn't sure if that was the only way new-from-template is used.