Installing FlinkML DenseVector dependency - why are there two different implementations?

Question

I'm a bit confused as to how to install the dependencies I actually need.

I'm new to both Java and Flink, and I think I'm missing something minor here. I'm doing a basic exercise where I need the DenseVector class, that supports basic mathematical operations.

I searched for flink docs and found this class. So it's artefact id is org.apache.flink.ml.common.linalg.DenseVector. This class supports operations such as dot product.

Now I go to the flinkml tutorial page and it says I need the following dependency

<dependency>
    <groupId>org.apache.flink</groupId>
    <artifactId>flink-ml-uber</artifactId>
    <version>2.1.0</version>
</dependency>

When I install that however, the only DenseVector I get installed is this one, which seems to be representing the same entity, but the API is very limited - it basically supports no mathematical operations.

My question is: I can't seem to find a way to install a dependency that would give me access to org.apache.flink.ml.common.linalg.DenseVector. What maven dependency do I need to get that? I've checked a few packages at https://mvnrepository.com/, but could not find the one I need. What am I missing?

Thanks!

Answer 1

There are two different implementations because there was a major refactor with FlinkML 2.0.

You can install the dependency according to https://nightlies.apache.org/flink/flink-ml-docs-release-2.1/docs/try-flink-ml/build-your-own-project/. Then you can get access to those mathematical operations by using this class https://github.com/apache/flink-ml/blob/master/flink-ml-core/src/main/java/org/apache/flink/ml/linalg/BLAS.java .

Maybe we can use the answer provided by the owner and close this thread.

Answer 2

Okay, so I figured out what was going on. Apparently, there was a major refactor with FlinkML 2.0, because the package I'm interested in is available all the way up to this version

<dependency>
    <groupId>org.apache.flink</groupId>
    <artifactId>flink-ml-lib_2.12</artifactId>
    <version>1.12.7</version>
</dependency>

With this version, the latest update that is

<dependency>
    <groupId>org.apache.flink</groupId>
    <artifactId>flink-ml-lib_2.12</artifactId>
    <version>2.0.0</version>
</dependency>

the codebase was refactored as the dependency I was looking for can now be find with this import: import org.apache.flink.ml.math.DenseVector; This vector implementation has all mathematical operations I was interested in.