Does one-class svm provide a probability estimate?

Question

I am using Libsvm for outlier detection (from Java), but I need a probability estimate not just a label. I traced the code and found that this is not possible. In particular, in the function svm_predict_values(..) I see the following code:

if(model.param.svm_type == svm_parameter.ONE_CLASS)
        return (sum>0)?1:-1;
else
        return sum;

I understand that one-class SVM tries to estimate the support of some probability distribution given samples or data points from the "normal" class. Given a new data point, and given that the model has learned the support of the normal class distribution, can I get an estimate of the probability that a new data point is "normal" or an outlier?. It seems that this is not possible and that is why Libsvm thresholds the sum above and returns only a membership label, but I do not understand why. If it is possible to get a probability estimate from a one-class svm, I do not see how to do that in Libsvm after spending a lot of time reading the code.

The reason I went this rout is that I do not believe kernel density estimation would work well in a high dimensional setting, but maybe the svm is prone to the same issue.

Answer 1

I understand that one-class SVM tries to estimate the support of some probability distribution given samples or data points from the "normal" class

The problem is this sentence is false for SVM. In general - yes, this would be a nice probabilistic approach to built a classifier, taken by models like logistic regression, neural nets, and many others. However, SVM is not one of them, there is no proper probabilistic interpretation of SVM, it does not really construct probability distribution but rather directly looks for a nice decision rule. There are more probabilistic alternatives, like Relevance Vector Machines (RVM), which are however, non-convex. The only reason why binary SVM can provide you with probability estimates is because there is a small "cheat" in many implementations, originated by Platt, where you simply fit another, probabilistic model on top of SVM - typically Logistic Regression on top of SVM projection.

So, what can you do? You can either go for other, more probabilistic model, or use similar cheat, and first project your data through SVM (this is what "sum" is in the code provided) and then fit Logistic Regression on top of it, which will be your probability estimate.