I would like to create a data frame that will present the accuracy of different seeds number and deep learning methods.I have created the code that contains two loops (see below) but I got an error, How can I create this loop correctly
Error in .h2o.doSafeREST(h2oRestApiVersion = h2oRestApiVersion, urlSuffix = page, :
ERROR MESSAGE:
Can only append one column
Attached is my code:
attach(iris)
train<-iris
test<-iris
invisible(capture.output(h2o.init(nthreads = -1))) # initalising with all cpu cores
trainHex <- as.h2o(train[1:200,])
testHex <- as.h2o(test)
x_names <- colnames(trainHex[1:4])
SEED<-c(123456789,12345678,1234567)
method<-c("Rectifier", "Tanh", "TanhWithDropout", "RectifierWithDropout", "Maxout", "MaxoutWithDropout")
Res<-data.frame()
for(i in 1:6){
for(j in 1:3){
system.time(ann <- h2o.deeplearning(
reproducible = TRUE,
seed = SEED[j],
x = x_names,
y = "Species",
training_frame = trainHex,epochs = 50,
standardize = TRUE,
nesterov_accelerated_gradient = T, # for speed
activation = method[i]
))
#ann
testHex$h20<-ifelse(predict(ann,newdata = testHex)>0.5,1,0)
testHex<-as.data.frame(testHex)
s<-xtabs(~Species +h20,data=testHex )
accuracy<-sum(diag(s))/sum(s)
tmp<-data.frame(seed=SEED[j],method=method[i],result=accuracy)
Res<-rbind(Res,tmp)
}
}
Error in .h2o.doSafeREST(h2oRestApiVersion = h2oRestApiVersion, urlSuffix = page, :
ERROR MESSAGE:
Can only append one column
You are doing a multinomial classification; i.e. the prediction is going to be one of three classes.
h2o.predict()
is therefore returning 4 columns:I'm not completely sure what you are doing, but given this to get the predictions:
You can do this to get a 1 for a correct answer, 0 for a mistake:
Or, doing it client-side:
More generally,
h2o.grid()
is better for experimenting with alternative parameters. I think this might be closer to your intention:(I've set epochs to 1 just to get it to finish quickly. Set to 50 if you want.)
I've used
splitFrame()
to use 80% as training data, 20% as test data; by assigning the test data tovalidation_frame
the grid will score on that unseen data automatically for us.