I am trying to find the best parameters for my Neural Network I want to create in R. I am using the h2o package and following the tutorial from
The code I have seems to run in 1 minute and from what I understood grid search should run multiple models until the best parameters are determined and that would take while to run. Please let me know where I am going wrong and how I can do the gird search to optimise my parameters.
h2o.init(nthreads=-1,max_mem_size='6G') testHex = as.h2o(test) trainHex = as.h2o(training) predictors <-colnames(training)[!(colnames(training) %in% c("responseVar"))] response = "responseVar" hyper_params <- list( activation=c("Rectifier","Tanh","Maxout","RectifierWithDropout","TanhWithDropout","MaxoutWithDropout"), hidden=list(c(20,20),c(50,50),c(75,75),c(100,100),c(30,30,30),c(25,25,25,25)), input_dropout_ratio=c(0,0.03,0.05), #rate=c(0.01,0.02,0.05), l1=seq(0,1e-4,1e-6), l2=seq(0,1e-4,1e-6) ) h2o.rm("dl_grid_random") search_criteria = list(strategy = "RandomDiscrete", max_runtime_secs = 360, max_models = 100, seed=1234567, stopping_rounds=5, stopping_tolerance=1e-2) dl_random_grid <- h2o.grid( algorithm="deeplearning", grid_id = "dl_grid_random", training_frame=trainHex, x=predictors, y=response, epochs=1, stopping_metric="RMSE", stopping_tolerance=1e-2, ## stop when logloss does not improve by >=1% for 2 scoring events stopping_rounds=2, score_validation_samples=10000, ## downsample validation set for faster scoring score_duty_cycle=0.025, ## don't score more than 2.5% of the wall time max_w2=10, ## can help improve stability for Rectifier hyper_params = hyper_params, search_criteria = search_criteria ) grid <- h2o.getGrid("dl_grid_random",sort_by="mae",decreasing=FALSE) grid grid@summary_table[1,] best_model <- h2o.getModel(grid@model_ids[[1]]) ## model with lowest logloss best_model 1 Answer
You have set max_runtime_secs = 360 via search_criteria in the grid settings, so the longest it will potentially run is 6 minutes.
If the grid is stopping prior to that, it means that your early stopping settings are triggering the grid to stop early. If you'd like it to run longer, then you can increase stopping_rounds for the grid and/or increase the stopping_tolerance and it should be less sensitive to stopping early and run longer (in the code above you have them set to 5 and 1e-2, respectively). You might also want to set stopping_metric = "RMSE", since the default value for regression is mean residual deviance ("deviance"). See more information about stopping metric in the user guide.
I noticed that you have stopping_rounds = 2 for the individual DNN models (you might want to try increasing that if your models are underfitting).
