Obtaining an AUC value from the KNN function

I am using the class package in order to use the KNN algorithm. I am also using the ROCR package to calculate the AUC value.

knn_one<-knn(train, test, train$Digit, k=1)

To calculate the AUC value for another method, e.g. classification trees, I used these series of commands:

treeTrain_Pred<-predict(Tree_Train, test , type = "prob")[,2] Pred<-prediction(treeTrain_Pred, test$Digit) Perf<-performance(Pred, "auc") [[1]] 

However, when I try

knn_one = predict(knn_one, test, type="prob")[,2] 

I get the following error:

Error in UseMethod("predict") : no applicable method for 'predict' applied to an object of class "factor" 

How can I fix this and obtain an AUC value for my KNN function?

1 Answer

There is no predict method for knn models, instead you train and receive predictions as part of a single call. Example on sonar data:

library(mlbench) data(Sonar) 

create data partition:

set.seed(1) tr_ind <- sample(1:nrow(Sonar), 150) train <- Sonar[tr_ind,] test <- Sonar[-tr_ind,] mod <- class::knn(cl = train$Class, test = test[,1:60], train = train[,1:60], k = 5, prob = TRUE) 

Now the probability of the predictions are in:

attributes(mod)$prob library(pROC) roc(test$Class, attributes(mod)$prob) #output Call: roc.default(response = test$Class, predictor = attributes(mod)$prob) Data: attributes(mod)$prob in 30 controls (test$Class M) < 28 cases (test$Class R). Area under the curve: 0.4667 plot(roc(test$Class, attributes(mod)$prob), print.thres = T, print.auc=T) 

lets try with k = 4

mod <- class::knn(cl = train$Class, test = test[,1:60], train = train[,1:60], k = 4, prob = TRUE) plot(roc(test$Class, attributes(mod)$prob), print.thres = T, print.auc = T, print.auc.y = 0.2)