I Come from a predominantly python + scikit learn background, and I was wondering how would one obtain the cross validation accuracy for a logistic regression model in R? I was searching and surprised that there's no easy way to this. I'm looking for the equivalent:
import pandas as pd from sklearn.cross_validation import cross_val_score from sklearn.linear_model import LogisticRegression ## Assume pandas dataframe of dataset and target exist. scores = cross_val_score(LogisticRegression(),dataset,target,cv=10) print(scores) For R: I have:
model = glm(df$Y~df$X,family=binomial') summary(model) And now I'm stuck. Reason being, the deviance for my R model is 1900, implying its a bad fit, but the python one gives me 85% 10 fold cross validation accuracy.. which means its good. Seems a bit strange... So i wanted to run cross val in R to see if its the same result.
Any help is appreciated!
22 Answers
R version using caret package:
library(caret) # define training control train_control <- trainControl(method = "cv", number = 10) # train the model on training set model <- train(target ~ ., data = train, trControl = train_control, method = "glm", family=binomial()) # print cv scores summary(model) Below I took an answer from here and made a few changes.
The changes I made were to make it a logit (logistic) model, add modeling and prediction, store the CV's results, and to make it a fully working example.
Also note that there are many packages and functions you could use, including cv.glm() from boot.
data(ChickWeight) df <- ChickWeight df$Y <- 0 df$Y[df$weight > 100] <- 1 df$X <- df$Diet df <- df[sample(nrow(df)),] folds <- cut(seq(1,nrow(df)),breaks=10,labels=FALSE) result <- list() for(i in 1:10){ testIndexes <- which(folds==i,arr.ind=TRUE) testData <- df[testIndexes, ] trainData <- df[-testIndexes, ] model <- glm(Y~X,family=binomial,data=trainData) result[[i]] <- predict(model, testData) } result You could add a line to calculate accuracy within the loop or just do it after the loop completes.
3
