41079

predict.glmnet: Some Factors Have Only One Level in New Data

Question:

I've trained an elastic net model in R using glmnet and would like to use it to make predictions off of a new data set.

But I'm having trouble producing the matrix to use as an argument in the predict() method because some of my factor variables (dummy variables indicating the presence of comorbidities) in the new data set only have one level (the comorbidities were never observed), which means I can't use

<blockquote>

model.matrix(RESPONSE ~ ., new_data)

</blockquote>

because it gives me the (expected)

<blockquote>

Error in contrasts<-(*tmp*, value = contr.funs[1 + isOF[nn]]) : contrasts can be applied only to factors with 2 or more levels

</blockquote>

I'm at a loss for how to get around this issue. Is there a way in R that I can construct an appropriate matrix for use in predict() in this situation, or do I need to prepare the matrix outside of R? In either case, how might I go about doing it?

Here is a toy example that reproduces the issue I'm having:

x1 <- rnorm(100) x2 <- as.factor(rbinom(100, 1, 0.6)) x3 <- as.factor(rbinom(100, 1, 0.4)) y <- rbinom(100, 1, 0.2) toy_data <- data.frame(x1, x2, x3, y) colnames(toy_data) = c("Continuous", "FactorA", "FactorB", "Outcome") mat1 <- model.matrix(Outcome ~ ., toy_data)[,-1] y1 <- toy_data$Outcome new_data <- toy_data new_data$FactorB <- as.factor(0) #summary(new_data) # Just to verify that FactorB now only contains one level mat2 <- model.matrix(Outcome ~ ., new_data)[,-1]

Answer1:

You can set the levels of your dataset to match the levels of the complete dataset in your example. A factor can have values present in the levels even when that value isn't present in the variable.

You can do this with the levels argument in factor():

new_data$FactorB <- factor(0, levels = levels(toy_data$FactorB))

Or by using the levels() function with assignment:

levels(new_data$FactorB) <- levels(toy_data$FactorB)

Using either approach, model.matrix() works properly once you have more than one level:

head( model.matrix(Outcome ~ ., new_data)[,-1] ) Continuous FactorA1 FactorB1 1 -1.91632972 0 0 2 1.11411267 0 0 3 -1.21333837 1 0 4 -0.06311276 0 0 5 1.31599915 0 0 6 0.36374591 1 0

Recommend

  • Special character not recognized
  • How to filter a column by multiple, flexible criteria
  • RGB range for cold and warm colors?
  • Find and replace word in text file
  • Can I construct a numpy object zero-d array from its value in a single expression?
  • Change multiple character columns to date
  • EF4 and Server Generated UniqueIdentifier Id
  • flatten multidimensional array into csv text with PHP
  • Merging rows with shared information
  • Type-safe way to divide a tuple into multiple tuples
  • Cythonized function unexpectedly slow
  • Passing parameter through “window.location.href”
  • why do I get the error when installing the gem 'pg'? [duplicate]
  • Paperclip, set path outside of rails root folder
  • SignalR .NET Client Invoke throws an exception
  • Using variable in a value field in jMeter
  • Using $this when not in object context
  • Array.prototype.includes - not transformed with babel
  • Volley JsonObjectRequest send headers in GET Request
  • Accessing IRQ description array within a module and displaying action names
  • Shallow update not allowed (git > 1.9)
  • Can a Chrome extension content script make an jQuery AJAX request for an html file that is itself a
  • When should I choose bucket sort over other sorting algorithms?
  • Build own AppleScript numerical error handling
  • Websockets service method fails during R startup
  • How do you troubleshoot character encoding problems?
  • Google cloud sdk not working when python points python3
  • Is there a mandatory requirement to switch app.yaml?
  • File upload with ng-file-upload throwing error
  • ExecuteAsync RestSharp to allow backgroundWorker CancellationPending c#
  • AngularJs get employee from factory
  • Hits per day in Google Big Query
  • Understanding cpu registers
  • FormattedException instead of throw new Exception(string.Format(…)) in .NET
  • Linking SubReports Without LinkChild/LinkMaster
  • XCode 8, some methods disappeared ? ex: layoutAttributesClass() -> AnyClass
  • costura.fody for a dll that references another dll
  • Observable and ngFor in Angular 2
  • UserPrincipal.Current returns apppool on IIS
  • java string with new operator and a literal