Load packages and data

library(tidyverse)
## ── Attaching packages ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── tidyverse 1.3.0 ──
## ✓ ggplot2 3.3.2     ✓ purrr   0.3.4
## ✓ tibble  3.0.3     ✓ dplyr   1.0.2
## ✓ tidyr   1.1.1     ✓ stringr 1.4.0
## ✓ readr   1.3.1     ✓ forcats 0.5.0
## ── Conflicts ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── tidyverse_conflicts() ──
## x dplyr::filter() masks stats::filter()
## x dplyr::lag()    masks stats::lag()
df <- readr::read_rds("df.rds")
step_2_b_df <- readr::read_rds("step_2_b_df.rds")

Overview

  1. Exploration
  2. Regression models
  3. Binary classification option b
  4. Binary classification option a
  5. Interpretation and optimization

This R Markdown file tackles part iii, specifically training, evaluating, tuning, and comparing models for the binary classifier outcome_2 as a function of xA, xB, response_1, x07:x11.

We will use the caret package to handle training, testing, and evaluation.

library(caret)
## Loading required package: lattice
## 
## Attaching package: 'caret'
## The following object is masked from 'package:purrr':
## 
##     lift

General

Throughout all methods, we will use 5-fold cross validation, as our resampling method, by specifying "repeatedcv" as the method argument to a caret::trainControl(). For this classification problem, we will use the Area under the ROC curve as our primary performance metric. We must specify the summaryFunction argument to be twoClassSummary within the trainControl() function in order to maximize the area under the ROC curve. We will also instruct caret to return the class predicted probabilities.

my_ctrl <- caret::trainControl(method = "repeatedcv", 
                               number = 5, 
                               repeats = 5, 
                               savePredictions = TRUE, 
                               summaryFunction = twoClassSummary, 
                               classProbs = TRUE)
roc_metric <- "ROC"

Logistic regression with additive terms

First we will train a logistic regression model with additive terms, using method = "glm" in caret::train. We will train the model for outcome_2 as a function of xA, xB, response_1, x07:x11.

The main purpose of this logistic regression model is to provide a baseline comparison to the other complex models we will train.

set.seed(12345)
mod_glm <- caret::train(outcome_2 ~ .,
                            method = "glm", 
                            metric = roc_metric, 
                            trControl = my_ctrl,
                            preProcess = c("center", "scale"),
                            data = step_2_b_df)

mod_glm
## Generalized Linear Model 
## 
## 2013 samples
##    8 predictor
##    2 classes: 'Fail', 'Pass' 
## 
## Pre-processing: centered (10), scaled (10) 
## Resampling: Cross-Validated (5 fold, repeated 5 times) 
## Summary of sample sizes: 1610, 1611, 1611, 1610, 1610, 1611, ... 
## Resampling results:
## 
##   ROC        Sens       Spec     
##   0.6230073  0.5429442  0.6488326

Look at confusion matrix associated with the mod_glm model.

confusionMatrix.train(mod_glm)
## Cross-Validated (5 fold, repeated 5 times) Confusion Matrix 
## 
## (entries are percentual average cell counts across resamples)
##  
##           Reference
## Prediction Fail Pass
##       Fail 26.6 17.9
##       Pass 22.4 33.1
##                            
##  Accuracy (average) : 0.597

Regularized regression with elastic net

We now try a regularization approach. Elastic net is a mixture between Lasso and Ridge penalties. We will train two different models with interactions, specifically one with all pair interactions between all step_2_b_df input variables, and one with all triplet interactions.

All pair interactions

Let’s first fit a regularized regression model with elastic net, on all pairwise interactions between all step_2_b_df inputs, using caret::train with method="glmnet". We specify centering and scaling as preprocessing steps.

set.seed(12345)
mod_glmnet_2 <- caret::train(outcome_2 ~ (.)^2,
                             method = "glmnet",
                             preProcess = c("center", "scale"),
                             metric = roc_metric,
                             trControl = my_ctrl,
                             data = step_2_b_df)

mod_glmnet_2
## glmnet 
## 
## 2013 samples
##    8 predictor
##    2 classes: 'Fail', 'Pass' 
## 
## Pre-processing: centered (52), scaled (52) 
## Resampling: Cross-Validated (5 fold, repeated 5 times) 
## Summary of sample sizes: 1610, 1611, 1611, 1610, 1610, 1611, ... 
## Resampling results across tuning parameters:
## 
##   alpha  lambda        ROC        Sens       Spec     
##   0.10   0.0002009229  0.6899008  0.5602030  0.7210097
##   0.10   0.0020092294  0.6863258  0.5526904  0.7186739
##   0.10   0.0200922935  0.6856803  0.5472081  0.7260725
##   0.55   0.0002009229  0.6910454  0.5612183  0.7196486
##   0.55   0.0020092294  0.6858822  0.5512690  0.7180914
##   0.55   0.0200922935  0.6879072  0.5352284  0.7350206
##   1.00   0.0002009229  0.6919745  0.5656853  0.7194525
##   1.00   0.0020092294  0.6867074  0.5510660  0.7190632
##   1.00   0.0200922935  0.6866352  0.5181726  0.7540876
## 
## ROC was used to select the optimal model using the largest value.
## The final values used for the model were alpha = 1 and lambda = 0.0002009229.

Create a custom tuning grid enet_grid to try out many possible values of the penalty factor (lambda) and the mixing fraction (alpha).

enet_grid <- expand.grid(alpha = seq(0.1, 0.9, by = 0.1),
                         lambda = exp(seq(-6, 0.5, length.out = 25)))

Now retrain the pairwise interactions model using tuneGrid = enet_grid.

set.seed(12345)
mod_glmnet_2_b <- caret::train(outcome_2 ~ (.)^2,
                               method = "glmnet",
                               preProcess = c("center", "scale"),
                               tuneGrid = enet_grid,
                               metric = roc_metric,
                               trControl = my_ctrl,
                               data = step_2_b_df)

mod_glmnet_2_b
## glmnet 
## 
## 2013 samples
##    8 predictor
##    2 classes: 'Fail', 'Pass' 
## 
## Pre-processing: centered (52), scaled (52) 
## Resampling: Cross-Validated (5 fold, repeated 5 times) 
## Summary of sample sizes: 1610, 1611, 1611, 1610, 1610, 1611, ... 
## Resampling results across tuning parameters:
## 
##   alpha  lambda       ROC        Sens         Spec     
##   0.1    0.002478752  0.6860739  0.552284264  0.7192583
##   0.1    0.003249784  0.6858461  0.551675127  0.7190651
##   0.1    0.004260651  0.6857079  0.550659898  0.7182856
##   0.1    0.005585954  0.6856006  0.551269036  0.7192565
##   0.1    0.007323502  0.6857026  0.550659898  0.7188700
##   0.1    0.009601525  0.6858698  0.550659898  0.7192583
##   0.1    0.012588142  0.6859417  0.549644670  0.7208165
##   0.1    0.016503766  0.6856930  0.548832487  0.7235387
##   0.1    0.021637371  0.6856744  0.547005076  0.7272394
##   0.1    0.028367816  0.6854550  0.544162437  0.7301577
##   0.1    0.037191811  0.6852410  0.539086294  0.7322965
##   0.1    0.048760568  0.6848695  0.536852792  0.7363827
##   0.1    0.063927861  0.6839911  0.531573604  0.7431901
##   0.1    0.083813041  0.6822212  0.522639594  0.7494132
##   0.1    0.109883635  0.6796847  0.510253807  0.7573971
##   0.1    0.144063659  0.6760792  0.493604061  0.7651774
##   0.1    0.188875603  0.6709093  0.467614213  0.7819124
##   0.1    0.247626595  0.6626101  0.428223350  0.8033152
##   0.1    0.324652467  0.6489469  0.387411168  0.8371613
##   0.1    0.425637741  0.6270342  0.332182741  0.8721695
##   0.1    0.558035146  0.6035123  0.225786802  0.9087568
##   0.1    0.731615629  0.5664170  0.120203046  0.9733545
##   0.1    0.959189457  0.5475750  0.005685279  0.9992224
##   0.1    1.257551613  0.5000000  0.000000000  1.0000000
##   0.1    1.648721271  0.5000000  0.000000000  1.0000000
##   0.2    0.002478752  0.6857670  0.551269036  0.7180933
##   0.2    0.003249784  0.6855144  0.551675127  0.7177021
##   0.2    0.004260651  0.6856550  0.551065990  0.7188681
##   0.2    0.005585954  0.6857729  0.550862944  0.7194516
##   0.2    0.007323502  0.6859659  0.550050761  0.7190651
##   0.2    0.009601525  0.6860676  0.549644670  0.7208193
##   0.2    0.012588142  0.6861723  0.548629442  0.7217864
##   0.2    0.016503766  0.6863817  0.547614213  0.7264561
##   0.2    0.021637371  0.6864606  0.543147208  0.7289870
##   0.2    0.028367816  0.6866722  0.537664975  0.7338546
##   0.2    0.037191811  0.6864140  0.532994924  0.7402785
##   0.2    0.048760568  0.6855325  0.525076142  0.7482520
##   0.2    0.063927861  0.6839517  0.515329949  0.7552583
##   0.2    0.083813041  0.6817590  0.499492386  0.7659616
##   0.2    0.109883635  0.6779782  0.477563452  0.7838541
##   0.2    0.144063659  0.6719985  0.432893401  0.8134189
##   0.2    0.188875603  0.6590478  0.368527919  0.8571868
##   0.2    0.247626595  0.6178985  0.249746193  0.9017457
##   0.2    0.324652467  0.5735423  0.176243655  0.9387184
##   0.2    0.425637741  0.5646142  0.090964467  0.9819114
##   0.2    0.558035146  0.5000000  0.000000000  1.0000000
##   0.2    0.731615629  0.5000000  0.000000000  1.0000000
##   0.2    0.959189457  0.5000000  0.000000000  1.0000000
##   0.2    1.257551613  0.5000000  0.000000000  1.0000000
##   0.2    1.648721271  0.5000000  0.000000000  1.0000000
##   0.3    0.002478752  0.6855262  0.551269036  0.7171196
##   0.3    0.003249784  0.6857567  0.551472081  0.7186758
##   0.3    0.004260651  0.6858984  0.550456853  0.7190623
##   0.3    0.005585954  0.6860252  0.549238579  0.7184816
##   0.3    0.007323502  0.6861626  0.550050761  0.7194554
##   0.3    0.009601525  0.6863108  0.547208122  0.7190670
##   0.3    0.012588142  0.6867950  0.547208122  0.7227611
##   0.3    0.016503766  0.6871967  0.545380711  0.7272337
##   0.3    0.021637371  0.6875365  0.539492386  0.7315188
##   0.3    0.028367816  0.6873415  0.533604061  0.7377457
##   0.3    0.037191811  0.6866385  0.525685279  0.7443561
##   0.3    0.048760568  0.6853485  0.515126904  0.7550604
##   0.3    0.063927861  0.6831125  0.498274112  0.7679005
##   0.3    0.083813041  0.6793538  0.472487310  0.7896888
##   0.3    0.109883635  0.6727592  0.420507614  0.8262628
##   0.3    0.144063659  0.6502934  0.315939086  0.8807265
##   0.3    0.188875603  0.5947134  0.198375635  0.9223727
##   0.3    0.247626595  0.5647183  0.156954315  0.9538944
##   0.3    0.324652467  0.5372668  0.019492386  0.9963022
##   0.3    0.425637741  0.5000000  0.000000000  1.0000000
##   0.3    0.558035146  0.5000000  0.000000000  1.0000000
##   0.3    0.731615629  0.5000000  0.000000000  1.0000000
##   0.3    0.959189457  0.5000000  0.000000000  1.0000000
##   0.3    1.257551613  0.5000000  0.000000000  1.0000000
##   0.3    1.648721271  0.5000000  0.000000000  1.0000000
##   0.4    0.002478752  0.6857773  0.551269036  0.7180924
##   0.4    0.003249784  0.6859614  0.550456853  0.7194525
##   0.4    0.004260651  0.6860992  0.548832487  0.7184807
##   0.4    0.005585954  0.6862911  0.550050761  0.7179001
##   0.4    0.007323502  0.6864629  0.548020305  0.7186787
##   0.4    0.009601525  0.6867617  0.547005076  0.7208146
##   0.4    0.012588142  0.6873807  0.545989848  0.7247066
##   0.4    0.016503766  0.6879338  0.543147208  0.7282074
##   0.4    0.021637371  0.6879212  0.537055838  0.7324925
##   0.4    0.028367816  0.6874136  0.529746193  0.7412437
##   0.4    0.037191811  0.6863198  0.519187817  0.7513635
##   0.4    0.048760568  0.6842918  0.504162437  0.7673152
##   0.4    0.063927861  0.6811679  0.481015228  0.7883239
##   0.4    0.083813041  0.6745089  0.425380711  0.8223737
##   0.4    0.109883635  0.6492606  0.305989848  0.8840322
##   0.4    0.144063659  0.5844083  0.194314721  0.9272328
##   0.4    0.188875603  0.5643279  0.159796954  0.9507781
##   0.4    0.247626595  0.5223748  0.019086294  0.9963022
##   0.4    0.324652467  0.5000000  0.000000000  1.0000000
##   0.4    0.425637741  0.5000000  0.000000000  1.0000000
##   0.4    0.558035146  0.5000000  0.000000000  1.0000000
##   0.4    0.731615629  0.5000000  0.000000000  1.0000000
##   0.4    0.959189457  0.5000000  0.000000000  1.0000000
##   0.4    1.257551613  0.5000000  0.000000000  1.0000000
##   0.4    1.648721271  0.5000000  0.000000000  1.0000000
##   0.5    0.002478752  0.6859227  0.549847716  0.7190632
##   0.5    0.003249784  0.6861373  0.549441624  0.7184798
##   0.5    0.004260651  0.6863195  0.549644670  0.7188719
##   0.5    0.005585954  0.6865409  0.550862944  0.7186777
##   0.5    0.007323502  0.6866983  0.548020305  0.7188709
##   0.5    0.009601525  0.6872111  0.546802030  0.7219825
##   0.5    0.012588142  0.6878915  0.545177665  0.7241203
##   0.5    0.016503766  0.6882423  0.540507614  0.7293763
##   0.5    0.021637371  0.6878747  0.534619289  0.7357982
##   0.5    0.028367816  0.6871196  0.525482234  0.7457229
##   0.5    0.037191811  0.6857026  0.510456853  0.7601193
##   0.5    0.048760568  0.6828236  0.492791878  0.7809339
##   0.5    0.063927861  0.6771403  0.447512690  0.8103121
##   0.5    0.083813041  0.6584815  0.338071066  0.8717765
##   0.5    0.109883635  0.5928573  0.197969543  0.9256765
##   0.5    0.144063659  0.5652270  0.170761421  0.9428037
##   0.5    0.188875603  0.5475750  0.047309645  0.9881317
##   0.5    0.247626595  0.5000000  0.000000000  1.0000000
##   0.5    0.324652467  0.5000000  0.000000000  1.0000000
##   0.5    0.425637741  0.5000000  0.000000000  1.0000000
##   0.5    0.558035146  0.5000000  0.000000000  1.0000000
##   0.5    0.731615629  0.5000000  0.000000000  1.0000000
##   0.5    0.959189457  0.5000000  0.000000000  1.0000000
##   0.5    1.257551613  0.5000000  0.000000000  1.0000000
##   0.5    1.648721271  0.5000000  0.000000000  1.0000000
##   0.6    0.002478752  0.6860662  0.549644670  0.7192574
##   0.6    0.003249784  0.6862759  0.549644670  0.7177021
##   0.6    0.004260651  0.6865044  0.550862944  0.7182894
##   0.6    0.005585954  0.6867741  0.549238579  0.7180924
##   0.6    0.007323502  0.6870432  0.547208122  0.7190642
##   0.6    0.009601525  0.6877801  0.547208122  0.7217864
##   0.6    0.012588142  0.6881916  0.545380711  0.7258735
##   0.6    0.016503766  0.6882200  0.539492386  0.7317149
##   0.6    0.021637371  0.6876132  0.529746193  0.7392991
##   0.6    0.028367816  0.6866873  0.518984772  0.7519526
##   0.6    0.037191811  0.6846122  0.502335025  0.7715974
##   0.6    0.048760568  0.6804390  0.473502538  0.7984438
##   0.6    0.063927861  0.6690011  0.394923858  0.8433786
##   0.6    0.083813041  0.6176788  0.213197970  0.9153692
##   0.6    0.109883635  0.5677658  0.182131980  0.9334615
##   0.6    0.144063659  0.5646142  0.135025381  0.9646005
##   0.6    0.188875603  0.5000000  0.000000000  1.0000000
##   0.6    0.247626595  0.5000000  0.000000000  1.0000000
##   0.6    0.324652467  0.5000000  0.000000000  1.0000000
##   0.6    0.425637741  0.5000000  0.000000000  1.0000000
##   0.6    0.558035146  0.5000000  0.000000000  1.0000000
##   0.6    0.731615629  0.5000000  0.000000000  1.0000000
##   0.6    0.959189457  0.5000000  0.000000000  1.0000000
##   0.6    1.257551613  0.5000000  0.000000000  1.0000000
##   0.6    1.648721271  0.5000000  0.000000000  1.0000000
##   0.7    0.002478752  0.6862795  0.549238579  0.7184807
##   0.7    0.003249784  0.6863470  0.550050761  0.7180942
##   0.7    0.004260651  0.6866220  0.550050761  0.7186787
##   0.7    0.005585954  0.6869401  0.548426396  0.7179010
##   0.7    0.007323502  0.6874660  0.548020305  0.7200379
##   0.7    0.009601525  0.6881534  0.546395939  0.7227601
##   0.7    0.012588142  0.6884292  0.542944162  0.7278210
##   0.7    0.016503766  0.6881562  0.536649746  0.7330751
##   0.7    0.021637371  0.6874909  0.525076142  0.7439697
##   0.7    0.028367816  0.6861433  0.513908629  0.7589534
##   0.7    0.037191811  0.6830915  0.493197970  0.7832688
##   0.7    0.048760568  0.6762752  0.440609137  0.8147819
##   0.7    0.063927861  0.6435505  0.288121827  0.8848061
##   0.7    0.083813041  0.5809687  0.190253807  0.9291793
##   0.7    0.109883635  0.5640952  0.167106599  0.9451366
##   0.7    0.144063659  0.5140434  0.019289340  0.9955226
##   0.7    0.188875603  0.5000000  0.000000000  1.0000000
##   0.7    0.247626595  0.5000000  0.000000000  1.0000000
##   0.7    0.324652467  0.5000000  0.000000000  1.0000000
##   0.7    0.425637741  0.5000000  0.000000000  1.0000000
##   0.7    0.558035146  0.5000000  0.000000000  1.0000000
##   0.7    0.731615629  0.5000000  0.000000000  1.0000000
##   0.7    0.959189457  0.5000000  0.000000000  1.0000000
##   0.7    1.257551613  0.5000000  0.000000000  1.0000000
##   0.7    1.648721271  0.5000000  0.000000000  1.0000000
##   0.8    0.002478752  0.6863824  0.550050761  0.7182865
##   0.8    0.003249784  0.6865248  0.550659898  0.7173166
##   0.8    0.004260651  0.6868808  0.550050761  0.7180942
##   0.8    0.005585954  0.6871951  0.548223350  0.7169254
##   0.8    0.007323502  0.6878127  0.548020305  0.7196495
##   0.8    0.009601525  0.6884802  0.546802030  0.7227611
##   0.8    0.012588142  0.6884157  0.541928934  0.7293791
##   0.8    0.016503766  0.6880747  0.534010152  0.7363817
##   0.8    0.021637371  0.6871249  0.522030457  0.7490296
##   0.8    0.028367816  0.6852552  0.505786802  0.7673142
##   0.8    0.037191811  0.6811506  0.474923858  0.7941634
##   0.8    0.048760568  0.6680624  0.393299492  0.8425982
##   0.8    0.063927861  0.6078368  0.208121827  0.9200350
##   0.8    0.083813041  0.5664792  0.182741117  0.9330713
##   0.8    0.109883635  0.5646142  0.131167513  0.9657665
##   0.8    0.144063659  0.5000000  0.000000000  1.0000000
##   0.8    0.188875603  0.5000000  0.000000000  1.0000000
##   0.8    0.247626595  0.5000000  0.000000000  1.0000000
##   0.8    0.324652467  0.5000000  0.000000000  1.0000000
##   0.8    0.425637741  0.5000000  0.000000000  1.0000000
##   0.8    0.558035146  0.5000000  0.000000000  1.0000000
##   0.8    0.731615629  0.5000000  0.000000000  1.0000000
##   0.8    0.959189457  0.5000000  0.000000000  1.0000000
##   0.8    1.257551613  0.5000000  0.000000000  1.0000000
##   0.8    1.648721271  0.5000000  0.000000000  1.0000000
##   0.9    0.002478752  0.6865801  0.550050761  0.7180924
##   0.9    0.003249784  0.6868156  0.551065990  0.7177059
##   0.9    0.004260651  0.6870774  0.549847716  0.7169264
##   0.9    0.005585954  0.6875774  0.548832487  0.7177059
##   0.9    0.007323502  0.6881523  0.546598985  0.7200388
##   0.9    0.009601525  0.6885486  0.545989848  0.7243211
##   0.9    0.012588142  0.6883489  0.540507614  0.7319091
##   0.9    0.016503766  0.6879464  0.531167513  0.7410504
##   0.9    0.021637371  0.6866768  0.517360406  0.7535060
##   0.9    0.028367816  0.6839671  0.500304569  0.7770429
##   0.9    0.037191811  0.6777720  0.452182741  0.8071930
##   0.9    0.048760568  0.6490495  0.302741117  0.8772190
##   0.9    0.063927861  0.5831544  0.190862944  0.9287900
##   0.9    0.083813041  0.5641494  0.173604061  0.9408563
##   0.9    0.109883635  0.5255309  0.028832487  0.9918257
##   0.9    0.144063659  0.5000000  0.000000000  1.0000000
##   0.9    0.188875603  0.5000000  0.000000000  1.0000000
##   0.9    0.247626595  0.5000000  0.000000000  1.0000000
##   0.9    0.324652467  0.5000000  0.000000000  1.0000000
##   0.9    0.425637741  0.5000000  0.000000000  1.0000000
##   0.9    0.558035146  0.5000000  0.000000000  1.0000000
##   0.9    0.731615629  0.5000000  0.000000000  1.0000000
##   0.9    0.959189457  0.5000000  0.000000000  1.0000000
##   0.9    1.257551613  0.5000000  0.000000000  1.0000000
##   0.9    1.648721271  0.5000000  0.000000000  1.0000000
## 
## ROC was used to select the optimal model using the largest value.
## The final values used for the model were alpha = 0.9 and lambda = 0.009601525.

Print out the non-zero coefficients, specifying the optimal value of lambda identified by resampling.

coef(mod_glmnet_2_b$finalModel, s = mod_glmnet_2_b$bestTune$lambda) %>% 
  as.matrix() %>% 
  as.data.frame() %>% 
  tibble::rownames_to_column("coef_name") %>% 
  tibble::as_tibble() %>% 
  purrr::set_names(c("coef_name", "coef_value")) %>%
  filter(coef_value != 0)
## # A tibble: 18 x 2
##    coef_name       coef_value
##    <chr>                <dbl>
##  1 (Intercept)        0.0262 
##  2 response_1        -0.112  
##  3 x09                0.00277
##  4 x10               -0.0393 
##  5 xAA2:xBB3          0.0210 
##  6 xBB3:response_1    0.718  
##  7 xBB4:response_1   -0.0894 
##  8 xBB4:x07          -0.102  
##  9 xBB4:x08          -0.00938
## 10 xBB2:x10           0.0319 
## 11 xBB4:x10          -0.162  
## 12 response_1:x07    -0.125  
## 13 response_1:x08    -0.323  
## 14 response_1:x10    -0.117  
## 15 response_1:x11    -0.0366 
## 16 x07:x09            0.166  
## 17 x08:x09            0.0229 
## 18 x10:x11           -0.0179

Visualize trends of metric AUC with respect to mixing percentage alpha and regularization parameter lambda, for model trained with our defined enet_grid.

plot(mod_glmnet_2_b, xTrans = log)

All triplet interactions

Now fit a regularized regression model with elastic net, on all triplet interactions between all step_2_b_df inputs, using tuneGrid = enet_grid, then displaying the optimal tuning parameters.

set.seed(12345)
mod_glmnet_3_b <- caret::train(outcome_2 ~ (.)^3,
                               method = "glmnet",
                               preProcess = c("center", "scale"),
                               tuneGrid = enet_grid,
                               metric = roc_metric,
                               trControl = my_ctrl,
                               data = step_2_b_df)

mod_glmnet_3_b$bestTune
##     alpha     lambda
## 207   0.9 0.01258814

Check number of coefficients:

# number of coefficients
mod_glmnet_3_b$coefnames %>% length()
## [1] 150
# check
(model.matrix(outcome_2 ~ (.)^3, data = step_2_b_df) %>% colnames() %>% length() - 1) - (mod_glmnet_3_b$coefnames %>% length())
## [1] 0

Visualize trends of metric AUC with respect to mixing percentage alpha and regularization parameter lambda, for model trained with our defined enet_grid.

plot(mod_glmnet_3_b, xTrans = log)