Fitting Contextualized Models

Let’s walk through an example of Contextualized analysis.

Download and Prepare Data

First, we will load the data into a standard pandas dataframe or a numpy array, and create a train / test split. There’s only 1 step of preprocessing required: deciding the context variables..

Note

Deciding context variables is an experiment-driven question.

Since we will typically interpret the contextualized models in terms of the predictor, it’s often helpful to use interpretable variables as the predictors.

In this example, we will use age, sex, and BMI as the contexts to look for context-specific predictors of diabetes progression.

import numpy as np
import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.datasets import load_diabetes

X, Y = load_diabetes(return_X_y=True, as_frame=True)
Y = np.expand_dims(Y.values, axis=-1)
C = X[['age', 'sex', 'bmi']]
X.drop(['age', 'sex', 'bmi'], axis=1, inplace=True)

seed = 1
C_train, C_test, X_train, X_test, Y_train, Y_test = train_test_split(C, X, Y, test_size=0.20, random_state=seed)

Train a Contextualized Model

Contextualized models follow an sklearn-like interface to make training easy.

Note

Common constructor keywords include:

• n_bootstraps: The integer number of bootstrap runs to fit.

• encoder_type: ‘mlp’ or ‘ngam’, which type of model to make as context encoder.

• alpha: non-negative float, regularization strength.

• mu_ratio: float in range (0.0, 1.0), governs how much the regularization applies to context-specific parameters or context-specific offsets.

• l1_ratio: float in range (0.0, 1.0), governs how much the regularization penalizes l1 vs l2 parameter norms.

Common fitting keywords include:

• max_epochs: positive number, the maximum number of epochs to fit. Early stopping is turned on by default.

• learning_rate: positive float, default is 1e-3.

%%capture
from contextualized.easy import ContextualizedRegressor
model = ContextualizedRegressor(n_bootstraps=3)
model.fit(C_train.values, X_train.values, Y_train,
          encoder_type="mlp", max_epochs=20,
          learning_rate=1e-3)

Save the trained model.

from contextualized.utils import save, load

save_path = './easy_demo_model.pt'
save(model, path=save_path)

The model can be re-loaded with:

model = load(save_path)

In the next step, we will analyze what this model has learned.