CoxPHFitter¶

class
lifelines.fitters.coxph_fitter.
CoxPHFitter
(baseline_estimation_method: str = 'breslow', penalizer: float = 0.0, strata: Union[List[str], str, None] = None, l1_ratio: float = 0.0, n_baseline_knots: int = 1, **kwargs)¶ Bases:
lifelines.fitters.RegressionFitter
,lifelines.fitters.mixins.ProportionalHazardMixin
This class implements fitting Cox’s proportional hazard model using Efron’s method for ties.
\[h(tx) = h_0(t) \exp((x  \overline{x})' \beta)\]The baseline hazard, \(h_0(t)\) can be modeled nonparametrically (using Breslow’s method) or parametrically (using cubic splines). This is specified using the
baseline_estimation_method
parameter.Parameters:  alpha (float, optional (default=0.05)) – the level in the confidence intervals.
 baseline_estimation_method (string, optional) – specify how the fitter should estimate the baseline.
"breslow"
or"spline"
 penalizer (float, optional (default=0.0)) – Attach a penalty to the size of the coefficients during regression. This improves
stability of the estimates and controls for high correlation between covariates.
For example, this shrinks the magnitude value of \(\beta_i\). See
l1_ratio
below. The penalty term is \(\frac{1}{2} \text{penalizer} \left( (1\text{l1_ratio}) \beta_2^2 + \text{l1_ratio}\beta_1\right)\).  l1_ratio (float, optional (default=0.0)) – Specify what ratio to assign to a L1 vs L2 penalty. Same as scikitlearn. See
penalizer
above.  strata (list, optional) – specify a list of columns to use in stratification. This is useful if a categorical covariate does not obey the proportional hazard assumption. This is used similar to the strata expression in R. See http://courses.washington.edu/b515/l17.pdf.
 n_baseline_knots (int) – Used when
baseline_estimation_method
is “spline”. Set the number of interior knots in the baseline hazard.
Examples
from lifelines.datasets import load_rossi from lifelines import CoxPHFitter rossi = load_rossi() cph = CoxPHFitter() cph.fit(rossi, 'week', 'arrest') cph.print_summary()

params_
¶ The estimated coefficients. Changed in version 0.22.0: use to be
.hazards_
Type: Series

hazard_ratios_
¶ The exp(coefficients)
Type: Series

confidence_intervals_
¶ The lower and upper confidence intervals for the hazard coefficients
Type: DataFrame

durations
¶ The durations provided
Type: Series

event_observed
¶ The event_observed variable provided
Type: Series

weights
¶ The event_observed variable provided
Type: Series

variance_matrix_
¶ The variance matrix of the coefficients
Type: numpy array

strata
¶ the strata provided
Type: list

standard_errors_
¶ the standard errors of the estimates
Type: Series

baseline_hazard_
¶ Type: DataFrame

baseline_cumulative_hazard_
¶ Type: DataFrame

baseline_survival_
¶ Type: DataFrame

check_assumptions
(training_df: pandas.core.frame.DataFrame, advice: bool = True, show_plots: bool = False, p_value_threshold: float = 0.01, plot_n_bootstraps: int = 10, columns: Optional[List[str]] = None) → None¶ Use this function to test the proportional hazards assumption. See usage example at https://lifelines.readthedocs.io/en/latest/jupyter_notebooks/Proportional%20hazard%20assumption.html
Parameters:  training_df (DataFrame) – the original DataFrame used in the call to
fit(...)
or a subsampled version.  advice (bool, optional) – display advice as output to the user’s screen
 show_plots (bool, optional) – display plots of the scaled schoenfeld residuals and loess curves. This is an eyeball test for violations. This will slow down the function significantly.
 p_value_threshold (float, optional) – the threshold to use to alert the user of violations. See note below.
 plot_n_bootstraps – in the plots displayed, also display plot_n_bootstraps bootstrapped loess curves. This will slow down the function significantly.
 columns (list, optional) – specify a subset of columns to test.
Examples
from lifelines.datasets import load_rossi from lifelines import CoxPHFitter rossi = load_rossi() cph = CoxPHFitter().fit(rossi, 'week', 'arrest') cph.check_assumptions(rossi)
Notes
The
p_value_threshold
is arbitrarily set at 0.01. Under the null, some covariates will be below the threshold (i.e. by chance). This is compounded when there are many covariates.Similarly, when there are lots of observations, even minor deviances from the proportional hazard assumption will be flagged.
With that in mind, it’s best to use a combination of statistical tests and eyeball tests to determine the most serious violations.
References
section 5 in https://socialsciences.mcmaster.ca/jfox/Books/Companion/appendices/AppendixCoxRegression.pdf, http://www.mwsug.org/proceedings/2006/stats/MWSUG2006SD08.pdf, http://eprints.lse.ac.uk/84988/1/06_ParkHendry2015ReassessingSchoenfeldTests_Final.pdf
 training_df (DataFrame) – the original DataFrame used in the call to

compute_followup_hazard_ratios
(training_df: pandas.core.frame.DataFrame, followup_times: Iterable[T_co]) → pandas.core.frame.DataFrame¶ Recompute the hazard ratio at different followup times (lifelines handles accounting for updated censoring and updated durations). This is useful because we need to remember that the hazard ratio is actually a weightedaverage of periodspecific hazard ratios.
Parameters:  training_df (pd.DataFrame) – The same dataframe used to train the model
 followup_times (Iterable) – a list/array of followup times to recompute the hazard ratio at.

compute_residuals
(training_dataframe: pandas.core.frame.DataFrame, kind: str) → pandas.core.frame.DataFrame¶ Compute the residuals the model.
Parameters:  training_dataframe (DataFrame) – the same training DataFrame given in fit
 kind (string) – {‘schoenfeld’, ‘score’, ‘delta_beta’, ‘deviance’, ‘martingale’, ‘scaled_schoenfeld’}

concordance_index_
¶ The concordance score (also known as the cindex) of the fit. The cindex is a generalization of the ROC AUC to survival data, including censoring.
For this purpose, the
score_
is a measure of the predictive accuracy of the fitted model onto the training dataset.References

fit
(df: pandas.core.frame.DataFrame, duration_col: Optional[str] = None, event_col: Optional[str] = None, show_progress: bool = False, initial_point: Optional[numpy.ndarray] = None, strata: Union[List[str], str, None] = None, step_size: Optional[float] = None, weights_col: Optional[str] = None, cluster_col: Optional[str] = None, robust: bool = False, batch_mode: Optional[bool] = None) → CoxPHFitter¶ Fit the Cox proportional hazard model to a dataset.
Parameters:  df (DataFrame) – a Pandas DataFrame with necessary columns duration_col and event_col (see below), covariates columns, and special columns (weights, strata). duration_col refers to the lifetimes of the subjects. event_col refers to whether the ‘death’ events was observed: 1 if observed, 0 else (censored).
 duration_col (string) – the name of the column in DataFrame that contains the subjects’ lifetimes.
 event_col (string, optional) – the name of the column in DataFrame that contains the subjects’ death observation. If left as None, assume all individuals are uncensored.
 weights_col (string, optional) – an optional column in the DataFrame, df, that denotes the weight per subject.
This column is expelled and not used as a covariate, but as a weight in the
final regression. Default weight is 1.
This can be used for caseweights. For example, a weight of 2 means there were two subjects with
identical observations.
This can be used for sampling weights. In that case, use
robust=True
to get more accurate standard errors.  show_progress (bool, optional (default=False)) – since the fitter is iterative, show convergence diagnostics. Useful if convergence is failing.
 initial_point ((d,) numpy array, optional) – initialize the starting point of the iterative algorithm. Default is the zero vector.
 strata (list or string, optional) – specify a column or list of columns n to use in stratification. This is useful if a
categorical covariate does not obey the proportional hazard assumption. This
is used similar to the
strata
expression in R. See http://courses.washington.edu/b515/l17.pdf.  step_size (float, optional) – set an initial step size for the fitting algorithm. Setting to 1.0 may improve performance, but could also hurt convergence.
 robust (bool, optional (default=False)) – Compute the robust errors using the Huber sandwich estimator, aka WeiLin estimate. This does not handle ties, so if there are high number of ties, results may significantly differ. See “The Robust Inference for the Cox Proportional Hazards Model”, Journal of the American Statistical Association, Vol. 84, No. 408 (Dec., 1989), pp. 1074 1078
 cluster_col (string, optional) – specifies what column has unique identifiers for clustering covariances. Using this forces the sandwich estimator (robust variance estimator) to be used.
 batch_mode (bool, optional) – enabling batch_mode can be faster for datasets with a large number of ties. If left as None, lifelines will choose the best option.
Returns: self – self with additional new properties:
print_summary
,hazards_
,confidence_intervals_
,baseline_survival_
, etc.Return type: Note
Tied survival times are handled using Efron’s tiemethod.
Examples
from lifelines import CoxPHFitter df = pd.DataFrame({ 'T': [5, 3, 9, 8, 7, 4, 4, 3, 2, 5, 6, 7], 'E': [1, 1, 1, 1, 1, 1, 0, 0, 1, 1, 1, 0], 'var': [0, 0, 0, 0, 1, 1, 1, 1, 1, 2, 2, 2], 'age': [4, 3, 9, 8, 7, 4, 4, 3, 2, 5, 6, 7], }) cph = CoxPHFitter() cph.fit(df, 'T', 'E') cph.print_summary() cph.predict_median(df)
from lifelines import CoxPHFitter df = pd.DataFrame({ 'T': [5, 3, 9, 8, 7, 4, 4, 3, 2, 5, 6, 7], 'E': [1, 1, 1, 1, 1, 1, 0, 0, 1, 1, 1, 0], 'var': [0, 0, 0, 0, 1, 1, 1, 1, 1, 2, 2, 2], 'weights': [1.1, 0.5, 2.0, 1.6, 1.2, 4.3, 1.4, 4.5, 3.0, 3.2, 0.4, 6.2], 'month': [10, 3, 9, 8, 7, 4, 4, 3, 2, 5, 6, 7], 'age': [4, 3, 9, 8, 7, 4, 4, 3, 2, 5, 6, 7], }) cph = CoxPHFitter() cph.fit(df, 'T', 'E', strata=['month', 'age'], robust=True, weights_col='weights') cph.print_summary() cph.predict_median(df)

fit_right_censoring
(*args, **kwargs)¶ Alias for
fit
See also
fit

hazard_ratios_

log_likelihood_ratio_test
() → lifelines.statistics.StatisticalResult¶ This function computes the likelihood ratio test for the Cox model. We compare the existing model (with all the covariates) to the trivial model of no covariates.

plot
(columns=None, hazard_ratios=False, ax=None, **errorbar_kwargs)¶ Produces a visual representation of the coefficients (i.e. log hazard ratios), including their standard errors and magnitudes.
Parameters:  columns (list, optional) – specify a subset of the columns to plot
 hazard_ratios (bool, optional) – by default,
plot
will present the loghazard ratios (the coefficients). However, by turning this flag to True, the hazard ratios are presented instead.  errorbar_kwargs – pass in additional plotting commands to matplotlib errorbar command
Examples
from lifelines import datasets, CoxPHFitter rossi = datasets.load_rossi() cph = CoxPHFitter().fit(rossi, 'week', 'arrest') cph.plot(hazard_ratios=True)
Returns: ax – the matplotlib axis that be edited. Return type: matplotlib axis

plot_covariate_groups
(covariates, values, plot_baseline=True, **kwargs)¶ Produces a plot comparing the baseline survival curve of the model versus what happens when a covariate(s) is varied over values in a group. This is useful to compare subjects’ survival as we vary covariate(s), all else being held equal. The baseline survival curve is equal to the predicted survival curve at all average values in the original dataset.
Parameters:  covariates (string or list) – a string (or list of strings) of the covariate(s) in the original dataset that we wish to vary.
 values (1d or 2d iterable) – an iterable of the specific values we wish the covariate(s) to take on.
 plot_baseline (bool) – also display the baseline survival, defined as the survival at the mean of the original dataset.
 kwargs – pass in additional plotting commands.
Returns: ax – the matplotlib axis that be edited.
Return type: matplotlib axis, or list of axis’
Examples
from lifelines import datasets, CoxPHFitter rossi = datasets.load_rossi() cph = CoxPHFitter().fit(rossi, 'week', 'arrest') cph.plot_covariate_groups('prio', values=arange(0, 15, 3), cmap='coolwarm')
# multiple variables at once cph.plot_covariate_groups(['prio', 'paro'], values=[ [0, 0], [5, 0], [10, 0], [0, 1], [5, 1], [10, 1] ], cmap='coolwarm')
# if you have categorical variables, you can do the following to see the # effect of all the categories on one plot. cph.plot_covariate_groups(['dummy1', 'dummy2', 'dummy3'], values=[[1, 0, 0], [0, 1, 0], [0, 0, 1]]) # same as: cph.plot_covariate_groups(['dummy1', 'dummy2', 'dummy3'], values=np.eye(3))

predict_cumulative_hazard
(X: Union[pandas.core.series.Series, pandas.core.frame.DataFrame], times: Union[numpy.ndarray, List[float], None] = None, conditional_after: Optional[List[int]] = None) → pandas.core.frame.DataFrame¶ Parameters:  X (numpy array or DataFrame) – a (n,d) covariate numpy array or DataFrame. If a DataFrame, columns can be in any order. If a numpy array, columns must be in the same order as the training data.
 times (iterable, optional) – an iterable of increasing times to predict the cumulative hazard at. Default is the set of all durations (observed and unobserved). Uses a linear interpolation if points in time are not in the index.
 conditional_after (iterable, optional) – Must be equal is size to X.shape[0] (denoted
n
above). An iterable (array, list, series) of possibly nonzero values that represent how long the subject has already lived for. Ex: if \(T\) is the unknown event time, then this represents \(T  T > s\). This is useful for knowing the remaining hazard/survival of censored subjects. The new timeline is the remaining duration of the subject, i.e. reset back to starting at 0.

predict_expectation
(X: pandas.core.frame.DataFrame, conditional_after: Optional[numpy.ndarray] = None) → pandas.core.series.Series¶ Compute the expected lifetime, \(E[T]\), using covariates X. This algorithm to compute the expectation is to use the fact that \(E[T] = \int_0^\inf P(T > t) dt = \int_0^\inf S(t) dt\). To compute the integral, we use the trapizoidal rule to approximate the integral.
Caution
If the survival function doesn’t converge to 0, then the expectation is really infinity and the returned values are meaningless/too large. In that case, using
predict_median
orpredict_percentile
would be better.Parameters:  X (numpy array or DataFrame) – a (n,d) covariate numpy array or DataFrame. If a DataFrame, columns can be in any order. If a numpy array, columns must be in the same order as the training data.
 conditional_after (iterable, optional) – Must be equal is size to X.shape[0] (denoted n above). An iterable (array, list, series) of possibly nonzero values that represent how long the subject has already lived for. Ex: if \(T\) is the unknown event time, then this represents \(T  T > s\). This is useful for knowing the remaining hazard/survival of censored subjects. The new timeline is the remaining duration of the subject, i.e. normalized back to starting at 0.
Notes
If X is a DataFrame, the order of the columns do not matter. But if X is an array, then the column ordering is assumed to be the same as the training dataset.
See also

predict_log_partial_hazard
(X: Union[numpy.ndarray, pandas.core.frame.DataFrame]) → pandas.core.series.Series¶ This is equivalent to R’s linear.predictors. Returns the log of the partial hazard for the individuals, partial since the baseline hazard is not included. Equal to \((x  \text{mean}(x_{\text{train}})) \beta\)
Parameters: X (numpy array or DataFrame) – a (n,d) covariate numpy array or DataFrame. If a DataFrame, columns can be in any order. If a numpy array, columns must be in the same order as the training data. Notes
If X is a DataFrame, the order of the columns do not matter. But if X is an array, then the column ordering is assumed to be the same as the training dataset.

predict_median
(X: pandas.core.frame.DataFrame, conditional_after: Optional[numpy.ndarray] = None) → pandas.core.series.Series¶ Predict the median lifetimes for the individuals. If the survival curve of an individual does not cross 0.5, then the result is infinity.
Parameters:  X (numpy array or DataFrame) – a (n,d) covariate numpy array or DataFrame. If a DataFrame, columns can be in any order. If a numpy array, columns must be in the same order as the training data.
 conditional_after (iterable, optional) – Must be equal is size to X.shape[0] (denoted
n
above). An iterable (array, list, series) of possibly nonzero values that represent how long the subject has already lived for. Ex: if \(T\) is the unknown event time, then this represents \(T  T > s\). This is useful for knowing the remaining hazard/survival of censored subjects. The new timeline is the remaining duration of the subject, i.e. normalized back to starting at 0.
See also

predict_partial_hazard
(X: Union[numpy.ndarray, pandas.core.frame.DataFrame]) → pandas.core.series.Series¶ Returns the partial hazard for the individuals, partial since the baseline hazard is not included. Equal to \(\exp{(x  mean(x_{train}))'\beta}\)
Parameters: X (numpy array or DataFrame) – a (n,d) covariate numpy array or DataFrame. If a DataFrame, columns can be in any order. If a numpy array, columns must be in the same order as the training data. Notes
If X is a DataFrame, the order of the columns do not matter. But if X is an array, then the column ordering is assumed to be the same as the training dataset.

predict_percentile
(X: pandas.core.frame.DataFrame, p: float = 0.5, conditional_after: Optional[numpy.ndarray] = None) → pandas.core.series.Series¶ Returns the median lifetimes for the individuals, by default. If the survival curve of an individual does not cross 0.5, then the result is infinity. http://stats.stackexchange.com/questions/102986/percentilelossfunctions
Parameters:  X (numpy array or DataFrame) – a (n,d) covariate numpy array or DataFrame. If a DataFrame, columns can be in any order. If a numpy array, columns must be in the same order as the training data.
 p (float, optional (default=0.5)) – the percentile, must be between 0 and 1.
 conditional_after (iterable, optional) – Must be equal is size to X.shape[0] (denoted
n
above). An iterable (array, list, series) of possibly nonzero values that represent how long the subject has already lived for. Ex: if \(T\) is the unknown event time, then this represents \(T  T > s\). This is useful for knowing the remaining hazard/survival of censored subjects. The new timeline is the remaining duration of the subject, i.e. normalized back to starting at 0.
See also

predict_survival_function
(X: Union[pandas.core.series.Series, pandas.core.frame.DataFrame], times: Union[numpy.ndarray, List[float], None] = None, conditional_after: Optional[List[int]] = None) → pandas.core.frame.DataFrame¶ Predict the survival function for individuals, given their covariates. This assumes that the individual just entered the study (that is, we do not condition on how long they have already lived for.)
Parameters:  X (numpy array or DataFrame) – a (n,d) covariate numpy array or DataFrame. If a DataFrame, columns can be in any order. If a numpy array, columns must be in the same order as the training data.
 times (iterable, optional) – an iterable of increasing times to predict the cumulative hazard at. Default is the set of all durations (observed and unobserved). Uses a linear interpolation if points in time are not in the index.
 conditional_after (iterable, optional) – Must be equal is size to X.shape[0] (denoted
n
above). An iterable (array, list, series) of possibly nonzero values that represent how long the subject has already lived for. Ex: if \(T\) is the unknown event time, then this represents \(T  T > s\). This is useful for knowing the remaining hazard/survival of censored subjects. The new timeline is the remaining duration of the subject, i.e. normalized back to starting at 0.

print_summary
(decimals: int = 2, style: Optional[str] = None, **kwargs) → None¶ Print summary statistics describing the fit, the coefficients, and the error bounds.
Parameters:  decimals (int, optional (default=2)) – specify the number of decimal places to show
 style (string) – {html, ascii, latex}
 kwargs – print additional metadata in the output (useful to provide model names, dataset names, etc.) when comparing multiple outputs.

score
(df: pandas.core.frame.DataFrame, scoring_method: str = 'log_likelihood') → float¶ Score the data in df on the fitted model. With default scoring method, returns the average partial loglikelihood.
Parameters:  df (DataFrame) – the dataframe with duration col, event col, etc.
 scoring_method (str) – one of {‘log_likelihood’, ‘concordance_index’} log_likelihood: returns the average unpenalized partial loglikelihood. concordance_index: returns the concordanceindex
Examples
from lifelines import CoxPHFitter from lifelines.datasets import load_rossi rossi_train = load_rossi().loc[:400] rossi_test = load_rossi().loc[400:] cph = CoxPHFitter().fit(rossi_train, 'week', 'arrest') cph.score(rossi_train) cph.score(rossi_test)

summary
¶ Summary statistics describing the fit.
Returns: df Return type: DataFrame