macrosynergy.learning.metrics

Collection of non-standard scikit-learn performance metrics for evaluation of machine learning model predictions.

panel_significance_probability(y_true, y_pred)

Function to create a linear mixed effects model between the ground truth returns and the predicted returns, returning the significance of the model slope. Period-specific random effects are included in the model to account for return cross-sectional correlations. This can be passed into sklearn’s make_scorer function to be used as a scorer in a grid search or cross validation procedure.

Parameters:

• y_true (Series) – Pandas series of ground truth labels. These must be multi-indexed by cross-section and date. The dates must be in datetime format.

• y_pred (Union[Series, ndarray]) – Either a pandas series or numpy array of predicted targets. This must have the same length as y_true.

Return <float> significance_prob:

1 - p-value of the regression slope parameter, given by the linear mixed effects model.

Return type:

float

regression_accuracy(y_true, y_pred)

Function to return the accuracy between the signs of the predictions and targets.

Parameters:

• y_true (Series) – Pandas series of ground truth labels. These must be multi-indexed by cross-section and date. The dates must be in datetime format.

• y_pred (Union[Series, ndarray]) – Either a pandas series or numpy array of predicted targets. This must have the same length as y_true.

Return <float>:

Accuracy between the signs of the predictions and targets.

Return type:

float

regression_balanced_accuracy(y_true, y_pred)

Function to return the balanced accuracy between the signs of the predictions and targets.

Parameters:

• y_true (Series) – Pandas series of ground truth labels. These must be multi-indexed by cross-section and date. The dates must be in datetime format.

• y_pred (Union[Series, ndarray]) – Either a pandas series or numpy array of predicted targets. This must have the same length as y_true.

Return <float>:

Balanced accuracy between the signs of the predictions and targets.

Return type:

float

sharpe_ratio(y_true, y_pred)

Function to return a Sharpe ratio for a strategy where we go long if the predictions are positive and short if the predictions are negative.

Parameters:

• y_true (Series) – Pandas series of ground truth labels. These must be multi-indexed by cross-section and date. The dates must be in datetime format.

• y_pred (Union[Series, ndarray]) – Either a pandas series or numpy array of predicted targets. This must have the same length as y_true.

Return <float>:

Sharpe ratio for the binary strategy.

Return type:

float

sortino_ratio(y_true, y_pred)

Function to return a Sortino ratio for a strategy where we go long if the predictions are positive and short if the predictions are negative.

Parameters:

• y_true (Series) – Pandas series of ground truth labels. These must be multi-indexed by cross-section and date. The dates must be in datetime format.

• y_pred (Union[Series, ndarray]) – Either a pandas series or numpy array of predicted targets. This must have the same length as y_true.

Return <float>:

Sortino ratio for the binary strategy.

Return type:

float