skbold.postproc.mvp_results module

class MvpAverageResults(mvp_results_list, identifiers=None)

Bases: object

Averages results from MVPA analyses on, for example, different subjects or different ROIs.

Parameters:

• mvp_results_list (list) – List with MvpResults objects (after updating across folds)
• identifiers (list of str) – List of identifiers (e.g. subject-name) that correspond to the different MvpResults objects

compute_statistics(metric='accuracy', h0=0.5)

Computes statistics across MvpResults objects

Parameters:

• metric (str) – Which metric should be used in the MvpResults dataframes
• h0 (float) – The null-hypothesis in terms of model performance (e.g. accuracy equals 1 / n_classes)

write(path, name='average_results')

class MvpResults(mvp, n_iter, type_model='classification', feature_scoring=None, confmat=False, verbose=False, **metrics)

Bases: object

Class to keep track of model evaluation metrics and feature scores. See the ReadTheDocs homepage for more information on its API and use.

Parameters:

• mvp (mvp-object) – Necessary to extract some metadata from.
• n_iter (int) – Number of folds that will be kept track of.
• type_model (str) – Either ‘classification’ or ‘regression’
• feature_scoring (str) – Which method to use to calculate feature-scores with. Can be: 1) ‘fwm’: feature weight mapping [1] - keep track of raw voxel-weights (coefficients) 2) ‘forward’: transform raw voxel-weights to corresponding forward- model [2].
• confmat (bool) – Whether to keep track of the confusion-matrix across folds (only relevant for type_model=’classification’)
• verbose (bool) – Whether to print extra output.
• **metrics (keyword-arguments) – Keyword arguments of the form: name_metric: metric_function; any metric from scikit-learn works (or other metrics, as long as they have two input args, y_true and y_pred).

References

[1]	Stelzer, J., Buschmann, T., Lohmann, G., Margulies, D.S., Trampel, R., and Turner, R. (2014). Prioritizing spatial accuracy in high-resolution fMRI data using multivariate feature weight mapping. Front. Neurosci., http://dx.doi.org/10.3389/fnins.2014.00066.

[2]	Haufe, S., Meineck, F., Gorger, K., Dahne, S., Haynes, J-D., Blankertz, B., and Biessmann, F. et al. (2014). On the interpretation of weight vectors of linear models in multivariate neuroimaging. Neuroimage, 87, 96-110.

compute_scores(multiclass='ovr', maps_to_tstat=True)

Computes scores across folds.

load_model(path, param=None)

Load model or pipeline from disk.

Parameters:

• path (str) – Absolute path to model.
• param (str) – Which, if any, specific param needs to be loaded.

save_model(model, out_path)

Method to serialize model(s) to disk.

Parameters:model (pipeline or scikit-learn object.) – Model to be saved.

update(test_idx, y_pred, pipeline=None)

Updates with information from current fold.

Parameters:

• test_idx (ndarray) – Indices of current test-trials.
• y_pred (ndarray) – Predictions of current test-trials.
• pipeline (scikit-learn Pipeline object) – pipeline from which relevant scores/coefficients will be extracted.

write(out_path, confmat=True, to_tstat=True, multiclass='ovr')

Writes results to disk.

Parameters:

• out_path (str) – Where to save the results to
• feature_viz (bool) – Whether to write out (and optionally return) feature-visualization information
• confmat (bool) – Whether to write out (and optionally return) the confusion-matrix (across folds).
• to_tstat (bool) – Whether to convert averaged feature-scores to t-tstats (by dividing them by sqrt(score.std(axis=0)).