Skip to content

[follow-up] Optionally wire transfer-learning splitter into CrossSubjectEvaluation #1088

Description

@bruAristimunha

Summary

Optional follow-up to #1077. The transfer-learning splitter is a standalone scikit-learn cross-validator and already works on its own (for train, test in CrossSubjectTransferSplitter(...).split(y, metadata)), so it needs no change here to be usable. This issue is only about the convenience of running it through CrossSubjectEvaluation, to reuse its caching, results DB, parallelism, and scoring. That path is currently blocked because the top-level splitter is hardcoded.

Not a prerequisite for the splitter PR; track separately.

Current behaviour

Every evaluation builds its top-level splitter in _create_splitter() and passes cv_class/cv_kwargs only to the inner cross-validator, not to the splitter itself:

  • WithinSessionEvaluation -> WithinSessionSplitter (evaluations.py:86)
  • CrossSessionEvaluation -> CrossSessionSplitter (evaluations.py:299)
  • CrossSubjectEvaluation -> CrossSubjectSplitter (evaluations.py:479)
  • WithinSubjectEvaluation -> WithinSubjectSplitter (evaluations.py:660)
def _create_splitter(self):
    ...
    cv_class, cv_kwargs = self._resolve_cv(default_class, default_kwargs)
    return CrossSubjectSplitter(
        cv_class=cv_class, random_state=self.random_state, **cv_kwargs
    )

And the runner does self.cv = self._create_splitter() (base.py:758-764), with no public hook to inject a splitter. So CrossSubjectEvaluation(cv_class=CrossSubjectTransferSplitter, ...) does not swap the splitter; cv_class is forwarded into CrossSubjectSplitter as its inner fold strategy, which is not what a transfer setup needs.

Proposal

Let CrossSubjectEvaluation accept and use a top-level splitter, keeping the scikit-learn contract (split(y, metadata) -> (train_idx, test_idx)) untouched so caching, scoring, and result handling stay as-is. Two options, smallest first:

  1. Allow passing a splitter instance. If cv (or a splitter= argument) is a BaseCrossValidator that yields (train_idx, test_idx) over metadata, use it directly in _create_splitter() instead of hardcoding CrossSubjectSplitter. This makes CrossSubjectTransferSplitter usable today and is additive.

  2. Let cv_class select the top-level splitter when it is a MOABB splitter (detected by its split(y, metadata) signature), falling back to current behaviour otherwise.

I lean towards option 1: it is the least surprising, does not change any default, and keeps the transfer splitter a plain cross-validator rather than a new evaluation engine. Background and the splitter sketch are in #1077.

Scope

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions