Backtesting guide
Combinatorially Symmetric Cross Validation
Combinatorially Symmetric Cross Validation, or CSCV, divides an aligned matrix of strategy trials into an even number of ordered blocks and evaluates symmetric in-sample and out-of-sample combinations. Its original use is to estimate Probability of Backtest Overfitting.
Read the practical guideKey Takeaways
- CSCV requires many aligned strategy trials across the same observations.
- Every split uses half the blocks in-sample and the complementary half out-of-sample.
- The in-sample winner is evaluated by its relative out-of-sample rank.
- CSCV is distinct from purged cross-validation and combinatorial purged CV.
Prepare a common trial matrix
Place the performance observations for each strategy configuration on the same ordered rows. Divide the rows into an even number of approximately equal blocks. The matrix should represent the full research competition because excluding losing trials changes the behavior of the selection process being measured.
- Align dates and the performance measure across every candidate.
- Include all variants considered in the same selection decision.
- Use blocks long enough to retain meaningful performance observations.
Run the symmetric combinations
Choose half of the blocks as the in-sample set and use the remaining blocks as its out-of-sample complement. Rank candidates in-sample and select the winner. Then rank that same candidate out-of-sample. Repeat the process across the unique symmetric combinations without changing the metric or selection rule.
- Select a candidate using only the in-sample blocks.
- Record its relative rank on the complementary blocks.
- Retain each split result so the final distribution can be audited.
Distinguish CSCV from related methods
CSCV was introduced to study overfitting through symmetric candidate rankings. Purged cross-validation removes training labels that overlap validation intervals. Combinatorial purged cross-validation constructs multiple purged train-test paths. Similar names do not make these procedures interchangeable, and a CSCV implementation should state whether any additional purging or embargo is applied.
- Use CSCV when the target is the reliability of selecting among strategy trials.
- Use purging when label intervals can leak across train and validation sets.
- Document blocks, combinations, rankings, and any leakage controls separately.
Related Pineify workflow
Generate consistent trials before an external CSCV study
Pineify Strategy Optimizer can run and compare related parameter configurations using consistent strategy and market assumptions. The retained trial matrix can then be transformed for a separate CSCV implementation.
Run comparable parameter configurations and inspect their historical metrics.
Open Strategy OptimizerBoundary: Strategy Optimizer does not construct symmetric block combinations, rank in-sample winners out-of-sample, or calculate PBO.
Primary sources
This page is educational and does not provide investment advice. Backtests are hypothetical, depend on their data and assumptions, and do not guarantee future results. Trading can result in substantial loss.