Purged Cross-Validation for Trading Strategies

Why a timestamp split can still leak

Suppose an observation predicts at bar 8 whether a price target will be reached by bar 11. Its row belongs to the period before a test block starting at bar 10, but its label requires prices from bars 10 and 11. Training on that label reveals part of the test outcome. A variable holding period makes a fixed row gap especially easy to mis-specify.

• Record an event ID, prediction timestamp, label-end timestamp, feature availability timestamps, and target value. Use one timezone and define whether timestamps denote bar opens or closes.
• A label interval is not a feature lookback. A causal test feature may legitimately use past prices, but future-fitted scalers, revised data, and centered windows still leak.
• Do not replace the label end with the prediction time. For barrier labels, record the actual barrier-touch or expiry time used to determine the outcome.

A bar-by-bar timeline: purge first, then embargo

This fictional fold has test labels T1 = [10, 12] and T2 = [11, 14]. Their combined outcome interval covers bars 10 through 14. Treat endpoints as inclusive: training interval [a, b] overlaps test interval [c, d] when a <= d and b >= c. For this example only, embargo training starts in (14, 16], a two-bar buffer after the final test label ends.

• Along the timeline: A ends at 8; test outcomes occupy 10 through 14; the embargo covers starts at 15 and 16; F starts at 17. Six candidate training rows become A and F after three purges and one embargo exclusion.
• With no embargo, E survives the overlap test. A training event starting at 16 is excluded by this buffer; one starting at 17 is not. A label ending exactly at 10 is purged under the stated endpoint rule.
• This two-bar choice illustrates the mechanism; it is not a recommended setting. Specify whether an implementation measures its buffer from the last test prediction or the last test label end, and whether it counts rows, bars, or elapsed time.
• For separated test blocks, check overlap against every test label or the union of their intervals. A single envelope from the first to last test timestamp can unnecessarily discard the training gap between blocks.

Choose the split for the question being asked

Chronological testing asks what could have been learned before each trading decision. Purged folds ask how a model performs on held-out blocks after controlling label overlap. These are different experiments. Purging cannot make a model trained on 2025 data behave as though it had been trained in 2020.

Reproduce CPCV with four groups and three paths

Split a fictional dataset into four ordered groups G1, G2, G3, G4 without shuffling. Hold out two groups per fit. There are C(4, 2) = 4! / (2! × 2!) = 6 splits. Name each split by its test groups: S12, S13, S14, S23, S24, S34. S13, for example, tests G1 and G3 and trains on the permitted rows of G2 and G4 after purging and embargo.

• Each group appears in three test splits: choose its other test group from the remaining three. Six splits × two test groups = twelve group predictions, enough for three paths of four groups each.
• In general, phi = (k / N) × C(N, k) = C(N - 1, k - 1) paths for N groups and k test groups. Require 1 <= k < N and enough training observations after exclusions.
• The table gives one valid assignment. Each cell identifies the fitted split supplying predictions for that group. Read each row in chronological group order, and use every (split, test-group) prediction once across the three paths.

Turn predictions into comparable path results

A reconstructed prediction path is not yet a trade ledger. Apply the same position-sizing, entry, exit, fee, and slippage rules to each chronological prediction sequence. Declare how model changes at group boundaries affect an open position. A reset-at-boundary policy must include its closing costs; carrying positions requires a consistent execution simulation.

• Calculate the metric from each completed path. Do not average six split Sharpes and call the result a path Sharpe: averaging ratios does not reconstruct a return series.
• For a numerical check, hypothetical path returns of +2%, -1%, +3%, and 0% compound to 1.02 × 0.99 × 1.03 × 1.00 - 1 = 4.0094%. Their arithmetic sum is 4%, a different quantity.
• Report path dispersion and all construction choices. Three paths reuse the same four historical groups and overlapping training sets; they are not three independent market histories or a confidence interval.
• CPCV does not by itself estimate Probability of Backtest Overfitting. CSCV evaluates the relative rank of an in-sample winner among candidate strategies and answers a separate selection question.

A reproducible experiment checklist

Save a versioned event table and a split manifest before fitting. The manifest should identify test IDs, candidate train IDs, purged IDs, embargoed IDs, retained train IDs, and the endpoint convention for every split. Keep an untouched chronological holdout outside model selection if deployment is the final question.

• 1. Freeze the dataset, labeling rule, features, model candidates, metric, costs, N, k, embargo, random seed, and minimum retained training size. Do not choose the embargo by maximizing test performance.
• 2. Build the six group combinations above. Compare the same folds without exclusions, with purging, and with purging plus embargo. Any score difference is a diagnostic on this dataset, not proof of how much live performance improves.
• 3. Assert that training IDs and test IDs are disjoint, every retained training label has zero interval overlap with every test label, and no retained training start lies in the declared embargo.
• 4. Fit imputers, scalers, feature selection, and the estimator using retained training rows only. Tune hyperparameters in a separate inner validation loop with the same leakage controls.
• 5. Store test predictions keyed by event ID and split ID. Construct the three paths using the table and verify that each group appears once per path and every test-group occurrence is used once.
• 6. Run the fixed execution simulation on each path, then report costs, trade count, returns, drawdown, and the distribution of the chosen metric. Preserve failed folds rather than silently dropping them.

Failure cases that a clean split does not repair

An empty or tiny training set is a failed design, not a successful validation result. Long labels, short groups, and large embargoes can consume most observations. Disclose the retained sample counts and revise the protocol before using test results for selection.

• Missing label-end timestamps make the overlap assertion unverifiable. Reconstruct them from the original labeling process or report that purging cannot be checked.
• Cross-asset observations can share information at the same time. A split per ticker may still leak across a jointly fitted model; define the information unit before splitting.
• Purging does not fix noncausal features, survivorship bias, unrealistic fills, repeated test inspection, or a market regime absent from the sample.
• No universal embargo percentage guarantees independence. Document the information horizon and examine sensitivity without turning the final test into another tuning set.

Inspect TradingView results after designing the experiment

Backtest Deep Report accepts TradingView List of Trades CSV files and calculates metrics including net profit, profit factor, and drawdown from the supplied closed trades. Import each separately prepared evaluation run and keep its split or path ID with the file. A trade CSV does not contain the feature availability and label metadata needed to establish a purged split.

• Compare the report with the original export using the same initial capital and trade selection. A closed-trade drawdown is not automatically the intrabar equity drawdown shown by another engine.
• Use Pine Script AI Coding Agent to draft or revise TradingView strategy rules, then review the generated code and test it in TradingView. Define any date filters and position treatment explicitly.
• The split manifest, purge checks, embargo, CPCV fitting, and path construction described here are external research steps. This workflow does not claim that Backtest Deep Report, the coding agent, or Strategy Optimizer executes them natively.