Nested Validation for Trading Models: Keep Tuning Inside Training
Nested cross validation separates model selection from performance evaluation. An inner loop chooses parameters using only the outer training history; an outer loop tests the resulting fitted rule on a later, untouched window. Repeat the entire selection process at each outer origin. For trading data, both loops must respect information availability and remove training labels that overlap the next evaluation window.
This hypothetical schedule uses equally spaced decision indices t = 0 through 57, with outcome data through index 59. A label for decision t uses the inclusive interval [t, t + 2] and becomes available at the end of t + 2. At an evaluation origin s, freeze the fit before observing s. Keep training starts t only when t + 2 < s. Thus starts s - 2 and s - 1 are excluded, while s - 3 is retained. The two-row exclusion follows from this label definition, not a default percentage.
Original hypothetical schedule. Bars show event-start indices; the outlined tails show the final two indices needed to observe evaluation labels. Exact ranges appear in the table below. Scroll the figure horizontally on smaller screens. View original source
For each outer origin s, use two inner validation origins v = s - 20 and v = s - 10. Each validation set contains starts v through v + 7. Fit that inner model on starts 0 through v - 3, excluding v - 2 and v - 1. The last validation label ends at v + 9, before the outer origin.
For outer fold one, inner fits contain 8 and 18 rows: 0 through 7 and 0 through 17. The final refit contains 28 rows: 0 through 27. The inner validation sets and outer test each contain 8 decision rows. Outcome-tail indices are not additional predictions in that evaluation window.
These small counts make the split auditable; they are not adequate-sample recommendations. If features require earlier warmup bars, supply them causally or revise the first usable index and all counts. Real irregular events require timestamp checks, not an assumption that two rows mean two bars.
Outer origin
Final refit starts
Inner validation starts
Outer test starts; labels known by
30
0 through 27
10 through 17; 20 through 27
30 through 37; end of 39
40
0 through 37
20 through 27; 30 through 37
40 through 47; end of 49
50
0 through 47
30 through 37; 40 through 47
50 through 57; end of 59
Repeat every choice inside each outer training history
Before opening any outer result, freeze the candidate set, inner schedule, objective, eligibility rules, tie-break, search budget and refit policy. At each origin, rebuild the full process from the information available there. This includes feature selection, scaling, missing-value treatment and any learned prediction model. A transformation fitted on the full dataset can leak even when the estimator uses the correct rows.
For each candidate, fit separately on each inner training set and score only that inner validation window. Use the same costs, sizing, execution assumptions and position-boundary policy for all candidates.
Aggregate the two validation scores using the declared rule, select one candidate, then refit its learned components on all eligible outer training rows. Freeze that fit for the outer window unless an online update rule was explicitly included in the protocol.
Save outer predictions and results without changing the procedure. At origin 40, outcomes through 39 are now available. Earlier outer observations may enter the next training history under the prespecified expanding-window policy; data that had not yet settled at origin 30 could not have entered that earlier fit.
A fixed Pine strategy may have no learned coefficients to refit. Its parameter search still belongs entirely inside the outer training history, and its chosen settings remain fixed during that outer test.
Recalculate a selection and its outer result
The following invented scores demonstrate the selection mechanism, not a measured strategy. Compare two prespecified candidates A and B by mean inner-window net P&L. Each evaluation window starts with $10,000, uses the same fixed sizing and costs, and starts flat. Only starts in the listed eight-index range may open trades; all positions settle or close by its label-tail end. A wins exact ties. For each outer fold, evaluate only the selected candidate on its outer window.
In fold one, B wins because (80 + 40) / 2 = $60 exceeds (40 + 60) / 2 = $50. That $60 is a selection score. The outer observation for the selected procedure is -$20.
The outer mean is (-20 + 60 + 10) / 3 = $16.67 per window, or 0.1667% of the common $10,000 starting balance. The individual returns are -0.2%, +0.6% and +0.1%. This average of reset-window returns is not a compounded portfolio return.
Repeated inner origins use the same history, candidate and fixed fitting seed, so their scores repeat. For example, B at origin 30 scores -$20 both as the first outer result and as a later inner validation result. This reuse is permitted only after that outcome is available under the frozen schedule.
The sequence B, A, B is the output of one selection rule at different origins. Replacing it afterward with whichever fixed candidate looks best on all outer windows evaluates another rule and consumes those windows for selection.
Two candidates × two inner folds × three outer folds require 12 candidate evaluations, followed by 3 selected outer evaluations. A fitted model also needs one final refit per outer origin. The example does not include extra trials, failed runs or alternative feature sets; a real experiment log must count them.
Outer fold
A inner P&L ($); mean
B inner P&L ($); mean
Selected
Outer net P&L ($)
1
40, 60; 50
80, 40; 60
B
-20
2
60, 70; 65
40, -20; 10
A
60
3
70, 60; 65
-20, 160; 70
B
10
Apply purge and embargo at both levels
With inclusive endpoints, training interval [a, b] overlaps evaluation interval [c, d] when a <= d and b >= c. Check actual label intervals at every inner and outer boundary. In the first outer fold, training start 27 ends at 29 and survives; start 28 ends at 30 and is purged; start 29 ends at 31 and is purged. Variable-horizon labels need individual end times, even when their prediction timestamps are adjacent.
An embargo excludes additional candidate training observations after an evaluation interval. If a two-sided fold has test starts 30 through 37 and last label end 39, an explicitly chosen two-bar post-label embargo would exclude training starts in (39, 41], namely 40 and 41. That buffer is a separate illustrative assumption, not a value implied by nested CV.
In the strictly forward schedule above, the current fit never includes training starts after its test window. A post-test embargo therefore removes no additional training rows. The two rows immediately before the test were purged for label overlap; calling them an embargo obscures which condition was checked.
Choose any additional exclusion from documented label horizons, feature dependencies and publication delays. State whether its anchor is the last prediction or last label end and whether its unit is bars or elapsed time. A universal fixed fraction of the dataset cannot establish that the relevant information intervals are disjoint.
Causal evaluation features can use earlier market prices. An overlapping lookback alone is not the same as an unavailable future label. Audit feature availability separately, including revised data and transforms that used later observations.
Report a procedure, not a retrospectively chosen winner
Nested evaluation estimates the performance of the frozen selection procedure over the tested historical origins. It does not certify future profitability or make overlapping training histories independent. Report each fold, the selected settings, retained row counts, failed fits and execution assumptions alongside any aggregate. A few outer scores do not support a precise confidence interval by treating folds as independent trials.
If you use outer scores to choose a model family, objective, feature set, purge rule or search budget, those scores become development evidence. Evaluate the revised procedure on genuinely unused later data; do not rename the same outer results a final evaluation.
After assessment, a deployment fit can rerun the prespecified selection procedure on all currently available eligible history. Its exact parameters may differ from every outer winner. The historical evidence belongs to that procedure and its earlier training sizes, not to a new fit evaluated on unseen future prices.
Keep an additional chronological holdout when research will continue after outer results are inspected. A holdout is only untouched while none of its outcomes influence selection. If it disappoints, preserve that observation rather than repeatedly revising and retesting on it.
If purging leaves too few observations, a score is undefined, or a candidate produces no eligible trades, follow a failure rule declared before testing. Never silently drop the difficult outer fold. Splitting cannot repair lookahead in signals, missing transaction costs or unrealistic fills.
Keep the split audit separate from the trade report
TradingView exports Strategy Report tabs separately. Export List of Trades for each selected outer run and attach its fold ID, script version, parameter set, symbol, timeframe, dates, costs and boundary policy. Pineify Backtest Deep Report parses trade IDs, entry and exit timestamps, prices, quantity, net P&L and excursion fields. Reconcile its trade count and net profit with the source export before comparing runs.
Backtest Deep Report calculates supported trade metrics including profit factor and drawdown from closed-trade equity. Closed-trade drawdown does not describe every intrabar equity move. If your download is a workbook, convert the relevant trade sheet to CSV and verify the resulting fields before importing.
Keep label endpoints, feature-availability times, candidate trials, inner scores and exclusion IDs in an external split manifest. A selected-trades CSV does not reconstruct all decision rows, rejected candidates or the information available to the search.
The diagram, nested selection, purge assertions and numerical example on this page are external research calculations. This page does not claim that Backtest Deep Report runs nested cross validation or prevents evaluation reuse.
Pine Script AI Coding Agent can help draft explicit parameter inputs and date rules. Review and validate the code, then reconcile the resulting trades. Code assistance does not validate the statistical protocol.
Where Pineify fits
Backtest Deep Report
Inspect supported metrics from each selected outer-run trade CSV and retain its fold manifest in your research records.
Also useful: Pine Script AI Coding Agent. Draft strategy parameters and date boundaries for review before running the fixed evaluation protocol.
No. Nesting separates tuning from assessment; purging removes training labels that overlap evaluation labels. A trading experiment can require both at each loop. Purging alone does not stop an analyst from choosing a model using its test score.
Shuffling lets inner models train on observations later than their validation predictions. If the question is how a past-only trading process would perform, preserve chronological ordering inside the inner loop as well as the outer loop.
Yes, under a frozen expanding-window protocol once its labels and features are available at the later origin. That does not permit changing the selection procedure after inspecting earlier outer scores or using later information in the earlier fit.
Bayesian optimization can choose candidates within the inner search. Its bounds, budget and stopping rule remain part of the procedure evaluated by the outer loop. A more efficient search does not make its repeatedly consulted validation scores independent performance evidence.
The split design and selection audit can run in an external research workflow. TradingView can supply strategy runs for declared parameters and windows, but exporting trades alone does not prove that the inner search or label boundaries were valid.
Educational research methods, not investment advice. All windows, label horizons and P&L values are hypothetical. Historical validation does not guarantee future returns or remove execution and market risk.
Published by Pineify. Pineify is an independent product and is not affiliated with TradingView. Sources support the technical explanations; examples do not establish trading performance.
