Bootstrap Intervals for Backtest Metrics: What to Resample

Trade outcomes and daily account returns answer different questions

Resampling complete closed trades can estimate uncertainty in average net P&L per trade under the declared trade-generating process. Resampling dated account returns targets a statistic per valuation period. Changing the unit changes the question, even when both datasets came from the same TradingView strategy.

• Without external cash flows, daily simple return is r(t) = E(t)/E(t-1) - 1, where E includes open positions and costs at the same valuation time. Deposits and withdrawals require an explicit adjustment. Preserve the calendar and flag missing valuations instead of inventing zero returns.
• A trade opened on Monday and closed on Friday has one realized trade outcome but several daily account returns. Booking all of its P&L on Friday omits the intervening exposure. A closed-trade CSV alone cannot reconstruct that mark-to-market path.
• Keep each trade record together, including its return denominator and relevant fields. Do not sample profits and position sizes separately. Several exits from one position or trades sharing market exposure may require a larger sampling unit.

Construct the percentile interval explicitly

For observations x(1), ..., x(n), first calculate the statistic theta_hat = T(x). Under an appropriate IID model, draw n indices independently and uniformly from 1 through n, allowing repeats. Apply the same function T to that resample. Repeat B times and sort the resulting statistics as z(1) ≤ ... ≤ z(B). For confidence level 1-alpha, the percentile interval is [Q(alpha/2), Q(1-alpha/2)].

• This page uses linear interpolation equivalent to R quantile type 7. For probability p, set h = 1 + (B-1)p, j = floor(h), and g = h-j. Then Q(p) = (1-g)z(j) + g z(j+1); at p = 1 use z(B). A nearest-rank rule can return different endpoints, so record the quantile convention.
• For B = 10,000 and a 95% interval, the lower endpoint interpolates between ordered statistics 250 and 251 with weight 0.975 on 251. The upper endpoint uses 9,750 and 9,751 with weight 0.025 on 9,751. These are one-based positions.
• Recalculate the full target statistic on each resample. A Sharpe ratio requires the chosen excess-return definition, standard deviation convention and annualization rule. Resampling a precomputed report ratio does not estimate its uncertainty.
• The percentile method is one interval construction. Basic and bias-corrected and accelerated (BCa) intervals use different endpoint calculations. Software defaults may select BCa, so specify the method when reproducing a percentile result.

Reproduce an interval by enumerating every resample

Use four constructed observations [-1, 0, 1, 2] in percentage points and T = arithmetic mean. The observed mean is 0.5%. There are 4^4 = 256 equally likely ordered resamples of length four. Enumerating them removes random simulation error. Their 95% percentile interval is [-0.5%, 1.5%] using the stated interpolation. This tiny dataset demonstrates arithmetic; it does not establish 95% coverage for real trading data.

from itertools import product
from math import floor, isfinite


def quantile_type7(values, p):
    z = sorted(float(v) for v in values)
    if not z or not all(isfinite(v) for v in z):
        raise ValueError("Need finite, nonempty values")
    if not isfinite(p) or not 0 <= p <= 1:
        raise ValueError("p must be between 0 and 1")
    h = (len(z) - 1) * p  # zero-based position
    j = floor(h)
    g = h - j
    return (1 - g) * z[j] + g * z[min(j + 1, len(z) - 1)]


x = [-1, 0, 1, 2]
means = [sum(sample) / len(x) for sample in product(x, repeat=len(x))]
interval = [quantile_type7(means, p) for p in (0.025, 0.975)]
print(len(means), sum(x) / len(x), interval)
# 256 0.5 [-0.5, 1.5]

Choose B for endpoint stability, not a preferred conclusion

For a larger sample, use random resampling rather than enumerating n^n index sequences. A reproducible working choice is B = 10,000 with a recorded random seed, followed by more replications if the endpoints still move enough to affect interpretation. This is a computational starting point, not a universal sufficient count.

• At the 2.5% tail, 1,000 replications supply roughly 25 draws in the tail; 10,000 supply roughly 250. Under independent Monte Carlo draws, the probability-scale sampling error at a fixed 2.5% cutoff is approximately sqrt(0.025 × 0.975 / B): 0.00494 at B = 1,000 and 0.00156 at B = 10,000. Those are probability units, not return-percentage-point errors in the endpoint.
• Endpoint uncertainty also depends on how densely the bootstrap statistics lie near the quantile. Compare independent seeds and larger B while retaining the same data and method. Record endpoint movement at a precision useful for the metric; do not keep the seed that gives the most favorable lower bound.
• Save n, B, the seed, software version, statistic definition, sampling unit, date range, costs, confidence level and quantile algorithm. A saved resample-index matrix gives an even more direct reproduction path.
• Increasing B cannot reveal a crash absent from the history, correct an unrepresentative regime, or turn four observations into a large sample. A narrow simulated interval can still reflect weak assumptions.

See how duplicated exposure makes IID intervals too narrow

Suppose four independent underlying outcomes are [-1, 0, 1, 2] pp, but each appears twice because two records share exactly the same exposure. The eight rows are [-1, -1, 0, 0, 1, 1, 2, 2]. Treating all eight rows as independent produces the same mean but a smaller estimated sampling spread. Resampling the four known pairs together preserves the dependence in this constructed example.

• The four distinct values each have empirical probability 1/4, so enumerating their eight-position sequences gives the exact same distribution as drawing from all eight duplicated rows. To reproduce the incorrect interval with the code above, use repeat=8 and divide each sum by 8.
• For the sample mean in this model, independent pair outcomes with variance sigma² give variance sigma²/4. Mistakenly treating eight copies as independent suggests sigma²/8. The corresponding standard error is too small by a factor of sqrt(2). This factor belongs to this exact duplication model, not every overlapping strategy.
• Real data rarely come with such convenient independent pairs. Check shared positions, overlapping holding periods, serial correlation and volatility clustering. Daily sampling does not by itself establish independence. A near-zero return autocorrelation does not rule out dependence in return magnitudes.
• When chronology matters, use a justified time-series resampling design that retains local dependence, such as a block bootstrap, and investigate sensitivity to its settings. Grouping arbitrary neighboring trades into pairs does not solve that problem. Structural change can invalidate the assumed common process even after dependence is addressed.

Interpret coverage and recognize interval failures

A nominal 95% confidence procedure aims to cover the fixed population quantity in about 95% of repeated datasets under its assumptions. Once calculated, the particular interval either contains that quantity or it does not. Its nominal confidence level is neither the probability that the next trade wins nor the fraction of future daily returns expected to lie inside the bounds.

• A prediction interval needs a defined future observation or horizon and a model for its randomness. The interval [-0.5%, 1.5%] above concerns an estimated mean; individual returns can lie outside it. A path simulation for future drawdowns is another target again.
• Percentile coverage can be poor for small samples, skewed sampling distributions and biased estimators. BCa or studentized methods may address some approximation errors, but neither repairs the wrong sampling unit, missing extreme events or nonstationarity.
• Profit factor can be undefined or infinite in a resample with no gross losses. Sharpe can be undefined when the resampled return variance is zero. Count and report such failures; silently deleting those replications changes the distribution being summarized.
• Identical observed returns give a zero-width percentile interval. That describes the empirical resampling distribution, not certainty about the market. Some interval methods can fail on degenerate samples.
• Bootstrapping the winning strategy after searching many variants does not account for selection bias. Keep the search history and reserve untouched evaluation data. A positive lower bound from a selected backtest is not proof of a tradable edge.

Reconcile TradingView inputs before calculating the interval

TradingView exports Strategy Report tabs separately. The List of Trades download supplies trade information; the Performance Summary download supplies metrics. Preserve the strategy version, symbol, chart interval, date range and cost assumptions with the export. For daily account statistics, obtain and reconcile the dated equity series separately.

• Pineify Backtest Deep Report parses trade entry and exit records, including timestamps, prices, quantity, net P&L and P&L percentage. Its supported summaries include net profit and profit factor. Use those fields to check that the chosen trade sample matches the original report.
• The current report also resamples trade P&L with replacement for Monte Carlo equity paths. That path analysis is distinct from the mean confidence interval and type-7 percentile calculation explained here. Perform this interval calculation independently; this page does not claim it is a built-in report output.
• Pine Script AI Coding Agent can help draft custom series output from a written specification. Define valuation time, costs, open-position treatment and missing-data rules. Review the generated code and compare its output with independent calculations before using it as statistical input.