Table of contents:
Introduction.
Limitations and risks when parameterizing models.
Parametrization challenges.
The illusion of the global optimum.
The foundational model.
Walk-forward framework.
Rolling windows with embargo.
Backtesting helpers.
Executing the Walk-Forward process.
K-Means clustering to identify parameter regimes.
Final selection of parameters.
Before you begin, remember that you have an index with the newsletter content organized by clicking on “Read full story” in this image.
Introduction
I want to start by saying that the key is in the data, not in the model or its parameters. Therefore, if your data is garbage, no matter how much you parameterize it, the results will still be garbage. If you parameterize a model, it's to fine-tune something that already works. Period.
Knowing that, we can proceed. The genesis of a strategy is often an elegant, compelling concept. This concept is then meticulously translated into a computational model, an architecture of logic replete with parameters. The immediate, almost instinctual, drive is to calibrate these parameters to what appears to be perfection, right?
This is the foundational dilemma of optimization: we are armed with a model, a finite history of market data, and an overwhelming incentive to tune the model's parameters to sculpt the most aesthetically pleasing performance curve on that historical data.
We can formalize this endeavor as a constrained optimization problem. Let S represent our trading strategy, which is a function of a parameter vector θ.
This vector θ belongs to a predefined parameter space Θ, such that:
Given a historical dataset D, which is a time-ordered sequence of market observations, our objective is to identify a specific parameter vector θ* that maximizes a chosen performance metric, our objective function f(⋅).
This function quantifies the desirability of a strategy’s outcome, encompassing metrics such as total return, Sharpe ratio, Calmar ratio, or other measures of risk-adjusted profitability. The mathematical representation of this search is:
This equation is the wet dream for quantitative analysts. It presents a deceptively linear path to success: discover the optimal parameters on past data, deploy, and profit. The procedure appears robust. Let's consider a simple model that depends on a single parameter: the lookback window N. The process involves iterating through all plausible integer values of N within a given range. For each candidate value, a backtest is executed on the historical dataset D. The objective function f(⋅) is computed for each resulting equity line, and the value of N that yields the maximum score is anointed as N.
To know more and see another example check this:
The outcome is frequently a spectacular equity curve, a testament to the power of optimization. It is a process that is algorithmically clean, mathematically precise, and almost invariably, a prelude to failure.
Limitations and risks when parameterizing models
The principal hazard, the latent variable in this deterministic optimization, is overfitting, a term more formally known in statistics as curve-fitting. This phenomenon occurs when a model, through the optimization process, becomes so intricately calibrated to the specific noise, outliers, and random fluctuations inherent in the training dataset that it forfeits its ability to generalize to new, unseen data. The distinction is analogous to that between genuine learning—internalizing the underlying principles of a system—and rote memorization. A model that has merely memorized the past is brittle; it has learned a specific sequence, not a general pattern.
Let's define this more rigorously. The performance we observe on our historical, in-sample data D gives us the in-sample error, Ein. The true test of a model, however, is its performance on future, out-of-sample data, which yields the out-of-sample error, Eout. The goal of any robust modeling process is to find a model that minimizes Eout. The process of maximizing performance on D is equivalent to minimizing Ein. Overfitting is the condition where we have successfully minimized Ein but at the cost of increasing Eout.
By selecting the single best parameter set θ*, we are, by construction, selecting the configuration that most effectively exploited the statistical artifacts of our specific historical sample. The risk is that this optimal performance is not a consequence of the model's intrinsic predictive power (its edge), but rather a statistical illusion born from data dredging. When this overfitted model is deployed into the live market, it confronts a data distribution for which it was not trained, and its performance rapidly and often decouples from the backtest.
The pivotal conflict that animates this entire analysis is this paradox: a strategy's viability is contingent on parameters that require optimization, yet the conventional act of optimization itself introduces a risk of self-deception and future failure. The central question becomes: how do we calibrate our model without rendering it a historical specialist, perfectly adapted for a market that no longer exists?
Parametrization challenges
I like to think of models as a deeper appreciation of the nature of their parameters. For example, in a breakout model, the parameter N (the lookback window) is the lens through which the model perceives market dynamics. A small N yields a model with high sensitivity, attuned to short-term oscillations, but also susceptible to being misled by market noise (high variance). A large N produces a model that is more inert, filtering out minor fluctuations to focus on established, durable trends, but at the risk of being slow to react to major structural shifts (high bias). Neither approach is axiomatically superior; their efficacy is regime-dependent. The purpose of parametrization is to endow our model with the requisite flexibility to adapt its logic.
The first practical obstacle is the dimensionality and topology of the parameter space θ. Our simple model has one dimension, N. Most institutional strategies, however, are multiparametric, involving thresholds for entry, rules for exit (profit targets, stop-losses), volatility filters, position sizing algorithms, and more. A strategy with, say, four parameters, each discretized into 20 potential values, results in a parameter space with 204=160,000 unique combinations.
This curse of dimensionality not only makes an exhaustive grid search computationally prohibitive but, more critically, it exponentially inflates the probability of discovering a spuriously performant parameter set through sheer chance. The more questions you ask of the data, the more likely it is to give you a favorable answer by coincidence.
The second, more important obstacle is the non-stationarity of financial time series. Markets are complex adaptive systems, not static physical ones. The statistical properties of market data—mean, variance, correlation—evolve over time. These shifts define distinct market regimes. The optimal parameter set for the quantitative easing era of low volatility is highly unlikely to be the optimal set for a period of geopolitical turmoil and high inflation. A single best parameter vector, optimized over a long history spanning multiple regimes, is often just a suboptimal compromise that is truly effective in none of them. The challenge, therefore, is not to unearth a single, immutable set of golden parameters, but to architect a process for systematically re-evaluating and selecting robust parameters that can adapt to a dynamic environment.
The resolution to this paradox lies in a paradigm shift: we must abandon the pursuit of a singular, perfect parameter and instead embrace a framework centered on robustness. A robust parameter is not necessarily the one that generated the highest peak performance in a backtest. It is a parameter that performs consistently well across a spectrum of market conditions, exhibits low sensitivity to minor perturbations in its value (i.e., it resides in a flat region of the performance landscape), and is validated through a process that simulates multiple paths.
To achieve this we will combine two well known methods. First, we employ WF, a technique tailored for time-series data, to simulate a realistic cycle of model optimization and deployment. This generates a distribution of historically optimal parameters.
Second, we apply an unsupervised machine learning algorithm, K-Means clustering, to dissect this distribution, identify stable parameter regimes, and select our final parameter based on out-of-sample profitability and stability.
This integrated approach directly confronts the core challenges. WF is our primary defense against overfitting, as it relentlessly tests the model on unseen (out-of-sample) data. K-means clustering addresses non-stationarity by revealing whether different market epochs naturally demand different parameterizations, guiding us to select a parameter from the most reliably profitable regime. We will now deconstruct this process into its constituent analytical and mathematical components.
The illusion of the global optimum
What was optimal yesterday is unlikely to be optimal tomorrow, and what is optimal tomorrow will be different the day after.
Searching for a single point—the global optimum—in the parameter performance landscape is a fool's errand. As we discussed, this landscape is rugged and treacherous. The highest peak you find after an exhaustive search is almost certainly an artifact of historical noise. It's a one-hit wonder. A strategy perched on this peak is brittle.
The more robust approach is to search for parameter stability. We are not looking for a single point, but for a wide, elevated plateau. A parameter set that lies in the middle of such a plateau has a crucial property: its performance is insensitive to small changes in the parameter values. If N=50 works well, but so do N=48, N=49, N=51, and N=52, then we have found something potentially robust. The strategy's logic is not dependent on a knife-edge parameter value. This implies that the market inefficiency it's capturing is real and persistent, not a statistical fluke.
Let's visualize this performance landscape. We'll use a slightly simple model and compute the Sharpe ratio for a grid of these parameter values and plot it as a 3D surface.
The resulting 3D plot is insightful. It would show a surface with various peaks and valleys. There would likely be a very sharp, isolated peak—this is the global optimum, the overfitting trap. Elsewhere on the plot, there would be a broader, flatter, but still elevated region. This is our robust plateau. A strategy with parameters chosen from this plateau is far more likely to survive contact with the future. The goal of a sophisticated optimization process is to identify these plateaus, not to conquer the highest peaks.
The foundational model
Before any optimization can occur, a model must be defined. The core of any model is a hypothesis about a market inefficiency. This hypothesis is then translated into a set of rules that generate trading signals. The logic is best encapsulated in a class for organizational clarity and computational efficiency.
The model's internal logic should be tailored to capture the specific market inefficiency you've identified, producing signals (e.g., 1 for long, -1 for short, and 0 for neutral). Below is a structural template for such a model, which uses a simple breakout strategy for demonstration purposes.
This would be an example of a hypothetical model:
import numpy as np
class TradingModel:
"""
A generic template for a trading model using NumPy.
This example implements a simple breakout strategy, but the internal logic
can be replaced with any custom signal generation rule.
Example Logic:
- A long signal (1) is generated if the current price exceeds the maximum of the previous N bars.
- A short signal (-1) is generated if the current price falls below the minimum of the previous N bars.
"""
def __init__(self, window=20, signal_type='long'):
self.window = window
if signal_type not in ('long', 'short'):
raise ValueError("signal_type must be 'long' or 'short'")
self.signal_type = signal_type
def predict(self, data):
"""
Generates trading signals based on the implemented logic.
This is where your custom model rules would go.
"""
data = np.asarray(data, dtype=float)
n = len(data)
signals = np.zeros(n, dtype=int)
# Start loop from self.window to ensure enough lookback data.
for i in range(self.window, n):
window_slice = data[i-self.window:i]
if self.signal_type == 'long':
# Example logic: A signal to go long (1).
if data[i] > np.max(window_slice):
signals[i] = 1
else: # 'short'
# Example logic: A signal to go short (-1).
if data[i] < np.min(window_slice):
signals[i] = -1
return signalsA signal is an instantaneous event, whereas a trading position is a state held over time. We often wish to maintain our position for a certain duration following a signal. This could require signal propagation.
A model is the artifact; it is the specific, tangible output generated when a learning algorithm is applied to a particular dataset. It is a data structure containing the specific parameters and rules that represent the crystallized knowledge extracted from data. The model is a simplified map of the complex market dynamics observed in the data it was trained on. Crucially, it embodies a specific, and often falsifiable, theory about how some part of the market works.
Besides these utility functions extend a signal forward for a specified number of periods, simulating the act of holding a position.
def propagate_ones(input_array, N):
result = np.array(input_array)
# We iterate N times, propagating the signal one step at a time.
for _ in range(N):
# np.roll shifts the array elements. -1 means a left shift.
shifted = np.roll(result, -1)
# Identify positions where a '1' is followed by a '0'. These are candidates for propagation.
can_propagate = (result == 1) & (shifted == 0)
# Apply the propagation by shifting the boolean mask back to the right and setting the value to 1.
result[np.where(np.roll(can_propagate, 1))] = 1
return result
def propagate_minus_ones(input_array, N):
result = np.array(input_array)
for _ in range(N):
shifted = np.roll(result, -1)
can_propagate = (result == -1) & (shifted == 0)
result[np.where(np.roll(can_propagate, 1))] = -1
return resultOur TradingModel provides a solid way to translate an inneficiency into signal, bt it has a critical vulnerability: hard-coded parameters like the window size. A window of 20 might perform well during a specific historical period, but is it truly the best choice? How can we find a more optimal value that adapts to changing market conditions? This is the central question of parameter optimization.