Before you begin, remember that you have an index with the newsletter content organized by clicking below.
From model estimation to production-ready feature sets
Embedded feature selection integrates variable selection into model training. Rather than ranking inputs beforehand or testing separate subsets, embedded methods decide which features survive while estimating coefficients, decision boundaries, tree structures, or stochastic gates. This connects feature selection to the model’s actual objective while controlling redundancy, instability, and implementation frictions.
What’s inside:
Lasso regression: Produces compact linear models by shrinking weak coefficients to zero. Its simplicity makes the selected support easy to interpret, although strongly correlated predictors can lead to unstable winner-takes-all choices.
Elastic Net: Combines sparse selection with ridge-style stabilization, allowing groups of correlated predictors to remain in the model. This makes it better suited to feature libraries containing similar signals across multiple horizons or transformations.
Group and sparse-group Lasso: Selects economically related feature families together while preserving sparsity within each family. The approach aligns model structure with predefined groups such as momentum, volatility, or event-driven variables.
Penalized Logistic regression: Selects inputs according to their contribution to a regularized probabilistic decision boundary. Its usefulness depends on aligning probability calibration with the confidence thresholds and asymmetric costs used during execution.
Sparse Support Vector Machines: Combines margin-based classification with coefficient sparsity, retaining variables that help separate classes. Feature survival depends on observations near the decision boundary rather than on overall fit alone.
Tree-induced feature selection: Builds nonlinear and conditional support through split selection, depth restrictions, minimum leaf sizes, and pruning. Out-of-sample permutation tests are needed because impurity-based importance can favor noisy variables with many possible split points.
SCAD and MCP: Apply nonconvex penalties that preserve sparsity while reducing the excessive shrinkage imposed on large coefficients by Lasso. They can recover stronger signals with less bias, but introduce initialization sensitivity and more difficult optimization.
Stochastic feature gates: Learn probabilistic switches that approximate an explicit feature budget inside differentiable models. They support nonlinear interactions and operational constraints, but require stable gate selection and deterministic deployment rules.
