Trade-off between predictive performance and FDR control
Posted on
tradeoff between predictive performance and FDR control for high-dimensional Gaussian model selection
in the context of high-dimensional Gaussian linear regression for ordered variables, study the variable selection procedure via the minimization of the penalized least-squares criterion
focus on model selection where the penalty function depends on an unknown multiplicative constant commonly calibrated for prediction.
propose a new proper calibration of this hyperparameter to simultaneously control predictive risk and false discovery rate
obtain non-asymptotic bounds on the False Discovery Rate with respect to the hyperparameter and provide an algorithm to calibrate it
Introduction
some variable selection procedures focus on the prediction of the response variable $Y$ through a control of the predictive risk.
others focus on limiting the number of selected nonactive variables through a control of the FDR
there also exists procedures where several cost functions are considered simultaneously
the goal of the paper: identify a set of variables from a model selection procedure by limiting the selection of non-active variables while maintaining accurate prediction performances
related work
in a varibale selection procedure, a cost function has to be defined
- the predictive risk and the false discovery rate are the common used cost functions
in this paper, consider a model selection procedure composed of three steps
- solve the Lasso criterion on a relevant grid $\Lambda$. Each $\lambda\in\Lambda$ defines a variable subset to get a collection $M$ of relevant subsets of variables with a wide range of sizes. each $\lambda\in \Lambda$ defines a variable subset to get a collection $\cM$ of relevant subsets of variables with a wide range of sizes
- the least-squares estimator onto each variable subset of $\cM$ is calculated leading to a collection of estimators $(\hat\beta_m)_{m\in \cM}$
- the penalized least-squares minimization is solved to select the best $m$ of $\cM$ \(\hat m = \argmin_{m\in\cM} \{ \Vert Y-X\hat\beta_m\Vert_2^2 + \text{pen}(D_m) \}\)
main challenge: definition of pen that achieves an optimal trade-off between goodness of fit and sparsity within $\cM$
- multiple testing methods to control the False Discovery Rate
- the simultaneous control of several cost functions
- multi-step algorithm where a threshold procedure is applied to some Lasso estimators computed for sepecific values of $\lambda$
- post-selection inference where the principle is to test the relevance of each selected variable by a model selection procedure. valid CIs are provided from conditional hypothesis tests for each model of the collection in addition to a PR control
- completely different direction: control the False Negative Rate (FNR) in addition to the FDR. a good FNR control ensures that most of the active variables close to the set of active variables
- combine FDR with the PR
- ordered variable selection: a large class of methods exists dealing with variables having a natural ranking. this assumption allows for drastically reducing the estimation complexity.
- it can be applied to data sets where the assumption that an ordering of the variables is available a priori (e.g., from suitable forms of prior knowledge) is appropriate
main contributions
assume variables are ranked according to their importance for the response variable $Y$.
$X_1$ being the most important one and $X_p$ being the least important one.
all the penalty functions defined by
\[\text{pen}(D_m) = K\sigma^2D_m, \forall m\in\cM\]- theoretical bounds on the FDR in model selection
- calibration of the hyperparameter $K$
- towards a non-ordering variable selection