-
Generalized Ridge Regression: Applications to Nonorthogonal Linear Regression Models
Authors:
Román Salmerón Gómez,
Catalina García García,
Guillermo Hortal Reina
Abstract:
This paper analyzes the possibilities of using the generalized ridge regression to mitigate multicollinearity in a multiple linear regression model. For this purpose, we obtain the expressions for the estimated variance, the coefficient of variation, the coefficient of correlation, the variance inflation factor and the condition number. The results obtained are illustrated with two numerical examp…
▽ More
This paper analyzes the possibilities of using the generalized ridge regression to mitigate multicollinearity in a multiple linear regression model. For this purpose, we obtain the expressions for the estimated variance, the coefficient of variation, the coefficient of correlation, the variance inflation factor and the condition number. The results obtained are illustrated with two numerical examples.
△ Less
Submitted 8 April, 2025;
originally announced April 2025.
-
Stepwise regression revisited
Authors:
Román Salmerón Gómez,
Catalina García García
Abstract:
This paper shows that the degree of approximate multicollinearity in a linear regression model increases simply by including independent variables, even if these are not highly linearly related. In the current situation where it is relatively easy to find linear models with a large number of independent variables, it is shown that this issue can lead to the erroneous conclusion that there is a wor…
▽ More
This paper shows that the degree of approximate multicollinearity in a linear regression model increases simply by including independent variables, even if these are not highly linearly related. In the current situation where it is relatively easy to find linear models with a large number of independent variables, it is shown that this issue can lead to the erroneous conclusion that there is a worrying problem of approximate multicollinearity. To avoid this situation, an adjusted variance inflation factor is proposed to compensate the presence of a large number of independent variables in the multiple linear regression model. It is shown that this proposal has a direct impact on variable selection models based on influence relationships, which translates into a new decision criterion in the individual significance contrast to be considered in stepwise regression models or even directly in a multiple linear regression model.
△ Less
Submitted 6 March, 2025;
originally announced March 2025.
-
Unraveling Residualization: enhancing its application and exposing its relationship with the FWL theorem
Authors:
Catalina García García,
Román Salmerón Gómez,
Claudia García García
Abstract:
The residualization procedure has been applied in many different fields to estimate models with multicollinearity. However, there exists a lack of understanding of this methodology and some authors discourage its use. This paper aims to contribute to a better understanding of the residualization procedure to promote an adequate application and interpretation of it among statistics and data science…
▽ More
The residualization procedure has been applied in many different fields to estimate models with multicollinearity. However, there exists a lack of understanding of this methodology and some authors discourage its use. This paper aims to contribute to a better understanding of the residualization procedure to promote an adequate application and interpretation of it among statistics and data sciences. We highlight its interesting potential application, not only to mitigate multicollinearity but also when the study is oriented to the analysis of the isolated effect of independent variables. The relation between the residualization methodology and the Frisch-Waugh-Lovell (FWL) theorem is also analyzed, concluding that, although both provide the same estimations, the interpretation of the estimated coefficients is different. These different interpretations justify the application of the residualization methodology regardless of the FWL theorem. A real data example is presented for a better illustration of the contribution of this paper.
△ Less
Submitted 23 October, 2024;
originally announced October 2024.
-
Generalized Ridge Regression: Biased Estimation for Multiple Linear Regression Models
Authors:
Román Salmerón Gómez,
Catalina García García,
Guillermo Hortal Reina
Abstract:
When the regressors of a econometric linear model are nonorthogonal, it is well known that their estimation by ordinary least squares can present various problems that discourage the use of this model. The ridge regression is the most commonly used alternative; however, its generalized version has hardly been analyzed. The present work addresses the estimation of this generalized version, as well…
▽ More
When the regressors of a econometric linear model are nonorthogonal, it is well known that their estimation by ordinary least squares can present various problems that discourage the use of this model. The ridge regression is the most commonly used alternative; however, its generalized version has hardly been analyzed. The present work addresses the estimation of this generalized version, as well as the calculation of its mean squared error, goodness of fit and bootstrap inference.
△ Less
Submitted 2 July, 2024;
originally announced July 2024.
-
Enlarging of the sample to address multicollinearity
Authors:
Román Salmerón Gómez,
Catalina García García,
Ainara Rodríguez Sánchez
Abstract:
The paper analyzes how the enlarging of the sample affects to the mitigation of collinearity concluding that it may mitigate the consequences of collinearity related to statistical analysis but not necessarily the numerical instability. The problem that is addressed is of importance in the teaching of social sciences since it discusses one of the solutions proposed almost unanimously to solve the…
▽ More
The paper analyzes how the enlarging of the sample affects to the mitigation of collinearity concluding that it may mitigate the consequences of collinearity related to statistical analysis but not necessarily the numerical instability. The problem that is addressed is of importance in the teaching of social sciences since it discusses one of the solutions proposed almost unanimously to solve the problem of multicollinearity. For a better understanding and illustration of the contribution of this paper, two empirical examples are presented and not highly technical developments are used.
△ Less
Submitted 1 July, 2024;
originally announced July 2024.
-
The Raise Regression: Justification, properties and application
Authors:
Román Salmerón Gómez,
Catalina García García,
José García Pérez
Abstract:
Multicollinearity produces an inflation in the variance of the Ordinary Least Squares estimators due to the correlation between two or more independent variables (including the constant term). A widely applied solution is to estimate with penalized estimators (such as the ridge estimator, the Liu estimator, etc.) which exchange the mean square error by the bias. Although the variance diminishes wi…
▽ More
Multicollinearity produces an inflation in the variance of the Ordinary Least Squares estimators due to the correlation between two or more independent variables (including the constant term). A widely applied solution is to estimate with penalized estimators (such as the ridge estimator, the Liu estimator, etc.) which exchange the mean square error by the bias. Although the variance diminishes with these procedures, all seems to indicate that the inference is lost and also the goodness of fit. Alternatively, the raise regression (\cite{Garcia2011} and \cite{Salmeron2017}) allows the mitigation of the problems generated by multicollinearity but without losing the inference and keeping the coefficient of determination. This paper completely formalizes the raise estimator summarizing all the previous contributions: its mean square error, the variance inflation factor, the condition number, the adequate selection of the variable to be raised, the successive raising and the relation between the raise and the ridge estimator. As a novelty, it is also presented the estimation method, the relation between the raise and the residualization, it is analyzed the norm of the estimator and the behaviour of the individual and joint significance test and the behaviour of the mean square error and the coefficient of variation. The usefulness of the raise regression as alternative to mitigate the multicollinearity is illustrated with two empirical applications.
△ Less
Submitted 29 April, 2021;
originally announced April 2021.
