Remedying Multicollinearity in Quantitative Analysis: A Simulation Studies

Adamu Abubakar, U. F. Abbas, K. E. Lasisi

Abstract


In multiple regression modeling, if there is a relationship between  and , which are all the explanatory variables of , then the relationship between  and  or  and  is not linear. Linear regression modeling is prone the problem of Multicollinearity as a result of violation of the assumption linearity. But, sometimes in real life data, it is difficult to satisfy the independence of predictor assumption due to the nature of the data leading to the problem of Multicollinearity. It causes interpretation misconceptions if the data set is contaminated with collinear variables. In this study, we restrict in the famous four methods which are Multiple Regression, Ridge Regression, Stepwise Regression and Partial Least Squares Regression. The aim of this study is to improve the remedial procedures for solving Multicollinearity problem in linear regression modeling. Findings from the simulation study shows that the partial least square method is the effective method when there is a change in sample size and number of predictors; it is capable to representing the data well when Multicollinearity problem is present.


Full Text:

PDF

References


Abdi, H. (2003), Partial Least Squares (PLS) Regression. The University of Texas at Dallas

Alcácer, J., Chung, W., Hawk, A., & Pacheco-de-Almeida, G. (2018) Applying random coefficient models to strategy research: Identifying and exploring firm heterogeneous effects. Strategy Science, 3(3): 533–553

Al-Hassan, Y.M. (2007) A Comparison Between Ridge Regression And Principle Component Regression Methods using Simulation Technique, AL al-Bayt University, 2007,(Unpublished)

Åke Björck and Ulf G. Indahl (2017) fast and stable partial least squares modeling: A benchmark study with theoretical comments; DOI: 10.1002/cem.2898

Allison, P. (2012). “When Can You Safely Ignore Multicollinearity?” Statistical Horizons. September 10, 2012. http://statisticalhorizons.com/Multicollinearity

Babar I, Ayed H, Chand S, Suhail M, Khan YA, Marzouki R (2021) Modified Liu estimators in the linear regression model: An application to Tobacco data. PLoS ONE 16(11): e0259991. https://doi.org/10.1371/journal.pone.0259991

Chadalavada, K. (2018) Partial Least Squares Regression in Football Projections; Senior Thesis in Mathematics.

Cohen, P., West, S. G., & Aiken, L. S. (2014) applied multiple regression/correlation analysis for the behavioral sciences. London: Psychology Press.

David A. W. (2004). Ridge Regression as an Alternative to Ordinary Least Squares: Improving Prediction Accuracy and the Interpretation of Beta Weights; Association for Institutional Research Number 92, summer, 2004; Enhancing knowledge. Expanding networks. Professional Development, Information Resources & Networking.

Greene, W. H. (2003). Econometric analysis. Pearson Education India.

Griffiths, W. E., Judge, G. G., Hill, R. C., Lütkepohl, H., & Lee, T.-C. (1985). The theory and practice of econometrics. New York: Wiley.

Hutmacher MM, Kowalski KG. (2014) Covariate Selection in Pharmacometric Analyses: A Review of Methods. Br J ClinPharmacol. 2014; 79(1):132–47.

Imdad, M. U. and Aslam, M. (2018a) mctest: An R Package for Detection of Collinearity among Regressors. URL https://cran.r-project.org/web/packages/mctest/. R package version 1.2. [p328]

Imdad, M. U. and Aslam, M. (2018b). lmridge: Linear Ridge Regression with Ridge Penalty and Ridge Statistics, URL https://cran.r-project.org/package=lmridge. R package version 1.2. [p326]

Ismail, B; Manjula, S (2016) Estimation of linear regression model with correlated regressors in the presence of autocorrelation: International Journal of Statistics and Applications 6 (2), 35 – 39.

Lukman, A. F., K. Ayinde, S. K. Sek, and E. Adewuyi. (2019). A modified new two-parameter estimator in a linear regression model. Modeling and Simulation in Engineering 2019:6342702. doi:10.1155/2019/6342702.

Neale, M. C. (1994). Mx: Statistical modeling (2nd Ed). Department of Psychiatry: Medical College of Virginia.

Ogee, A., Ellis, M., Scibilia, B. and Pammer, C. (2020) enough is enough! Handling Multicollinearity in Regression Analysis. The Minitab Blog.

Olanrewaju, SO; Yahaya, HU; Nasiru, MO (2017). Effects of Multicollinearity and on some estimators in a system of regression equation. European Journal of Statistics and Probability 5 (3), 1 – 15.

Rothman KJ, Greenland S, Lash TL. (2008) Modern Epidemiology. Philadelphia: Wolters Kluwer Health/Lippincott Williams & Wilkins; 2008

Rupa Singh (2022) Multicollinearity in Linear Regression: AI for Human Dignity | Thought Leader| Machine learning… | Founder and CEO at AI-Beehive | Building Published Jun 18, 2022

Wondola, D.W. et al (2020) Partial Least Square (PLS) Method of Addressing Multicollinearity Problems in Multiple Linear Regressions (Case Studies: Cost of electricity bills and factors affecting it); IOP Conf. Series: Journal of Physics: Conf. Series 1463 (2020) 012006; IOP Publishing doi:10.1088/1742-6596/1463/1/012006

Yizho Jiang (2009) Contributions to partial least squares regression and supervised principal component analysis modeling; Follow this and additional works at: https://digitalrepository.unm.edu/math_etds


Refbacks

  • There are currently no refbacks.