Comparison of Different Estimation Methods Used in Confirmatory Factor Analyses in Non-Normal Data A Monte Carlo Study
Author :
Abstract
Keywords
Abstract
In social sciences, education, and psychology, it is difficult to obtain distributional assumption. For these reasons, in situations where normality assumptions are not met, it is important to compare the effects of estimation methods used in CFA. So, in this study, it was aimed to compare CFA results obtained from ML, MLR and WLS estimation methods in data sets with different sample sizes where normality assumption is not met. The study was conducted with data produced after Monte Carlo simulations. In this direction, non-normal data sets were produced 300, 500, 1000, 1500 and 2000 sample sizes including 14 items in total; 7 items in the first factor and including 7 items in the second factor. In the study, fit indices were examined in order to find the efficiency of the same model which was tested with different estimation methods and changing sample sizes. Additionally, as a result of CFA conducted using different estimation methods, the average relative bias related to factor loads were calculated. When analyses conducted with three estimation methods with the sample sizes are compared, it was found that the best fit is obtained with the MLR method in small sample size (N=300 and 500). In the comparison of WLS and MLR prediction methods in big sample sizes (1000, 1500 and 2000) where distribution is not normal, WLS estimations were found to give less biased and truer factor estimations compared to MLR estimations. ML showed high level of bias under all sample size in non-normality.
Keywords
- Baghdarnia, M., Soreh, R. F., & Gorji, R. (2014). The comparison of two methods of maximum likelihood (ML) and diagonally weighted least squares (DWLS) in testing construct validity of achievement goals. Journal of Educational and Management Studies, 4(1), 22-38.
- Bandalos, D. L. (2014). Relative performance of categorical diagonally weighted least squares and robust maximum likelihood estimation. Structural Equation Modeling: A Multidisciplinary Journal, 21, 102 – 116.
- Beauducel, A., & Herzberg, P. Y. (2006). On the performance of maximum likelihood versus average and variance adjusted weighted least squares estimation in CFA. Structural Equation Modeling, 13, 186–203.
- Bentler, P. M., & Chou, C.-P. (1987). Practical issues in structural equation modeling. Sociological Methods and Research, 16, 78–117.
- Bentler, P. M., & Dudgeon, P. (1996). Covariance structure analysis: statistical practice, theory and directions. Annual Review of Psychology, 47(1), 563-592. doi:10.1146/annurev.psych.47.1.563
- Boomsma, A., & Hoogland, J. J. (2001). The robustness of LISREL modeling revisited. In R. Cudeck, S. du Toit, & D. Sörbom (Eds.), Structural equation models: Present and future. A Festschrift in honor of Karl Jöreskog (pp. 139–168). Chicago: Scientific Software.
- Browne, M. W. (1984). Asymptotically distribution‐free methods for the analysis of covariance structures. British Journal of Mathematical and Statistical Psychology, 37, 62-83.
- Curran, P. J., West, S. G., & Finch, G. F. (1996). The robustness of test statistics to nonnormality and specification error in confirmatory factor analysis. Psychological Methods, 1, 16-29. doi:10.1037/1082989X.1.1.16
- Çelik, H. E., Saraçlı, S., & Yılmaz, V. (2011). An application of structural equaiton modeling under the assumption of multivariate normality. e-Journal of New World Sciences Academy Physical Sciences, 6(4), 112-123.
- Çelik, H. E., & Yılmaz, V. (2016). LISREL 9.1 ile Yapısal Eşitlik Modellemesi. Ankara: Anı Yayıncılık.
- DiStefano, C. (2002). The impact of categorization with confirmatory factor analysis. Structural Equation Modeling, 9, 327–346.
- Finch, J. F., West, S. G., & MacKinnon D. P. (1997) Effects of sample size and nonnormality on the estimation of mediated effects in latent variable models. Structural Equation Modeling: A Multidisciplinary Journal, 4:2, 87-107, DOI:10.1080/10705519709540063
- Finney, S. J., & DiStefano, C. (2013). Non-normal and categorical data in structural equation modeling. G. R. Hancock ve R. O. Mueller (Ed.), Structural Equation Modeling: A Second Course (pp. 439–492). Greenwich, CT: Information Age Publishing.
- Finney, S. J., Distefano, C., & Kopp, J. P. (2016). Overview of estimation methods and preconditions for their application with structural equation modeling. In K. Schweizer & C. Distefano (Eds.), Principles and methods of test construction: Standards and recent advances (pp. 135-165). Boston: Hogrefe.
- Fitriana, E. (2011). Confirmatory factor analysis of the Bandung family relation test: A simulation study comparing ML, DWLS, and WLS estimation. Ph.D. thesis, Radboud University Nijmegen, Nijmegen. Retrieved from https://repository.ubn.ru.nl/handle/2066/87163
- Flora, D.B., & Curran, P.J. (2004). An empirical evaluation of alternative methods of estimation for confirmatory factor analysis with ordinal data. Psychological methods, 9(4), 466-491.
- Hoogland, J. J., & Boomsma, A. (1998). Robustness studies in covariance structure modeling: An overview and a meta-analysis. Sociological Methods & Research, 26, 329–367.
- Jöreskog, K., & Sörbom, D. (1993). LISREL 8: Structural equation modeling with the SIMPLIS command language. Chicago, IL: Scientific Software International Inc.
- Kılıç, A. F., Uysal, İ., & Atar, B. (2017). Doğrulayıcı faktör analizinde kullanılan kestirim yöntemlerinin karşılaştırmalı olarak incelenmesi. IVth International Eurasian Educational Research Congress Conference Proceedings, Ankara: Anı Yayıncılık.
- Kline, B.R. (2005, 2011, 2016). Principles and practice of structural modeling. New York-London: The Guilford Press.
- Koğar, H., & Yılmaz-Koğar, E. (2015). Comparison of different estimation methods for categorical and ordinal data in confirmatory factor analysis. Journal of Measurement and Evaluation in Education and Psychology, 6(2), 351-364.
- Lei, P. (2009). Evaluating estimation methods for ordinal data in structural equation modeling. Quality & Quantity, 43, 495 – 507.
- Li, C. H. (2014). The performance of MLR, USLMV, and WLSMV estimation in structural regression models with ordinal variables (doctoral disertation). Michigan State University, USA.
- Li, C. H. (2016). Confirmatory factor analysis with ordinal data: Comparing robust maximum likelihood and diagonally weighted least squares. Behav Res. 48: 936–949. DOI 10.3758/s13428-015-0619-7
- Mindrila, D. (2004). Maximum Likelihood (ML) and Diagonally Weighted Least Squares (DWLS) Estimation Procedures: A Comparison of Estimation Bias with Ordinal and Multivariate Non-Normal Data. International Journal of Digital Society (IJDS), 1(1): 1-7.
- Muthén, B. O., & Kaplan, D. (1992). A comparison of some methodologies for the factor-analysis of non-normal Likert variables: A note on the size of the model. British Journal of Mathematical and Statistical Psychology, 45, 19–30.
- Muthén, L. K., & Muthén, B. O. (2002). How to use a Monte Carlo study to decide on sample size and determine power. Structural Equation Modeling: A Multidisciplinary Journal, 9, 599-620.
- Nalbantoğlu-Yılmaz, F. (2019). Gerçek bir veride doğrulayıcı faktör analizinde kullanılan alternatif kestirim yöntemlerinin karşılaştırılması. Vith International Eurasian Educational Research Congress Abstract Book, Ankara: Anı Yayıncılık.
- Sayın, A. (2016). The Examination of model fit indexes with different estimation methods under different sample sizes in confirmatory factor analysis. Journal of Measurement and Evaluation in Education and Psychology, 7(2), 432-442.
- Schermelleh-Engel, K., Moosbrugger, H., & Müller, H., (2003). Evaluating the fit of structural equation models: Test of significance and descriptive goodness-of-fit measures. Methods of Psychological Research - Online, 8(2), 23-74.
- Schumacker, R. E., & Beyerlein, S. T. (2000). Confirmatory factor analysis with different correlation types and estimation methods. Structural Equatıon Modeling, 7(4), 629–636.
- Schumacker, R. E., & Lomax, R. G. (2016). A beginner’s quide to structural equaiton modeling. New York: Routledge.
- Sümer, N. (2000). Yapısal eşitlik modelleri: Temel kavramlar ve örnek uygulamalar. Türk Psikoloji Yazıları, 3(6), 49-74.
- Şimşek, Ö.F. (2007). Yapısal eşitlik modellemesine giriş: Temel ilkeler ve LISREL uygulamaları. Ankara: Ekinoks Yayınları.
- Mindrila, D. (2010). Maximum likelihood (ML) and diagonally weighted least squares (DWLS) estimation procedures: A comparison of estimation bias with ordinal and multivariate non-normal data. International Journal of Digital Society (IJDS), 1(1): 60-66.
- Wang, W. C., & Cunningham, E. G. (2005). Comparison of alternative estimation methods in confirmatory factor analyses of the general health questionnaire. Psychological Reports, 97(1), 3-10.
