Author :
Abstract
This study investigated empirical evidences for construct validity of the TIMSS 2011 Mathematics test by using sample of fourth and eighth grade Turkish students (n=4091). The purpose of this study was to analyze construct validity of the TIMSS 2011 mathematics test by using Turkish students’ data. A confirmatory factor analysis model was formed with using cognitive domain definitions for each mathematics items presented by TIMSS assessment framework. In this three-factor model, observed variables (items) were associated to three latent variables (mathematics cognitive domain), namely knowing, applying and reasoning. Fit indices of confirmatory factor analysis, conducted by LISREL, showed mixed results. For eighth grade booklets, the proposed model was supported by the data, however, for fourth grade booklets the proposed model was not supported completely. Besides, eight of the items were found to be correlated with other cognitive domains than with the defined one. These results showed that for eighth grade, the mathematics cognitive domain defined by TIMSS also works for the Turkish sample and that a scale score for each cognitive domain could be used confidently. However, for fourth grade, a scale score for each cognitive domain should be used with caution for Turkish students.
Keywords
Abstract
This study investigated empirical evidences for construct validity of the TIMSS 2011 Mathematics test by using sample of fourth and eighth grade Turkish students (n=4091). The purpose of this study was to analyze construct validity of the TIMSS 2011 mathematics test by using Turkish students’ data. A confirmatory factor analysis model was formed with using cognitive domain definitions for each mathematics items presented by TIMSS assessment framework. In this three-factor model, observed variables (items) were associated to three latent variables (mathematics cognitive domain), namely knowing, applying and reasoning. Fit indices of confirmatory factor analysis, conducted by LISREL, showed mixed results. For eighth grade booklets, the proposed model was supported by the data, however, for fourth grade booklets the proposed model was not supported completely. Besides, eight of the items were found to be correlated with other cognitive domains than with the defined one. These results showed that for eighth grade, the mathematics cognitive domain defined by TIMSS also works for the Turkish sample and that a scale score for each cognitive domain could be used confidently. However, for fourth grade, a scale score for each cognitive domain should be used with caution for Turkish students.
Keywords
- Akyüz, G. & Berberoğlu, G. (2010). Teacher and classroom characteristics and their relations to mathematics achievement of the students in the TIMSS. New Horizons in Education, 58(1), 77-95.
- Arvey, R. D. (1992). Constructs and construct validation: Definitions and issues. Human Performance, 5(1&2), 59-69.http://dx.doi.org/10.1080/08959285.1992.9667924
- Baykal, A. & Circi, R. (2010). Item revision to improve construct validity: A study on released science items in Turkish PISA 2006. Procedia Social and Behavioral Sciences, 2, 19311935.http://dx.doi.org/10.1016/j.sbspro.2010.03.1012
- Berberoglu, G., Celebi, O., Ozdemir, E., Uysal, E., Yayan, B. (2003). Factors effecting achievement level of Turkish students in the third international mathematics and science study (TIMSS). Eğitim Bilimleri ve Uygulama, 2 (3), 3-14.
- Bilican, S., Demirtaşlı, R. N., Kilmen, S. (2011). The attitudes and opinions of the students towards mathematics course: The comparison of TIMSS 1999 and TIMSS 2007. Educational Sciences: Theory & Practice, 11(3), 1277-1283.
- Byrne, B. M. (1998). Structural equation modeling with LISREL, PRELIS, and SIMPLIS: Basic concepts, applications, and programming. New Jersey: Lawrence Erlbaum Associates, Inc.
- Ceylan, E. & Berberoğlu, G. (2007). Factors related with students' science achievement: A modeling study. Education and Science, 32 (144), 36-48.
- Cronbach, L. J. (1971). Test validation. In R. L. Thorndike (Ed.), Educational measurement (2nd ed., pp. 443- 507). Washington, DC: American Council on Education.
- Demir, İ. & Kılıç, S. (2010). Using PISA 2003, examining the factors affecting students’ mathematics achievement. Hacettepe Üniversitesi Eğitim Fakültesi Dergisi (H. U. Journal of Education), 38, 44-54.
- Eckert, J. M. (2008). Trends in mathematics and science study (TIMSS) international accountability and implications for science instruction. Research in Comparative and International Education, 3 (2), 202210.http://dx.doi.org/10.2304/rcie.2008.3.2.202
- Eklöf, H. (2007). Test-taking motivation and mathematics performance in TIMSS 2003. International Journal of Testing, 7(3), 311-326.http://dx.doi.org/10.1080/15305050701438074
- Foy, P., Arora, A. and Stanco, G. M. (2013). TIMSS 2011 user guide for the international database. Chestnut Hill, MA: TIMSS & PIRLS International Study Center, Boston College.
- Fraenkel, J. R. & Wallen, N. E. (2005). How to design and evaluate research in education. New York: McGraw- Hill.
- George, D., & Mallery, P. (2003). SPSS for Windows step by step: A simple guide and reference. 11.0 update (4th ed.). Boston: Allyn & Bacon.
- Hu, L.-T. & Bentler, P.M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives, Structural Equation Modeling, 6, 1–55. http://dx.doi.org/10.1080/10705519909540118
- Hubley, A. M., & Zumbo, B. D. (2011). Validity and the consequences of test interpretation and use. Social Indicators Research, 103(2), 219-230.http://dx.doi.org/10.1007/s11205-011-9843-4
- Incikabi, L., Ozgelen S., Hartono, T. (2012). A comparative analysis of numbers and biology content domains between Turkey and the USA. International Journal of Environmental & Science Education. 7 (4), 523-536.
- Joncas, M. & Foy, P. (2012). Sample design in TIMSS and PIRLS. Retrieved January 10, 2014, from http://pirls.bc.edu/methods/t-sample-design.html.
- Jöreskog, K. G., & Sörbom, D. (1999). LISREL 8.30. Chicago: Scientific Software International.
- Jöreskog, K. G. (2004). Structural Equation modeling with ordinal variables using LISREL. Retrieved June 30, 2013, from http://202.129.48.214/CD4007/LISREL88S/PDFfiles/ordinal.pdf.
- Kline, R.B. (1998). Principles and practice of structural equation modeling, New York: The Guilford Press.
- McDonald, R. P., & Moon-Ho, R. H. (2002). Principles and practice in reporting structural equation analyses. Psychological Methods, 7(1), 64-82.http://dx.doi.org/10.1037//1082-989X.7.1.64
- Messick, S. (1989). Validity. In R. L. Linn (Ed.), Educational measurement (pp. 13–103). New York, NY: Macmillan
- MONE (2009). İlköğretim matematik dersi 1-5. sınıflar öğretim programı. Ankara: Talim ve Terbiye Kurulu Başkanlığı.
- Mullis, I. V., Martin, M. O., Ruddock, G. J., O'Sullivan, C. Y., & Preuschoff, C. (2009). TIMSS 2011 assessment frameworks. Amsterdam: International Association for the Evaluation of Educational Achievement.
- Mullis, I. V. & Martin, M. O. (2012). TIMSS 2011 Item writing guidelines. Retrieved January 10, 2014, from http://pirls.bc.edu/methods/pdf/T11_Item_writing_guidelines.pdf.
- Mullis, I. V., Martin, M. O., Foy, P. Arora, A. (2012). TIMSS 2011 International Results in Mathematics. Amsterdam: International Association for the Evaluation of Educational Achievement.
- Olkun, S. & Aydoğdu, T. (2003). Üçüncü uluslararası matematik ve fen araştırması (TIMSS) nedir? Neyi sorgular? Örnek geometri soruları ve etkinlikler. İlköğretim Online, 2 (1), 28-35.
- Ozturk, D., & Ucar, S. (2010). TIMSS verileri kullanarak Tayvan ve Türkiye’ deki 8.sınıf öğrencilerinin fen başarısına etki eden faktörlerin belirlenmesi ve karşılaştırılması. Cukurova University Journal of Social Sciences, 19(3), 241-256.
- Schumacker, R. E. & Lomax, R. G. (2004). A beginners guide to structural equation modeling. New Jersey: Lawrence Erlbaum Associates, Inc.
- Sharma, S. (1996). Applied multivariate techniques. New York: J. Wiley.
- Sun, J. (2005). Assessing goodness of fit in confirmatory factor analysis. Measurement and Evaluation in Counseling and Development, 37, 240-256.
- TIMSS & PIRLS International Study Center (2013). Retrieved July 31, 2014, from http://timss.bc.edu/
- Ullman, J. B. (2001). Structural equation modeling. In B. Tabachnick & L. S. Fidell (Eds.), Using multivariate statistics (4th ed., pp.653-771). Boston: Allyn & Bacon.
- Yayan, B. (2009). Uluslararası matematik ve fen çalışması (TIMSS 2007) ve Türk öğrencilerin TIMSS 2007’deki matematik performanslarının değerlendirilmesi. Cito Eğitim: Kuram ve Uygulama Dergisi, 3, 39-52.
- Yıldırım, Ö., Demirtaşlı, Ç. N., Akbaş, U. (2012). The opinions of mathematics teachers on homework and ın- class assessment: TIMSS 1999 and TIMSS 2007 periods. Eğitim ve Bilim, 37 (163), 126-142.
- Yücel, C., Karadağ, E., & Turan, S. (2013). TIMSS 2011 ulusal ön değerlendirme raporu. (Eğitimde Politika Analizi Raporlar Serisi I). Eskişehir: Osmangazi Üniversitesi Eğitim Fakültesi.
- Zumbo, B. D. (2005). Structural equation modeling and test validation. In B. S. Everitt & D. C. Howell (Eds), Encyclopedia of statistics in behavioral science (pp. 1951-1958). Hoboken, N.J.: John Wiley & Sons.
- TIMSS 2011 Sample Released Items
- TIMSS 2011 Problematic Items