Author :
Abstract
Keywords
Abstract
When attributes are hierarchically structured, modifying the Q-matrix or prior distribution in the estimation process yields more accurate and precise item and person parameter estimates. Modification of the prior distribution and the Q-matrix depend on the assumed hierarchical structure, as such, identifying the correct hierarchical structure is of the essence. To address the subjectivity in the conventional methods for attribute structure identification (i.e., expert opinions via content analysis and verbal data analyses such as interviews and think-aloud protocols), this study proposes a likelihood-ratio test based exhaustive empirical search for identifying hierarchical structures. It further suggests a likelihood-ratio approach for selection of the most accurate hierarchical structure when multiple candidates are present. Results show that the likelihood ratio test based exhaustive search yields an R-matrix that specifies all the prerequisite relationships among all attributes. Thus, the method is promising and can be used for exploratory purposes in attribute hierarchy identification.
Keywords
- Akbay, L., & de la Torre, J. (2015, April). Effect of Q-matrix design under hierarchical attribute structures. Poster presented at the meeting of National Council on Measurement in Education, Chicago, IL - USA. Akbay, L., Terzi, R., Kaplan, M., & Karaaslan, K. G. (2017). Expert-based attribute identification and validation on fraction subtraction: A cognitively diagnostic assessment application. Journal on Mathematics Education, 8(1), 103-120.
- Chi, M. T. (1997). Quantifying qualitative analyses of verbal data: A practical guide. The journal of the learning sciences, 6, 271-315. https://doi.org/10.1007/s11336-009-9125-0
- Cui, Y., & Leighton, J. P. (2009). The hierarchy consistency index: Evaluating person fit for cognitive diagnostic assessment. Journal of Educational Measurement, 46, 429-449.
- de la Torre, J. (2009a). A cognitive diagnosis model for cognitively based multiple-choice options. Applied Psychological Measurement, 33, 163-183. https://doi.org/10.1177/0146621608320523
- de la Torre, J. (2009b). DINA model and parameter estimation: A didactic. Journal of Educational and Behavioral Statistics, 34, 115-130. https://doi.org/10.3102/1076998607309474
- de la Torre, J. (2011). The generalized DINA model framework. Psychometrika, 76, 179-199. https://doi.org/10.1007/s11336-011-9207-7
- de la Torre, J., & Douglas, J. A. (2004). Higher-order latent trait models for cognitive diagnosis. Psychometrika, 69, 333-353. https://doi.org/10.1007/BF02295640
- de la Torre, J., & Minchen, N. (2014). Cognitive diagnostic assessments and the cognitive diagnosis model framework. Psicologia Educativa, 20, 89-97. https://doi.org/10.1016/j.pse.2014.11.001
- Doornik, J. A. (2011). Object-oriented matrix programming using Ox (Version 6.20) [Computer software]. London: Timberlake Consultants Press.
- Embretson, S. E. (1998). A cognitive design system approach to generating valid tests: Application to abstract reasoning. Psychological Methods, 3, 380-396. http://dx.doi.org/10.1037/1082-989X.3.3.380
- Gierl, M. J., Wang, C., & Zhou, J. (2008). Using the attribute hierarchy method to make diagnostic inferences about examinees' cognitive skills in algebra on the SAT. Journal of Technology, Learning, and Assessment, 6(6), 1-50.
- Junker, B. W., & Sijtsma, K. (2001). Cognitive assessment models with few assumptions, and connections with nonparametric item response theory. Applied Psychological Measurement, 25, 258-272. https://doi.org/10.1177/01466210122032064
- Leighton, J. P., Gierl, M. J., & Hunka, S. M. (2004). The attribute hierarchy method for cognitive assessment: A variation on Tatsuoka’s rule-space approach. Journal of Educational Measurement, 41, 205-237. https://doi.org/10.1111/j.1745-3984.2004.tb01163.x
- Linn, M. C., Eylon, B. S., & Davis, E. A. (2004). The knowledge integration perspective on learning. In: M.C. Linn, E.A. Davis, & P. Bell (Eds.), Internet environments for science education (pp. 29-46). Mahvah, NJ: Lawrence Erlbaum Associates.
- Maris, E. (1995). Psychometric latent response models. Psychometrika, 60, 523-547. https://doi.org/10.1007/BF02294327
- Maris, E. (1999). Estimating multiple classification latent class models. Psychometrika, 64, 187-212. https://doi.org/10.1007/BF02294535
- Robitzsch, A., Kiefer, T., George, A. C., & Uenlue, A. (2019). CDM: Cognitive diagnosis modeling [Computer software]. Retrieved from https://CRAN.R-project.org/package=CDM
- Rupp, A. A., & Templin, J. L. (2008). Unique characteristics of diagnostic classification models: A comprehensive review of the current state-of-the-art. Measurement, 6, 219-262. https://doi.org/10.1080/15366360802490866
- Schmidt, W. H., Wang, H. C., & McKnight, C. C. (2005). Curriculum coherence: An examination of US mathematics and science content standards from an international perspective. Journal of Curriculum Studies, 37, 525-559.
- Smith, C. L., Wiser, M., Anderson, C. W., & Krajcik, J. (2006). Implications of research on children's learning for standards and assessment: A proposed learning progression for matter and the atomic-molecular theory. Measurement: Interdisciplinary Research & Perspective, 4, 1-98.
- Templin, J. L., & Bradshaw, L. (2014). Hierarchical diagnostic classification models: A family of models for estimating and testing attribute hierarchies. Psychometrika, 79, 317-339. https://doi.org/10.1007/s11336013-9362-0
- Templin, J. L., & Henson, R. A. (2006). Measurement of psychological disorders using cognitive diagnosis models. Psychological methods, 11(3), 287-305. http://dx.doi.org/10.1037/1082-989X.11.3.287
- Tjoe, H., & de la Torre, J. (2014). The identification and validation process of proportional reasoning attributes: An application of a cognitive diagnosis modeling framework. Mathematics Education Research Journal, 26, 237-255.
- Vosniadou, S., & Brewer, W. F. (1992). Mental models of the earth: A study of conceptual change in childhood. Cognitive psychology, 24, 535-585.
