A comparison of discriminant logistic regression and Item Response Theory Likelihood-Ratio Tests for Differential Item Functioning (IRTLRDIF) in polytomous short tests

  1. María Dolores Hidalgo 1
  2. María Dolores López-Martínez 1
  3. Juana Gómez-Benito 2
  4. Georgina Guilera 2
  1. 1 Universidad de Murcia
    info

    Universidad de Murcia

    Murcia, España

    ROR https://ror.org/03p3aeb86

  2. 2 Universitat de Barcelona
    info

    Universitat de Barcelona

    Barcelona, España

    ROR https://ror.org/021018s57

Journal:
Psicothema

ISSN: 0214-9915

Year of publication: 2016

Volume: 28

Issue: 1

Pages: 83-88

Type: Article

DIALNET GOOGLE SCHOLAR lock_openOpen access editor

More publications in: Psicothema

Abstract

Antecedentes: en ciencias sociales, del comportamiento y de salud es habitual usar tests breves. El tamaño del test puede afectar a la correcta identificación de ítems con DIF. Este trabajo compara la eficacia relativa de la Regresión Logística Discriminante (RLD) e IRTLRDIF en la detección del DIF en tests cortos politómicos. Método: se diseñó un estudio de simulación. Se manipuló tamaño del test, tamaño de la muestra, cantidad DIF y número de categorías de respuesta al ítem. Se evaluó el Error Tipo I y la potencia.Resultados: en las condiciones de no-DIF IRTLRDIF y RLD mostraron tasas de Error Tipo I cercanas al nivel nominal. En tests con DIF las tasas de Error Tipo I dependieron del tamaño del test, de la muestra, cantidad de DIF, contaminación del test y número de categorías del ítem. RLD presentó mayor tasa de Error Tipo I que IRTLRDIF. La potencia estuvo afectada por la cantidad de DIF y tamaño de la muestra. En tests muy cortos RLD mostró mayor potencia que IRTLRDIF. Conclusiones: en tests cortos y con DIF las tasas de Error Tipo I fueron elevadas. La potencia de IRTLRDIF y RLD fue relativamente baja en tests cortos y tamaños muestrales pequeños.

Bibliographic References

  • Balluerka, N., Gorostiaga, A., Gómez-Benito, J., & Hidalgo, M. D. (2010). Use of multilevel logistic regression to identify the causes of differential item functioning. Psicothema, 22, 1018-1025.
  • Benítez, I., Padilla, J. L., Hidalgo, M. D., & Sireci, S. G. (in press). Using mixed methods to interpret differential item functioning. Applied Measurement in Education.
  • Bolt, D. M. (2002). A Monte-Carlo comparison of parametric and nonparametric polytomous DIF detection methods. Applied Psychological Measurement, 15, 113-141.
  • Bradley, J. V. (1978) Robustness? British Journal of Mathematical and Statistical Psychology, 31, 144-152.
  • Camilli, G., & Shepard, L. A. (1994). Methods for identifying biased test items. Newbury Park, CA: Sage.
  • DeMars, C. E. (2008). Polytomous differential item functioning and violations of ordering of the expected latent trait by the Raw Score. Educational and Psychological Measurement, 68, 379-386.
  • Donoghue, J. R., Holland, P. W., & Thayer, D. T. (1993). A Monte Carlo study of factors that affect the Mantel-Haenszel and standardization measures of differential item functioning. In P. W. Holland & H. Wainer (Eds.), Differential item functioning (pp. 137-166). Hillsdale, NJ: Lawrence Erlbaum.
  • Finch, W. H., & French, B. F. (2008). Anomalous Type I Error rates for identifying one type of differential item functioning in the presence of the other. Educational and Psychological Measurement, 68, 742-759.
  • French, A. W., & Miller, T. R. (1996). Logistic regression and its use in detecting differential item functioning in polytomous items. Journal of Educational Measurement, 33, 315-332.
  • Gelin, M. N., & Zumbo, B. D. (2007). Operating characteristics of the DIF MIMIC approach using Jöreskog’s covariance matrix with ML and WLS estimation for short scales. Journal of Modern Applied Statistical Methods, 6, 573-588.
  • Gómez-Benito, J., Hidalgo, M. D., & Zumbo, B. (2013). Effi ciency of combination of R-square and Odds-ratio using discriminant logistic function for detecting DIF in polytomous items. Educational and Psychological Measurement, 73, 875-897.
  • González-Betanzos, F., & Abad, F. J. (2012). The effects of purifi cation and the evaluation of Differential Item Functioning with the likelihood ratio test. Methodology, 8, 134-145.
  • Guilera, G., Gómez-Benito, J., Hidalgo, M. D., & Sánchez-Meca, J. (2013). Type I Error and statistical power of Mantel-Haenszel procedure for detecting DIF: A meta-analysis. Psychological Methods, 18, 553-571.
  • Hidalgo, M. D., Gómez-Benito, J., & Padilla, J. L. (2005). Regresión logística: alternativas de análisis en la detección del funcionamiento diferencial del ítem [Logistic regression: Analytic strategies in differential item functioning detection]. Psicothema, 17, 509-515.
  • Miller, T. R., & Spray, J. A. (1993). Logistic discriminant function analysis for DIF identifi cation of polytomously scored items. Journal of Educational Measurement, 30, 107-122.
  • Paek, I., & Wilson, M. (2011). Formulating the Rasch Differential Item functioning model under the marginal maximum Likelihood estimation context and its comparison with Mantel-Haenszel procedure in short test and small sample conditions. Educational and Psychological Measurement, 71, 1023-1046.
  • Penfield, R. D. (2007). An approach for categorizing DIF in polytomous items. Applied Measurement in Education, 20, 335-355.
  • Penfield, R. D., Alvarez, K., & Lee, O. (2009). Using a taxonomy of Differential Step Functioning to improve the interpretation of DIF in polytomous items: An illustration. Applied Measurement in Education, 22, 61-78.
  • Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores. Psychometric Monograph Supplement, 17.
  • Scott, N. W., Fayer, P. M., Aaronson, N. K., Bottomley, A., Graeff, A., Groenvold, M., …, Sprangers, M. A. G. (2009). A simulation study provided sample size guidance for differential item functioning (DIF) using short scales. Journal of Clinical Epidemiology, 62, 288-295.
  • Scott, N. W., Fayers, P. M., Aaronson, N. K., Bottomley, A., Graeff, A., Groenvold, M., …, Sprangers, M. A. G. (2010). Differential Item Functioning (DIF) analyses of health related quality of life instruments using logistic regression. Health and Quality of Life Outcomes, 8, 81.
  • Sireci, S. G., & Ríos, J. A. (2013). Decisions that make a difference in detecting differential item functioning. Educational Research and Evaluation: An International Journal of Theory and Practice, 19, 170-187.
  • Teresi, J. A. (2006). Overview of quantitative measurement methods: Equivalence, invariance, and differential item functioning in health applications. Medical Care, 44, 39-49.
  • Thissen, D. (2001). IRTLRDIF v.2.0b: Software for the computation of the statistics involved in item response theory likelihood-ratio tests for differential item functioning [Computer software and manual]. University of North Carolina at Chapel Hill.
  • Thissen, D., Steinberg, L., & Wainer, H. (1988). Use of item response theory in the study of group differences in trace lines. In Wainer, H. & Braun, H. I. (Eds.), Test Validity, pp 147-169. Hillsdale, N.J.: Erlbaum.
  • Wang, W. C., & Yeh, Y. L. (2003). Effects of anchor item methods on differential item functioning detection with the likelihood ratio test. Applied Psychological Measurement, 27, 479-498.
Data source: Dialnet