Development and validation of the test of spelling competence (TCORT) in incoming university students

Desarrollo y validación del test de competencia ortográfica TCORT en estudiantes universitarios de nuevo ingreso

Juan F. Luesia, Juan F. Plaza, Isabel Benítez, & Milagrosa Sánchez-Martín


This study aims to present a test to assess spelling competence (TCORT) in incoming Spanish university students and to obtain validity evidence to support its use. The starting point was a bank of 73 items covering the most relevant spelling areas of the Spanish language, and the 37 most representative items were selected from the information provided by six experts. Afterwards, a pilot study of the TCORT was carried out with 602 students. The final version consisted of 31 items, with sufficient internal consistency (α = .90; ω = .89) and an invariant unidimensional structure between sexes. It also correlated significantly with other variables, such as previous performance (r = .37, p <.01), verbal reasoning (r = .27, p <.01) and spelling, measured with another tool not adapted to the target population (r = .45, p <.01). Validity evidence based on the test content, internal structure and relations to other variables supported the use of the test to assess spelling competences in incoming university students. The usefulness of TCORT in providing relevant data to facilitate the creation of university policies aimed at promoting spelling competence training in university students is discussed.

Please, cite this article as follows: Luesia, J. F., Plaza, J. F., Benítez, I., & Sánchez-Martín, M. (2023). Desarrollo y validación del test de competencia ortográfica TCORT en estudiantes universitarios de nuevo ingreso [Development and validation of the test of spelling competence (TCORT) in incoming university students]. Revista Española de Pedagogía, 81 (286), 601-636.

Alsina, Á., Ayllón, S., Colomer, J., Fernández-Peña, R., Noell, J. F., Díaz, M. P. I., Pérez-Burriel, M., & Serra, L. (2017). Improving and evaluating reflective narratives: A rubric for higher education students. Teaching and Teacher Education, 63, 148-158.

American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014). Standards for educational and psychological testing. AERA.

Arribas-Águila, D., Santamaría, P., Sánchez-Sánchez, F., & Fernández-Pinto, I. (2013). BAT-7. TEA ability battery. TEA Ediciones.

Arteagoitia, I., Howard, E. R., Loguit, M., Malabonga, V., & Kenyon, D. M. (2005). The spanish developmental contrastive spelling test: An instrument for investigating intra-linguistic and crosslinguistic influences on Spanish-spelling development. Bilingual Research Journal, 29 (3), 541-560.

Ayre, C., & Scally, A. J. (2014). Critical values for Lawshe’s content validity ratio: revisiting the original methods of calculation. Measurement and Evaluation in Counseling and Development, 47 (1), 79-86.

Binkley, M., Erstad, O., Herman, J., Raizen, S., Ripley, M., Miller-Ricci, M., & Rumble, M. (2012). Defining twenty-first century skills. In P. Griffin, B. McGaw, & E. Care (Eds.), Assessment and teaching of 21st century skills (pp. 17-66). Springer.

Byrne, B. M. (2012). Structural equation modeling with Mplus: Basic concepts, applications, and programming. Routledge.

Camps, A., Milian, M., Bigas, M., Camps, M., & Cabré, P. (2004). La enseñanza de la ortografía [Teaching spelling] (3rd ed.). Graó.

Cejudo, J., Salido-López, J. V., & Rodrigo-Ruiz, D. (2017). Efecto de un programa para la mejora en competencia en comunicación lingüística de alumnado de educación secundaria [Effect of a programme to enhance proficiency in linguistic competence in secondary education]. Revista de Psicodidáctica22 (2), 135–141.

Chen, F. (2007). Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling, 14 (3), 464-504.

Child, D. (2006). The essentials of factor analysis (3rd ed.)Continuum.

Consejo de Europa. (2002). Marco Común Europeo de Referencia para las Lenguas: aprendizaje, enseñanza, evaluación [Common European Framework for Languages: Learning, teaching, assessment]. Secretaría General Técnica del Ministerio de Educación, Cultura y Deporte; Subdirección General de Información y Publicaciones; Anaya.

Cuadro, A., Costa Ball, D., Palombo, A. L, & von Hagen, A. (2013). Propiedades psicométricas de una prueba experimental de dictado de palabras para la evaluación de la ortografía en escolares uruguayos [Psychometric properties of an experimental test of word dictation for spelling assessment of uruguayan school children]. Ciencias Psicológicas, 7 (1), 57-66.

Cuetos, F. (2008). Revisión de libros: Evaluación del lenguaje: BLOC (Screening-R)Psicothema20 (4), 975-981.

Cuetos, F., Arribas, D., & Ramos, J. L. (2016). PROLEC-SE-R. Batería de evaluación de los procesos lectores en secundaria y bachillerato​ [PROLEC-SE-R. Battery for the assesment of reading processes in secondary school and baccalaureate]. TEA Ediciones.

Daffern, T., Mackenzie, N. M., & Hemmings, B. (2015). The development of a spelling assessment tool informed by triple word form theory. Australian Journal of Language and Literacy, 38 (2), 72-82.

Dunn, T. J., Baguley, T., & Brunsden, V. (2013). From alpha to omega: A practical solution to the pervasive problem of internal consistency estimation. British Journal of Psychology, 105 (3), 399-412.

Evans, J. D. (1996). Straightforward statistics for the behavioral sciences. Thomson Brooks/Cole Publishing Co.

Fernández-Rufete, A. (2015). Enseñanza de la ortografía, tratamiento didáctico y consideraciones de los docentes de educación primaria de la provincia de Almería [Teaching spelling, didactic treatment and considerations of primary school teachers in the province of Almería]. Investigaciones Sobre Lectura, (4), 7-24.

Ferrando, P. J., & Lorenzo-Seva, U. (2018). Assessing the quality and appropriateness of factor solutions and factor score estimates in exploratory item factor analysis. Educational and Psychological Measurement, 78 (5), 762-780.

Gentry, R., McNeel, J., & Wallace-Nesler, V. (2014). Fostering writing in today's classroom. Shell Education.

Gómez, A. (2005). Enseñar ortografía a universitarios andaluces [Teaching spelling to Andalusian university students]. EA, Escuela Abierta, (8), 129-148.

Gómez, A. (2007). La ortografía del español y los géneros electrónicos [Spanish spelling and electronic writing]. Comunicar, 15 (29), 157-164.

Gómez, A. (2014). La norma disortográfica en la escritura digital [The dysorthographic norm in digital writing]. Didac, (63), 19-25.

Gómez, A., & Gómez, M. T. (2015). Escritura ortográfica y mensajes de texto en estudiantes universitarios [Spelling and texting in university students]. Perfiles Educativos37 (150), 91-104.

Gómez-Torrego, L. (2015). Ortografía de uso español actual [Spelling in current Spanish usage]. Ediciones SM.

Gorsuch, R. L. (2003). Factor analysis. En J. A. Schinka, & W. F. Velicer (Eds.), Handbook of psychology. Volume 2: Research methods in psychology (pp. 143–164). John Wiley & Sons.

Hancock, G. R., & Mueller R. O. (2000). Rethinking construct reliability within latent variable systems. En R. Cudek, S. H. C. duToit, & D. F. Sorbom (Eds.), Structural equation modeling: Present and future (pp. 195-216). Scientific Software Internacional.

Hooper, D., Coughlan, J., & Mullen, M. R. (2008). Structural equation modelling: Guidelines for determining model fit. Electronic Journal of Business Research Methods, 6 (1), 53-60.

Hu, L. y Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6 (1), 1-55.

Hubley, A. y Zumbo, B. (2011). Validity and the consequences of test interpretation and use. Social Indicators Research, 103 (2), 219-230.

Izura, C., Cuetos, F., & Brysbaert, M. (2014). Lextale-Esp: A test to rapidly and efficiently assess the Spanish vocabulary size. Psicológica, 35 (1), 49-66.

Job, J. M., & Klassen, R. M. (2012). Predicting performance on academic and non-academic tasks: A comparison of adolescents with and without learning disabilities. Contemporary Educational Psychology, 37(2), 162-169.

Kellogg, R. T. y Raulerson, B. A. (2007). Improving the writing skills of college students. Psychonomic Bulletin and Review, 14 (2), 237-242.

Khoiriyah, U., Roberts, C., Jorm, C., & Van der Vleuten, C. P. M. (2015). Enhancing students’ learning in problem based learning: validation of a self-assessment scale for active learning and critical thinking. BMC Medical Education, 15, 140.

Lloret-Segura, S., Ferreres-Traver, A., Hernández-Baeza, A., & Tomás-Marco, I. (2014). El análisis factorial exploratorio de los ítems: una guía práctica, revisada y actualizada [Exploratory item factor analysis: A practical guide revised and updated]. Anales de Psicología, 30 (3), 1151-1169.

Mardia, K. V. (1970). Measures of multivariate skewness and kurtosis with applications. Biometrika, 57 (3), 519-530.

Marín-García, I. (2021). Método Signo: ortografía con LSE [Sign method: orthography with LSE]. REVLES, 3, 77-101.

Martín, R. A. (2015). La didáctica de la lengua y la literatura españolas en el máster en Formación del Profesorado de Educación Secundaria. Revisión y renovación [The teaching of Spanish language and literature in the master’s degree in Secondary Education Teacher Training. Review and renewal]. Revista Española de Pedagogía73 (261), 365-379.

Martín-Sánchez, M. A. (2010). Notes to the history of teaching foreign languages: the teaching of grammar. TEJUELO. Didactics of Language and Literature. Education, 8, 59-76.

Mingle, J., & Adams, M. (2015). Social media network participation and academic performance in senior high schools in Ghana. Library Philosophy and Practice (e-journal), 1286.

Muñiz, J., & Fonseca-Pedrero, E. (2019). Diez pasos para la construcción de un test [Ten steps for test development]. Psicothema, 31 (1), 7-16.

Níkleva, D. (2015). La cortesía en los correos electrónicos de estudiantes universitarios como parte de la competencia pragmático-discursiva [Politeness in university students' e-mails as part of pragma-discursive competence]. Spanish in Context, 12 (2), 280-303.

Nunnally, J. C., & Bernstein, I. H. (1994). The assessment of reliability. En Psychometric theory (pp. 248-292). McGraw-Hill.

Orellana-García, P., Valenzuela-Hasenohr, M. F., Kung, M., Elmore, J., & Stenner, A. J. (2020). EVOC: un instrumento para evaluar vocabulario en español en Chile [EVOC: An instrument to evaluate the Spanish lexicon in Chile]. Magis. Revista Internacional de Investigación en Educación, 13, 1-26.

Oren, C., Kennet-Cohen, T., Turvall, E., & Allalouf, A. (2014). Demonstrating the validity of three general scores of PET in predicting higher education achievement in Israel. Psicothema, 26 (1), 117-126.

OECD. (2015). OECD skills strategy diagnostic report: Spain. OECD.

OECD. (2018). Trends shaping education spotlight 16: Writing in a changing world. OECD.

Palacios, A., Arias, V., & Arias, B. (2014). Las actitudes hacia las matemáticas: construcción y validación de un instrumento para su medida [Attitudes towards mathematics: construction and validation of a measurement instrument]. Revista de Psicodidáctica, 19 (1), 67-91.

Pascual-Gómez, I., & Carril-Martínez, I. (2017). Relación entre la comprensión lectora, la ortografía y el rendimiento: un estudio en educación primaria [Relationship between reading comprehension, spelling and academic performance. A study in primary school students]. Ocnos: Revista de Estudios Sobre Lectura, 16 (1), 7-17.

Ramírez-Uclés, I., Flores, P., & Castro, E. (2013). Análisis de las capacidades visuales e intelectuales en los alumnos con talento matemático [Analysis of spatial visualization and intellectual capabilities in mathematically gifted students]. Revista Mexicana de Psicología, 30 (1), 24-31.

Real Academia Española, & Asociación de Academias de la Lengua Española (2010). Ortografía de la lengua española [Spelling of the Spanish language]. Espasa Calpe.

Rhemtulla, M., Brosseau-Liard, P. É., & Savalei, V. (2012). When can categorical variables be treated as continuous? A comparison of robust continuous and categorical SEM estimation methods under suboptimal conditions. Psychological Methods, 17 (3), 354–373.

Rico-Martín, A. M. (2002). Breve análisis de los factores que intervienen en el aprendizaje ortográfico [A brief analysis of the factors involved in spelling learning]. Publicaciones, 32, 71-84.

Rico-Martín, A. M., & Níkleva, D. (2016). Análisis de la competencia lingüístico-discursiva escrita de los alumnos de nuevo ingreso del grado de Maestro en educación primaria [Analysis of written linguistic and discursive competence of first year students in primary education teacher degree]. Revista Signos, 49 (90), 48-70.

Rios, J., & Wells, C. (2014). Validity evidence based on internal structure. Psicothema, 26 (1), 108-116.

Rizzo, M. F. (2022). La regulación del español en contextos digitales: el dispositivo normativo de la Fundéu [The regulation of Spanish in digital contexts The Fundéu's regulatory mechanism]. Spanish in Context19 (2), 314-336.

Romero-González, A. N., & Álvarez-Álvarez, M. D. (2020). La escritura académica de estudiantes universitarios de humanidades a partir de sus producciones. Estudio transversal [The academic writing of university students in the humanities, based on their productions. A transversal study]. Revista Mexicana de Investigación Educativa, 25 (85), 395-418.

Sæle, R. G., Sørlie, T., Nergård-Nilssen, T., Ottosen, K. O., Goll, C. B., & Friborg, O. (2016). Demographic and psychological predictors of grade point average (GPA) in North-Norway: A particular analysis of cognitive/school-related and literacy problems. Educational Psychology, 36 (10), 1886-1907.

Salvador-Mata, F., Gallego, J. L., & Mieres, C. G. (2007). Habilidades lingüísticas y comprensión lectora: una investigación empírica [Language skills and reading comprehension: An empirical investigation]. Bordón, 59 (1), 153-166.

Sánchez-Rivero, R., Alves, R. A., Limpo, T., & Fidalgo, R. (2021). Análisis de una encuesta sobre la enseñanza de la escritura en la educación obligatoria [Analysis of a survey on the teaching of writing in compulsory education: Teachers’ practices and variables]. Revista Española de Pedagogía, 79 (279), 321-340.

Sánchez-Sánchez, F., & Arribas-Águila, D. (2014). BAT-7, batería de aptitudes de TEA: descripción y datos psicométricos [BAT-7, TEA abilities battery: Description and psychometric properties]. International Journal of Developmental and Educational Psychology, 2 (1), 353-363.

Saneleuterio, E. (2018). Solo en caso de ambigüedad o la coherencia en la enseñanza de la tilde [Only in case of ambiguity or the consistency in the teaching of the tilde accent]. Boletín de Filología, 53 (1), 279-289.

Seifart, F. (2006). Orthography development. En J. Gippert, N. P. Himmelmannm, & U. Mosel (Eds.), Essentials of language documentation (pp. 275.299). De Gruyter Mouton.

Sireci, S. G., & Faulkner-Bond, M. (2014). Validity evidence based on test content. Psicothema, 26 (1), 100-107.

Suárez, S. R., Suárez, A. M., Guisado, I. S., & Suárez, M. R. (2019). La ortografía en el ámbito universitario: una realidad que exige la reflexión del alumnado y la implicación del profesorado [Spelling in the university education level: A reality that requires the reflection of students and the involvement of teachers]. Didáctica. Lengua y Literatura, 31, 135-145.

Suárez, S. R., Cremades, R., & Suárez-Ramírez, M. R. (2021). La ortografía como factor de calidad en la educación superior. Datos para una reflexión [The orthography as a quality factor in higher education. Data for a reflection]. Foro de Educación, 19 (2), 387-401.

Svetina, D., Rutkowski, L., & Rutkowski, D. (2019). Multiple-group invariance with categorical outcomes using updated guidelines: An illustration using Mplus and the lavaan/semTools packages. Structural Equation Modeling: A Multidisciplinary Journal, 27 (1), 1-20.

Timmerman, M. E. y Lorenzo-Seva, U. (2011). Dimensionality assessment of ordered polytomous items with parallel analysis. Psychological Methods, 16 (2), 209-220.

Vanderswalmen, R., Vrijders, J., & Desoete, A. (2010). Metacognition and spelling performance in college students. En A. Efklides, & P. Misailidi (Eds.), Trends and prospects in metacognition research (pp. 367–394). Springer.

Venkatesan, S., & Holla, J. M. (2011). Validation of graded spelling list for children with learning disabilities. Journal of Psychology, 2 (2), 73-82.

Wilson, F. R., Pan, W., & Schumsky, D. A. (2012). Recalculation of the critical values for Lawshe’s content validity ratio. Measurement and Evaluation in Counseling and Development, 45 (3), 197-210.

Zhou, Y. (2019). A mixed methods model of scale development and validation análisis.  Measurement: Interdisciplinary Research and Perspectives, 17 (1), 38-47.

Juan F. Luesia is a Research Assistant in the Psychology Department of Universidad Loyola Andalucía. Graduate in Psychology of UNED (Spain), where he specialised in the field of methodology through the inter-university master’s degree in Behavioural Sciences and Health Methodology. He is currently undertaking his thesis focused on the assessment of relevant academic competences in university performance through a comprehensive approach to the phenomenon and the application of mixed methods. His scientific interest covers the applied study of methodology from an interdisciplinary perspective.

Juan F. Plaza is an Associate Professor of the Faculty of Social Sciences at Universidad Loyola Andalucía (Spain), where he teaches the Written Communication subject in the Communication degree. Doctor of Communication of Universidad Pontificia de Salamanca. He has been deputy dean of the Communication degree in the Faculty of Social and Human Sciences at Universidad Loyola Andalucía (Spain) and is currently his general vice-secretary. His research focuses on the study of male and female representations in the mass media (stereotypes, models, language). His work as an assessor in Spanish communication journals should also be noted.

Isabel Benítez is an Associate Professor in the Department of Behavioural Sciences Methodology at Universidad de Granada (Spain). Doctor of Psychology of Universidad de Granada. She specialised in psychometry and pretesting methods to assess survey questionnaires. Her research focuses on the application of qualitative processes, such as cognitive interviewing and behavioural coding, in order to assess the quality of assessment tools within the framework of mixed validation studies. In the same context, she has also taken part in the assessment of bias and equivalence in transcultural studies.

Milagrosa Sánchez-Martín is an Associate Professor and Dean in the Faculty of Social and Human Sciences at Universidad Loyola Andalucía (Spain). Doctor of Psychology of Universidad de Sevilla. Over the years, her scientific production has centred on the development of methodological processes related to the assessment of programmes and methodological quality. Her scientific and technical interests are currently focused on continuing to implement rigorous and multidisciplinary methodologies that make it possible to obtain valid and reliable results in order to address problems that have a major impact on social sciences and health.

Data collected by PlumX Metrics. More information on the metrics collected can be found at