Development and validation of a questionnaire about determinants of academic success in secondary school students

Jordi Longás Mayayo
Elena Carrillo Álvarez
Albert Fornieles Deu

Abstract

Introduction: School failure is a major challenge at individual and social levels, owing to its negative impact on social cohesion, citizen participation, economic development, and sustainability and also its impact on health habits. Various studies have shown the link between family socio-economic level and educational results, to the point that this factor has been regarded as their principal
determinant. Nonetheless, the essentially attributing school failure to the students’ milieu is only descriptive in value and could impede the development of more appropriate responses to this challenge. A better systemic and integral understanding of the phenomenon might contribute to a better approach to the problem through the development of new proposals for public policy and socio-educational actions. To this end, it is necessary to have appropriate measurement instruments that evaluate different dimensions in an integrated manner. Method: The objective of this study is to evaluate the psychometric properties of the Questionnaire on Determinants of Success at School (QDSS) in a sample of 858 secondary-school students resident in vulnerable socio-demographic settings from 5 Spanish cities. The questionnaire combines items about factors that specialist literature has significantly linked to educational attainment, excluding those that already have specific validated questionnaires. Results: The principal component analysis gave 7 factors: school environment, relationships with classmates, personal expectations, social capital, ITC study resources, climate in class, and family support. The goodness of fit indices show good properties for the questionnaire. Conclusion: The QDSS is an instrument that has appropriate psychometric properties for use in identifying factors that support educational success in secondary-school students.

Cite this article as: Longás Mayayo, J., Carrillo Álvarez, E., Fornieles Deu, A. and Riera i Romani, J. (2018). Desarrollo y validación del cuestionario sobre condicionantes de éxito escolar en alumnos de secundaria | Development and validation of a questionnaire about determinants of academic success in secondary school students. Revista Española de Pedagogía, 76 (269), 55-82. doi: 10.22550/REP76-1-2018-03

Referencias | References

Archambault, I., Janosz, M., Morizot, J. and Pagani, L. (2009). Adolescent Behavioral, Affective and Cognitive Engagement in School: Relationship to Dropout. Journal of School Health, 79 (9), 408-415.

Belfield, C. (2008). The cost of early school-leaving and school failure. Washington, DC: World Bank.

Berliner, D. C. (2009). Poverty and Potential: OutofSchool Factors and School Success. Boulder and Tempe: Education and the Public Interest Center & Education Policy Research Unit. Retrieved from http://epicpolicy.org/publication/poverty-and-potential (Consulted on March 28, 2015).

Boada, C., Herrera, D., Mas, E., Miñarro, E., Olivella, M. and Riudor, X. (Dir.) (2011). Informe

sobre el risc de fracàs escolar a Catalunya. Col·lecció Estudis i Informes, (26). Barcelona:

Consell de Treball, Econòmic i Social de Catalunya (Generalitat de Catalunya).

Bonal, X., Tarabini, A., Constans, M., Kliczkowski, F. and Valiente, O. (2010). Ser pobre en la escuela. Habitus de pobreza y condiciones de educabilidad. Barcelona: Miño y Dávila.

Bravo, M. and Verdugo, S. (2007). Gestión escolar y éxito académico en condiciones de pobreza. Revista Electrónica Iberoamericana sobre Calidad, Eficacia y Cambio en Educación, 5,121- 144.

Buja, A. and Eyuboglu, N. (1992). Remarks on Parallel Analysis. Multivariate Behavioural

Research, 27 (4), 509-540.

Brunello, G. and Da Paola, M. (2013). The costs of early school leaving in Europe, IZA Discussion Paper, 7791. Bonn: IZA.

Calero, J. and Escardibul, J. O. (2007). Evaluación de servicios educativos: el rendimiento en los centros públicos y privados medido en PISA-2003. Hacienda Pública Española/ Revista de Economía Pública, 183, 33-66.

Calero, J., Chois, K. and Waisgrais, S. (2010). Determinantes del riesgo de fracaso escolar

en España: una aproximación a través de un análisis logístico multinivel aplicado a PISA-2006. Revista de Educación, special number 2010, 225-256.

Canals, J. (2002). Estudi de fiabilitat i validesa del CSI-4. Doctoral thesis, Tarragona: Universitat Rovira i Virgili.

Carpenter, H., Cummings, C., Dyson, A., Jones, L., Laing, K., Oseman, D. and Todd, L. (2010). Extended services evaluation: end of year one report. Research report, dfe-rr016. London:

Department for Education. Retrieved from https://www.education.gov.uk/publications/eorderingdownload/dfe-rr016.pdf

Cattell, R. B. (1966). The Scree test for the number of factors. Multivariate Behavioral Research, 1 (2), 245–276.

Choi, A. and Calero, J. (2013). Determinantes del riesgo de fracaso escolar en España en PISA-2009 y propuestas de reforma. Revista de Educación, 362, 562-593.

Civís, M. and Longás, J. (2015). La colaboración interinstitucional como respuesta al desafío

de la inclusión socioeducativa. Análisis de 4 experiencias de trabajo en red a nivel local en Cataluña. Educación XX1, 18 (1), 213-236. doi:10.5944/educxx1.18.1.12318

Coleman, J. C. (1988). Social capital in the creation of human capital, American Journal of Sociology, 94, S95-S120.

Collet, J. and Tort, A. (2011). Famílies, escola I èxit. Millorar els vincles per millorar els resultats. Barcelona: Fundació Jaume Bofill.

Cordero, J. M., Crespo, E. and Santín, D. (2010). Factors affecting educational attainment: evidence from Spanish PISA 2006 results. Regional and Sectoral Economic Studies, 10, 55-76.

Costa, P. T. and McCrae, R. (2008). Inventario de personalidad NEO reducido de cinco factores (NEO-FFI). (3rd. Ed.) Madrid: TEA Ediciones.

Costa-Tutusaus, Ll. and Guerra-Balic, M. (2016). Development and psychometric validation of a scoring questionnaire to assess healthy lifestyles among adolescents in Catalonia. BMC

Public Health, 16 (89).

Depalma, R., Matusov, E. and Smith, M. (2009). Smuggling Authentic Learning into the School Context: Transitioning From an Innovative Elementary to a Conventional High School. Teachers College Record, 111 (4), 934-972.

Escudero, J. and Martínez, B (2012). Las políticas de lucha contra el fracaso escolar: ¿programas especiales o cambios profundos del sistema y la educación? Revista de Educación, special number 2012, 174-193.

Eurostat (2015). Europe 2020 education indicators in 2014. The EU is moving closer to

its Europe 2020 goals on education. Retrieved from http://ec.europa.eu/eurostat/documents/2995521/6787423/3-20042015-BP-EN.pdf/b2f295ba-2e15-409c-bec9-91c4e49c5d32 (Consulted on May 5, 2015).

European Commission (2010). Europe 2020: A strategy for smart, sustainable and inclusive

growth. Brussels: European Commission. Retrieved from http://eurlex.europa.eu/LexUriServ/LexUriServ.do?uri=COM:2010:2020:FIN:ES:PDF

European Commission (2011). Comunicación de la comisión al Parlamento Europeo, al Consejo, al Comité Económico y Social Europeo y al Comité de las Regiones. Abordar el abandono escolar prematuro: una contribución clave a la agenda europea 2020. Brussels: European Commission. Retrieved from: http://ec.europa.eu/education/schooleducation/doc/earlycom_es.pdf

Fernández-Enguita, M., Mena, L. and Riviere, J. (2010). Fracaso y abandono escolar en España. Barcelona: Colección Estudios Sociales, Obra Social La Caixa.

Ferrer, F. and Albaigés, B. (2008). L’estat de l’esducació a Catalunya. Anuari 2006. Volum II.

Barcelona: Fundació Jaume Bofill and Editorial Mediterrània.

Flecha, R., García, R., Gómez, A. and Latorre, A. (2009). Participación en escuelas de éxito:

una investigación comunicativa del proyecto Includ-ed. Cultura y Educación, 21, 183-196. Flecha, R. and L. Puigvert, L. (2002). La Comunidad de Aprendizaje: Una Apuesta por la igualdad educativa. REXE: Revista de Estudios y Experiencias En Educación, 1, 11–20.

Geddes, H. (2010). El apego en el aula. Barcelona: Grao.

Gil-Flores, J. (2011). Estatus socioeconómico de las familias y resultados educativos logrados por el alumnado. Cultura y Educación, 23 (1), 141-154.

Goldthorpe, J. (2000). On sociology. Oxford: Oxford University Press.

Guio, A. C. and Marlier, E. (2013). Alternative vs. current measures of material deprivation at EU level: What differences does it make? ImPRovE Discussion Paper No. 13/07. Antwerp: Improve.

Hanushek, E. and Woessmann, L. (2010). The Economics of International Differences in Educational Achievement. Working Paper 15949.

Cambridge, MA: National Bureau of Economic Research. Hatcher, R. and Leblond, I. (2001). Education Action Zones and Zones d’Education Prioritaires. Conferene Travelling Policy / Local Spaces: Globalisation, Identities and Education Policy in Europe. United Kingdom: Keele University.

Hernando, A., Oliva, A. and Pertegal, M. A. (2012). Variables familiares y rendimiento académico en la adolescencia. Estudios de Psicología, 33, 51-56.

Hollingshead, A. A. (1975). Four-factor index of social status. Unpublished manuscript. New

Haven, CT: Yale University.

Horn, J. L. (1965). A rationale and test for the number of factors in factor analysis. Psycometrika, 30, 179-185.

Instituto Nacional de Estadística. (2015a). Tasa de paro por nivel de formación alcanzado, sexo y grupo de edad. Encuesta de Población Activa. Resultados nacionales, 1r trimestre 2015.Madrid: INE. Retrieved from http://www.ine.es/jaxiT3/Tabla.htm?t=6393

Instituto Nacional de Estadística. (2015b). Tasa de riesgo de pobreza o exclusión social (estrategia Europa 2020) por nivel de formación alcanzado (personas de 16 y más años). Encuesta de Condiciones de Vida. Series Base 2013. Resultados nacionales. Published on 26 May 2015.

Madrid: INE. Retrieved from http://www.ine.es/jaxiT3/Tabla.htm?t=4674&L=0

Izquierdo, I., Olea, J. and Abad, F. (2014). Exploratory factor analysis in validation studies: Uses and recommendations. Psicothema, 26 (3), 395-400.

Jeynes, W. H. (2007). The relation between parental involvement and urban secondary school student academic achievement: A metaanalysis. Urban Education 42, 82-110. 10.1177/0042085906293818

Jolonch, A. (2008). Exclusió social. Dels marges al cor de la societat. Barcelona: Fundació Lluís Carulla-ESADE, Col·lecció Observatori dels Valors.

León, O. G. and Montero, I. (2015). Métodos de investigación en Psicología y Educación (5th

Ed.). Madrid: McGraw Hill.

Longás, J., Cussó, I., de Querol, R. and Riera, J. (2016). Análisis de factores de apoyo a trayectorias de éxito escolar en la enseñanza secundaria en contextos de pobreza y vulnerabilidad social en España. Un estudio de casos múltiples. Revista de Estudios y Experiencias en Educación, 15 (28), 107-127.

Maccoby, E. and Martín, J. (2006). Socialization in the context of the family: Parent-child interaction. In Hetherington, E. (Ed.), Handbook of child psychology (pp. 1-101). NY: Wiley.

Marchesi, A. and Hernández, C. (Coord.) (2003). El fracaso escolar. Una perspectiva internacional. Madrid: Alianza.

Marcus, R. F. and Sanders-Reio, J. (2001). The influence of attachment on school completion. School Psychology Quarterly, 16 (4), 427-444.

Mena, L., Fernández-Enguita, M. and Riviere, J. (2010). Desenganchados de la educación: procesos, experiencias, motivaciones y estrategias del abandono y del fracaso escolar. Revista de Educación, special number 2010, 119-145.

Miller, P. (2016). Liderazgo de intermediación en entornos complejos. Pedagogía Social Revista Interuniversitaria, 28, 17-24.

Moliner, O. (2008). Condiciones, procesos y circunstancias que permiten avanzar hacia la inclusión educativa: retomando las aportaciones de la experiencia canadiense. Revista Electrónica Iberoamericana Sobre Calidad, Eficacia y Cambio En Educación, 6, 27–44.

Murillo, J. F. and Román, M. (2011). School infrastructure and resources do matter: analysis of the incidence of school resources on the performance of Latin American students. School effectiveness and school improvement, 22 (1), 29-50.

OECD (2013). PISA 2012 Results: Excellence Through Equity: Giving Every Student the Chance to Succeed (Volume II), PISA, OECD Publishing. doi: 10.1787/19963777

WHO (2013). Review of social determinants and the health divide in the WHO European Region: final report. Retrieved from http://www.euro.who.int/__data/assets/pdf_file/0004/251878/Review-of-social-determinants-and-the-health-divide-in-the-WHO-European-Region-FINAL-REPORT.pdf?ua=1

Pàmies, J. (2013). The impact of groupings in school. Spaces of learning and sociability for

Moroccan youth in Barcelona. Revista de Educación, 362, 133–158.

Pedró, F. (2012). Políticas públicas sobre apoyo y refuerzo educativo: evidencias internacionales. Revista de Educación, special number 2012, pp. 22-45.

Pozo, M. T., Suárez, M. and García-Cano, M. (2012). Logros educativos y diversidad en la

escuela: hacia una definición desde el consenso. Revista de Educación, 358, 59-84.

Renée, M. and McAlister, M. (2011). The strengths and challenges of community organizing as an education reform strategy: what the research says. Annenberg institute for school reform at brown university, Quincy, MA: Nellie Mae Education Foundation.

Rivkin, S., Hanushek, E. and Kain, J. (2005). Teachers, school and academic achievement. Econometrica, 73 (2), 417-458.

Rodríguez, H., Rios, O. and Racionero, S. (2012). Reframing Compensatory Education through Scientific Evidence. Inclusive Actions for Equality of Results. Revista de Educación,

special number 2012, 67-87.

Roman, S., Cuestas, P. J. and Fenollar, P. (2008). An examination of the interrelationships between self-esteem, others’ expectations, family support, learning approaches and academic achievement. Studies in higher education, 33 (2), 127-138.

Roorda, D., Koomen, H., Spilt, J. and Oort, F. (2011). The influence of affective teacher-student relationships on students’ school engagement and achievement: A meta-analytic

approach. Review of Educational Research, 81 (4), 493-529.

Sánchez, E. and García-Rodicio, H. (2006). Re-lectura del estudio PISA: qué y cómo se evalúa e interpreta el rendimiento de los alumnos en la lectura. Revista de Educación, special number 2006, 195-226.

Santana, L.E. and Feliciano, L. (2011). Parents’ and teachers' perceived support, self-concept and decision making in high school students. Revista de Educación, 355, 493-519.

Sarasa, S. and Sales, A. (2009). Itineraris i Factors d’Exclusió Social. Barcelona: Ajuntament de Barcelona, Síndic de Greuges de Barcelona.

Smith, R. (2003). Research and Revelation: What Really Works. In: P. Smeyers and M. Depaepe (Eds.), Beyond Empiricism: On Criteria for Educational Research (pp. 129-140). Leuven: Leuven University Press.

Thompson, B. (2004). Exploratory and Confirmatory Factor Analysis: Understanding Concepts and Applications. Washington, D.C.: American Psychological Association.

Torres-Acosta, N.D., Rodríguez-Gómez, J. and Acosta-Vargas, M. (2013). Personalidad, aprendizaje y rendimiento académico en medicina. Investigación en Educación Médica, 2 (8), 193-201.

Author Biography

Jordi Longás Mayayo is a Lecturer of the Blanquerna-Universitat Ramon Llull Faculty of Psychology, Educational Sciences and Sports. He coordinated the R+D on Education Success in Contexts of Vulnerability and he is the Executive Director of the Scientific Management Team of the CaixaProinfancia program at Obra Social «La Caixa». His main lines of research focus on academic success, socio-educational networks, school management and teachers welfare.

Elena Carrillo Álvarez is Lecturer at the Blanquerna-Universitat Ramon Llull Faculty of Health Sciences, Main Researcher of the research group Global Research on Wellbeing (GRoW) and member of the Spanish Academy of Nutrition and Dietetics. Her research focuses on: the promotion of a healthy diet in different socioeconomic groups and the development of socio-educational governance networks that connect social, education and health agents, from a health determinants perspective.

Albert Fornieles Deu holds a PhD in Psychology from Universidad de Barcelona (1995) with bachelor's degrees in Psychology (UB 1989) and Social and Cultural Anthropology (UNED 2006). He is currently working with the Blanquerna Foundation in the project «Research on school success in contexts of poverty and social vulnerability at the Spanish State level». He also participates in the project «Food insecurity, weight discrimination, eating disorders and psychological well-being in Spanish adolescents».

Jordi Riera Romaní is Professor of Education at Blanquerna-Universitat Ramon Llull, in the Blanquerna Faculty of Psychology, Educational Sciences and Sports. He has been Head of the Department of Education and Dean of the same Faculty (1996-2004), and is currently Vice Chancellor of Academic Policy and Deputy Rector of Universidad Ramón Llull. He is also Principal Investigator of the Consolidated Research Group of «Pedagogy, Society and Innovation with the support of ICT (PSITIC)».

Citación recomendada | Recommended citation

Mayayo, J. L., Álvarez, E. C., & Fornieles Deu, A. (2018) . Development and validation of a questionnaire about determinants of academic success in secondary school students. Revista Española de Pedagogía, 76(269). https://www.revistadepedagogia.org/rep/vol76/iss269/18