DOI
10.22550/REP79-3-2021-07
Abstract
Introduction: interest in evaluating the improvements resulting from external evaluation processes in higher education has revealed that there is a lack of objective instruments available for this purpose. Consequently, this study presents the design and validation of an instrument for evaluating the improvement effect of the accreditation system of Spanish university degrees. Methodology: a 108-item questionnaire was prepared and was applied to a sample of 1964 subjects from different university groups (students, teachers, management, etc.). Its reliability and construct validity were analysed using exploratory factor analysis (EFA). Results: analysis of the instrument’s technical characteristics showed high reliability, both overall and at the dimensional level, with Cronbach’s α and McDonald’s ω being greater than .95. The EFA identified eight factors that contained the 105 items finally included, explaining 77.37% of the variance in Discussion and conclusions: these results all indicate that the instrument designed is reliable and valid, with a solid multidimensional structure that makes it possible to evaluate the impact of the accreditation system on various aspects of Spanish university degrees.
Please, cite this article as follows: García-Domingo, B., Rodríguez-Mantilla, J. M., & Martínez-Zarzuelo, A. (2023). Instrumento para valorar el impacto del sistema de acreditación en educación superior: validación mediante análisis factorial exploratorio [An instrument to evaluate the impact of the higher education accreditation system: Validation through exploratory factor analysis]. Revista Española de Pedagogía, 81 (286), 555-578. 10.22550/REP81-3-2023-06
Referencias | References
Cetzal, R. S. P., Delgado, M. L., & Reche, P. C. (2012). El poder en las organizaciones escolares… todos lo buscan, pocos lo consiguen [The power in school organizations… all seek it, few get it]. Revista Educación y Ciencia, 2 (40), 33-46.
Díaz, F., Cuevas, M., Fernández, M., Gijón, J., Lizarte, E. J., Ibáñez, P., El Homrani, M., Ávalos, I., & Rodríguez, R. J. (2019). Liderazgo y calidad en la educación superior [Leadership and quality in higher education]. EDMETIC. Revista de Educación Mediática y TIC, 8 (2), 52-72. 10.21071/edmetic.v8i2.12120
Egido, I., Fernández, F. J., & Fernández, M. J. (2016). Evaluation of the impact of quality management systems on school climate. International Journal of Educational Management, 30 (4), 474-492. 10.1108/IJEM-01-2015-0010
ENQA, ESU, EUA, EURASHE, EI, BUSINESS EUROPE, & EQAR. (2015). Standards and guidelines for quality assurance in the European Higher Education Area (ESG). European Association of Institutions in Higher Education. https://www.enqa.eu/wp-content/ uploads/2015/11/ESG_2015.pdf
EQAR. (n.d). European Quality Assurance Register for Higher Education. https://www.eqar.eu
Fernández-Cruz, F. J., Egido, I., & Carballo, R. (2016). Impact of quality management systems on teaching-learning processes. Quality Assurance in Education, 24 (3), 394-415. https://doi. org/10.1108/QAE-09-2013-0037
Fernández-Díaz, M. J. (2013). Evaluación de impacto para un cambio sostenible en las organizaciones educativas [Impact assessment for sustainable change in educational organisa- tions]. Revista Española de Pedagogía, 71 (254), 119-138. https://reunir.unir.net/ handle/123456789/3777
Fernández-Díaz, M. J., Rodríguez-Mantilla, J. M., & Fontana-Abad, M. (2016). Impact of implementation of quality management systems on internal communications and external rela- tions at schools. Total Quality Management & Business Excellence, 27 (1), 97-110. https://doi. org/10.1080/14783363.2014.954365
Ferreiro, V. V., Brito, J., & Garambullo, A. I. (2020). Modelo de gestión de calidad como estrategia de planeación en procesos de acreditaciones internacionales [Quality management model as a planning strategy in international accreditation processes]. RIDE. Revista Iberoamericana para la Investigación y el Desarrollo Educativo, 10 (20), 1-27. 10.23913/ride.v10i20.606
Galindo, M. A. (s. f.). Agencia de calidad: ANECA [Quality agency: ANECA] [Presentation]. https://www.farmaceuticos.com/wp-content/ uploads/2019/12/2014-Presentacion-Farmacia- ANECA-Nov.ppt
Guzmán-Puentes, S., & Guevara-Ramírez, R. (2022). Configuración de la acreditación de la calidad como campo de estudio. Una revisión sistemática de la investigación internacional (1998-2016) [Shaping quality accreditation as a field of study. A systematic review of international research (1998-2016)]. RIES. Revista Iberoamericana de Educación Superior, 13 (36), 160-180. 10.22201/iisue.20072872e.2022.36.1189
Hair, J. F., Black, W. C., Babin, B. J., & Anderson, R. (2014). Multivariate data analysis (7th ed.). Pearson Education.
Ibáñez-López. F. J., Hernández-Pina, F., & Monroy-Hernández, F. (2020). Análisis del conocimiento y la percepción del profesorado sobre los procesos de evaluación y acreditación de titulaciones universitarias en educación [Teachers’ opinions about the evaluation and the accreditation processes of education degrees]. Bordón. Revista de Pedagogía, 72 (4), 61-78. 10.13042/ Bordon.2020.79417.
Izquierdo, I., Olea, J., & Abad, F. J. (2014). Exploratory factor analysis in validation studies: Uses and recommendations. Psicothema, 26 (3), 395-400. 10.7334/psicothema2013.349
Organic Law 2/2023, of 22 March, on the University System. Spanish Official State Gazette, 70, 23 March 2023. https://www.boe.es/eli/es/ lo/2023/03/22/2/con
López-Aguado, M., & Gutiérrez-Provecho, L. (2019). Cómo realizar e interpretar un análisis factorial exploratorio utilizando SPSS [How to perform and interpret an exploratory factor analysis using SPSS statistics]. REIRE. Revista d’Innovació i Recerca en Educació, 12 (2), 1-14. http://doi.org/10.1344/reire2019.12.227057
Lorenzo, M. (2011). Organización de centros educativos. Modelos emergentes [Organisation of educational centres. Emerging models]. La Muralla.
Márquez, E., & Zeballos, Z. R. (2017). El impacto de la acreditación en la mejora de la calidad de los programas educativos que ofrece la Universidad Autónoma de Tamaulipas: un estudio de caso [The impact of accreditation on improving the quality of educational programs offered by the Autonomous University of Tamaulipas: A case study]. Revista Iberoamericana de Evaluación Educativa, 10 (2), 65-83. 10.15366/riee2017.10.2.004
Martínez, J. E., Tobón, S., & Romero, A. (2017). Problemáticas relacionadas con la acreditación de la calidad de la educación superior en América Latina [Problems related with the quality accreditation of higher education in Latin America]. Revista Innovación Educativa, 17 (73), 79-96. http://repositoriodigital.ipn.mx/handle/123456789/23409
Martínez, J. E., & Tobón, S. (2019). Cuestionario de percepción sobre el impacto de los procesos de acreditación de la calidad educativa en instituciones de educación superior [Perception questionnaire on the impact of educational quality accreditation processes in higher education institutions]. Conrado, 15 (70), 242-247. https://conrado.ucf.edu.cu/index.php/conrado/article/view/1132
Martínez, J. E., Ponce, S., Moreno, I., & Zamora, L. (2022). Evaluación y acreditación en una universidad pública de México: experiencias docentes [Evaluation and accreditation in public universities in Mexico: Teaching experiences]. Revista Venezolana de Gerencia, 27 (Especial 7), 443-459. 10.52080/rvgluz.27.7.29
Martínez-Iñiguez, J. E., Tobón, S., López Ramírez, E., & Manzanilla-Granados, H. M. (2020). Calidad educativa: un estudio documental desde una perspectiva socioformativa [Educational quality: Documentary research from a socio-formative perspective]. Revista Latinoamericana de Estudios Educativos, 16 (1), 233-258. https://revistasojs.ucaldas.edu.co/index.php/latinoamericana/article/view/2296
Martínez-Zarzuelo, A., Rodríguez-Mantilla, J. M., & Fernández-Díaz, M. J. (2022). Improvements in climate and satisfaction after implementing a quality management system in education. Evaluation and Program Planning, 94, 102119. https://doi.org/10.1016/j.evalprogplan.2022.102119
Matarranz, M. (2021). El Espacio Europeo de Educación Superior y su sello de calidad [The European Higher Education Area and its quality seal]. Revista Española de Educación Comparada, (37), 153-173. 10.5944/reec.37.2021.27728
Monarca, H., Thoilliez, B., Garrido, R., & Prieto, M. (2018). Percepciones del profesorado sobre el seguimiento y la evaluación de la calidad en la Universidad Autónoma de Madrid [Teaching staff perceptions of quality monitoring and evaluation at the Autonomous University of Madrid]. In H. Monarca, & M. Prieto (coords.), Calidad de la educación supe- rior en Iberoamérica [Quality of higher education in Ibero-America] (pp. 123-161). Dykinson.
Parra, K. L., Vargas, Y., & Soto, J. A. (2019). Acreditación en instituciones de educación superior. Experiencias. Logros y retos [Accreditation in higher education institutions. Experiences. Achievements and challenges]. Qartuppi.
Royal Decree 1112/2015, of 11 December, approving the Statute of the Autonomous Body of the National Agency for Quality Assessment and Accreditation. Spanish Official State Gazette, 302, 18 December 2015, pp. 119 063 to 119 081. https://www.boe.es/eli/es/rd/2015/12/11/1112
Royal Decree 822/2021, of 28 September, which establishes the organisation of university education and the procedure for quality assurance. Spanish Official State Gazette, 233, 29 September 2021, pp. 119 537 to 119 578. https://www. boe.es/buscar/act.php?id=BOE-A-2021-15781
Resolution of 7 March 2018, of the General Secretariat for Universities, issuing instructions on the procedure for the institutional accreditation of public and private university centres. Spanish Official State Gazette, 63, of 13 March 2018. https://www.boe.es/eli/es/res/2018/03/07/(1)/con
Resolution of 3 March 2022, of the General Secretariat for Universities, issuing instructions on the procedure for the institutional accreditation of public and private university centres, and publishing the Protocol for the certification of internal quality assurance systems of university centres and the Protocol for the evaluation procedure for the renewal of the institutional accreditation of university centres, approved by the General Conference on University Policy. Spanish Official State Gazette, 58, 9 March 2022. https://www.boe.es/eli/es/res/2022/03/03/(2)
Rodríguez, D. (2006). Dimensión institucional, cultural y micropolítica: claves para entender las organizaciones educativas [Institutional, cultural and micropolitical dimensions: Keys to understand educational organizations]. Revista Electrónica Interuniversitaria de Formación del Profesorado, 9 (1), 1-14.
Rodríguez-Mantilla, J. M., Fernández-Cruz, F. J., & Fernández-Díaz, M. J. (2021a). Evaluación para la innovación y mejora de centros educativos [Evaluation for innovation and improvement of educational centres]. Editorial Síntesis. Rodríguez-Mantilla, J. M., León, V., & Martínez-Zarzuelo, A. (2021b). ISO 9001 standard and their impact on school management and planning and support systems. Comparative study on perception between heads-teachers. The TQM Journal, 33 (6), 1610-1630. https://doi. org/10.1108/TQM-04-2020-0071
Sarasola, M., Delgado, P., & Lasida, J. (2015). El impacto de la implementación del sistema de calidad educativa PCI a través de la perspectiva del profesorado [The impact of PCI education quality system implementationthrough teachers’ perspective]. Páginas de Educación, 8 (2), 121-143. https://revistas.ucu.edu.uy/index.php/paginasdeeducacion/article/view/695
Smart-Qual. (2022). Quality indicator scoreboard (QIS). Quality indicator set & structured catalogue. Smart-Qual. https://smartqual.eu/wp-content/uploads/2022/04/QIS-indicator-set-and-catalog-IO1-SMARTQUAL-2.pdf
Thurler, M. G., & Maulini, O. (2010). La organización del trabajo escolar: una oportunidad para repensar la escuela [The organisation of school work: An opportunity to rethink school]. Graó.
Torres-Salas, M. I., García-Rojas, A., & Alvarado-Arguedas, A. (2018). La evaluación externa: un mecanismo para garantizar la calidad de la educación superior en Costa Rica [The external evaluation: A mechanism to ensure the quality of higher education in Costa Rica]. Revista Electrónica Educare, 22 (2), 1-16. https://dx.doi.org/10.15359/ree.22-2.16
Trujillo, J. M. (2007). Análisis interno de la organización: liderazgo y micropolítica en un centro educativo [Internal analysis of the organisation: Leadership and micro-politics in an educational centre]. Innovación educativa, 17, 37-52. http://hdl.handle.net/10347/4361
Ulker, N., & Bakioglu, A. (2018). An international research on the influence of accreditation on academic quality. Studies in Higher Education, 44 (9), 1507-1518. 10.1080/03075079.2018.1445986
Vázquez, J. A. (2015). Nuevos escenarios y tendencias universitarias [New trends and scenarios in tertiary education]. Revista de Investigación Educativa, 33 (1), 13-26. http://dx.doi.org/10.6018/rie.33.1.211501
Weaver, S. S. (2015). Measurement theory. In W. Donsbach (Ed.), The international encyclopedia of communication. Wiley Online Library. http://dx.doi.org/10.1002/9781405186407.wbiecm016.pub3
Author Biography
Begoña García-Domingo. Doctor of Psychology from the Universidad Autónoma de Madrid (UAM). She was an associate professor in the Department of Evolutionary Psychology and Education of the CES Don Bosco in Madrid higher education institute (affiliated to the Universidad Complutense de Madrid [UCM]) until 2019. Since then, she has carried out her teaching and research activity in the Department of Research and Psychology of Education (in the research methods area) of the Faculty of Education of the UCM. She is a member of the “Quality and evaluation of educational institutions” research group (UCM). She has participated in different research and innovation projects, especially taking an interest in measuring and evaluating the competences of teachers, emotional intelligence in formative settings, and educational quality.
https://orcid.org/0000-0002-5784-0460
Jesús-M. Rodríguez-Mantilla. Doctor of Education from the UCM. He teaches in the Department of Educational Research and Psychology (in the research methods area) of the Faculty of Education of the UCM. He is director of the “Quality and evaluation of educational institutions” research group (UCM). He has two recognised six-year periods of research activity. He is associate director of the Bordón. Revista de Pedagogía journal. He is currently coordinator of the degree in Pedagogy and coordinator of the School Pedagogy Practicum in the Faculty of Education of the UCM.
https://orcid.org/0000-0002-0885-3484
Angélica Martínez-Zarzuelo. Doctor of Education from the UCM. She teaches and researches in the Department of Didactics of Experimental Sciences, Social Sciences and Mathematics (in the didactics of mathematics area) of the Faculty of Education-Teacher Training Centre of the UCM. Her research interests include educational evaluation and educational quality. She is a member of the “Quality and evaluation of educational institutions” research group (UCM).
Citación recomendada | Recommended citation
García-Domingo, B., Rodríguez-Mantilla, J. M., y Martínez-Zarzuelo, A. (2023). Instrumento para valorar el impacto del sistema de acreditación en educación superior: validación mediante análisis factorial exploratorio [An instrument to evaluate the impact of the higher education accreditation system: Validation through exploratory factor analysis]. Revista Española de Pedagogía, 81 (286), 555-578. 10.22550/REP81-3-2023-06
Licencia Creative Commons | Creative Commons License
Esta obra está bajo una licencia internacional Creative Commons Atribución-NoComercial 4.0.
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License
Palabras clave | Keywords
Accreditation, evaluation of impact, exploratory factor analysis, measurement instrument
