Design and validation of a rubric to evaluate educational actions and projects on youth empowerment

Diseño y validación de una rúbrica para evaluar acciones y proyectos educativos de empoderamiento juvenil

Laura Corbella Molina, Carme Trull Oliva, M.ª Pilar Rodrigo-Moriche and Xavier Úcar Martínez

DOI: https://doi.org/10.22550/REP79-3-2021-05

In recent years, there has been a signifi­cant increase in international debates, re­search and socio-educational programmes focusing on youth empowerment. One of the main issues with this concept is related to how it is measured and evaluated. Evaluating so­cio-educational actions and projects is crucial in order to design, implement and improve educational practices that help young people to empower themselves. This article presents the process of building and validating a ru­bric, within the framework of the HEBE Pro­ject, for the evaluation of youth empowerment actions and projects. The methodological pro­cess consists of three phases: (1) Design of the rubric; (2) Expert validation by 17 practition­ers from different fields, 3 experts in evalua­tion and 5 young people; (3) And a compari­son by means of a pilot test with 20 projects or socio-educational services aimed at youth empowerment, in which 63 professionals par­ticipate. The results show evidence of validity and reliability of the rubric in order to evalu­ ate the quality of socio-educational practices, and also to design and implement actions that focus on youth empowerment. It is noted for being a validated and useful instrument for making educational assessments related to youth empowerment, and for its usefulness in generating processes of reflection that become the basis for rethinking and improving peda­gogical practices.

 


 

Please, cite this article as follows: Corbella Molina, L., Trull Oliva, C., Rodrigo-Moriche, M. P. y Úcar Martínez, X. (2021). Diseño y validación de una rúbrica para evaluar acciones y proyectos educativos de empoderamiento juvenil | Design and validation of a rubric to evaluate educational actions and projects on youth empowerment. Revista Española de Pedagogía, 79 (280), 537-555. https://doi.org/10.22550/REP79-3-2021-05

Andueza, A. (2019). Evaluación de la escritura académica: construcción y validación de un instrumento integrado basado en tareas para evaluar habilidades específicas de escritura [Evaluation of academic writing: Construc­tion and validation of an integrated task-based instrument to evaluate specific writing skills]. Revista Electrónica de Investigación y Eva­luación Educativa, 25 (2), art. 5. https://doi. org/10.7203/relieve.25.2.11163

Bacqué, M.-H., & Biewener, C. (2016). El empodera­miento: una acción progresiva que ha revolucio­nado la política y la sociedad [Empowerment: Progressive action that has revolutionised poli­tics and society]. Editorial Gedisa, S. A.

Beatty, P. C., & Willis, G. (2007). Research synthe­sis: The practice of cognitive interviewing. Pub-lic Opinion Quarterly, 71 (2), 287-311. https:// doi.org/10.1093/poq/nfm006

Bharuthram, S., & Patel, M. (2017). Co-construct-ing a rubric checklist with first year univer­sity students: A self-assessment tool. Apples – Journal of Applied Language Studies, 11 (4), 35-55. https://doi.org/10.17011/apples/ urn.201708073430

Cevallos, B., & Paladines, B. (2016). La autonomía personal: un indicador del empoderamiento Juvenil [Personal autonomy: An indicator of youth empowerment]. In P. Soler, J. Bellera, & A. Planas (Eds.), Pedagogía Social, Juventud y Transformaciones Sociales (pp. 370-377). Uni­versity of Girona.

Cubillos-Veja, C., & Ferran-Aranaz, M. (2018). Di­seño y validación de una rúbrica para valorar la resolución de casos prácticos relativos a de­rechos humanos [Design and validation of a rubric for assessing the resolution of human rights case studies]. Revista Iberoamericana de Evaluación Educativa, 11 (2), 35-52. https:// doi.org/10.15366/riee2018.11.2.002

Dimova, Y., & Loughran, J. (2009). Developing a big picture understanding of reflection in pedagogi­cal practice. Reflective Practice, 10 (2), 205-217. https://doi.org/10.1080/14623940902786214

Doğan, C. D., & Uluman, M. (2017). A comparison of rubrics and graded category rating scales with various methods regarding raters’ relia­bility. Educational Sciences: Theory & Prac­tice, 17 (2), 631-651. https://doi.org/10.12738/ estp.2017.2.0321

Fernández-Cruz, F. J., Fernández-Díaz, M. J., & Rodríguez-Mantilla, J. M. (2018). Diseño y validación de un instrumento de medida del perfil de formación docente en tecnologías de la información y comunicación | Design and validation of an instrument to measure tea­cher training profiles in information and com­munication technologies. revista española de pedagogía, 76 (270), 247-270. https://doi. org/10.22550/REP76-2-2018-03

Froufe, S. (1997). Los ámbitos de intervención en la educación social [The areas of intervention in Social Education]. Aula, 9, 179-200. https:// revistas.usal.es/index.php/0214-3402/article/ view/3491

García-Sanz, M. P. (2014). La evaluación de compe­tencias en educación superior mediante rúbri­cas: un caso práctico [Assessing competencies in higher education by rubrics: a case study]. Revista Electrónica Interuniversitaria de For­mación del Profesorado, 17 (1), 87-106. https:// doi.org/10.6018/reifop.17.1.198861

Gatica-Lara, F., & Uribarren-Berrueta, T. del N. J. (2013). ¿Cómo elaborar una rúbrica? [How to make a rubric?] Investigación en Educación Médica, 2 (1), 61-65. https://doi.org/10.1016/ S2007-5057(13)72684-X

Gil, J. (2007). La evaluación de competencias labo­rales [Assessment of work-related competen­cies]. Educación XX1, 10, 83-106. https://doi. org/10.5944/educxx1.1.10.298

Hilton, C. E. (2015). The importance of pretest-ing questionnaires: A field research example of cognitive pretesting the Exercise referral Quality of Life Scale (ER-QLS). International Journal of Social Research Methodology, 20 (1), 21-34. https://doi.org/10.1080/13645579. 2015.1091640

Janer, À., & Úcar, X. (2020). Social pedagogy in the world today: An analysis of the academic, trai­ning and professional perspectives. The British Journal of Social Work, 50 (3), 701-721. https:// doi.org/10.1093/bjsw/bcz025

León, V., & Fernández, M. J. (2019). Diseño y vali­dación de una escala para evaluar el funciona­miento de las tutorías en educación secunda­ria [Design and validation of a scale to assess the performance of tutorials in Secondary Education]. Revista de Investigación Educa­tiva, 37 (2), 525-541. https://doi.org/10.6018/ rie.37.2.345251

 

Llena-Berñe, A., Agud-Morell, I., Páez de la Torre, S., & Vila-Mumbrú, C. (2017). Explorando momentos clave para el empoderamiento de jóvenes a partir de sus relatos [Exploring key moments for youth empowerment based on young people’s stories]. Pedagogía Social. Re­vista Interuniversitaria, 30, 81-94. https://doi. org/10.7179/PSRI_2017.30.06

Luttrell, C., Quiroz, S., Scrutton, C., & Bird, K. (2009). Understanding and operationalising empowerment. https://www.1decada4.es/plu­ginfile.php/189/course/section/47/understan­ding.pdf

Miranda, M., Burguera, J. L., Arias, J. M., & Peña, E. (2019). Inclusión, diversidad y equidad: diseño y validación de un cuestionario de opi­nión dirigido al profesorado de orientación educativa (IDEC-O) [Inclusion, diversity and equality: Design and validation of an opinion questionnaire for educational guidance teach-ers (IDEC-O)]. Revista de Investigación Edu­cativa, 37 (2), 505-524. https://doi.org/10.6018/ rie.37.2.333891

Morton, M. H., & Montgomery, P. (2013). Youth empowerment programs for improving adoles­cents’ self-efficacy and self-esteem. Research on Social Work Practice, 23 (1), 22-33. https:// doi.org/10.1177/1049731512459967

Murphy, D. L., & Ermeling, B. A. (2016). Feedback on reflection: Comparing rating-scale and forced-choice formats for measuring and facilitating teacher team reflective practice. Reflective Practice, 17(3), 317-333. https://doi.org/10.108 0/14623943.2016.1164681

Neuert, C. E., & Lenzner, T. (2015). Incorporating eye tracking into cognitive interviewing to pretest survey questions. International Jour­nal of Social Research Methodology, 19 (5), 501-519. https://doi.org/10.1080/13645579.201 5.1049448

Panadero, E., & Jonsson, A. (2013). The use of scor-ing rubrics for formative assessment purposes revisited: A review. Educational Research Re­view, 9, 129-144. https://doi.org/10.1016/j.edu­rev.2013.01.002

Petrie, P. (2005). Extending ‘pedagogy’. Journal of Education for Teaching, 31 (4), 293-296. ht­tps://doi.org/10.1080/02607470500280126

Planas, A., Trilla, J., Garriga, P., Alonso, A., & Mon­seny, M. (2016a). ¿Qué dimensiones conforman el empoderamiento juvenil? Una propuesta de indicadores [What dimensions make up youth empowerment? A proposal for indicators]. In P. Soler, J. Ballera, and A. Planas (Eds.), Pe­dagogía Social, Juventud y Transformaciones Sociales (pp. 311-318). University of Girona. http://hdl.handle.net/10256/13324

Planas, A., Úcar, X., Páez de la Torre, S., Trilla, J., & Garriga, P. (2016b). Aproximación a los indi­cadores de empoderamiento juvenil [Approach to youth empowerment indicators]. In C. Gon­zález & M. Castro Morera (Coords.), Libro de Actas del XVI Congreso Nacional y VII Congre­so Iberoamericano de Pedagogía: Democracia y Educación en el Siglo XXI. La obra de John Dewey 100 años después (p. 386). Universidad Complutense de Madrid. https://biblioteca. ucm.es/data/cont/docs/405-2016-10-05-Libro­DeActas_SEP2016.pdf

Proyecto HEBE (2019) [HEBE project]. Rúbri­ca para la evaluación de acciones y proyectos socioeducativos de empoderamiento juvenil [Rubric for the evaluation of socio-educational youth empowerment actions and projects]. Universitat de Girona. http://hdl.handle. net/10256/17643

Reddy, Y. M., & Andrade, H. (2010). A review of rubric use in higher education. Assessment and Eval-uation in Higher Education, 35 (4), 435-448. https://doi.org/10.1080/02602930902862859

Richez, J. C., Labadie, F., & De Linares, C. (2012). «Youth empowerment» dans l’espace euro­méditerranéen. Rapport d’étude [Youth em­powerment in the Euro-Mediterranean region. A study report]. https://injep.fr/wp-content/ uploads/2019/03/rapport-2012-01_Em­power-Youth.pdf

Soler, P. (2017). Presentación monográfico empo­deramiento juvenil y pedagogía social [Mono-graphic presentation youth empowerment and social pedagogy ]. Pedagogía Social. Re­vista Interuniversitaria, 30, 13-16. https://doi. org/10.7179/PSRI_2017.30.01

Soler, P. (2020). El ocio como recurso privilegiado para el empoderamiento juvenil [Leisure as a privileged resource for youth empowerment]. In I. Lazcano & Á. De-Juanas (Eds.), Ocio y juventud. Sentido, potencial y participación co­munitaria (pp. 161-180). Universidad Nacional de Educación a Distancia (UNED).

Soler, P., Planas, A., Ciraso-Cali, A., & Ribot-Horas, A. (2014). Empoderamiento en la comunidad. El diseño de un sistema abierto de indicadores a partir de procesos de evaluación participati­va [Community empowerment. Designing an open indicator system based on participatory evaluation processes]. Pedagogía Social: Re­vista Interuniversitaria, 24, 49-77. https://doi. org/10.7179/PSRI_2014.24.03

Soler, P., Trilla, J., Jiménez-Morales, M., & Úcar, X. (2017). La construcción de un modelo pedagó­gico del empoderamiento juvenil: espacios, mo­mentos y procesos [The construction of a peda­gogical model of youth empowerment: spaces, moments and processes]. Pedagogía Social. Revista Interuniversitaria, 30, 19-32. https:// doi.org/10.7179/PSRI_2017.30.02

Soler, P., Trull, C., Rodrigo-Moriche, M. P., & Cor­bella, L. (2019). El reto educativo del empode­ramiento juvenil [The educational challenge of youth empowerment]. In I. Alonso & K. Artetxe (Eds.), Educación en el tiempo libre: la inclusión en el centro (pp. 129-142). Ediciones OCTAEDRO, S.L.

Tinsley, S., & Sheats, S. (2020). The role of teacher educators and university supervisors to help student teachers reflect: From monological re­flection toward dialogical conversation. Reflec­tive Practice, 21 (2), 171-182. https://doi.org/10. 1080/14623943.2020.1716710

Tourón, J., Martín, D., Navarro, E., Pradas, S., & Íñigo, V. (2018). Validación de constructo de un instrumento para medir la competencia digital docente de los profesores (CDD) | Construct validation of a questionnaire to measure teachers’ digital competence (TDC). revista española de pedagogía, 76 (269), 25-54. doi: https://doi.org/10.22550/REP76-1-2018-02

Úcar, X., Jiménez-Morales, M., Soler, P., & Trilla, J. (2016a). Exploring the conceptualization and research of empowerment in the field of youth. International Journal of Adolescence and You­th, 22 (4), 405-418. https://doi.org/10.1080/026 73843.2016.1209120

Úcar, X. (Coord.), Planas, A., Novella, A., Rodrigo, P., Vila, C., Núñez, H., Paladines, B., Paéz de la Torre, S., Turón, N., Paéz, J., & Arenillas, M. (2016b). Informe del proceso de evaluación participativa del empoderamiento juvenil con 4 grupos de jóvenes [Report on the participatory evaluation process of youth empowerment with 4 groups of young people]. Dipòsit Digital de Documents. Autonomous University of Barce­lona. https://ddd.uab.cat/record/167223

Wagaman, M. A. (2011). Social empathy as a fra­mework for adolescent empowerment. Journal of Social Service Research, 37 (3), 278-293. ht­tps://doi.org/10.1080/01488376.2011.564045

Zimmerman, M. A., Eisman, A. B., Reischl, T. M., Morrel-Samuels, S., Stoddard, S., Miller, A. L., Hutchison, P., Franzen, S., & Rupp, L. (2018). Youth empowerment solutions: Evaluation of an after-school program to engage middle school students in community change. Health Education and Behavior, 45 (1), 20-31. https:// doi.org/10.1177/1090198117710491

Laura Corbella Molina is a Pre-doc­toral Researcher forming part of the PhD Programme in Education in the De­partment of Theories of Education and Social Pedagogy at the Universitat Au­tònoma de Barcelona (UAB) and collabo­rating lecturer at Universitat Oberta de Catalunya. She is a social educator and holds a Master’s Degree in Educational Research from the UAB. Her research activity focuses on social pedagogy and, specifically, the ethical dimension of so­cio-educational relations.

 https://orcid.org/0000-0002-2151-089X

Carme Trull Oliva is Associate Pro­fessor of the Area of Theory and History of Education and Specialized Research Tech­nician of the Area of Studies of Education and Psychology at the University of Girona (UdG). She is a social educator and holds an  Inter-university Master's Degree in Youth and Society, and the Doctorate in Education. Her research activity focuses on social peda­gogy and, specifically, the socio-educational interventions of the juvenile justice system.

 https://orcid.org/0000-0003-2777-0315

Mª Pilar Rodrigo-Moriche is As­sociate Professor in the Department of Pedagogy at the Universidad Autóno­ma de Madrid (UAM) and Collaborating Lecturer in the Faculty of Education at the Universidad Nacional de Educación a Distancia (UNED). She has worked in early childhood education centres, youth areas, organisations for the social and la­bour inclusion of socially disadvantaged people and organisations for people with intellectual disabilities. She is a member of the EMIPE Research Group and the TABA International Research Group.

 https://orcid.org/0000-0002-6513-0899

Xavier Úcar Martínez is Professor of Social Pedagogy at the Universidad Autónoma de Barcelona. He specialises in Social Education and in the Design and Evaluation of Community Development Programmes and Socio-cultural Anima­tion Programmes. President of the “So­ciedad Iberoamericana de Pedagogía So­cial” [Ibero-American Society of Social Pedagogy] (SIPS) (2012-2018). President of the “Coordinadora per l’Animació So­ciocultural de Cataluña” [Coordinating Committee for Socio-Cultural Animation of Catalonia] (CASC_CAT) (2009-2014). Member of the GIPE research group on Education Policy and of the GPSEJ group “Pedagogía Social y Empoderamiento Ju­venil” [Social Pedagogy and Youth Em­powerment].

 https://orcid.org/0000-0003-3678-8277