Educational and social implications of Service-Learning using mixed methods: a meta-analysis

Implicaciones educativas y sociales del Aprendizaje-Servicio con métodos mixtos a través de un meta-análisis

Carolina Bonastre, Celia Camilli, Desirée García-Gil and Laura Cuervo


Service-Learning (SL) aims to improve students’ academic knowledge and perfor­mance, as well as their civic and social com­petencies. This study uses the meta-analy-sis technique to summarise up-to-date re­search findings on SL in scientific papers that used mixed methods research. The findings indicate that SL leads to signifi­cant improvements in students, especially in terms of knowledge acquisition (d=1.07), with a similar effect size for the few stud­ies using a control group (d=0.89), although this effect size is smaller in terms of atti­tude toward service (d=0.45). There are no significant effects when these variables are compared with dependent variables such as programme duration, gender, or sample size. Moreover, there is high heterogeneity in the evaluation methods and measurement instruments used in the articles. Although research on SL is increasing significantly, the methodological quality of most of the studies reviewed is quite low. It can be con­cluded that SL is beneficial to the students involved in the service, however, in order to properly appreciate the different scenar­ios of the findings, more rigorous research designs are required. We would recommend creating uniform research protocols and ex­tending SL using mixed methodology across every discipline, including the arts, and at every level of education.

Please, cite this article as follows: Bonastre, C., Camilli, C., García-Gil, D., & Cuervo, L. (2021). Implicaciones educativas y sociales del Aprendizaje-Servicio con métodos mixtos a través de un meta-análisis | Educational and social implications of Service-Learning using mixed methods: a meta-analysis. Revista Española de Pedagogía, 79 (279), 269-287.

Barrera-Algarín, E., Malagón-Bernal, J. L., & Sarasola-Sánchez, J. L. (2017). El meta-aná­lisis como elemento de innovación educativa de la práctica basada en la evidencia en tra­bajo social [The meta-analysis as an element of educational innovation of the evidence-based practice in social work]. Profesorado, Revista de Currículum y Formación del Pro­fesorado, 21 (3), 259-75.

Batlle, R. (2011). ¿De qué hablamos cuando ha­blamos de Aprendizaje Servicio? [What do we talk about when we talk about Service Learning?] Crítica, 972, 49-54.

Botella, J., & Sánchez-Meca, J. (2015). Me­ta-análisis en ciencias sociales y de la salud [Meta-analysis in health and social scien­ces]. Síntesis.

Bringle, R. G., Hatcher, J. A., & Williams, M. J. (2011). Quantitative approaches to research on international Service Learning: Design, measurement and theory. In R. G. Bringle, J. A. Hatcher, & S. G. Jones (Eds.), Inter­national Service Learning: Conceptual fra­meworks and research (pp. 275-290). Stylus Editorial.

Brizee, A., Pascual-Ferrá, P., & Caranante, G. (2019). High-impact civic engagement: out­comes of community-based research in tech­nical writing courses. Journal of Technical Writing and Communication, 50 (3), 224-251.

Camilli, C., García, M., & Galán, A. (2018). ¿Es posible evaluar los resultados de proyectos de Aprendizaje-Servicio? Las evidencias como clave del éxito [Is it possible to eva­luate the results of Service-Learning pro­jects? Evidence as a key to success]. In M. Ruiz-Corbella & J. García-Gutiérrez (Eds.), Aprendizaje-Servicio. Los retos de la evalua­ción (pp. 7-12). Narcea.

Celio, C., Durlak, J., & Dymnicki, A. (2011). A Meta-analysis of the impact of Service-Learning on students. Journal of Experiential Education, 34 (2), 164-181.

Chiva-Bartoll, O., Salvador-García, C., Ferran­do-Félix, S., & Cabedo-Mas, A. (2019). Service learning in musical education: Literature review and recommendations for practice. Revista Electrónica Complutense de Inves­tigación en Educación Musical, 16, 57-74.

Ciesielkiewicz, M., & Nocito, G. (2018). Moti­vation in service-learning: An improvement over traditional instructional methods. Teknokultura, 15 (1), 55-67. http://dx.doi. org/10.5209/TEKN.58258

Cohen, J. (1988). Statistical power analysis for the behavioral sciences. Erlbaum Editorial.

Cone, N. (2009). A bridge to developing effica­cious science teachers of all students: Com­munity-based service-learning supplemented with explicit discussions and activities about diversity. Journal of Science Teacher Education, 20 (4), 365-383.

Conway, J., Amel, E., & Gerwien, P. (2009). Teaching and learning in the social con­text: A meta-analysis of service learning’s effects on academic, personal, social, and citizenship outcomes. Teaching of Psy­chology, 36 (4), 233-245.

Creswell, J., & Plano, V. (2017). Designing and conducting mixed methods research. SAGE.

Cumberland, D. M., Sawning, S., Church-Nally, M., Shaw, M. A., Branch, E., & LaFaver, K. (2019). Experiential learning: Transforming Theory into practice through the parkin­son’s disease buddy program. Teaching and Learning in Medicine, 31 (4), 453‐465.

Deeley, S. J. (2015). Critical perspectives on Ser­vice-Learning in higher education. Palgrave MacMillan.

Dolgon, C., Mitchell, T. D., & Eatman, T. K. (Eds.) (2017). The Cambridge handbook of service learning and community engage­ment. Cambridge University Press.

Eyler, (2011). What international Service Learning research can learn from research on Service Learning. In R. G. Bringle, J. A. Hatcher, & S. G. Jones (Eds.), International Service Learning: Conceptual frameworks and research (pp. 225-371). Stylus.

Fitch, P., Steinke, P., & Hudson, T. (2013). Re­search and theoretical perspectives on cog­nitive outcomes of Service Learning. In P. Clayton, R. Bringle, & J. Hatcher (Eds.), Re­search on Service-Learning. Conceptual Fra­meworks and Assessment (pp. 57-85). Stylus.

Frels, R., & Onwuegbuzie, A. (2013). Administering quantitative instruments with qualita­tive interviews: A mixed research approach. Journal of Counseling and Development, 91, 184-194.

Gerholz, K.-H., Liszt, V., & Klingsieck, K. B. (2017). Effects of learning design patterns in service learning courses. Active Learning in Higher Education, 19 (1), 47-59.

Gómez-Lanier, L. (2016). The Effects of an Ex­periential Service-Learning Project on Resi­dential Interior Design Students’ Attitudes toward Design and Community. International Journal for the Scholarship of Teaching and Learning, 10 (2), 1-7.

Hamui-Sutton, A. (2013). Un acercamiento a los métodos mixtos de investigación en educa­ción médica [A mixed methods approach to medical education research]. Investigación en Educación Médica, 2 (8), 211-216.

Hayward, L. M., & Li, L. (2017). Sustaining and improving an international service-learning partnership: Evaluation of an evidence-based service delivery model. Physiotherapy Theory and Practice, 33 (6), 475-489.

Hébert, A., & Hauf, P. (2015). Student learning through service-learning: Effects on academic development, civic responsibility, inter­personal skills and practical skills. Active Learning in Higher Education, 16 (1), 37-49.

Hesse-Biber, S. N. (2016). Introduction: Naviga­ting a turbulent research landscape: working the boundaries, tensions, diversity, and contradictions of multimethod and mixed methods inquiry. In S. N. Hesse-Biber & B. Johnson (Eds.), The Oxford Handbook of Multimethod and Mixed Methods Research Inquiry (pp. XXXIII-LIII). Oxford Universi­ty Press.

Hirschinger-Blank, N. B., Simons, L., & Ken­yon, A. (2009). An Evaluation of a Service-Learning Model for Criminal Justice Un­dergraduate Students. Journal of Experien­tial Education, 32 (1), 61-78.

Howlett, K. M., Rao, A., Allred, J., & Beck, D. (2019). An English learner Service-lear­ning project: Preparing education majors using technology and the SAMR model. CALLEJ, 20 (2), 128-149.

Hutton, B., Catalá-López, F., & Moher, D. (2016). La extensión de la declaración PRIS­MA para revisiones sistemáticas que incor­poran metaanálisis en red: PRISMA-NMA [The extension of the PRISMA statement for systematic reviews incorporating ne­twork meta-analyses: PRISMA-NMA]. Me­dicina clínica, 147 (6), 262-266.

Hwang, H.-L., Wang, H.-H., Tuc, C.-T., Chena, S., & Changa, S.-H. (2014). Reciprocit of service learning among students and paired residents in long-term care facilities. Nurse Education Today, 34 (5), 854-859.

Johnson, R., Onwuegbuzie, A., & Turner, L. (2007). Toward a definition of mixed methods research. Journal of Mixed Methods Research, 1 (2), 112-133.

Knight-McKenna, M., Hollingsworth H. L., & Ammerman, N. (2019). Fostering partnerships with families: Academic service-learning in the Little Village. Journal of Early Childhood Teacher Education, 40 (2), 57-73.

Lewellyn, A., & Kiser, P. (2014). Conceptualizing criticality as a guiding principle for high quality academic Service Learning. Interna­tional Journal of Teaching and Learning in Higher Education, 26 (1), 147-156.

Li, Y., Yao, M., Song, F., Fu, J., & Chen, X. (2019). Building a just world: The effects of Service-Learning on social justice beliefs of Chinese college students. Educational Psy­chology, 39 (5), 591-616.

Lowe, L. A., & Medina, V. L. (2010). Service-Learning collaborations: A Formula for Recipro­city. Families in Society, 91 (2), 127-134.

Ocal, A., & Altınok, A. (2016). Developing social sensitivity with Service-Learning. Social Indicators Research, 129 (1), 61-75.

Peralta, L., O’Connor, D., Cotton, W., & Bennie, A. (2015). Preservice physical education teachers’ indigenous knowledge, cultural competency and pedagogy: A Service Learning intervention. Teaching Education, 27 (3), 248-266.

Ponce, O., & Pagán, N. (2015). Mixed methods re­search in education: Capturing the complex-ity of the profession. International Journal of Educational Excellence, 1 (1), 111-135.

Rodríguez-Izquierdo, R. M. (2020). Service Learning and academic commitment in higher education. Revista de Psicodidáctica, 25 (1), 45- 51.

Rubio, L., Puig, J. M., Martín, X., & Palos, J. (2015). Analizar, repensar y mejorar pro­yectos: una rúbrica para la autoevaluación de experiencias de Aprendizaje Servicio [Analyze, rethink and improve projects: Ru­bric for self-assessment and enhancement of Service Learning projects]. Profesorado, Revista del Currículum y formación del pro­fesorado, 19 (1), 111-126.

Schmid, C. H., White, I. R., & Stijnen, T. (2020). Introduction to systematic reviews and meta-analysis. In C. H. Schmid, I. R. White, & T. Stijnen (Eds.), Handbook of Meta-Analysis (pp. 1-18). Taylor and Francis.

Schneider-Cline, W. (2018). Guided vs. open-ended journals: A comparison of two reflective writing models for undergraduate Service-Learning experiences. Journal of Service Learning in Higher Education, 8, 1-21.

Schvaneveldt, P., & Spencer, T. (2016). Impact of international humanitarian Service-Learning on emerging adult social competence. A mixed-methods evaluation. Gateways: International Journal of Community Research and Engagemen, 9 (1), 113-131.

Seider, S. C. (2012). The influence of pa­rental support on the community Service Learning experiences of American college students. Citizenship and So­cial Justice, 7 (3), 217-288.

Seider, S. C., Gillmor, S., & Rabinowicz, S. (2012). Differential outcomes for Ameri­can college students engaged in communi­ty Service-Learning involving youth and adults. Journal of Experiential Education, 35 (3), 447-463.

Seider, S. C., Rabinowicz, S. A., & Gillmor, S. (2011). The impact of philosophy and theology Service-Learning experiences upon the public service motivation of participating college students. The Journal of Higher Education, 82 (5), 597-628.

Selmo, L. (2015). The Narrative Approach in Service-Learning Methodology: A Case Study. International Journal of Research on Service-Learning and Community Engage­ment, 3 (1).

Shostak, S., Corral, M., Ward, A. G., & Willet, A. (2019). Integrating community-based re­search into a senior capstone seminar: Les­sons learned from a mixed-methods study. Teaching Sociology, 47 (3), 191-203.

Simons, L., & Cleary, B. (2005). Student and community perceptions of the “Value added” for service learners. Journal of Experiential Education, 28 (2), 164-188.

Sterk Barrett, M. C. (2016). Dimensions of spirituality fostered through the PULSE program for Service Learning. Journal of Catholic Education, 20 (1), 114-135.

Stevahn, L., Anderson, J., & Hasart, T. (2016). Community-based research (CBR) in the education doctorate: Lessons learned and promising practices. International Journal of Doctoral Studies, 11, 441-465.

Tashakkori, A., & Teddlie, C. (2010). Handbook of mixed methods in social and behavioral research. Sage.

Wang, C., Zhang, X., & Yao, M. (2019). Enhancing Chinese college students’ transfer of learning through Service-Learning. Studies in Higher Education, 44 (8), 1316-1331.

Waterman, A., & Billig, S. (2003). Studying service-learning: innovations in education re­search methodology. Routledge.

Warren, J. (2012). Does Service-Learning in­crease student learning?: A Meta-Analysis. Michigan Journal of Community Service Learning, 18 (2), 56-61.

Wartenweiler, T. (2018). Serious play in edu­cation for social justice – An exploratory study. Journal of New Approaches in Edu­cational Research, 7 (1), 61-69.

White, A. (2001). A meta-analysis of Service Learning research in middle and high schools [Unpublished doctoral dissertation]. 67. University of Nebraska.

Yoiro, P., & Ye, F. (2012). A Meta-Analysis on the effects of Service-Learning on the so­cial, personal, and cognitive outcomes of learning. Academy of Management Learning & Education, 11 (1), 9-27.

Zimmerman, A. S. (Ed.) (2020). Preparing Stu­dents for Community-Engaged Scholarship in Higher Education. IGI Global

Carolina Bonastre. International PhD in Teaching Methods and Theory of Education from the Universidad Autóno­ma de Madrid with extensive teaching ex­perience as a teacher in music schools in the Region of Madrid and as a university lecturer. Her most noteworthy lines of re­search focus on music and emotion, inno­vation in educational methodologies, and digital contexts. She participates in vari­ous national and international competitive research projects.

Celia Camilli. Degree in Early Child­hood Education. PhD in Education. Assis­tant lecturer (National Agency for Quality Assessment and Accreditation [ANECA]). She currently teaches in the Department of Research and Psychology in Education (Universidad Complutense de Madrid). Member of the Adaptive Pedagogy re­search group. Her lines of research focus on qualitative research, systematic re­views and mixed methods research.