Quantitative methods for capturing processes and contexts in educational research

Abstract

Technological and methodological advances enable new substantive research questions to be posed, and new study designs to be implemented, in educational research. In this paper I review emerging methods relevant for capturing learning and teaching processes over time —the sequences of learning events— which take place in multiple contexts.

To do so, the concepts of nomothetic and ideographic research are traced through the use of Cattell’s (1952) cube, posing persons, variables and time as the three key dimensions for determining study-designs. For educational research, a fourth dimension —context— is important to consider given the nested structures (e.g. student-teacher dyads, peerrelations, student-groups, classrooms, teachers, and schools) learning and teaching occurs in. Several developments of quantitative methods enable researchers to a) establish quality of measurement (e.g. factor analysis, item response models), b) across sequences of time-points (e.g. autoregressive models), c) in complex multilevel structures (e.g. multilevel models, random effects models), also using estimators which are robust for small-n studies (e.g. Bayesian models). Educational researchers are encouraged to design studies fitting multilevel models for hierarchically and cross-classified data, and to think in terms of intraindividual learning processes.

Cite this article as: Malmberg, L. (2018). Métodos cuantitativos para el registro de procesos y contextos en la investigación educativa | Quantitative Methods for Capturing Processes and Contexts in Educational Research. Revista Española de Pedagogía, 76 (271), 449-462. doi: 10.22550/REP76-3-2018-03

Referencias | References

Allport, G. W. (1937). Personality. A psychological interpretation. Londres: Constable and company.

Bates, D., Mächler, M., Bolker, B. y Walker, S. (2014). Fitting Linear Mixed-Effects Models using lme4. Journal of Statistical Software, 67 (1). doi: 10.18637/jss.v067.i01

Cattell, R. B. (1952). Personality and motivation. Structure and measurement. Yonkers-on-Hudson, NY: World Book Company.

Deboeck, P. R. (2013). Dynamical Systems and Models of Continuous Time. En Todd D. Little (Ed.), The Oxford handbook of quantitative methods in psychology: Vol. 2: Statistical analysis. Oxford: Oxford University Press. doi: 10.1093/oxfordhb/9780199934898.013.0019

Duncan, T., Duncan, S. y Strycker, L. (2006). An Introduction to Latent Variable Growth Curve Modeling: Concepts, Issues, and Application. Mahwah, NJ: Lawrence Erlbaum.

Goldstein, H. (1997). Methods in School Effectiveness Research. School Effectiveness and School Improvement, 8 (4), 369-395. doi: 10.1080/0924345970080401

Hamaker, E. L. y Wichers, M. (2017). No Time Like the Present: Discovering the Hidden Dynamics in Intensive Longitudinal Data. Current Directions in Psychological Science, 26 (1), 10-15. doi: 10.1177/0963721416666518

Henderson, C. R. (1982). Best Linear Unbiased Estimation and Prediction under a Selection Model. Biometrics, 31 (2), 423-447.

Kaplan, D. y Depaoli, S. (2012). Bayesian Structural Equation Modeling. En R. H. Hoyle (Ed.), Handbook of structural equation modeling (pp. 650-673). Nueva York, NY: Guilford.

Kratochwill, T. R. y Levin, J. R. (2010). Enhancing the Scientific Credibility of Single-Case Intervention Research: Randomization to the Rescue. Psychological Methods, 15 (2), 124-144. doi: 10.1037/a0017736

Lamiell, J. T. (1998). ‘Nomothetic’ and ‘idiographic’: contrasting Windelband’s understanding with contemporary usage. Theory & Psychology, 8 (1), 23-38. doi: 10.1177/0959354398081002

Little, T. D. (2013). Longitudinal structural equation modeling. Nueva York, NY: Guilford.

Mainhard, T., Oudman, S., Hornstra, L., Bosker, R. J. y Goetz, T. (2018). Student emotions in class: The relative importance of teachers and their interpersonal relations with students. Learning and Instruction, 53, 109-119. doi: 10.1016/j.learninstruc.2017.07.011

Malmberg, L. E., Lim, W. H. T., Tolvanen, A. y Nurmi, J.-E. (2016). Within-students variability in learning experiences, and teachers’ perceptions of students’ task-focus. Frontline Learning Research, 4 (5), 62-82.

Marsh, H. W., Guo, J., Nagengast, B., Parker, P. D., Asparouhov, T., Muthén, B. y Dicke, T. (2017). What to do When Scalar Invariance Fails: The Extended Alignment Method for Multi-Group Factor Analysis Comparison of Latent Means Across Many Groups. Psychological Methods, Jan 12. doi: 10.1037/met0000113

Marsh, H. W. y Hau, K. T. (2007). Applications of latent-variable models in educational psychology: The need for methodological-substantive synergies. Contemporary Educational Psychology, 32 (1), 151-170. doi: 10.1016/j.cedpsych.2006.10.008

Matuschek, H., Kliegl, R., Vasishtha, S., Baayen, H. y Bates, D. (2017). Balancing Type I Error and Power in Linear Mixed Models. Journal of Memory and Language, 94, 305-315. doi: 10.1016/j.jml.2017.01.001

Mehta, P. (2013). N-level structural equation modeling. xxM user’s guide version 1. En Y. M. Petscher, Ch. Schatschneider y D. L. Compton (Eds.), Applied Quantitative Analysis in Education and the Social Sciences (p. 329). Londres: Routledge.

Moerbeek, M. y Teerenstra, S. (2016). Power analysis of trials with multilevel data. Boca Ratón: Chapmans CRC Press.

Morin, A. J., Marsh, H. W., Nagengast, B. y Scalas, L. F. (2014). Doubly latent multilevel analyses of classroom climate: An illustration. Journal of Experimental Education, 82 (2), 143-167. doi: 10.1080/00220973.2013.769412

Murayama, K., Goetz, T., Malmberg, L. E., Pekrun, R., Tanaka, A. y Martin, A. J. (2017). Within-person analysis in educational psychology: Importance and illustrations. British Journal of Educational Psychology, Series II, 12.

Muthén, B. (1994). Multilevel covariance structure analysis. Sociological Methods & Research, 22 (3), 376-398.

Muthén, B. y Asparouhov, T. (2012). Bayesian structural equation modeling: A more flexible representation of substantive theory. Psychological Methods, 17 (3), 313-335. doi: 10.1037/a0026802

Natesan, P. y Hedges, L. V. (2017). Bayesian unknown change-point models to investigate immediacy in single case designs. Psychological Methods, 4 (22), 743-759. doi: 10.1037/met0000134

Ortega, L., Malmberg, L. y Sammons, P. (2018). School effects on Chilean children’s achievement growth in language and mathematics: An accelerated growth curve model. School Effectiveness and School Improvement, 29 (2), 308-337. doi: 10.1080/09243453.2018.1443945

Ortega, L., Malmberg, L.-E. y Sammons, P. (2014). Teacher effects on Chilean children’s achievement growth: a cross-classified multiple membership accelerated growth curve model. Artículo presentado en la conferencia «Advances in Multilevel Modelling for Educational Research», en la Universidad de Maryland, EE.UU.

Perels, F., Gurtler, T. y Schmitz, B. (2005). Training of self-regulatory and problem-solving competence. Learning and Instruction, 15 (2), 123- 139. doi: 10.1016/j.learninstruc.2005.04.010

Praetorius, A. K., Koch, T., Scheunpflug, A., Zeinz, H. y Dresel, M. (2017). Identifying determinants of teachers’ judgment (in)accuracy regarding students’ school-related motivations using a Bayesian cross-classified multi-level model. Learning and Instruction, 52, 148-160. doi: 10.1016/j.learninstruc.2017.06.003

Rakoczy, K., Pinger, P., Hochweber, J., Klieme, E., Schütze, B. y Besser, M. (2018). Formative assessment in mathematics: Mediated by feedback’s perceived usefulness and students’ self-efficacy. Learning and Instruction, febrero de 2017. doi: 10.1016/j.learninstruc.2018.01.004

Ram, N. y Grimm, K. (2007). Using simple and complex growth models to articulate developmental change: Matching theory to method. International Journal of Behavioral Development, 31 (4), 303-316. doi: 10.1177/0165025407077751

Rasbash, J., Steele, F., Goldstein, H. y Browne, W. (2017). A User’s Guide to MLwiN Version 3.01. Bristol: Centre for Multilevel Modelling, Bristol.

Robinson, W. S. (1950). Ecological correlations and the behavior of individuals. American Sociological Review, 15 (3), 351-357.

Scherer, R., Nilsen, T. y Jansen, M. (2016). Evaluating individual students’ perceptions of instructional quality: An investigation of their factor structure, measurement invariance, and relations to educational outcomes. Frontiers in Psychology, 7, 1-16. doi: 10.3389/fpsyg.2016.00110

Schmiedek, F. (2016). Experimental manipulation «in the wild»: Proposing a within-person encouragement design. Seminario presentado en el ciclo «Network on Intrapersonal Research in Education (NIRE)» en la Universidad de Oxford. Recuperado de http://www.education.ox.ac.uk/network-on-intrapersonalresearch-in-education-nire/seminar-5-oxfordmayjune-2016/florian-schmiedek/ (Consultado el 12/06/2018).

Schmitz, B. (2006). Advantages of studying processes in educational research. Learning and Instruction, 16 (5), 433-449. doi: 10.1016/j.learninstruc.2006.09.004

Schmitz, B. (2015). The study of learning processes using time-series analyses. Seminario presentado en el ciclo «Network on Intrapersonal Research in Education (NIRE)» en la Universidad de Oxford. Recuperado de http://www.education.ox.ac.uk/network-on-intrapersonalresearch-in-education-nire/seminar-1/bernhard-schmitz/ (Consultado el 12/06/2018).

Singer, J. D. y Willett, J. B. (2003). Applied Longitudinal Data Analysis: Modeling Change and Event Occurrence. Oxford: Oxford University Press.

Skinner, E. A., Zimmer-Gembeck, M. J. y Connell, J. P. (1998). Individual differences and the development of perceived control. Monographs of the Society for Research in Child Development, 63 (2/3). doi: 10.2307/1166220

Van de Schoot, R., Kaplan, D., Denissen, J., Asendorpf, J. B., Neyer, F. J. y van Aken, M. A. G. (2014). A gentle introduction to Bayesian analysis: application to developmental research. Child Development, 85 (3), 842-860. doi: 10.1111/cdev.12169

Voelkle, M. C., Oud, J. H. L., von Oertzen, T. y Lindenberger, U. (2012, jul.). Maximum Likelihood Dynamic Factor Modeling for Arbitrary N and T Using SEM. Structural Equation Modeling, 19 (3), 329-350. doi: 10.1080/10705511.2012.687656

Von Davier, M., González, E. y Mislevy, R. J. (2009). What are plausible values and why are they useful? En M. von Davier y D. Hastedt (Eds.), IERI monograph series: Issues and methodologies in large scale assessments: Volume 2 (pp. 9-36). Hamburgo, Alemania: IERI Institute.

Walls, T. A., Barta, W. D., Stawski, R. S., Collyer, C. S. y Hofer, S. M. (2013). Time-scale dependent longitudinal designs. En B. Laursen, T. D. Little y N. A. Card (Eds.), Handbook of developmental research methods (pp. 45-64). Nueva York: Guilford Press.

Walls, T. A. y Schafer, J. S. (2006). Models for intensive longitudinal data. Nueva York: Oxford University Press.

Zitzmann, S., Lüdtke, O., Robitzsch, A. y Marsh, H. W. (2016). A Bayesian Approach for Estimating Multilevel Latent Contextual Models. Structural Equation Modeling, 23 (5), 661- 679. doi: 10.1080/10705511.2016.1207179

Author Biography

Lars-Erik Malmberg es Profesor Agregado de Métodos Cuantitativos en Educación en el Departamento de Educación de la University of Oxford. Tiene más de 70 publicaciones (artículos revisados por pares, capítulos de libros e informes) y es el redactor jefe de la revista Journal of Learning and Instruction. Su labor investigadora se centra actualmente en las perspectivas intraindividuales en procesos de aprendizaje y en el modelado de datos intraindividuales utilizando modelos de ecuaciones estructurales multinivel.

https://orcid.org/0000-0002-5309-7403

Citación recomendada | Recommended citation

Malmeberg, L. (2018) . Quantitative methods for capturing processes and contexts in educational research. Revista Española de Pedagogía, 76(271). https://www.revistadepedagogia.org/rep/vol76/iss271/16