Análisis de redesuna alternativa a los enfoques clásicos de evaluación de los sistemas educativos

  1. Marcos Álvarez-Díaz 1
  2. César Gallego-Acedo 1
  3. Rubén Fernández-Alonso 1
  4. José Muñiz 2
  5. Eduardo Fonseca-Pedrero 3
  1. 1 Universidade Autónoma de Lisboa
    info

    Universidade Autónoma de Lisboa

    Lisboa, Portugal

    ROR https://ror.org/01ryrwk91

  2. 2 Universidade do Minho, Braga, Portugal
  3. 3 Universidade de Lisboa
    info

    Universidade de Lisboa

    Lisboa, Portugal

    ROR https://ror.org/01c27hj86

Show affiliations +
Journal:
Psicología educativa

ISSN: 1135-755X

Year of publication: 2022

Volume: 28

Issue: 2

Pages: 165-173

Type: Article

DOI: 10.5093/PSED2021A16 DIALNET GOOGLE SCHOLAR lock_openOpen access editor

More publications in: Psicología educativa

Institutional repository: lock_openOpen access Editor

Abstract

Widespread research on educational systems has made it possible to identify the main factors that define students’ educational success. Nevertheless, these studies have not been very efficient when it comes to transferring their conclusions to the applied contexts where those who teach or make decisions about educational systems move. The main reason for this failure lies on the unfamiliarity of education professionals with the sophisticated statistical and psychometric methodology used by most researchers. The goal of this paper is to analyze educational systems using the Network Analysis methodology, which is very affordable, clear, and intuitive for teachers and educational leaders without a great methodological sophistication. A sample of 7,882 eighth-graders was used. Students’ mathematical skills were evaluated and data were obtained from their educational context. Network Analysis was used calculating indicators of centrality, as well as precision and stability of the network. The results indicate that academic self-concept and academic expectations have an important effect on performance in mathematics. The findings match with those previously obtained with other approaches. The Network Analysis methodology offers an ideal combination of analytical rigor and interpretative simplicity, that provides a great potential to be used in applied educational contexts for evidence-based decision making.

Bibliographic References

  • Adams, R. J. y Wu, M. L. (2002). Technical report for the OECD programme for international student assessment. OECD Publications. http://www. oecd.org/dataoecd/53/19/33688233.pdf
  • Alonso-Tapia, J. y Nieto, C. (2019). Classroom emotional climate: Nature, measurement, effects and implications for education. Revista de Psicodidáctica, 24(2), 79-87. https://doi.org/10.1016/j.psicod.2018.08.002
  • Beaton, A. E., Mullis, I. V. S., Martin, M.O., Gonzalez, E. J., Kelly, D. L. y Smith, T. A. (1996). Mathematics achievement in the middle school years: IEA’s third international Mathematics and Science Report. TIMSS & PIRLS International Study Center, Boston College. https://eric.ed.gov /?id=ED406419
  • Boonk, L., Gijselaers, H. J. M., Ritzen, H. y Brand-Gruwel, S. (2018): A review of the relationship between parental involvement indicators and academic achievement. Educational Research Review, 24, 10-30. https:// doi.org/10.1016/j.edurev.2018.02.001
  • Borgatti, S. P., Mehra, A., Brass, D. J. y Labianca, G. (2009). Network analysis in the social sciences. Science, 323, 892-896. https://doi.org/10.1126/ science.1165821
  • Borsboom, D. (2017). A network theory of mental disorders. World Psychiatry, 16(1), 5-13. https://doi.org/10.1002/wps.20375
  • Borsboom, D. y Cramer, A. O. (2013). Network analysis: an integrative approach to the structure of psychopathology. Annual Review of Clinical Psychology, 9, 91-121. https://doi.org/10.1146/annurevclinpsy-050212-185608
  • Castro, M., Expósito-Casas, E., López-Martín, E., Lizasoain, L., NavarroAsencio, E. y Gaviria, J. L. (2015). Parental involvement on student achievement: A meta-analysis. Educational Research Review, 14, 33- 46. https://doi.org/10.1016/j.edurev.2015.01.002
  • Consejería de Educación, Cultura y Deporte del Principado de Asturias. (2012). Evaluación de diagnóstico Asturias 2011. Oviedo.
  • Costantini, G., Epskamp, S., Borsboom, D., Perugini, M., Mõttus, R., Waldorp, L. J., y Cramer, A. O. J. (2015). State of the art personality research: A tutorial on network analysis of personality data in R. Journal of Research in Personality, 54, 13-29. https://doi.org/10.1016/j.jrp.2014.07.003
  • Cramer, A. O. J., van Borkulo, C. D., Giltay, E. J., van der Maas, H. L. J., Kendler, K. S., Scheffer, M. y Borsboom, D. (2016). Major depression as a complex dynamic system. PLOS ONE, 11(12), e0167490. https://doi. org/10.1371/journal.pone.0167490
  • Epskamp, S., Borsboom, D. y Fried, E. I. (2018). Estimating psychological networks and their accuracy: A tutorial paper. Behavior Research Methods, 50(1), 195-212. https://doi.org/10.3758/s13428-017-0862-1
  • Epskamp, S., Kruis, J. y Marsman, M. (2017). Estimating psychopathological networks: Be careful what you wish for. PLoS ONE, 12(6). https://doi. org/10.1371/journal.pone.0179891
  • Epskamp, S., Maris, G., Waldorp, L. J. y Borsboom, D. (2018). Network psychometrics. En P. Irwing, D. Hughes y T. Booth (Eds.), Handbook of psychometrics. Wiley. https://doi.org/10.1002/9781118489772.ch30
  • Fan, X. y Chen, M. (2001). Parental involvement and students’ academic achievement: a meta-analysis. Educational Psychology Review, 13(1), 1-22. https://doi.org/10.1023/A:1009048817385
  • Fernández-Alonso, R. (2004). Evaluación del rendimiento matemático (tesis doctoral). Universidad de Oviedo, Oviedo. http://hdl.handle. net/10651/16615
  • Fernández-Alonso, R., Álvarez-Díaz, M., Woitschach, P., Suárez-Álvarez, J. y Cuesta, M. (2017). Parental involvement and academic performance: Less control and more communication. Psicothema, 29(4), 453-461. https://doi.org/10.7334/psicothema2017.181
  • Fernández-Alonso, R y Muñiz, J. (2011). Diseño de cuadernillos para la evaluación de las competencias básicas. Aula Abierta, 39(2). 3-34.
  • Fernández-Alonso, R. y Muñiz, J. (2019). Calidad de los sistemas educativos: modelos de evaluación. Propósitos y Representaciones, 7(SPE), e347. https://doi.org/10.20511/pyr2019.v7nSPE.347
  • Fernández-Alonso, R. y Muñiz, J. (en prensa). Homework: Facts facts and fictions. En T. Nilsen, A. Stancel-Pi tak y J. Gustafsson (Eds). International handbook of comparative large-scale studies in education: Perspectives, methods and findings. Springer International Publishing.
  • Fernández-Alonso, R., Suárez-Álvarez, J. y Muñiz, J. (2012). Imputación de datos perdidos en las evaluaciones diagnósticas educativas [Imputation methods for missing data in educational diagnostic evaluation]. Psicothema, 24(1), 167-175. https://psycnet.apa.org/record/2012-02525-026
  • Fernández-Alonso, R., Suárez-Álvarez, J. y Muñiz, J. (2015). Adolescents’ homework performance in mathematics and science: Personal factors and teaching practices. Journal of Educational Psychology, 107(4), 1075-1085. https://doi.org/10.1037/edu0000032
  • Fonseca-Pedrero, E. (2017). Análisis de redes: ¿una nueva forma de comprender la psicopatología? Revista de Psiquiatría y Salud Mental, 10(4), 206-215. https://doi.org/10.1016/j.rpsm.2017.06.004
  • Fonseca-Pedrero, E. (2018). Análisis de redes en psicología. Papeles del Psicólogo, 39(1), 1-12. https://doi.org/10.23923/pap.psicol2018.2852
  • Fried, E. y Cramer, A. (2017). Moving forward: Challenges and directions for psychopathological network theory and methodology. Perspectives on Psychological Science, 12(6), 999-1020. https://doi. org/10.1177/1745691617705892
  • Froemel, J. E. (2009). La efectividad y la eficacia de las mediciones estandarizadas y de las evaluaciones en educación. Revista Iberoamericana de Evaluación Educativa, 2(1), 10-28. http://hdl.handle. net/10486/661543
  • García-Crespo, F. J., Fernández-Alonso, R. y Muñiz, J. (2019). Resilient and low performer students: Personal and family determinants in European countries. Psicothema, 31(4), 363-375. https://doi.org/10.7334/ psicothema2019.245
  • Golino, H. F. y Epskamp, S. (2017). Exploratory graph analysis: A new approach for estimating the number of dimensions in psychological research. PLoS ONE, 12(6). https://doi.org/10.1371/journal.pone.0174035
  • Govorova, E., Benítez, I. y Muñiz, J. (2020). Predicting student well-being: Network analysis based on PISA 2018. International Journal of Environmental Research and Public Health, 17(11), 4014. https://doi. org/10.3390/ijerph17114014
  • Guloksuz, S., Pries, L.-K. y van Os, J. (2017). Application of network methods for understanding mental disorders: pitfalls and promise. Psychological Medicine, 1-10. https://doi.org/10.1017/S0033291717001350
  • Han, F. (2019). Longitudinal relations between school self-concept and academic achievement. Revista de Psicodidáctica, 24(2), 95-102. https://doi.org/10.1016/j.psicod.2019.03.001
  • Haslbeck, J. (2019). Mgm. Estimating time-varying k-order mixed graphical models. https://Cran.r-Project.Org/Web/Packages/Mgm/Mgm.Pdf
  • Haslbeck, J. M. B. y Fried, E. I. (2017). How predictable are symptoms in psychopathological networks? A reanalysis of 17 published datasets. Psychological Medicine, 47(16), 2767-2776. https://doi.org/10.1017/ S0033291717001258
  • Hattie, J. (2009). Visible learning. A synthesis of over 800 meta-analyses relating to achievement. Routledge.
  • Hill, N. E. y Tyson, D. F. (2009). Parental involvement in middle school: A meta-analytic assessment of the strategies that promote achievement. Developmental Psychology, 45(3), 740-763. https://doi.org/10.1037/ a0015362
  • Huang, C. (2011). Self-concept and academic achievement: A meta-analysis of longitudinal relations. Journal of School Psychology, 49(5), 505-528. https://doi.org/10.1016/j.jsp.2011.07.001
  • IBM Corp. (2013). IBM SPSS Statistics for Windows, Version 22.0. IBM Corp. Ikeda, M. y García, E. (2014). Grade repetition. A comparative study of academic and non-academic consequences. OCDE Journal: Economic Studies, 2014(1). https://doi.org/10.1787/eco_studies-2013- 5k3w65mx3hnx
  • Jeynes, W. H. (2018): A practical model for school leaders to encourage parental involvement and parental engagement. School Leadership and Management, 38(2), 147-163. https://doi.org/10.1080/13632434.2018. 1434767
  • Jones, P. J., Mair, P. y McNally, R. J. (2018). Visualizing psychological networks: A tutorial in R. Frontiers in Psychology, 9, 1742. https://doi. org/10.3389/fpsyg.2018.01742
  • Jornet, J. M. (2016). Análisis metodológico del Proyecto PISA como evaluación internacional. RELIEVE, 22(1), M1. https://doi.org/10.7203/relieve22.1.8293
  • Kim, S. W. y Hill, N. E. (2015). Including fathers in the picture: A metaanalysis of parental involvement and students’ academic achievement. Journal of Educational Psychology, 107(4), 919-934. https://doi. org/10.1037/edu0000023
  • Lane, J., Lane, A. M. y Kyprianou, A. (2004). Self-efficacy, self-esteem and their impact on academic performance. Social Behavior and Personality, 256(10), 247-256. https://doi.org/10.2224/sbp.2004.32.3.247
  • Marchesi, A. y Martín, E. (1998). Calidad de la enseñanza en tiempos de cambio. Alianza.
  • McNally, R. J. (2016). Can network analysis transform psychopathology? Behaviour Research and Therapy, 86, 95-104. https://doi.org/10.1016/j. brat.2016.06.006
  • Ministerio de Educación. (2011). Evaluación general de diagnóstico 2010. Informe de resultados. Instituto de Evaluación. Madrid. https://www. educacionyfp.gob.es/inee/dam/jcr:134d0cfd-1090-4c21-8b10-66a6b07b565a/informe-egd-2010.pdf
  • Möller, J., Pohlmann, B., Köller, O. y Marsh, H. W. (2009). A Meta-analytic path analysis of the internal/external frame of reference model of academic achievement and academic self-concept. Review of Educational Research, 79(3), 1129-1167. https://doi.org/10.3102/0034654309337522
  • Möller, J., Zitzmann, S., Helm, F., Machts, N. y Wolff, F. (2020). A metaanalysis of relations between achievement and self-concept. Review of Educational Research, 90(3), 376-419. https://doi. org/10.3102/0034654320919354
  • Mullis, I. V. S., Martin, M. O. y Foy, P. (2005). IEA’s TIMSS 2003 international report on achievement in the Mathematics cognitive domains. TIMSS & PIRLS International Study Center, Boston College.
  • Mullis, I. V. S., Martin, M. O. y Foy, P. (2008). TIMSS 2007 international Mathematics report: Findings from IEA’s trends in international Mathematics and Science study at the fourth and eighth grades. TIMSS & PIRLS International Study Center, Boston College.
  • Mullis, I. V. S., Martin, M. O., Foy, P. y Arora, A. (2012). The TIMSS 2011. International results in Mathematics. TIMSS & PIRLS International Study Center, Boston College.
  • Mullis, I. V. S., Martin, M. O., Foy, P. y Drucker, K. T. (2012). The PIRLS 2011 international results in reading. TIMSS & PIRLS International Study Center, Boston College.
  • Mullis, I. V. S., Martin, M. O., Foy, P. y Hooper, M. (2016). TIMSS 2015 international results in Mathematics. TIMSS & PIRLS International Study Center, Boston College.
  • Mullis, I. V. S., Martin, M. O., Gonzalez, E. J., Gregory, K. D., Garden, R. A., O’Connor, K. M., Chrostowski, S. J. y Smith, T. A. (2000). TIMSS 1999 international Mathematics report. TIMSS & PIRLS International Study Center, Boston College. Chestnut Hill, MA.
  • Mullis, I. V. S., Martin, M. O., Kennedy, A. M. y Foy. P. (2007). IEA’s progress in international reading literacy study in primary school in 40 coun- tries. TIMSS & PIRLS International Study Center, Boston College. Chestnut Hill, MA.
  • Mullis, I. V. S., Martin, M. O., Smith, T. A., Garden, R. A., Gonzalez, E. J., Chorostowski, S. J. y O’Connor, K. M. (2002). TIMSS assessment framewoks and specificacions 2003 (2 ed.). Boston College. Chestnut Hill, MA.
  • Muñiz, J. (2018). Introducción a la psicometria. Pirámide Newman.
  • Press. OECD. (2007). PISA 2006: Science competencies for tomorrow’s world: Volume 1: Analysis. OECD Publishing. https://doi. org/10.1787/9789264040014-en
  • OECD. (2013). PISA 2012 results: What makes schools successful? Resources, policies and practices (Vol. IV). OECD Publishing. https://doi. org/10.1787/9789264201156-en
  • OECD. (2014). PISA 2012 results: What students know and can do: Student performance in Mathematics, Reading and Science (Volume I) [Revised edition February 2014]. OECD Publishing. https://doi. org/10.1787/9789264201118-en
  • Palardy, G., Rumberger, R. y Butler, T. (2015). The effect of high school socioeconomic, racial, and linguistic segregation on academic performance and school behaviors. Teachers College Record, 117(12), 1-52.
  • Patall, E. A., Cooper, H. y Robinson, J. C. (2008). Parent involvement in homework: A research synthesis. Review of Educational Research, 78(4), 1039-1101. https://doi.org/10.3102/0034654308325185
  • Peña-Suárez, E., Fernández-Alonso, R. y Muñiz, J. (2009). Estimación del valor añadido de los centros escolares. Aula Abierta, 37(1), 3-18.
  • Petscher, Y. (2010). A meta-analysis of the relationship between student attitudes towards reading and achievement in reading. Journal of Research in Reading, 33(4), 335-355. https://doi.org/10.1111/j.1467- 9817.2009.01418.x
  • Pinquart, M. y Ebeling, M. (2020). Students’ expected and actual academic achievement – A meta-analysis. International Journal of Educational Research, 100, 101524. https://doi.org/10.1016/j.ijer.2019.101524
  • Pomerantz, E. M., Moorman, E. A. y Litwack, S. D. (2007). The how, whom, and why of parents’ involvement in children’s academic lives: More is not always better. Review of Educational Research, 77(3), 373-410. https://doi.org/10.3102/003465430305567
  • R Core Team. (2016). R: A language and environment for statistical computing. Vienna, Austria.
  • Richardson, M., Abraham, C. y Bond, R. (2012). Psychological correlates of university students’ academic performance: A systematic review and meta-analysis. Psychological Bulletin, 138(2), 353-387. https://doi. org/10.1037/a0026838
  • Rodríguez-Rodríguez, D. y Guzmán, R. (2019). Socio-familial risk factors and personal protective variables of academic performance in Secondary Education students. Psicothema, 31(2), 142-148. https:// doi.org/10.7334/psicothema2018.213
  • Sainz Martínez, A., Arregi Martínez, A. y Angulo Vargas, A. (2010). Evaluación de diagnóstico 2009. Análisis de factores y variables: 2º ESO. Instituto Vasco de Evaluación e Investigación Educativa (ISEI-IVEI). Bilbao.
  • Scheerens, J. (2016). Educational effectiveness and ineffectiveness. A critical review of the knowledge base. Springer.
  • Schmittmann, V. D., Cramer, A. O. J., Waldorp, L. J., Epskamp, S., Kievit, R. A. y Borsboom, D. (2013). Deconstructing the construct: A network perspective on psychological phenomena. New Ideas in Psychology, 31(1), 43-53. https://doi.org/10.1016/j.newideapsych.2011.02.007
  • Sirin, S. R. (2005). Socioeconomic status and academic achievement: A meta-analytic review of research. Review of Educational Research, 75(3), 417-453. https://doi.org/10.3102/00346543075003417
  • Stankov, L. (2013). Noncognitive predictors of intelligence and academic achievement: An important role of confidence. Personality and Individual Differences, 55(7), 727-732. https://doi.org/10.1016/j. paid.2013.07.006
  • Stankov, L., Morony. S. y Lee, Y. P. (2014) Confidence: the best non-cognitive predictor of academic achievement? Educational Psychology: An International Journal of Experimental Educational Psychology, 34(1), 9-28. https://doi.org/10.1080/01443410.2013.814194
  • Suárez-Álvarez, J., Fernández-Alonso, R. y Muñiz, J. (2014). Self-concept, motivation, expectations, and socioeconomic level as predictors of academic performance in mathematics. Learning and Individual Differences, 30, 118-123. https://doi.org/10.1016/j.lindif.2013.10.019
  • Tan, C. Y., Lyu, M. y Peng, B. (2019). Academic benefits from parental involvement are stratified by parental socioeconomic status: A metaanalysis. Parenting, 20(4), 241-287. https://doi.org/10.1080/15295192. 2019.1694836
  • Taut, S. y Palacios, D. (2016). Interpretaciones no intencionadas e intencionadas y usos de los resultados de PISA: una perspectiva de validez consecuencial. RELIEVE, 22(1), M8. https://doi.org/10.7203/ relieve22.1.8294
  • Towsend, T. (2007). International handbook of school effectiveness and improvement. Springer.
  • UNESCO-OREALC y LLECE. (2000). Primer estudio internacional comparativo sobre lenguaje, matemática y factores asociados, para alumnos del tercer y cuarto grado de la educación básica. Segundo informe. UNESCO.
  • UNESCO-OREALC y LLECE. (2016). Informe de resultados del tercer estudio regional comparativo y explicativo. Factores asociados. UNESCO.
  • Valle, A., Piñeiro, I., Rodríguez, S., Regueiro, B., Freire, C. y Rosário, P. (2019). Time spent and time management in homework in elementary school students: A person-centered approach. Psicothema, 31(4), 422-428.
  • White, K. R. (1982). The relation between socioeconomic status and academic achievement. Psychological Bulletin, 91(3), 461-481. https:// doi.org/10.1037/0033-2909.91.3.461
  • Wilder, S. (2014). Effects of parental involvement on academic achievement: A meta-synthesis. Educational Review, 66(3), 377-397. https://doi.org/ 10.1080/00131911.2013.780009
  • Woitschach, P., Fernández-Alonso, R., Martínez-Arias, R. y Muñiz, J. 2017). Influencia de los centros escolares sobre el rendimiento académico en Latinoamérica. Revista de Psicología y Educación, 12(2), 138-154. https://doi.org/10.23923/rpye2017.12.152
  • Woitschach, P., Zumbo, B. D. y Fernández-Alonso, R. (2019). An ecological view of measurement: Focus on multilevel model explanation of differential item functioning. Psicothema, 31(2), 194-203.
  • Wu, M. L., Adams, R. J., Wilson, M. R. y Haldane, S. (2007). ConQuest (Version 2.0) [Computer software]. ACER.
  • Xia, N. y Kirby, S. N. (2009). Retaining students in grade: A literature review of the effects of retention on students’ academic and non-academic outcomes (Technical Report No. 678). RAND Coorporation. http:// www.rand.org/ pubs/technical_reports/TR678/
  • M. E. J. (2010). Networks: An introduction. Oxford University
Data source: Dialnet