How Can Generative AI Support Education?

C Delgado Kloos, C Alario-Hoyos, I Estévez-Ayres, P Callejo Pinardo, …
2024 IEEE Global Engineering Education Conference (EDUCON)

Enhancing Research on Engineering Education: Empowering Research Skills Through Generative Artificial Intelligence for Systematic Literature Reviews

P Castillo Segura, C Fernandez-Panadero, C Alario-Hoyos, …
2024 IEEE Global Engineering Education Conference (EDUCON)

Student and teacher impact on the use of telepresence classrooms

A Carruana Martín, C Alario-Hoyos, PM Moreno-Marcos, …
Education and Information Technologies

Personalized Strategies for Academic Success in Learning Anatomy: Exploring Metacognitive and Technological Adaptation in Medical Students

M Stambuk Castellano, F Reina, A Carrera, RS Tubbs, C Alario-Hoyos, …
Clinical Anatomy


November 2023

EMOOCs 2023

S van Esveld, N de Vries, S Becchetti, S Dopper, W van Valkenburg, …
Conference Proceedings EMOOCS 2023


EMOOCs 2023: Post-Covid Prospects for Massive Open Online Courses-Boost or Backlash?

C Meinel, S Schweiger, T Staubitz, R Conrad, CA Hoyos, M Ebner, …
Universitätsverlag Potsdam


Innovat MOOC

CA Hoyos, CD Kloos, D Kiendl, L Terzieva
EMOOCs 2023, 229


Analysis and Prediction of Students’ Performance in a Computer-Based Course Through Real-Time Events

L Uguina-Gadella, I Estévez-Ayres, JA Fisteus, C Alario-Hoyos, CD Kloos
IEEE Transactions on Learning Technologies

Construcción de Centros de Enseñanza y Aprendizaje para las Universidades de Latinoamérica en el Siglo XXI

C Alario-Hoyos, CD Kloos, M Morales, H Amado, R Hernández, …
Congreso de Docencia en Educación Superior CODES 5

Leveraging the Potential of Generative AI to Accelerate Systematic Literature Reviews: An Example in the Area of Educational Technology

P Castillo-Segura, C Alario-Hoyos, CD Kloos, CF Panadero
2023 World Engineering Education Forum-Global Engineering Deans Council …

Transforming Education in the 21st Century: The Role of PROF-XXI Project in Developing Teaching Competencies

MM Chan, CA Hoyos, HR Amado-Salvatierra, M Pérez-Sanagustin, …
2023 IEEE Learning with MOOCS (LWMOOCS), 1-5

Statoodle: A Learning Analytics Tool to Analyze Moodle Students’ Actions and Prevent Cheating

PM Moreno-Marcos, J Barredo, PJ Muñoz-Merino, C Delgado Kloos
European Conference on Technology Enhanced Learning, 736-741

A Study of Student and Teacher Challenges in Smart Synchronous Hybrid Learning Environments

A Carruana Martín, C Alario-Hoyos, C Delgado Kloos
Sustainability 15 (15), 11694

Analysis of Orchestration Load and Teacher Agency in Smart Synchronous Hybrid Learning Environments

AC Martín, C Alario-Hoyos, A Martínez-Monés, CD Kloos

Recreation of different educational exercise scenarios for exercise modeling

A Jiménez-Macías, PJ Muñoz-Merino, CD Kloos
2023 IEEE Global Engineering Education Conference (EDUCON), 1-9

Recognizing the Value of Recognition in Education

CD Kloos, C Alario-Hoyos, MB Ibáñez, PM Moreno-Marcos, …
2023 IEEE Global Engineering Education Conference (EDUCON), 1-5

ABENEARIO: A system for learning early maths with ABN

A Martín Díaz, C Alario-Hoyos, I Estévez-Ayres, C Delgado Kloos, …
Education and Information Technologies, 1-23

Experiences with Micro-Credentials at UC3M: Academic and Technological Aspects

CA Hoyos, CD Kloos
2023 IEEE World Engineering Education Conference (EDUNINE), 1-6

Analyzing feature importance for a predictive undergraduate student dropout model

A Jiménez-Macias, PM Moreno-Marcos, PJ Muñoz-Merino, M Ortiz-Rojas, …
Computer Science and Information Systems 20 (1), 175-194

Smart Groups: A system to orchestrate collaboration in hybrid learning environments. A simulation study

A Carrruana Martín, C Alario-Hoyos, C Delgado Kloos
Australasian Journal of Educational Technology 38 (6), 150-168


October 2022

Adaptation of a process Mining Methodology to Analyse Learning Strategies in a Synchronous Massive Open Online Course

Jorge Maldonado-Mahauad, Carlos Alario-Hoyos, Carlos Delgado Kloos, Mar Perez-Sanagustin

The study of learners’ behaviour in Massive Open Online Courses (MOOCs) is a topic of great interest for the Learning Analytics (LA) research community. In the past years, there has been a special focus on the analysis of students’ learning strategies, as these have been associated with successful academic achievement. Different methods and techniques, such as temporal analysis and process mining (PM), have been applied for analysing learners’ trace data and categorising them according to their actual behaviour in a particular learning context. However, prior research in Learning Sciences and Psychology has observed that results from studies conducted in one context do not necessarily transfer or generalise to others. In this sense, there is an increasing interest in the LA community in replicating and adapting studies across contexts. This paper serves to continue this trend of reproducibility and builds upon a previous study which proposed and evaluated a PM methodology for classifying learners according to seven different behavioural patterns in three asynchronous MOOCs of Coursera. In the present study, the same methodology was applied to a synchronous MOOC on edX with N = 50,776 learners. As a result, twelve different behavioural patterns were detected. Then, we discuss what decision other researchers should made to adapt this methodology and how these decisions can have an effect on the analysis of trace data. Finally, the results obtained from applying the methodology contribute to gain insights on the study of learning strategies, providing evidence about the importance of the learning context in MOOCs.


October 2022

The impacts of scaffolding socially shared regulation on teamwork in an online project-based course

Catalina Cortázar, Miguel Nussbaum, Carlos Alario-Hoyos, Julián Goñi, Danilo Alvares

Employers now consider teamwork one of the essential skills for students to acquire during their academic life. However, COVID-19 has accelerated the transition towards online learning, affecting how we work in teams. This study looked at how scaffolding socially shared regulation of learning can influence teamwork in an online, project-based course. Intra-group peer assessment was used to analyze three variables during a first-year engineering course. By following the proposed scaffolding, students found an optimum balance in their contribution to team meetings. They also managed to establish a positive working environment earlier in the semester. This study contributes to the field by showing that scaffolding socially shared regulation in an online, project-based course allows for an interplay between collaboration during class and cooperation outside of it. This interplay ultimately leads teams to achieve better results on their final project.


April 2022

H2O Learn – Hybrid and Human-Oriented Learning: Trustworthy and Human-Centered Learning Analytics (TaHCLA) for Hybrid Education

Carlos Delgado-Kloos, Yannis Dimitriadis, Davinia Hernández-Leo, Carlos Alario-Hoyos, Alejandra Martínez-Monés, Patricia Santos, Pedro J. Muñoz-Merino, Juan I. Asensio-Pérez, Lluis Vicent-Safont

This paper presents the H2O Learn (Hybrid and Human-Oriented Learning) project, a coordinated research project funded by the Spanish Research Agency, which just started in 2021 and will last for three years. The main goal of the H2O Learn project is to build Trustworthy and Human-Centered Learning Analytics (TaHCLA) solutions to support human stakeholders when designing, orchestrating and (self-,socially- or co-) regulating learning in Hybrid Learning (HL). The contributions of H2O Learn consider key requirements for trustworthy Artificial Intelligence (AI), as defined by the European Commission: 1) fostering human (i.e., teachers, learners…) agency; 2) enabling transparency of the Learning Analytics (LA) systems; 3) seeking socio-emotional and inclusive wellbeing; and 4) promoting accountability by enabling the assessment of algorithms and design processes.


April 2022

Programming Teaching Interaction

Carlos Delgado Kloos, Carmen Fernández-Panadero, Carlos Alario-Hoyos, Pedro M Moreno-Marcos, María Blanca Ibáñez, Pedro J. Muñoz-Merino, Boni García, Iria Estévez-Ayres

There are many applications that help orchestrate face-to-face, online, or hybrid classes through the cloud. Examples are polling apps like Wooclap or Mentimeter, digital boards like Jamboard or Miro, or diagramming tools like, Coggle, or Kialo. However, one can go one step further and integrate some of these tools in order to use the most appropriate tool for each teaching moment and have information flow across tools preserving consistency without having to do this manually. In this paper, we present a specific experience and extrapolate it to highlight the power of programming the teaching interaction.


February 2022

Transforming the learning experience in pre-service teacher training using the flipped classroom

Camila Barahona, Miguel Nussbaum, Pablo Espinosa, Alejandra Meneses, Carlos Alario-Hoyos, Mar Pérez-Sanagustín

The lack of information and communication technology skills among teachers is highlighted as being one of the main barriers to increasing the use of technology in teaching and learning. This study shows how technology used in a flipped classroom can improve learning among pre-service teachers, as well as boosting perceptions of their own technology skills. The students covered the instructional content before class through videos complemented by formative assessment. The data resulted in a report that the teacher then used to adapt their lecture, as well as defining the questions to be asked using an in-class response system. The responses to these questions were then discussed with the students. The results show that it is possible to change pre-service teachers’ perceptions of using technology in the classroom, both as a student and as a future teacher.


January 2022

A Competency Framework for Teaching and Learning Innovation Centers for the 21st Century: Anticipating the Post-COVID-19 Age

Mar Pérez-Sanagustín, Iouri Kotorov, António Teixeira, Fernanda Mansilla, Julien Broisin, Carlos Alario-Hoyos, Óscar Jerez, Maria do Carmo Teixeira Pinto, Boni García, Carlos Delgado Kloos, Miguel Morales, Mario Solarte, Luis Magdiel Oliva-Córdova, Astrid Helena Gonzalez Lopez

During the COVID-19 pandemic, most Higher Education Institutions (HEIs) across the globe moved towards “emergency online education”, experiencing a metamorphosis that advanced their capacities and competencies as never before. Teaching and Learning Centers (TLCs), the internal units that promote sustainable transformations, can play a key role in making this metamorphosis last. Existing models for TLCs have defined the competencies that they could help develop, focusing on teachers’, students’, and managers’ development, but have mislead aspects such as leadership, organizational processes, and infrastructures. This paper evaluates the PROF-XXI framework, which offers a holistic perspective on the competencies that TLCs should develop for supporting deep and sustainable transformations of HEIs. The framework was evaluated with 83 participants from four Latin American institutions and used for analyzing the transformation of their teaching and learning practices during the pandemic lockdown. The result of the analysis shows that the PROF-XXI framework was useful for identifying the teaching and learning competencies addressed by the institutions, their deficiencies, and their strategic changes. Specifically, this study shows that most institutions counted with training plans for teachers before this period, mainly in the competencies of digital technologies and pedagogical quality, but that other initiatives were created to reinforce them, including students’ support actions.

Carlos Alario Hoyos, Iria Manuela Estévez Ayres, Carlos Delgado Kloos Árbol académico, Julio Villena Román, Pedro J. Muñoz Merino, Enrique Llorente Pérez

MOOCs have made it possiblenot only to provide quality open education for any learnerworldwide, but also to rethink theway on-campusteachingis delivered. Thematerials produced for a MOOC canbeconsumed byon-campusstudents beforearriving to the classroom, using class time to do activities that promote active learning, following this way a flippedclassroom strategy. This paper presents the experience of redesigning a first-year engineering course with a large number of students (over 400 each year), in which MOOCs are reused, and a flipped classroom strategy is implemented, dedicatingmost of traditional lecture time to do hands-on, interactive activities. The results show an increase in students’ motivation,both in the use of MOOC content outside the classroom, and in the realization of hands-on, interactive activities inside theclassroom. In relation to the teacher, having information on students’ previous work outside the classroom, and onstudents’ work in the hands-on, interactive activities carried out inside the classroom, allows understanding better thedifferences between groups, tailoring the explanations during class time, and providing proper reinforcement activities tobe done after class.

M. Elena Alonso-Mencía, Carlos Alario-Hoyos, Jorge Maldonado-Mahauad, Iria Estévez-Ayres, Mar Pérez-Sanagustín & Carlos Delgado Kloos

Learners in massive open online courses (MOOCs) are required to be autonomous during their learning process, and thus they need to self-regulate their learning to achieve their goals. According to existing literature, self-regulated learning (SRL) research in MOOCs is still scarce. More studies which build on past works regarding SRL in MOOCs are required, as well as literature reviews that help to identify the main challenges and future research directions in relation to this area. In this paper, the authors present the results of a systematic literature review on SRL in MOOCs, covering all the related papers published until the end of 2017. The papers considered in this review include real experiences with at least a MOOC (other learning scenarios sometimes claimed as MOOCs, such as blended courses, or online courses with access restrictions, are out of the scope of this analysis). Most studies on SRL in MOOCs share some common features: they are generally exploratory, based on one single MOOC and tend not to specify in which SRL model they are grounded. The results reveal that high self-regulators engage in non-linear navigation and approach MOOCs as an informal learning opportunity. In general, they prefer setting specific goals based on knowledge development and control their learning through assignments.


Pedro Manuel Moreno-Marcos, Carlos Alario-Hoyos, Pedro J Munoz-Merino, Iria Estevez-Ayres, Carlos Delgado Kloos

One of the characteristics of MOOCs (Massive Open Online Courses) is that the overall number of social interactions tend to be higher than in traditional courses, hindering the analysis of social learning. Learners typically ask or answer questions using the forum. This makes messages a rich source of information, which can be used to infer learners behaviour and outcomes. It is not feasible for teachers to process all forum messages and automated tools and analysis are required. Although there are some tools for analysing learners interactions, there is a need for methodologies and integrated tools that help to interpret the learning process based on social interactions in the forum. This work presents the 3S (Social, Sentiments, Skills) learning analytics methodology for analysing forum interactions in MOOCs. This methodology considers a temporal analysis combining the social, sentiments and skill dimensions …

Armin Weinberger, Carlos Alario-Hoyos, Poline Bala, Dennis Batangan, Carlos Delgado Kloos, Narayanan Kulathuramaiyer, John Carlo Navera, Josenh Palis, Alwin Melkie Sambul, Peter Sy, Tat-Chee Wan

This Paper explores the current use of Massive Open Online Courses (MOOCs) as a means of educational outreach among identified remote populations in Southeast Asia. Often excluded from traditional educational outreach, these groups are targeted through the COMPETEN-SEA Project, a Capacity Building in Higher Education project funded by the Erasmus+ programme of the European Commission and implemented in partnership with European and Southeast Asian universities. It is hoped that the Project will aid participating Southeast Asian countries address societal needs and attain national development goals.

Cristina Catalán Aguirre, Carlos Delgado Kloos, Carlos Alario-Hoyos, Pedro J Muñoz-Merino

This paper presents an initial design of a conversational agent for educational purposes built for Google Assistant and its first prototype. Recent studies suggest that people will get more and more attached to voice assistant because they can easily use technology without being forced to learn it. Speech recognition might facilitate a more efficient work environment without being overly rigid and overly domineering. Modern frameworks allow harnessing natural language understanding as well a machine learning tools, thereby making it easy to build conversational agents. In this paper, we present first design decisions and a prototype for building an agent for learning Java that complements a MOOC for programming with Java.

Jorge Maldonado-Mahauad, Mar Pérez-Sanagustín, Pedro Manuel Moreno-Marcos, Carlos Alario-Hoyos, Pedro J Muñoz-Merino, Carlos Delgado-Kloos

In the past years, predictive models in Massive Open Online Courses (MOOCs) have focused on forecasting learners’ success through their grades. The prediction of these grades is useful to identify problems that might lead to dropouts. However, most models in prior work predict categorical and continuous variables using low-level data. This paper contributes to extend current predictive models in the literature by considering coarse-grained variables related to Self-Regulated Learning (SRL). That is, using learners’ self-reported SRL strategies and MOOC activity sequence patterns as predictors. Lineal and logistic regression modelling were used as a first approach of prediction with data collected from N = 2,035 learners who took a self-paced MOOC in Coursera. We identified two groups of learners: (1) Comprehensive, who follow the course path designed by the teacher; and (2) Targeting, who seek …

Pedro Manuel Moreno-Marcos, Carlos Alario-Hoyos, Pedro J Muñoz-Merino, Carlos Delgado Kloos

This paper surveys the state of the art on prediction in MOOCs through a Systematic Literature Review (SLR). The main objectives are: (1) to identify the characteristics of the MOOCs used for prediction, (2) to describe the prediction outcomes, (3) to classify the prediction features, (4) to determine the techniques used to predict the variables, and (5) to identify the metrics used to evaluate the predictive models. Results show there is strong interest in predicting dropouts in MOOCs. A variety of predictive models are used, though regression and Support Vector Machines stand out. There is also wide variety in the choice of prediction features, but clickstream data about platform use stands out. Future research should focus on developing and applying predictive models that can be used in more heterogeneous contexts (in terms of platforms, thematic areas, and course durations), on predicting new outcomes and making …

Christian M Stracke, Rocael Hernández, Carlos Delgado Kloos, Mar Pérez Sanagustín, António Moreira Teixeira

Presentation at OE Global 2018, Delft, The Netherlands, by Stracke, C. M., et al. (2018, 24 April) on: «How to make MOOCs better for specific target groups and developing countries?»

Pedro Manuel Moreno-Marcos, Carlos Alario-Hoyos, Pedro J Muñoz-Merino, Iria Estévez-Ayres, Carlos Delgado Kloos

Forum messages in MOOCs (Massive Open Online Courses) are the most important source of information about the social interactions happening in these courses. Forum messages can be analyzed to detect patterns and learners’ behaviors. Particularly, sentiment analysis (e.g., classification in positive and negative messages) can be used as a first step for identifying complex emotions, such as excitement, frustration or boredom. The aim of this work is to compare different machine learning algorithms for sentiment analysis, using a real case study to check how the results can provide information about learners’ emotions or patterns in the MOOC. Both supervised and unsupervised (lexicon-based) algorithms were used for the sentiment analysis. The best approaches found were Random Forest and one lexicon based method, which used dictionaries of words. The analysis of the case study also showed an …

Carlos Alario-Hoyos, Iria Estévez-Ayres, Jesús M Gallego-Romero, Carlos Delgado Kloos, Carmen Fernández-Panadero, Raquel M Crespo-García, Florina Almenares, María Blanca Ibáñez, Julio Villena-Román, Jorge Ruiz-Magaña, Jorge Blasco

Many MOOCs are being designed replicating traditional passive teaching approaches but using video lectures as the means of transmitting information. However, it is well known that learning-by-doing increases retention rates and, thus, allows achieving a more effective learning. To this end, it is worth exploring which tools fit best in the context of each MOOC to enrich learners’ experience, including built-in tools already available in the MOOC platform, and third-party external tools which can be integrated in the MOOC platform. This paper presents an example of the integration of a software development tool, called Codeboard, in three MOOCs which serve as an introduction to programming with Java. We analyze the effect this tool has on learners’ interaction and engagement when running the MOOCs in synchronous (instructor-paced) or asynchronous (self-paced) modes. Results show that the overall use of the …


José A Ruipérez-Valiente, Pedro J Muñoz-Merino, José A Gascón-Pinedo, C Delgado Kloos

The emergence of massive open online courses (MOOCs) has caused a major impact on online education. However, learning analytics support for MOOCs still needs to improve to fulfill requirements of instructors and students. In addition, MOOCs pose challenges for learning analytics tools due to the number of learners, such as scalability in terms of computing time and visualizations. In this work, we present different visualizations of our “Add-on of the learNing AnaLYtics Support for open Edx” (ANALYSE), which is a learning analytics tool that we have designed and implemented for Open edX, based on MOOC features, teacher feedback, and pedagogical foundations. In addition, we provide a technical solution that addresses scalability at two levels: first, in terms of performance scalability, where we propose an architecture for handling massive amounts of data within educational settings; and, second, regarding.

Alario-Hoyos, C., Estévez-Ayres, I., Delgado Kloos, C., Villena-Román, J.

The concept of SPOCs (Small Private Online Courses) emerged as a way of describing the reuse of MOOCs (Massive Open Online Courses) for complementing traditional on-campus teaching. But SPOCs can also drive an entire methodological change to make a better use of face-to-face time between students and teachers in the classroom. This paper presents the redesign and evaluation of a first-year programming course in several engineering degrees, with over 400 students overall, through the reuse of MOOCs as SPOCs on campus, combined with a flipped classroom strategy aimed at promoting active learning. Results from a students’ self-reported questionnaire show a very positive acceptance of the SPOC, which includes both videos and complementary formative activities, and an increase of motivation through the combination of the SPOC and activities implemented in lectures to flip the classroom.


Ruipérez-Valiente, J. A., Muñoz-Merino, P. J., Gascón-Pinedo, J. A., & Kloos, C. D.

The emergence of massive open online courses (MOOCs) has caused a major impact on online education. However, learning analytics support for MOOCs still needs to improve to fulfill requirements of instructors and students. In addition, MOOCs pose challenges for learning analytics tools due to the number of learners, such as scalability in terms of computing time and visualizations. In this work, we present different visualizations of our “Add-on of the learNing AnaLYtics Support for open Edx” (ANALYSE), which is a learning analytics tool that we have designed and implemented for Open edX, based on MOOC features, teacher feedback, and pedagogical foundations. In addition, we provide a technical solution that addresses scalability at two levels: first, in terms of performance scalability, where we propose an architecture for handling massive amounts of data within educational settings; and, second, regarding the representation of visualizations under massiveness conditions, as well as advice on color usage and plot types. Finally, we provide some examples on how to use these visualizations to evaluate student performance and detect problems in resources.

Ruipérez-Valiente, J. A., Muñoz-Merino, P. J., Kloos, C. D., Niemann, K., Scheffel, M., & Wolpers, M

Presentación del Profesor Carlos Delgado Kloos en la Jornada de Educación Abierta celebrada el 11 de marzo de 2013 en la Universidad Carlos III de Madrid en el marco de la Open Education Week promovida por el Consorcio OpenCourseWare.