Аналитика мультимодальная

Описание	Направление учебной аналитики подчёркивает, что современные цифровые средства позволяют собирать данные сразу по нескольким каналам и такое многоканальное объединение данные позволяет глубже понимать динамику обучения.
Область знаний	NetSci, Информатика, Управление
Авторы	Worsley, Blikstein
Поясняющее видео	https://www.youtube.com/watch?v=AWvCIDGaKwU
Близкие понятия	Аналитика учебная
Среды и средства для освоения понятия	R, Julia, Python, Chronoviz

Multimodal learning analytics (MMLA) sits at the intersection of three ideas: multimodal teaching and learning, multimodal data, and computer-supported analysis.

Definitions

1. a set of techniques that can be used to collect multiple sources of data in high frequency (video, logs, audio, gestures, biosensors), synchronize and code the data, and examine learning in realistic, ecologically valid, social, mixed-media learning environments
   • Blikstein P. Multimodal learning analytics // Proceedings of the third international conference on learning analytics and knowledge LAK ’13. New York, NY, USA: Association for Computing Machinery, 2013. С. 102–106.
2. three main operative processes of MmLA: use of diverse sources of learning traces (multimodal data), processing and integration of these traces (multimodal analysis and fusion), and the study of human behavior in real learning environments (learning behavior detection and learning construct estimation).
   • Ochoa X. Multimodal learning analytics // The handbook of learning analytics. 2017. Т. 1. С. 129–141.
3. MMLA can monitor the learning activity at the micro-level and help us model humans' cognitive, affective and social factors associated with learning processes. MMLA can provide information in a temporal and unobtrusive manner. In this way, MMLA has the capacity to challenge established ‘truths’ and allow different theoretical lenses to further our knowledge of how humans learn.
   • Giannakos M., Cukurova M. The role of learning theory in multimodal learning analytics // British Journal of Educational Technology. 2023. Т. 54. № 5. С. 1246–1267.

Поделить деятельность ученика на отдельные кусочки действий и потом на основании этих кусочков "разломал, соединил, повторил, вернул" прогнозировать результаты. Разложить действия на кластеры.

Средства MMLA

https://ivohub.com/coding-analysing-tools/

1. https://archive.mpi.nl/tla/elan ELAN
   • Description: With ELAN a user can add an unlimited number of textual annotations to audio and/or video recordings. An annotation can be a sentence, word or gloss, a comment, translation or a description of any feature observed in the media. Annotations can be created on multiple layers, called tiers. Tiers can be hierarchically interconnected. An annotation can either be time-aligned to the media or it can refer to other existing annotations. The content of annotations consists of Unicode text and annotation documents are stored in an XML format (EAF).
2. Anvil
   • ANVIL http://www.anvil-software.de describes itself as a ‘free video annotation tool’. It allows for multi-layered information to be coded along a series of different level ‘tracks ‘which can also be colour-coded as per the user’s preference.
3. Praat
   • Praat http://www.praat.org/ is a free software tool developed by Paul Boersma and David Weenink with many resources to aid speech analysis. It is widely used for phonetic and phonological research. https://www.fon.hum.uva.nl/praat/manual/Intro.html The latest version of Praat (6.3.03) was released in December 2022.
4. Transana
   • Transana https://www.transana.com/products/transana-basic/ is a multi-modal analysis software capable of dealing with text, still-frame images, video and audio within a single analytical frame.
5. Dynapad
   • https://hci.ucsd.edu/lab/dynapad.htm - This is a multiscale interface and visualization software. It makes scale a first-class parameter of objects, supports navigation in multiscale workspaces, and provides special mechanisms to maintain interactivity while rendering large numbers of graphical items

Литература

1. Abrahamson, D., Worsley, M., Pardos, Z.A., Ou, L.: Learning analytics of embodied design: Enhancing synergy. International Journal of Child-Computer Interaction. 32, 100409 (2022). https://doi.org/10.1016/j.ijcci.2021.100409.
2. Crescenzi-Lanna, L.: Multimodal Learning Analytics research with young children: A systematic review. British Journal of Educational Technology. 51, 1485–1504 (2020). https://doi.org/10.1111/bjet.12959.
3. DeLiema, D., Kwon, Y.A., Chisholm, A., Williams, I., Dahn, M., Flood, V.J., Abrahamson, D., Steen, F.F.: A Multi-dimensional Framework for Documenting Students’ Heterogeneous Experiences with Programming Bugs. Cognition and Instruction. 41, 158–200 (2023). https://doi.org/10.1080/07370008.2022.2118279.
4. DeLiema, D., Kwon, Y.A., Chisholm, A., Williams, I., Dahn, M., Flood, V.J., Abrahamson, D., Steen, F.F.: A Multi-dimensional Framework for Documenting Students’ Heterogeneous Experiences with Programming Bugs. Cognition and Instruction. 41, 158–200 (2023). https://doi.org/10.1080/07370008.2022.2118279.
5. Emerson, A., Cloude, E.B., Azevedo, R., Lester, J.: Multimodal learning analytics for game-based learning. British Journal of Educational Technology. 51, 1505–1526 (2020). https://doi.org/10.1111/bjet.12992.
6. Giannakos, M., Cukurova, M.: The role of learning theory in multimodal learning analytics. British Journal of Educational Technology. 54, 1246–1267 (2023). https://doi.org/10.1111/bjet.13320.
7. Giannakos, M., Spikol, D., Di Mitri, D., Sharma, K., Ochoa, X., Hammad, R.: Introduction to Multimodal Learning Analytics. In: Giannakos, M., Spikol, D., Di Mitri, D., Sharma, K., Ochoa, X., and Hammad, R. (eds.) The Multimodal Learning Analytics Handbook. pp. 3–28. Springer International Publishing, Cham (2022). https://doi.org/10.1007/978-3-031-08076-0_1.
8. Mu, S., Cui, M., Huang, X.: Multimodal Data Fusion in Learning Analytics: A Systematic Review. Sensors. 20, 6856 (2020). https://doi.org/10.3390/s20236856.
9. Ochoa, X.: Multimodal learning analytics. The handbook of learning analytics. 1, 129–141 (2017).
10. Sharma, K., Giannakos, M.: Multimodal data capabilities for learning: What can multimodal data tell us about learning? British Journal of Educational Technology. 51, 1450–1484 (2020). https://doi.org/10.1111/bjet.12993.
11. Shvarts, A., Abrahamson, D.: Coordination Dynamics of Semiotic Mediation: A Functional Dynamic Systems Perspective on Mathematics Teaching/Learning. Constructivist Foundations. 18, 220–234 (2023).
12. Worsley, M., Abrahamson, D., Blikstein, P., Grover, S., Schneider, B., Tissenbaum, M.: Situating multimodal learning analytics. (2016).

Библиометрический анализ

• 245 Results for: All: "multimodal learning analytics"
• Searched The ACM Guide to Computing Literature (3,631,985 records)|Limit your search to The ACM Full-Text Collection (725,591 records)

Авторы (без тезауруса)

• https://app.vosviewer.com/?json=https://drive.google.com/uc?id=1YifIGeViC4RgOzLOr3tgJDabv97S1W1i

Авторы с тезаурусом

• https://app.vosviewer.com/?json=https://drive.google.com/uc?id=15xV4F_rK30GTmdRnnVgYRjFDSSUzEcI7

Ключевые слова

+Тезаурус -MultiModalLA - LA

• https://app.vosviewer.com/?json=https://drive.google.com/uc?id=1PUFDSEFfOQaczrgcUR9iuGl3sMudeCRc

• https://app.vosviewer.com/?json=https://drive.google.com/uc?id=10HrZyuaLf04eVc4eDdrA5AaqRBkp6nyk