<h1 style="text-align: center;"><br><br><br>-></h1> --- background-image: url(archipelago.jpg) background-size: cover .title-slide[ # Joint Visual Attention<br>and Collaboration<br>in Minecraft .authors[ .author[  Chris Proctor .affiliation[ University at Buffalo SUNY ]] .author[  Dalia Antonia Caraballo Muller .affiliation[ University at Buffalo SUNY ] ]]] .refs[ Slides available at [chrisproctor.net/slides/2022-cscl-minecraft](https://chrisproctor.net/slides/2022-cscl-minecraft) ] --- class: black ### Introduction .full-bleed[ <video controls src="cscl_interview.mp4" width="100%"> ] ??? So the big interdisciplinary challenge of this research is to see whether we can bridge the disciplinary gap, identifying where CSCL constructs such as collaoboration and particularly intersubjectivity, meet Relacion and common places. (Embed video here) --- # Intersubjectivity and collaboration in CSCL - **Collaboration** is "a process of building and maintaining a shared conception of a problem.cite[1]." Two paradigms of CSCL: cognitive/individualistic and socio-cultural. - Widely-adopted **collaboration rating scheme**.cite[2] defines collaboration in terms of nine dimensions of process. - **Intersubjectivity** is an essential basis for a shared conception of a problem and for collaboration. - **Joint visual attention** (JVA) is one of multiple modalities recruited together in establishing intersubjectivity.cite[3,4,5]. .refs[ .anchor[1] Akkerman, et al., 2007, p. 39 .anchor[2] Meier, Spada, and Rummel, 2007 .anchor[3] Schneider et al, 2019 .anchor[4] Olsen, et al., 2020 .anchor[5] Sharma, et al, 2021 ] ??? - **Collaboration** is "a process of building and maintaining a shared conception of a problem." Two paradigms of CSCL: cognitive/individualistic and socio-cultural.cite[1]. Closely related to my recent work with Kafai in CS education, including March 2022 Ed Researcher article on computational literacies. - Widely-adopted **collaboration rating scheme**.cite[2] defines collaboration in terms of nine dimensions of process. These include: sustaining mutual understanding, dialogue management, information pooling, reaching consensus, task division, time management, technical coordination, reciprocal interaction, and individual task orientation. - **Intersubjectivity** is an essential basis for a shared conception of a problem and for collaboration. - **Joint visual attention** (JVA) is one of multiple modalities recruited together in establishing intersubjectivity. Analytics have been used as a measure of collaoboration quality, though there is some debate around the validity of their use in an unsupervised, atheoretical manner. --- class: black background-image: url(collaboration.jpg) background-size: cover .heading-bottom[ # Collaboration<br>in Minecraft ] ??? - Minecraft is a multiplayer online game in which players inhabit and modify a blocky, pixellated world - Although it is evident that players of Minecraft engage in extensive in-game collaboration, it is unclear whether JVA plays the same coordinating role in Minecraft or other immersive online worlds that it plays in offline collaboration. - While playing Minecraft, players are embodied first in their corporeal bodies and secondly in their in-game avatars. - Visual attention has two layers here: first, how a player’s gaze moves across the screen, and second how the player orients her avatar to look around the world. Prior JVA research has tracked eye saccades, or quick darting shifts of focus around the visual field. In this research we assume the player’s ocular gaze is fixated on the center of her screen, and that she relies entirely on moving her avatar to look around. This is not an outrageous assumption: qualitative analysis of participants' gaze using Zoom video feeds shows them usually fixated on the center of the screen. One research assistant who is an avid gamer affirmed that, in contrast to first-person shooter games where situational awareness is paramount, it is common in Minecraft to remain fixated on the center of the screen, where a small crosshairs orients the eye to the location where block- building or block-removing actions will have their effect. - If JVA does function similarly in Minecraft as in other contexts, a potential advantage to online collaboration research is greater ecological validity (in an admittedly synthetic ecology). No further instrumentation is needed, players are free to may move around, manipulate their environment, and collaborate in groups larger than the dyads typically engaged in JVA research. The artifice of Minecraft could actually be helpful for research, as modalities of communication (e.g. gaze, gesture, movement and position, discourse, the manipulation of artifacts) can also be perfectly captured by the research setup. Given the limited articulation of avatars compared to the human body, face, and eyes, it may be easier to study the emergence of genres of multimodal interaction as players figure out how to work with what they have. --- # Research questions 1. Do high-collaboration groups in Minecraft engage in more joint visual<br>attention? 2. Does joint visual attention interact with other modalities in Minecraft? Is coordination between modalities dependent on effective collaboration? .image-inset[ <img src="collaboration.jpg" style="width: 40%; display: inline;"> <img src="hypotheses_0.jpg" style="width: 55%; display: inline;"> ] ??? 1. Do high-collaboration groups in Minecraft engage in more joint visual attention? We hypothesize that groups which collaborate more successfully will also exhibit more joint visual attention. 2. Does joint visual attention interact with other modalities in Minecraft? Is coordination between modalities dependent on effective collaboration? We hypothesize that the rate of block activity (placing and removing blocks) will be different in JVA and non-JVA intervals, with a greater difference in high-collaboration groups. --- count: false # Research questions 1. **Do high-collaboration groups in Minecraft engage in more joint visual attention?** 2. Does joint visual attention interact with other modalities in Minecraft? Is coordination between modalities dependent on effective collaboration? .image-inset[ <img src="collaboration.jpg" style="width: 40%; display: inline;"> <img src="hypotheses_1.jpg" style="width: 55%; display: inline;"> ] ??? While our research questions are not causal, they are consistent with the following diagram. --- count: false # Research questions 1. Do high-collaboration groups in Minecraft engage in more joint visual<br>attention? 2. **Does joint visual attention interact with other modalities in Minecraft? Is coordination between modalities dependent on effective collaboration?** .image-inset[ <img src="collaboration.jpg" style="width: 40%; display: inline;"> <img src="hypotheses_2.jpg" style="width: 55%; display: inline;"> ] --- # Context - Eight-week summer workshop for ten-year-old children focused on collaboration in Minecraft. - Entirely online: Participants logged into customized Minecraft server and Zoom. - Curriculum posed collaborative challenges in groups of two or three. ### Data sources .image-inset[ <img src="data_segments.png" style="width: 90%;"> ] .refs[ Repository link: [chrisproctor.net/research/proctor_2022_joint](https://chrisproctor.net/research/proctor_2022_joint) ] ??? - Eight-week summer workshop for ten-year-old children focused on collaboration in Minecraft. 10 participants, recruited through existing social ties; all were experienced players of MC. - Entirely online: Participants logged into customized Minecraft server and Zoom. - Curriculum posed collaborative challenges in groups of two or three. In the collaborative episodes studied in this paper: - Build a single structure which represents everyone in your group. - Build a prototype structure for the world’s spawn point (the starting location) which represents everyone in the workshop and the qualities we want for this world. - Build a prototype of a structure or object which creates a sense of unity and inclusiveness for our world. - One of this paper's major contributions is a package for analyzing We don't have time to go into the details today, but please check out the paper, the git repository --- background-color: #333 class: black ### Case study: High-collaboration group .full-bleed[ <video controls src="case_study.mp4" width="100%"> ] --- # Analysis - Identified and bounded episodes of collaboration. - Exported videos of episodes for coding and qualitative analysis. - Coded episodes as "high collaboration" or "low collaboration" using Meier, Spada, and Rummel’s (2007) collaboration measure. - Used Schneider and Pea’s (2013) JVA measure, considering pairwise distance between participant gaze. - Extracted block activity per second from server logs. - Following Jermann, et al. (2011) and Schneider, et al. (2018), we used cross-recurrence diagrams to illustrate how a dyad’s visual attention at different points in the collaboration coincided spatially. --- ### RQ1: Do high-collaboration groups in Minecraft engage in more joint visual attention? .image-inset[ <img src="rq1_results.png" style="width: 80%;"> ] t=2.03; p < 0.031 --- ### RQ1: Do high-collaboration groups in Minecraft engage in more joint visual attention? .image-inset[ <img src="w1g1_joint_attention.png" style="width: 90%;"> <img src="w1g3_joint_attention.png" style="width: 90%;"> ] --- ### RQ2: Does joint visual attention interact with other modalities in Minecraft? Is coordination between modalities dependent on effective collaboration? .image-inset[ <img src="rq2_expected.jpg" style="width: 29%; display: inline; vertical-align: top;"> <img src="rq2_results.png" style="width: 70%; display: inline;"> ] --- ### RQ2: Does joint visual attention interact with other modalities in Minecraft? .image-inset[ <img src="cross_recurrence.png" style="width: 60%;"> ] --- # Discussion - The results from RQ1 suggest that JVA plays an important role in collaboration in Minecraft just as it does offline. - While the results for RQ2 do not support our hypothesis, we inadvertently observed a sizable gap in block activity between high-collaboration and low-collaboration groups. - This paper demonstrates the feasibility of analyzing JVA alongside other multimodal analytics in Minecraft. We extend research on JVA and collaboration to groups larger than dyads and to three dimensional space in which players are free to move around. This could be helpful in future multimodal analysis of collaboration, especially on group roles and how members respond to interaction between other members. - Exploring how subjectivity and intersubjectivity emerge through multimodal sensemaking will support our research on Relación, or the work of building and maintaining a shared conception of intersubjectivity from which social futures can be organized. --- background-image: url(archipelago.jpg) background-size: cover .title-slide[ # Joint Visual Attention<br>and Collaboration<br>in Minecraft .authors[ .author[  Chris Proctor .affiliation[ University at Buffalo SUNY ]] .author[  Dalia Antonia Caraballo Muller .affiliation[ University at Buffalo SUNY ] ]]] .refs[ Slides available at [chrisproctor.net/slides/2022-cscl-minecraft](https://chrisproctor.net/slides/2022-cscl-minecraft) ]