.title[ ## SUPPORTING<br> CRITICAL COMPUTATIONAL LITERACIES<br> THROUGH INTERACTIVE STORYTELLING ] Chris Proctor<br> December 16, 2019<br> University at Buffalo <img src="/images/slides/brands/gse.png" alt="Stanford GSE" style="width:160px;"> <img src="/images/slides/sketch_4_squat.png" style="width:425px;position:absolute;right:0;bottom:0;"> <!-- <img src="images/slides_qr.png">--> .refs[ chrisproctor.net/slides/2019-job-talk-buffalo<br> Illustrations by Chris Proctor, based on fieldnotes and video ] ??? Hi, I'm Chris Proctor, a former secondary CS and English teacher, currently finishing my PhD in Learning Sciences and Technology Design at Stanford University. Today I would like to share with you my research on Supporting Critical Computational Literacies through Interactive Storytelling. If you would like to follow along on your device, the slides are avialable on my website. --- .full-bleed[ .half-left[<img height="100%" src="/images/slides/sketch_2_pointing.png">] .half-right[ .half-frame[ # <span style="color:var(--sketch-line-purple);">Outline</span> Background on my research Comparing Cognitive and Situated Assessments of Learning in Middle School Computer Science Future plans & funding ]]] ??? I'm going to tell you a bit about my broader research trajectory, and then share one study with you in detail. I will close by telling you about my future plans and how I plan to raise funds for them. --- background-position: top background-image: url(/images/slides/sketch_1_designing_futures.png) background-size: contain .bottom-third[ .half-frame[ ### <span style="color:var(--sketch-line-ruby);">Vision</span><br> An educational system which equitably supports all youth in designing and realizing possible futures by cultivating critical computational literacies ] ] ??? My research is guided by a vision of an educational system which equitably supports all children in designing and realizing possible futures by cultivating critical computational literacies. The language about possible futures I draw from the Connected Learning Framework. I will tell you more about "critical computational literacies" in a moment. --- ## Problem Many rationales for K-12 Computer Science .cite[1, 2, 3, 4] But what is Computer Science? And how will we teach it? .cite[5, 6] Instead of one answer, there will be a "patchwork." .cite[7] K-12 education and CS have much to gain from each other, but communicating across disciplines is hard. Learning Sciences may play a central role, with important lessons from Literacies. .cite[8, 9] .refs[ .anchor[1] Papert (1980) .anchor[2] Wing (2006) .anchor[3] Blikstein (2018) .anchor[4] Santo, Vogel, & Ching (2019) .anchor[5] Barr & Stephenson (2011) .anchor[6] Proctor, Bigman, & Blikstein (2018) .anchor[7] Sfard (1998) .anchor[8] Vossoughi & Gutiérrez (2016) .anchor[9] Vakil (2018) ] ??? This is in the context of our educational system which today remains deeply inequitable and which is still largely unresponsive to the digital transformation of our society. People have been dreaming of K-12 computer science for half a century, but we have seen a dramatic increase of interest over the last few years. There are many different rationales for teaching K-12 Computer Science, including economic opportunities, democratic participation, and social justice. There are two core questions--what is CS and how will we teach it. For a long time, the CS ed community has stressed the need for an "operational definition" of CS around which we can organize curricula, teacher licensing, and other resources. However, with my background as an English teacher, I am skeptical about the project of defining the right way to do sensemaking and communication. Efforts to narrowly define Standard English have done a lot of harm in marginalizing nondominant language practices. Instead, there will be a patchwork of understandings, reflecting the diversity of communities which practice various forms of computational literacy. This is broadly where CS and K-12 have a lot to learn from each other--CS has been grounded in university; K-12 grounded in diverse communities where schooling means many things, not always opportunity. This is where the learning sciences, with their focus on learning in context, have an important role to play. And literacies, with important perspectives on identity, power, and privilenge. --- ## Critical computational literacies .cite[1, 2] <img width="100%" src="images/literacy_axes_clean.png"> .refs[ .anchor[1] Lee & Garcia (2014) .anchor[2] Vee (2017)<br> *situated cognition*: Brown, Collins, & Duguid, (1989); Collins & Greeno (2011)<br> *material intelligence*: diSessa (2001) ] ??? The central construct in my research is "critical computational literacies." Literacy is a widely-used construct in education, so I want to explain what I mean by it. Broadly, I see literacy as closely-aligned with other learning sciences constructs for thinking about how our practices and identities are shaped by context and medium. For example, activity systems, situated cognition, figured worlds, distributed and embodied cognition. There are two main reasons why I prefer literacy: first because it builds a bridge to critical work from Litearcies, and second because it emphasizes the need for dialogue between different frames. I think of literacy in terms of two dimensions. The first dimension is radial, different scales of contextualized practice. The everyday idea of literacy is knowing how to read and write. However, we know that reading and writing involves more than just the cognitive skills of getting meaning from text and encoding meaning into text. It means participating in a community which practices reading and writing in particular ways, and which value particular identities. Becoming a lawyer means learning how to read and write legal text in the world of the law, and how to perform a lawyerly identity. You can be disbarred if you don't act the way lawyers are supposed to act. Zooming out even further, these worlds of practice do not have any sort of objective legitimacy. To be clear, there is no objectively right way to do English. Instead, the recognition and normative legitimacy of literacy practices is contested and political. Some people argue that the primary social function of literacy is to enact and justify hierarchy. When I talk about critical literacy, I mean becoming aware that our worlds of practice, the systems of signification and value, and the identites enacted within them, could be otherwise. When literacy is oppressive, this is the first step to finding strategies for resistance. - Second axis: diSessa's material intelligence So that's the first, radial, axis of literacy. The second axis maps practice to infrastructure. Here, we need to expand our notion of literacy again, because we read and write with many different technologies. I'm most interested in print text, computer code, and digital interfaces implemented by code. We can think about the relationship between practice and infrastructure at each level of practice. Just as there are specific cognitive skills associated with reading and writing, so there are different specific cognitive skills associated with computer programming. Andy diSessa calls this ability to interact with an external literacy medium "material intelligence", and he distinguishes between cognitive material intelligence and social material intelligence. We can also trace how critical practice is shaped by its infrastructure. So these two axes are how I think about literacy. This is informed by congruent work by Jim Wertsch, Paulo Freire, Annette Vee, Jim Gee, Andy diSessa, Roz Ivanic, Kris Gutierrez, and especially Dorothy Holland. --- ### Three framings of computational thinking <img src="images/icer_fig2.png" style="width: 550px; margin: -30px auto;" class="center"> .refs[ Kafai, Proctor, & Lui (2019) <br>International Computing Education Research **Best Paper Award** ] ??? One way this framework has been useful is in helping to sort out what we mean by "computational thinking." If you aren't familiar with this particular debate, as CS Ed has begun to get real traction at the K-12 level, computer scientists have been trying to articulate what it is about computing that makes it so powerful. Yasmin Kafai and I gave a paper, which won the Chair's award, at this year's International Computing Education Conference in Toronto. We argued that we can recognize three distinct framings of computational thinking: cognitive, situated, and critical, and that we need our research community to engage in theory dialogue between them. The vast majority of research, assessments, and pedagogy in CS education is cognitive. As a former English teacher, I am aware of how powerful it can be to weave together all three framings. --- ## Unfold Studio Web application for interactive storytelling using Ink .cite[1] programming language Print text and social media affordances support connecting computation to existing literacies, enact *literacy place* .cite[2, 3] Workshops at Computer Science Teachers Association, MozFest, Philly National Writing Project. Used in 6 states, internationally; nearly 10k student-authored stories <img src="images/literacy_axes_unfold.png" class="center" style="margin:0 auto; width: 400px;"> .refs[ .anchor[1] Inkle (2016) .anchor[2] Gee (2004) .anchor[3] Dourish (2006) ] ??? So, what might literacy-based computer science education look like? There has been a lot of research on interfaces for beginners, but not for these goals. I wanted a medium that supported integrating computation into print text and social media, which could create communities of computational practice grounded in the identities and cultures that already matter to them. My dissertation comes at the tail end of three years of design-based research exploring what kinds of tools and what kinds of pedagogy might support literacy-based computer science. Through this work, I developed a web application for interactive storytelling called Unfold Studio. In Unfold Studio, students can write interactive stories in a programming language called Ink, which you can see on the left. They enact choose-your-own-adventure-style games, which you can see running on the right. Users of Unfold Studio can write and share stories, follow users, and keep up with their community via a social feed. --- background-position: top background-image: url(/images/slides/unfold.png) background-size: contain ??? Here is a story written by a high school senior in a sociology class. Note the three framings of practice. Note the infrastructural affordances. This goes beyond personally-meaningful projects, to projects with rhetorical and critical possibilities. "Critical discourse model" (Proctor & Blikstein, 2019). At the same time, builds on familiar infrastructural affordances. Has been used in 6 US states; almost 10k student stories. --- ## Dissertation overview <img src="images/conjecture_mapping_0.png" width="100%" style="margin-top: -50px;"> .refs[ *Conjecture mapping*, Sandoval (2014) ] ??? The Learning Sciences are fundamentally concerned with learning as it happens in real life. The axes I described before are focused on the sociocultural context of learning and the way learning is mediated. Conjecture mapping is a way of A bit on DBR One point we made in our ICER paper was that a lot of CS Ed Research only connects the design to the mediating processes--for example, showing that course grades went up--without making theoretically-grounded claims about learning. --- count: false ## Dissertation overview <img src="images/conjecture_mapping_1.png" width="100%" style="margin-top: -50px;"> .refs[ *Conjecture mapping*, Sandoval (2014) ] --- count: false ## Dissertation overview <img src="images/conjecture_mapping_2.png" width="100%" style="margin-top: -50px;"> .refs[ *Conjecture mapping*, Sandoval (2014) ] --- count: false ## Dissertation overview <img src="images/conjecture_mapping_3.png" width="100%" style="margin-top: -50px;"> .refs[ *Conjecture mapping*, Sandoval (2014), *Figured worlds*, Holland, et al. (1998) ] --- count: false ## Dissertation overview <img src="images/conjecture_mapping_4.png" width="100%" style="margin-top: -50px;"> .refs[ *Conjecture mapping*, Sandoval (2014), *Figured worlds*, Holland, et al. (1998) ] --- count: false ## Dissertation overview <img src="images/conjecture_mapping_5.png" width="100%" style="margin-top: -50px;"> .refs[ *Conjecture mapping*, Sandoval (2014), *Figured worlds*, Holland, et al. (1998) ] --- ## Context 10-week classroom study (27 hours total) set in "Riverton," a small urban/rural midwestern city. 5 sections of 6th grade students, 50/149 students participating in research. Worked closely with 2 teachers & district CS education specialist. School population: 57% white, 25% black, and 10% two or more races. Title I (55% of students eligible for free or reduced lunch). Tenth percentile on state test results. Very little prior exposure to CS. <img src="images/cs_ed_research_chloropleth.png" class="center" style="margin: -20px auto; width: 400px;"> .refs[ *Tweet* @CSEdResearch (2019, December 12) ] ??? The students had computer science for an 80-minute block period two or three times a week, for a total of 27 hours of classroom time for each student. The school’s students are reported as 57% white, 25% black, and 10% two or more races. 55% of students are eligible for free or reduced lunch and the school is in the tenth percentile for state test results. 50 out of 149 students across six sections, participated. Very few had prior exposure to computer science. Very little research takes place in this kind of context, and I think my literacy framework is particularly important for studying the challenges of actually connecting these students with opportunity through CS. <!-- ## Scratch If you're not a Learning Scientist, I would like to say a word about design-based reserach. DBR is a research methodology which emerged from the Learning Sciences' desire for external validity. Design-based research aims to solve practical problems--innovating and improving learning environments-- but it also makes theoretical contributions. If we understand learning as taking place in complex systems, everything potentially affects everything else in nonlinear ways. Sometimes this can be decomposed into subsystems which can be modeled in a manageable way, but often they can't. I think it has particular promise for CS education is in the relationship between theoretical and practical problems. DBR gives us design research with theoretically-grounded outcomes, and a way of making theory actionable. TODO: Conjecture mapping (Sandoval, 2014) --> --- class:full-bleed-layout invert-slide-number-color <iframe src="https://research.unfold.studio/stories/8606" style="width: 100%; height: 100%; border: none;"> </iframe> ??? Let's see what this looks like in practice. Here is a story, "Egg-Hatching Simulator," by zdev. You can see the code (left) and the running story (right) of “Egg Hatching Simulator,” a story by zdev (a pseudonym chosen by the student). In this game the player hatches new pets from eggs, inspired by Pokémon. While it is not necessary to read the code in detail, the code does illustrate two elements of syntax which will be analyzed later. Divert statements (->) redirect the story’s flow to another part of the story. Lines beginning with a tilde (~) contain code which interacts with the execution environment, rather than emitting story output. Most often, code lines are used to manipulate state: initializing, updating and checking variables to keep track of what has happened in the story. The code excerpt in Figure 1 generates a random number between 0 and 1 and then cascades through cases to determine which pet the player receives. If the random number is above 0.999, the player sees “Soo, this is the secret pet. You got an Electric Shock. This is not meant to be in the game yet. If you hatch this and have proof EXAMPLE: Take Screenshot. Come find me, i will give you 10 Bear Paws.!” The story then redirects to the ending, which outputs, “If you made it to this, the Ending you are the luckiest person ever. The chances of hatching this were 1 in 1,000 (I think)............. Props to you!!!!!! .” This text would indeed be shown as output one time in a thousand. Therefore, this text is likely intended to be read by peers who choose to read the game’s source code in addition to playing. Important computational concepts are expressed and framed in the context of speaking to an audience of gamer-programmers, as insiders in-the-know. In positioning the player as being extremely lucky (“1 in 1,000”), zdev makes a probabilistic assertion grounded in a fairly complex code structure, and does so in an interactional context which positions him as an authoritative explainer and the reader as an interested colleague. In the rest of this paper we argue that these literacy interactions, in which students are positioned as authors and as audience, were the basis for a kind of computer science meaning-making for and with others. --- background-position: center background-image: url(images/unfold_zdev.png) background-size: contain class: invert-slide-number-color count: false ??? A static slide in case there's trouble loading Unfold Studio. Let's see what this looks like in practice. Here is a story, "Egg-Hatching Simulator," by zdev. You can see the code (left) and the running story (right) of “Egg Hatching Simulator,” a story by zdev (a pseudonym chosen by the student). In this game the player hatches new pets from eggs, inspired by Pokémon. While it is not necessary to read the code in detail, the code does illustrate two elements of syntax which will be analyzed later. Divert statements (->) redirect the story’s flow to another part of the story. Lines beginning with a tilde (~) contain code which interacts with the execution environment, rather than emitting story output. Most often, code lines are used to manipulate state: initializing, updating and checking variables to keep track of what has happened in the story. The code excerpt in Figure 1 generates a random number between 0 and 1 and then cascades through cases to determine which pet the player receives. If the random number is above 0.999, the player sees “Soo, this is the secret pet. You got an Electric Shock. This is not meant to be in the game yet. If you hatch this and have proof EXAMPLE: Take Screenshot. Come find me, i will give you 10 Bear Paws.!” The story then redirects to the ending, which outputs, “If you made it to this, the Ending you are the luckiest person ever. The chances of hatching this were 1 in 1,000 (I think)............. Props to you!!!!!! .” This text would indeed be shown as output one time in a thousand. Therefore, this text is likely intended to be read by peers who choose to read the game’s source code in addition to playing. Important computational concepts are expressed and framed in the context of speaking to an audience of gamer-programmers, as insiders in-the-know. In positioning the player as being extremely lucky (“1 in 1,000”), zdev makes a probabilistic assertion grounded in a fairly complex code structure, and does so in an interactional context which positions him as an authoritative explainer and the reader as an interested colleague. In the rest of this paper we argue that these literacy interactions, in which students are positioned as authors and as audience, were the basis for a kind of computer science meaning-making for and with others. --- .full-bleed[ .half-left[ .half-frame[ # <span style="color:var(--sketch-line-cornflower)">Research questions</span> 1. Is participation in interactive story-based literacy associated with computer science learning? - Authorship? - Audience? 2. If so, is this association mediated by individual student practice in writing their own stories? ]] .half-right[<img height="100%" src="/images/slides/sketch_0.png">] ] ??? With this context, I would like to share with you a particular research study. Late-state design-based research tends to move more toward an emphasis on instumentation and research efficiency (CITE Schwartz...) I'm focused on two research questions here: 1. Is participation in interactive story-based literacy associated with computer science learning? 2. If so, is this association mediated by individual student practice in writing their own stories? --- background-position: center background-image: url(images/rq1_path.png) background-size: contain ## Research question 1 ??? Here's RQ1 --- background-position: center background-image: url(images/rq2_path.png) background-size: contain ## Research question 2 ??? And here's RQ2. --- ## Data source: CS learning Interactive story portfolio submission, assessed using a rubric aligned with K-12 CS Framework's .cite[1] Control and Variables concepts. <style>table {border-collapse: collapse;} th, td {border: 2px solid; font-size: 0.8em;padding: 4px;} .rubric {font-family: Garamond;}</style> .rubric[ Level | Control criteria | Variable criteria ----- | ------------- | -------------- Advanced<br> (4 points) | Meets criteria for Proficient AND use of control adds meaning to the story. Uses an advanced control structure. | Meets criteria for Proficient AND use of state adds meaning to the story. Uses at least one declared variable. Proficient<br> (3 points) | Uses diverts correctly and meaningfully to control story execution. | Uses variables (either built-in or declared) to keep track of something in the story and using it to change what happens in the future. Basic <br>(2 points) | The use of control might be based closely on another story. The use of control might “check the boxes” but not have much effect on the story. May include minor errors in usage. | The use of state might be based closely on another story. The use of state might “check the boxes” but not have much effect on the story. May include minor errors in usage. Below basic (1 point) | Does not meet criteria for Basic. | Does not meet criteria for Basic. ] .refs[ .anchor[1] *K-12 CS Framework* (2016) ] ??? Don't need to read the whole table. But this is what I used to assess students' summative performance. Students were familiar with the rubric; we worked with it in class. --- .full-bleed[ .half-left[<img height="100%" src="images/literacy_log.png">] .half-right[ .half-frame[ ## Data Source: Literacy participation 500k events from Unfold Studio log Build literacy graph: interactions between users and stories Author score: Number of other user interactions with a user's stories Audience score: Number of interactions with other users' stories ]]] ??? We operationalize literacy events as actions taken by users in the process of reading and writing stories, as well as browsing, searching, following other users, and commenting on stories. In this study, we consider only those literacy events in which one user views, loves, or forks (makes a copy of) another user’s story. These interactions feature two important, reciprocally-connected roles, those of author and audience. As described in the background, we view these as important learning opportunities within a literacy place grounded in, but extending beyond, the classroom. Each literacy event can be considered as a link in a bipartite network of authors and stories. We define a user’s author score as the number of literacy events in which another user interacted with one of the user’s stories. Similarly, a user’s audience score is the number of literacy events in which that user interacted with a story written by another user. Figure 2 shows a histogram of participants’ author and audience scores. Thirty of the fifty study participants have author scores of zero because they chose not to make any of their stories publicly visible to their peers. (While this group wrote fewer stories on average than authors with positive author scores, they still wrote an average of 8 stories, including stories for the summative portfolio.) Note that the sums of all author and audience scores are not equal because these scores consider interactions with all "Literacy App" users. Some participants wrote stories which became popular on the site beyond the classes involved in this study, and they were occasionally inspired by stories written by external authors. For example, a student at another school wrote a story in which the player walks through an imagined monument to LGBTQ heroes from history. Several students pointed this story out as they were planning their own writing. --- class: full-bleed-layout <video controls autoplay="true" src="images/literacy_graph.mp4" width="100%" style="margin-top: -20px;"> .caption-bottom-left[ <span style="color:#833457;">⬤</span> User<br> <span style="color:#715884;">⬤</span> Story ] --- ## Authorship and audience as forms of participation <img src="images/author_audience_hist_a.png" width="100%"> ??? We operationalize literacy events as actions taken by users in the process of reading and writing stories, as well as browsing, searching, following other users, and commenting on stories. In this study, we consider only those literacy events in which one user views, loves, or forks (makes a copy of) another user’s story. These interactions feature two important, reciprocally-connected roles, those of author and audience. As described in the background, we view these as important learning opportunities within a literacy place grounded in, but extending beyond, the classroom. Each literacy event can be considered as a link in a bipartite network of authors and stories. We define a user’s author score as the number of literacy events in which another user interacted with one of the user’s stories. Similarly, a user’s audience score is the number of literacy events in which that user interacted with a story written by another user. Figure 2 shows a histogram of participants’ author and audience scores. Thirty of the fifty study participants have author scores of zero because they chose not to make any of their stories publicly visible to their peers. (While this group wrote fewer stories on average than authors with positive author scores, they still wrote an average of 8 stories, including stories for the summative portfolio.) Note that the sums of all author and audience scores are not equal because these scores consider interactions with all "Literacy App" users. Some participants wrote stories which became popular on the site beyond the classes involved in this study, and they were occasionally inspired by stories written by external authors. For example, a student at another school wrote a story in which the player walks through an imagined monument to LGBTQ heroes from history. Several students pointed this story out as they were planning their own writing. --- count:false ## Authorship and audience as forms of participation <img src="images/author_audience_hist_b.png" width="100%"> --- .full-bleed[ .half-left[ .half-frame[ ## Data source: Stories <img src="images/state_flow_hist_vertical_a.png" style="margin-top: -24px;"> ]] .half-right[<img width="100%" src="images/unfold_zdev_code_focus_a.png" style="margin-top: 36px;">] ] ??? Finally, we consider the content of students’ interactive stories, which are the primary artifacts created on "Literacy App". Over the course of the unit, the 48 authors participating in the research wrote 640 stories. In this study, we conduct static program analysis of the code from the final state of each story. (The left half of Figure 1 shows an excerpt of a story’s code.) Following a common strategy of counting syntactic elements which map to concepts (e.g. Brennan & Resnick, 2012; Fields, et al., 2016), we count the use of syntactic elements which correspond to flow and state, the two primary content knowledge goals of the unit. We chose to count the number of diverts in each story as a measure of practicing flow. An interactive story can be visualized as a directed graph, where each knot, or chunk of textual content, is connected to other knots by edges. Each divert (->) implements an edge, so the number of diverts in a story corresponds to the number of edges in its story graph. We defined a students' flow practice score as the logarithm of the maximum number of diverts in any of an author’s stories. (Using the sum across an author’s stories would be artificially inflated when authors repeatedly forked their own stories, and using an average would be artificially deflated for authors who made numerous throwaway stories for notes or to test out constructs.). We conducted a similar analysis for stories' use of state which will be reported in a subsequent publication. --- count:false .full-bleed[ .half-left[ .half-frame[ ## Data source: Stories <img src="images/state_flow_hist_vertical_b.png" style="margin-top: -24px;"> ]] .half-right[<img width="100%" src="images/unfold_zdev_code_focus_b.png" style="margin-top: 36px;">] ] ??? --- ## RQ1 Results Is participation in interactive story-based literacy associated with CS learning? <img src="images/technical_score_regplots_a.png" width="100%"> ??? Our first research question asks whether there is an association between participation in the literacy place, either as author or as audience, and performance on the summative assessment. Using standard OLS regression, we found a statistically-significant association between both author and audience scores and summative performance. Plots of these associations are shown in Figure 3 and regression tables are shown in Table 2. We additionally tried several models including measures of students’ prior interest and experience with Computer Science and English/Language Arts (using the survey instrument from blinded). These covariates both had statistically-significant associations with summative performance. However, when they were added to the models shown in Figure 3 and Table 2, author and audience scores remained statistically-significant and their coefficients did not change much. Therefore, we do not include these covariates in the following results. Figure 3 shows the positive association between technical score and both author score and audience score. Students who participated more in the literacy place, as authors and as audience, tended to have higher scores on the summative assessment of Computer Science content. This suggests that writing for an audience, or participating as an audience of others’ work, was associated with better performance on the technical summative assessment. There was a substantial correlation between author score and audience score (r2 = 0.36), which explains the collapse of model (3) in Table 2 due to collinearity. In other words, students with high author scores were reasonably likely to also have high audience scores. Intuitively, this is not surprising, as we hypothesize that these are reciprocal, dialogic relationships. --- count:false ## RQ1 Results Is participation in interactive story-based literacy associated with CS learning? <img src="images/technical_score_regplots_b.png" width="100%"> --- background-position: center background-image: url(images/results_1_path.png) background-size: contain ## RQ1 Results .caption-bottom-left[ <span style="width: 30px;display:inline-block;">\*</span>p < 0.1<br> <span style="width: 30px;display:inline-block;">\*\*</span>p < 0.05<br> <span style="width: 30px;display:inline-block;">\*\*\*</span>p < 0.01<br><br> ] --- background-position: center background-image: url(images/results_2_path.png) background-size: contain ## RQ2 Results .caption-bottom-left[ <span style="width: 30px;display:inline-block;">\*</span>p < 0.1<br> <span style="width: 30px;display:inline-block;">\*\*</span>p < 0.05<br> <span style="width: 30px;display:inline-block;">\*\*\*</span>p < 0.01<br><br> ] --- background-position: center background-image: url(images/results_author_context_path.png) background-size: contain ## Authorship and CS learning<br>Controlling for prior interest .cite[1] .caption-bottom-left[ <span style="width: 30px;display:inline-block;">\*</span>p < 0.1<br> <span style="width: 30px;display:inline-block;">\*\*</span>p < 0.05<br> <span style="width: 30px;display:inline-block;">\*\*\*</span>p < 0.01<br><br> ] .refs[ .anchor[1] Friend (2015) ] --- background-position: center background-image: url(images/results_audience_context_path.png) background-size: contain ## Audience and CS learning<br> Controlling for prior interest .cite[1] .caption-bottom-left[ <span style="width: 30px;display:inline-block;">\*</span>p < 0.1<br> <span style="width: 30px;display:inline-block;">\*\*</span>p < 0.05<br> <span style="width: 30px;display:inline-block;">\*\*\*</span>p < 0.01<br><br> ] .refs[ .anchor[1] Friend (2015) ] --- ## Discussion (RQ1) Literacy participation, as an author and as audience, was associated with better performance on a summative assessment of Computer Science content. (RQ2) Both associations were significantly mediated by flow practice. Supports hypothesis that a literacy-based approach to introductory Computer Science can be an effective learning environment. Part of a larger argument that this can also support criticality. ??? In this paper, we have shown that literacy participation, as an author and as audience, was associated with better performance on a summative assessment of Computer Science content. Furthermore, we showed that both associations were mediated by flow practice, a measure of students individually engaging with computer science concepts in their own stories. These results support our broad hypothesis that a literacy-based approach to introductory Computer Science can be an effective learning environment. The fact that these results were not substantially affected by the inclusion of covariates measuring students’ prior interest in Computer Science and writing suggests that this approach could be particularly effective for broadening participation in computing practice. Indeed, in the exit survey, numerous students related that they had not expected to enjoy programming. Even though these associations remain when controlling for prior interest in Computer Science and English/Language Arts, it is possible that we have missed hidden variables accounting for both students’ participation and their scores on the summative assessment. Moreover we have so far only provided a sketch of an argument for these results’ external validity. Our next steps will involve rigorous qualitative analysis showing that the measures used here accurately describe students’ experiences of participation and learning. In the course of this analysis, we developed author score and audience score as measures of participation in the classroom literacy place. In this study, we treated these as contextual factors and the summative portfolio evaluation as the final measure of learning. In that respect, this study is similar to other research showing the importance of sociocultural factors. However, we can also view these results as showing alignment between a traditional cognitive (or competency-based) measure of learning, and two measures based on students' participation in a community of computational practice. In future research, we intend to center participation in a community of practice as a primary form of learning, producing quantitative measures which can be held up against cognitive assessments. The challenge then will be to justify that the participation, the community of practice, and participants' enacted identities are legitimate forms of Computer Science. Social learning analytics combined with qualitative analysis will be invaluable tools in this task, as they will provide a high-granularity view of the nature of students' practice. It seems likely that author and audience scores are a coarse view on emergent dynamics in students’ trajectories of participation; our future research will further explore these dynamics. --- ## Implications Author and audience scores likely a coarse view on emergent dynamics in students’ trajectories of participation. Future work using learning analytics, social network analysis. There has been a lot of research showing **that** sociocultural factors are important, but very little quantititative research assessing participation as learning. In this study, we still used a cognitive measure as the outcome variable. Future research might assess participation and identity-building as outcomes in and of themselves. ??? TODO clean up the text here --- class: center background-position: top background-image: url(/images/slides/sketch_3_talk.png) background-size: contain .bottom-third.center[ # <span style="color:var(--sketch-line-green);">Next steps</span> ] ??? Now I would like to turn to where I'm going. I will tell you about emergent projects, stuff I'm excited to keep working on--and then turn to how I plan to fund them. --- background-image: url(images/debugging_markov.png) background-size: contain ## Debugging ??? In closing, I would like to show a few snapshots of other projects. You will probably notice a few similar themes. This is a wireframe from participatory design research project in which student journalists developed a mobile app to distribute their content and reconfigure the student body's news space. --- background-image: url(images/transliteracies.png) background-size: 100% 100% class: invert-slide-number-color ## <span style="color:white;">Computational & <br>Spatial Literacies</span> --- ## Student Journalism & Mobile Media <img width="100%" src="images/student_journalism.png"> --- ## Making with Code: A Constructionist ninth-grade CS course <img width="100%" src="images/constructionism.png" style="margin-top: -40px;"> ??? --- ## Funding: Strategic plan **Vision** An educational system which equitably supports all youth in designing and realizing possible futures by cultivating critical computational literacies **Research Agenda** - Efficacy, effectiveness, and scale-up research of Unfold Studio - Feedback loop within university CS Ed community of practice - Work with local schools, community organizations: pedagogy and tools **Next Steps** - Get to know local CS Ed on campus, in Buffalo, regionally. Join/develop partnerships. - Establish lab focused on pedagogy and tools for K-16 CS education - Target and apply for major grants ??? I was asked to talk about my plans for funding future research. I want to return to my vision statement, so that I can put my funding proposals in the service of long-range goals. Luckily for me, these are closely-aligned with long-term strategic plans of the NSF, IES, and numerous private foundations. --- ## Funding #### Grantwriting experience Year | Amount | Status | Source ---- | ------ | ------- | ------ 2016 | $8k | Awarded | TELOS (Stanford) 2016 | $17k | Awarded | MediaX (Stanford) 2017 | $4k | Awarded | Lopatin Fellowship (Stanford) 2017 | $138k | Hon. Mention | NSF Graduate Research Fellowship 2019 | $8k | Awarded | TELOS (Stanford) 2019 | $10k | Awarded | SPLICE (NSF) (with others) 2019 | $1M | Finalist | Spencer Foundation (with Prof. Dan McFarland) #### Other experience - 2015-2019 Member of Paulo Blikstein's TLTLab and FabLearn. Funding from NSF, Stanford, foundations, schools (within US and international), private grants. - 2019 Advisory board member, $500k NSF EHR Track III: Research on STEM Workforce Development (in submission). Awarded $50k from Facebook Research. --- ## Funding: Future sources - NSF - CAREER - STEM Learning and Learning Environments (EHR) - Broadening Participation in STEM Fields (EHR) - Building Capacity for STEM Education Research (EHR) - STEM + Computing K-12 Education (EHR) - CS4All:RPP (DLR) - Discovery Research PreK-12 (EHR-DLR) - INCLUDES Alliance (CISE-CNS) - IES - National Center for Education Research unsolicited proposal - Foundations: Spencer, Sloan, Ford, Grant, Hewlitt, Google, Facebook - State, local, and university grants ??? My strategic plan for the first year of my professorship: -- get to know campus resources (including plans for upgraded, refocused PhD cohort) -- get to know local schools (or less local if necessary), -- November: Have in place plans for NSF grants to pursue FOLLOW THE MONEY NSF 10 BIG IDEAS Future of Work at the Human-Technology Frontier NSF INCLUDES --- ### A turning point for the Learning Sciences "We [the Computer-Supported Collaborative Learning research community] should be honest with ourselves: if we have not managed to achieve any measurable impact by this point, why should we believe that we will be able to do so in the future?".cite[1] <br><br><br> "We applaud this vision of a scientifically literate citizenry in which many and diverse people act in socially compassionate and democratically responsible ways. However we disagree with the implicit assumption that teaching key concepts and methods will necessarily lead to socially responsible use or to a larger and more diverse citizenry who participate in discussion and debate of scientific issues." .cite[2] .refs[ .anchor[1] Wise & Schwarz (2017) .anchor[2] Eisenhart, Finkel, & Marion (1996) ] ??? --- .title[ ## SUPPORTING<br> CRITICAL COMPUTATIONAL LITERACIES<br> THROUGH INTERACTIVE STORYTELLING ] Chris Proctor<br> December 16, 2019<br> University at Buffalo <img src="/images/slides/brands/gse.png" alt="Stanford GSE" style="width:160px;"> <img src="/images/slides/sketch_4_squat.png" style="width:425px;position:absolute;right:0;bottom:0;"> <!-- <img src="images/slides_qr.png">--> .refs[ cproctor@stanford.edu<br> chrisproctor.net/slides/2019-job-talk-buffalo ] ??? Now, I would be delighted to take your questions.