Title: 2020 CSCL Guest Lecture
Date: 2020-04-07
template: slides

.title[
## COMPUTER-SUPPORTED COLLABORATIVE LEARNING AND COMPUTATIONAL PARTICIPATION
]

Chris Proctor 
April 7, 2020 
Teachers College, Columbia University

???
- Back-and-forth: Critique and design

---
.full-bleed[
.half-left[<img height="100%" src="/media/slides/2019-buffalo/sketch_2_pointing.png">]
.half-right[
.half-frame[
# Agenda

**Guest Lecture** Unfold Studio, CSCL theories and methods

**Breakout groups** Activity systems and design-based research in your work

**Discussion**

**Q&A**

]]]

---
## Introduction

- Many rationales for K-12 Computer Science .cite[1, 2, 3, 4]

- Sociocultural aspects of CS education (e.g. identity, practice, agency) central for equity .cite[5, 6]

- Not well-represented in quantitative CS Education research .cite[7]

.refs[
.anchor[1] Papert (1980) .anchor[2] Wing (2006) .anchor[3] Blikstein (2018) .anchor[4] Santo, Vogel, & Ching (2019) .anchor[5] Margolis & Fisher (2003) .anchor[6] Margolis (2008) .anchor[7] McGill, et al. (2019)
]

???

I'm going to structure this talk using a conventional academic argument. But my
goal is also to share some key theories and methods from CSCL

People have been dreaming of K-12 computer science for half a century, but we
have seen a dramatic increase of interest over the last few years. 
There are many different rationales for teaching K-12 Computer Science,
including economic opportunities, democratic participation, and social justice.

In my research, I consistently argue that sociocultural aspects are central but
understudied. Everyone needs identity support, but some people get it already.
When we ignore it in our research and just focus on results, it erases the
resource gap and contributes to inequity.

Thus the importance of quantitatve sociocultural research.

---
## Background: Unfold Studio

Web application for interactive storytelling using Ink .cite[1] programming language

Print text and social media affordances support connecting computation to existing
literacies, enact *literacy place* .cite[2, 3]

Workshops at Computer Science Teachers Association, MozFest, Philly National Writing Project. Used in 6 states, internationally; nearly 10k student-authored stories

.refs[
.anchor[1] Inkle (2016) .anchor[2] Gee (2004) .anchor[3] Dourish (2006)
]

???
So, what might literacy-based computer science education look like? There has
been a lot of research on interfaces for beginners, but not for these goals. 
I wanted a medium that supported integrating computation into print text and
social media, which could create communities of computational practice grounded
in the identities and cultures that already matter to them.

My dissertation comes at the tail end of three years of design-based research
exploring what kinds of tools and what kinds of pedagogy might support
literacy-based computer science. Through this work, I developed a web
application for interactive storytelling called Unfold Studio. In Unfold Studio, 
students can write interactive stories in a programming language called Ink,
which you can see on the left. They enact choose-your-own-adventure-style games,
which you can see running on the right. Users of Unfold Studio can write and
share stories, follow users, and keep up with their community via a social feed.

---
background-position: top
background-image: url(/media/slides/unfold.png)
background-size: contain

???
Here is a story written by a high school senior in a sociology class. Note the three framings of
practice. Note the infrastructural affordances. This goes beyond
personally-meaningful projects, to projects with rhetorical and critical
possibilities. "Critical discourse model" (Proctor & Blikstein, 2019). 
At the same time, builds on familiar infrastructural affordances.
Has been used in 6 US states; almost 10k student stories.

---
# Designing Unfold Studio

- Three iterations of design-based research.cite[1] in middle and high schools.
- Affordances and their design rationales

## Design-based research
- Method for studying what doesn't yet exist .cite[2]
- Complex system: Everything affects everything else .cite[3, 4]
- Moving from innovation to efficiency .cite[5]

.refs[
.anchor[1] Proctor & Blikstein, 2019 .anchor[2] Simon (1969) .anchor[3] Brown (1992) .anchor[4] Collins, Joseph, & Bielaczyc (2004) .anchor[5] Schwartz, Chang, & Martin (2005)
]

---
## Conjecture mapping .cite[1]
<img src="/media/slides/2019-buffalo/conjecture_mapping_0.png" width="100%" style="margin-top: -50px;">
.refs[
.anchor[1] Sandoval (2014)
]

---
## Background: Critical computational literacies .cite[1, 2]

.refs[
.anchor[1] Lee & Garcia (2014) .anchor[2] Vee (2017) 
*situated cognition*: Brown, Collins, & Duguid, (1989); Collins & Greeno (2011) 
*material intelligence*: diSessa (2001)
]

???
The central construct in my research is "critical computational literacies." 
Literacy is a widely-used construct in education, so I want to explain 
what I mean by it. Broadly, I see literacy as closely-aligned with other
learning sciences constructs for thinking about how our practices and identities
 are shaped by context and medium. For example, activity systems, situated
cognition, figured worlds, distributed and embodied cognition. There are two
main reasons why I prefer literacy: first because it builds a bridge to
critical work from Litearcies, and second because it emphasizes the need for
dialogue between different frames.

I think of literacy in terms of two dimensions. The first dimension is radial, different scales of 
contextualized practice. The everyday idea of literacy is knowing how to read and write. However, we
know that reading and writing involves more than just the cognitive skills of
getting meaning from text and encoding meaning into text. It means participating
in a community which practices reading and writing in particular ways, and which
value particular identities.

Becoming a lawyer means learning how to read and write legal text in the world
of the law, and how to perform a lawyerly identity. You can be disbarred if you
don't act the way lawyers are supposed to act.

Zooming out even further, these worlds of practice do not have any sort of
objective legitimacy. To be clear, there is no objectively right way to do
English. Instead, the recognition and normative legitimacy of literacy practices
is contested and political. Some people argue that the primary social function 
of literacy is to enact and justify hierarchy.

When I talk about critical literacy, I mean becoming
aware that our worlds of practice, the systems of signification and value, and
the identites enacted within them, could be otherwise. When literacy is
oppressive, this is the first step to finding strategies for resistance.

- Second axis: diSessa's material intelligence

So that's the first, radial, axis of literacy. The second axis maps practice to
infrastructure. Here, we need to expand our notion of literacy again, because we
read and write with many different technologies. I'm most interested in print
text, computer code, and digital interfaces implemented by code. We can think
about the relationship between practice and infrastructure at each level of
practice.

Just as there are specific cognitive skills associated with reading and writing,
so there are different specific cognitive skills associated with computer programming.
Andy diSessa calls this ability to interact with an external literacy medium 
"material intelligence", and he distinguishes between cognitive material
intelligence and social material intelligence. We can also trace how critical 
practice is shaped by its infrastructure.

So these two axes are how I think about literacy. This is informed by congruent
work by Jim Wertsch, Paulo Freire, Annette Vee, Jim Gee, Andy diSessa, Roz Ivanic, 
Kris Gutierrez, and especially Dorothy Holland.

---
### Three framings of computational thinking

.refs[
Kafai, Proctor, & Lui (2019)
 International Computing Education Research **Best Paper Award**
]

???
One way this framework has been useful is in helping to sort out what we mean by
"computational thinking." If you aren't familiar with this particular debate,
as CS Ed has begun to get real traction at the K-12 level, computer scientists
have been trying to articulate what it is about computing that makes it so powerful. 
Yasmin Kafai and I gave a paper, which won the Chair's award, 
at this year's International Computing Education 
Conference in Toronto. We argued that we can recognize three distinct framings of
computational thinking: cognitive, situated, and critical, and that we need our
research community to engage in theory dialogue between them.

The vast majority of research, assessments, and pedagogy in CS education is
cognitive. As a former English teacher, I am aware of how powerful it can be to
weave together all three framings.

---
## Background: Sociocultural alternatives to "literacy"

- Activity systems .cite[1]
- Figured worlds .cite[2]
- Learning ecologies .cite[3, 4]

.refs[
.anchor[1] Leont'ev (1974) .anchor[2] Holland, et al. (2001) .anchor[3] Barron,
2004) .anchor[4] Hecht & Crowley (2019)
]

---
background-position: top
background-image: url(/media/slides/2019-buffalo/sketch_1_designing_futures.png)
background-size: contain

.bottom-third[
.half-frame[
### Is participation in interactive story-based literacy associated with CS learning?.cite[1, 2]
]
]

.refs[
.anchor[1] Proctor (2019) .anchor[2] Proctor, Zheng, & Blikstein (2020)
]

---
.full-bleed[
.half-left[
.half-frame[
### RQ1: Does writing interactive stories support development of debugging practices?

]]
.half-right[<img height="100%" src="/media/slides/2019-buffalo/sketch_0.png">]
]

---
#RQ1
<img width="100%" src="/media/slides/2020_04_07_tc/rq1_debugging.png">

---
# RQ1: Results
<img width="100%" src="/media/slides/2020_04_07_tc/rq1_results.png">

---
.full-bleed[
.half-left[
.half-frame[
### RQ2: Does reading other stories support development of debugging practices?

]]
.half-right[<img height="100%" src="/media/slides/2019-buffalo/sketch_0.png">]
]

???
I'm going to present the better version

---
# RQ2: Results?
<img width="100%" src="/media/slides/2020_04_07_tc/rq2.png">

???
This is where mixed methods are necessary.

---
.full-bleed[
.half-left[<img height="100%" src="/media/slides/2019-buffalo/literacy_log.png">]
.half-right[
.half-frame[
## Data Source: Logs

500k events from Unfold Studio log

Build literacy graph: interactions between users and stories

Author score: Number of other user interactions with a user's stories

Audience score: Number of interactions with other users' stories

]]]

???
We operationalize literacy events as actions taken by users in the process of reading and writing stories, as well as browsing, searching, following other users, and commenting on stories. In this study, we consider only those literacy events in which one user views, loves, or forks (makes a copy of) another user’s story. These interactions feature two important, reciprocally-connected roles, those of author and audience. As described in the background, we view these as important learning opportunities within a literacy place grounded in, but extending beyond, the classroom. Each literacy event can be considered as a link in a bipartite network of authors and stories. We define a user’s author score as the number of literacy events in which another user interacted with one of the user’s stories. Similarly, a user’s audience score is the number of literacy events in which that user interacted with a story written by another user. Figure 2 shows a histogram of participants’ author and audience scores.

Thirty of the fifty study participants have author scores of zero because they chose not to make any of their stories publicly visible to their peers. (While this group wrote fewer stories on average than authors with positive author scores, they still wrote an average of 8 stories, including stories for the summative portfolio.) Note that the sums of all author and audience scores are not equal because these scores consider interactions with all "Literacy App" users. Some participants wrote stories which became popular on the site beyond the classes involved in this study, and they were occasionally inspired by stories written by external authors. For example, a student at another school wrote a story in which the player walks through an imagined monument to LGBTQ heroes from history. Several students pointed this story out as they were planning their own writing.

---
class: full-bleed-layout

<video controls autoplay="true" src="/media/slides/2019-buffalo/literacy_graph.mp4" width="100%" style="margin-top: -20px;">
.caption-bottom-left[
&#11044; User 
&#11044; Story
]

---
## Authorship and audience as forms of participation

---
count:false

## Authorship and audience as forms of participation

---
## RQ2 (v2) Results

Is participation in interactive story-based literacy associated with CS learning?

???
Our first research question asks whether there is an association between participation in the literacy place, either as author or as audience, and performance on the summative assessment. Using standard OLS regression, we found a statistically-significant association between both author and audience scores and summative performance. Plots of these associations are shown in Figure 3 and regression tables are shown in Table 2. We additionally tried several models including measures of students’ prior interest and experience with Computer Science and English/Language Arts (using the survey instrument from blinded). These covariates both had statistically-significant associations with summative performance. However, when they were added to the models shown in Figure 3 and Table 2, author and audience scores remained statistically-significant and their coefficients did not change much. Therefore, we do not include these covariates in the following results.

Figure 3 shows the positive association between technical score and both author score and audience score. Students who participated more in the literacy place, as authors and as audience, tended to have higher scores on the summative assessment of Computer Science content. This suggests that writing for an audience, or participating as an audience of others’ work, was associated with better performance on the technical summative assessment. There was a substantial correlation between author score and audience score (r2 = 0.36), which explains the collapse of model (3) in Table 2 due to collinearity. In other words, students with high author scores were reasonably likely to also have high audience scores. Intuitively, this is not surprising, as we hypothesize that these are reciprocal, dialogic relationships.

---
count:false

## RQ2 (v2) Results

Is participation in interactive story-based literacy associated with CS learning?

---
class: center
background-position: top
background-image: url(/media/slides/2019-buffalo/sketch_3_talk.png)
background-size: contain

.bottom-third.center[
# Conclusions
]

---
.title[
## COMPUTER-SUPPORTED COLLABORATIVE LEARNING AND COMPUTATIONAL PARTICIPATION
]

Chris Proctor 
April 7, 2020 
Teachers College, Columbia University

---
## Data source: CS learning

Interactive story portfolio submission, assessed using a rubric aligned with K-12 CS Framework's
.cite[1] Control and Variables concepts.

.rubric[
Level | Control criteria | Variable criteria
----- | ------------- | --------------
Advanced (4 points) | Meets criteria for Proficient AND use of control adds meaning to the story. Uses an advanced control structure. | Meets criteria for Proficient AND use of state adds meaning to the story. Uses at least one declared variable.
Proficient (3 points) | Uses diverts correctly and meaningfully to control story execution. | Uses variables (either built-in or declared) to keep track of something in the story and using it to change what happens in the future.
Basic (2 points) | The use of control might be based closely on another story. The use of control might “check the boxes” but not have much effect on the story. May include minor errors in usage. | The use of state might be based closely on another story. The use of state might “check the boxes” but not have much effect on the story. May include minor errors in usage.
Below basic (1 point) | Does not meet criteria for Basic. | Does not meet criteria for Basic.
]

.refs[
.anchor[1] *K-12 CS Framework* (2016)
]

???
Don't need to read the whole table. But this is what I used to assess students'
summative performance. Students were familiar with the rubric; we worked with it
in class.

---
.full-bleed[
.half-left[
.half-frame[
## Data source: Stories
<img src="/media/slides/2019-buffalo/state_flow_hist_vertical_a.png" style="margin-top: -24px;">
]]
.half-right[<img width="100%" src="/media/slides/2019-buffalo/unfold_zdev_code_focus_a.png" style="margin-top: 36px;">]
]

???
Finally, we consider the content of students’ interactive stories, which are the primary artifacts created on "Literacy App". Over the course of the unit, the 48 authors participating in the research wrote 640 stories. In this study, we conduct static program analysis of the code from the final state of each story. (The left half of Figure 1 shows an excerpt of a story’s code.) Following a common strategy of counting syntactic elements which map to concepts (e.g. Brennan & Resnick, 2012; Fields, et al., 2016), we count the use of syntactic elements which correspond to flow and state, the two primary content knowledge goals of the unit.

We chose to count the number of diverts in each story as a measure of practicing flow. An interactive story can be visualized as a directed graph, where each knot, or chunk of textual content, is connected to other knots by edges. Each divert (->) implements an edge, so the number of diverts in a story corresponds to the number of edges in its story graph. We defined a students' flow practice score as the logarithm of the maximum number of diverts in any of an author’s stories. (Using the sum across an author’s stories would be artificially inflated when authors repeatedly forked their own stories, and using an average would be artificially deflated for authors who made numerous throwaway stories for notes or to test out constructs.). We conducted a similar analysis for stories' use of state which will be reported in a subsequent publication.

---
count:false

.full-bleed[
.half-left[
.half-frame[
## Data source: Stories
<img src="/media/slides/2019-buffalo/state_flow_hist_vertical_b.png" style="margin-top: -24px;">
]]
.half-right[<img width="100%" src="/media/slides/2019-buffalo/unfold_zdev_code_focus_b.png" style="margin-top: 36px;">]
]