Title: 2018 SFUSD Research
Date: 2018-07-30
template: slides

class: center, middle

#San Francisco Unified School District
Chris Proctor<br/>
Stanford Graduate School of Education<br>
cproctor@stanford.edu

.refs[
Slides & resources at chrisproctor.net
]

---
# Research Overview:<br>Toward critical computational literacies

- Longitudinal studies on emergence of computational literacies
    - 3-year study of middle-school computer science learners
    - Follow-up interviews documenting the transition to high school
- Computer Science implementation research
    - Case study of a school district designing a K-12 CS program
    - How talk shapes stereotypes on Hacker News
- Design-based research developing tools to support computational literacies
    - Unfold Studio: a platform for interactive storytelling
    - A mobile app to support student journalism

---
## How Broad is Computational Thinking?<br/>A Longitudinal Study of Practices Shaping Learning in Computer Science.cite[1]

- Longitudinal study of a school cohort (N=48) across a three-year computer science curriculum in grades 6-8.
- Analyzed students' Scratch projects in terms of elaboration and computational thinking content
- Modeled association of Scratch project metrics with performance on a summative open-ended assessment of computational thinking.

.refs[
.anchor[1] Proctor & Blikstein, 2018
]

---
# Scratch Project Metrics

Static code analysis of Scratch projects.cite[1]

<table class="rubric">
<tr><td>Elaboration</td><td>log(# blocks)</td></tr>
<tr><td>Computational content</td><td>(# data, events, control, sensing, and function blocks)/(# blocks).cite[2]</td></tr>
</table>

.refs[
.anchor[1] Brennan & Resnick, 2012; Boechler, et al, 2014; Fields, et al, 2016; Moreno-Leon, et al, 2017
.anchor[2] Brennan & Resnick, 2012
]

???
Is automatic assessment legit? Moreno-Leon, et al, 2017 found a close correlation between automatic assessment
and expert evaluation. The teacher's assessment was coarser, but also followed the same trend.

I'm going to show you a couple of examples from different ends of the distribution

Literacies perspective (which attends to identity and culture) may be associated with long-term gains.
   Equity means critical questioning of CS practices, not just efforts at inclusion. 
   Need to ensure continuity of offerings and equity support, or gains may be lost.

---
.full-bleed[
.half-left[<img src="/media/slides/scratch_portrait_code.png" style="height:100%;">]
.half-right[
.half-frame[
## Low computational content (z= -2.03),<br/>high elaboration (z=1.49)
<img src="/media/slides/scratch_portrait.png" style="width: 100%;">
]]]

???
Here's an example that might not look very successful from a computer science point of view--
there is very basic decomposition, but basically everything is an imperative list. At the same time, 
though, this took a lot of work. And any time an early adolescent is drawing a self-portrait, representing
her body position, facial expression, clothing, and gesture, she's going to be paying a lot of attention
to just where the lines go.

---
.full-bleed[
.half-left[<img src="/media/slides/scratch_house_code.png" style="height:100%;">]
.half-right[
.half-frame[
## high computational content (z=1.40),<br/>medium elaboration (z=0.34)
<img src="/media/slides/scratch_house.png" style="width: 100%;">
]]]

???
This example is a fairly formulaic drawing, almost exactly like the teacher's drawing. Much more attention
to crafting an elegant algorithm. Note, for example, how functions build up incrementally. This is a quality 
of some really nicely-written libraries, like underscore.js

---
# Results

<table class="regression">
<tr>
<th></th>
<th colspan="4" style="text-align: center;">Summative assessment rubric score</th>
</tr>
<tr class="overbar">
    <td>Computational content</td>
    <td>0.80*</td>
    <td></td>
    <td>0.57</td>
    <td>0.01</td>
</tr>
<tr>
    <td>Elaboration</td>
    <td></td>
    <td>1.21***</td>
    <td>1.10***</td>
    <td>1.01**</td>
</tr>
<tr>
    <td>Prior CT</td>
    <td></td>
    <td></td>
    <td></td>
    <td>-0.21</td>
</tr>
<tr>
    <td>6th grade quant evaluation</td>
    <td></td>
    <td></td>
    <td></td>
    <td>0.49</td>
</tr>
<tr class="overbar underbar">
    <td>Adj. r<sup>2</sup>
    <td>0.082</td>
    <td>0.244</td>
    <td>0.278</td>
    <td>0.300</td>
</tr>

.refs[
Note: \*p<0.05; \*\*p<0.01; \*\*\*p<0.001
]

---
# Elaboration over computational content

- Personally-meaningful projects, Computational literacy practices
- A broader definition of CT may be more useful for characterizing learners' early trajectories in CS.
- Future research will use students' reflective writing to interpret this finding.

---
# Crossing to high school (Ongoing)

- Follow-up interviews with the same cohort in 10th, 11th grade, at several different high schools
- Most schools provide few CS offerings and little identity support
- Most students 
- Students who continue with CS often cite a supportive mentor
- Some students contesting presumed gender identities

---
# Research Overview:<br>Toward critical computational literacies

- Longitudinal studies on emergence of computational literacies
    - 3-year study of middle-school computer science learners
    - Follow-up interviews documenting the transition to high school
- .new[Computer Science implementation research]
    - Case study of a school district designing a K-12 CS program
    - How talk shapes stereotypes on Hacker News
- Design-based research developing tools to support computational literacies
    - Unfold Studio: a platform for interactive storytelling
    - A mobile app to support student journalism
---
# Computer Science Implementation research (Ongoing)

- Interviews with committee members of a small school district, tasked with designing a K-12 CS curriculum
- Need to define a vision for what CS is going to be in your district.
- Need to attend to inclusion in this process. This might mean a process of educating people about CS so they can participate.

---
# How talk shapes stereotypes on Hacker News (Ongoing)
- Computational linguistics approach to studying how word meanings change within Hacker News, an onling forum for computer scientists and people interested in startups.
- Corpus: 12m comments over 10 years; 300k users.

---
<img src="/media/slides/good_evil.png" style="width:100%;">
---
<img src="/media/slides/black_white.png" style="width:100%;">

---
# Research Overview:<br>Toward critical computational literacies

- Longitudinal studies on emergence of computational literacies
    - 3-year study of middle-school computer science learners
    - Follow-up interviews documenting the transition to high school
- Computer Science implementation research
    - Case study of a school district designing a K-12 CS program
    - How talk shapes stereotypes on Hacker News
- .new[Design-based research developing tools to support computational literacies]
    - Unfold Studio: a platform for interactive storytelling
    - A mobile app to support student journalism

---
# Research questions
1. What kind of medium and pedagogy might support textual-computational multiliteracy?.cite[1]
2. What practices might emerge in such a textual-computational multiliteracy space?
3. How might such practices support critical awareness and resistance to racism, sexism, and 
other oppressive ideologies?.cite[2]

.refs[
.anchor[1] New London Group, 1996
.anchor[2] Fairclough, 1992; Morrell, 2015
<br/>Slides & resources at chrisproctor.net
]

???
Answers:
1. interactive storytelling webapp. Codesign at multiple levels:
    - affordances (unicode, line-wrap)
    - social features (private stories, feed)
2. critical discourse models, for one. (Go back to prior slide to show example)
    - As Deana mentioned, dialogic model helpful
    - Immersion and interactivity (microworlds)
        - Simon mentioned identifying discourse markers in text. These can be enacted in critical discourse models.
        - Shibani: rhetorical moves.
    - 4-layer analytical framework: affordances, techniques, figured meanings, critical possibilities
3. conceptual framework. The importance of critical computational literacy
    - particularly dialogue between Learning Sciences and Critical Literacies
    - This echoes the socio-cognitive / socio-cultural divide Hilde mentioned
---
# Literacy space
A network of actors collaborate to do semiotic work.
<img src="/media/slides/csta/diagrams/literacy_space.png" width="70%">

---
# Textual literacy
<img src="/media/slides/csta/diagrams/textual_literacy.png" width="70%">

---
# Computational literacy
<img src="/media/slides/csta/diagrams/computational_literacy.png" width="70%">

---
# Microworld Simulation
<img src="/media/slides/csta/diagrams/microworld.png" width="70%">

---
# Figured meaning
<img src="/media/slides/csta/diagrams/figured_meaning.png" width="70%">

---
# Model of personhood / Subjectivity
<img src="/media/slides/csta/diagrams/subjectivity.png" width="70%">

---
# Identity
<img src="/media/slides/csta/diagrams/identity.png" width="70%">

A model of personhood one authors and occupies in a literacy space, which exists at the interface "between intimate discourses, inner speaking, and bodily practices formed in the past and the discourses and practices to which people are exposed, willingly or not, in the present."

---
#Immersion
<img src="/media/slides/csta/diagrams/immersion.png" width="70%">

---
#Interactivity

<img src="/media/slides/csta/diagrams/interactivity.png" width="70%">
---
# Interactive storytelling
<img src="/media/slides/csta/diagrams/interactive_storytelling.png" width="70%">

---
# Conceptual framework

- Literacy as a form of situated learning.cite[1] in 
which a network of actors.cite[2] collaborate to do semiotic work.
- Figured meanings.cite[3] within literacy spaces.cite[4].
- Identities self-authored at the interface between unconstrained inner speech and an external subject position made 
available by social meanings.cite[5]

.refs[
.anchor[1] Collins & Greeno, 2011
.anchor[2] Latour, 2005
.anchor[3] Holland, et al, 2001
.anchor[4] Gee, 1990
.anchor[5] Vygotsky, 1980; Bakhtin, 1981; Holland, et al, 2001
]
---
# Sociology workshop

- Create an oppressive social world where the possibilities of speech are limited for the main character. 
- Create a world where the main character is enregistered: s/he is assigned a model of personhood based on the way s/he speaks and performs identity.
- Create a world where the main character subverts a model of personhood s/he is assigned

---
<iframe src="https://unfold.studio/stories/1063" style="width: 100%; height: 100%; border: none;">
</iframe>

---
<iframe src="https://unfold.studio/stories/1133" style="width: 100%; height: 100%; border: none;">
</iframe>

---
# Next study: Tracing collaborative writing practices

A cohort of 70 students (55 consented) used Unfold Studio in their CS class for 3 months. Captured:
- 700 stories; average of 23 revisions per story
- 500 social events: loving stories, following users, forking stories
- Server events: page navigation, session lengths
- summative assessment, modified Fairy assessment.cite[1]
- summative reflection

Three units of analysis: 
- How did individual stories develop?
- How did authors change develop?
- How did the community develop?

.refs[
.anchor[1] Werner, et al, 2012
<br/>Slides & resources at chrisproctor.net
]

---
# Student Journalism & Mobile Media

- 2+ year design-based research project to design a mobile app for a high school journalism program. Goals:
    - Connect with the whole student body
    - Express student voice
    - provide a forum for discussing difficult topics on campus
- Reserach goal: reconceptualize a school's relationship to its digital space

---
# Student Journalism & Mobile Media