JLS Review of Utopian methodology

Review

This article proposes “utopian methodology,” theoretical and methodological extensions of design-based research (DBR) informed by efforts at radical social change. The authors make a compelling case for the urgency of this topic in their summary of existing critiques of DBR, most importantly that DBR too often aims at short-term, small-scale change irrelevant to the most important social and educational problems. This article is put in productive dialogue with recent Learning Sciences emphasis on activism and social movements. The article analyzes three long-term case studies of utopian design-based research projects set in Finland, Spain, and the United States. The findings are four methodological principles for a utopian methodology: organizing the design process around a utopian goal, creating the conditions for the goal’s enactment, analysis of challenges, and self-critique and collaborative redesign. The article’s claimed contributions are an integration of design-based research and cultural historical activity theory, and an argumentative grammar for a utopian methodology.

The proposed theory-building in the introduction (p. 3-6) is specific, well-documented, and quite exciting. I particularly liked how equity-oriented critiques of DBR were aligned with Adorno’s concerns that “concrete and positive suggestions for social change” were unlikely to succeed due to backlash and their tendency to be coopted or domesticated. This provides a clear path to the history of utopianism which follows.

However, I would have liked to see this thread further developed in the background section. Considering that “none of the three cases began as a self-consciously utopian methodological study” (p. 29), it feels important to clearly operationalize utopianism: what qualifies a project as utopian, and how prior utopian methodologies can be understood in terms of the definition used. The background subsection on CHAT (p. 7-8), consisting of two paragraphs, is also inadequate. I do not think we can assume a reader’s familiarity with CHAT; particularly considering the earlier reference to how Vygotsky’s revolutionary potential has been blunted, it would be helpful to head the authors’ construal of CHAT.

The article’s argument is based on synthesis of three long-term design-based research projects with disparate contexts and settings. While I do not think it succeeds in this draft, this kind of research synthesizing across heterogeneous contexts is much-needed in the Learning Sciences and I hope the authors will submit a revision. A central weakness in this draft is its relationship with prior work. Because this article is focused on theoretical and methodological development, it must rely on the analysis conducted elsewhere; there is not enough space here to fully-reiterate that analysis. However, the excerpts and summaries of prior work given here provide both too much detail and too little. Long excerpts of transcripts from case 1 or descriptions of organizational politics in case 3 provide too much detail in the sense that they are not sufficiently connected back to the article’s theory-building work. The reader doesn’t know what to do with these details. At the same time, there is too little detail in the sense that major contextual questions are left unanswered for a reader who has not read the referenced works.

One way of addressing this weakness would be to include a methods section contextualizing the case studies and explaining how their findings would be synthesized into a broader argument. Perhaps the ideas introduced in the introduction and background section could be used to analyze each case study in a consistent and methodological way, helping the reader understand the importance of each case study’s evidence to the broader argument. For example, each case study might clearly articulate its radial utopian goal (this is missing in all three cases) and then focus on the components, relations, and routines (Penuel’s (2018) “infrastructuring”) supporting its enactment. If CHAT has a relationship to how challenges are analyzed and critique and collaborative re-design is conducted, this could be more clearly theorized in the background and then analyzed in each case in turn.

As a final structural critique, the discussion only very-superficially addresses the framing set out in the beginning of the article, and there is no conclusion at all. The paper’s claimed contributions are reiterated in the discussion: “we continue the recently established fruitful dialogue between DBR and CHAT” and “specifying an argumentative grammar (Kelly, 2004) for a utopian methodology as a form of DBR.” However, neither CHAT nor an emergent structure of an argumentative grammar was mentioned in the case studies nor are examples from the case studies clearly synthesized in the discussion.

Overall, the article was thoughtful and well-written. Below, I note some more local issues I would encourage the authors to address in revision:

All three case studies are deeply engaged in racial projects but there is no theorization of race, either in terms of the nature of the racial utopian goal or in terms of how race is conceptualized as participating in the activity systems described. This lacuna underscored for me the absence of racial utopian projects in the background section.
In excerpt 1, major questions about positionality came up for me throughout the case study. Researchers are portrayed as very active participants and interpretants (e.g. “the research group was left with too few resources to assist Jaana” (p. 15-16)). What was Author 1’s role as a leader, organizer, and participant in the excerpts, and how were they accounted for theoretically and methodologically? Positionality is addressed in the final paragraph (p. 16), but these questions are not answered. Rather, positionality is addressed primarily in terms of how the research group gained access to people who were initially reluctant to participate.
At times, I had trouble understanding arguments due to opaque prose and introduction of dense theoretical concepts not sufficiently unpacked or previously discussed, such as hegemony, praxis, and identity construction (e.g. first paragraph of p. 21)
There is an unfinished sentence spanning pages 23 and 24.

Notes on first reading

It’s about doing DBR which is impactful.

Focus: critical moments in the process of conducting DBR.

Findings:

a) The design process is organized around a utopian goal; b) Create conditions for sustaining and re-generating the radical content of the utopian goal; c) Examine the recurring challenges to viability and achievability of the utopian design in its learning ecology that emerge for observation over multiple times scales; d) Self-critique and collaborative re-design for a new iteration.

Contribution: An argumentative grammar for DBR. Has this paper

Prior issues with DBR:

Lack of attention to organizational structures;
lack of long-term RPP commitment -> can’t evaluate suitability of designs;
focused on small goals, not equity.

So they’re basically operationalizing radical theory.

Possible areas of consideration:

Positionality.
Schools as adversarial.

For my own interest:

“mesogenetic methodology”
Adorno: Adorno denounced concrete and positive suggestions for social change for two inter-related reasons. First, he did not believe that radical action could avoid repression by the status quo. Second, those that deviated and endured, would, over time, become domesticated and assimilated within the prevailing institutional constraints and power relations.

Possible issues to address:

The proposed theory-building in the introduction (p. 3-6) is specific, well-documented, and quite exciting. I particularly liked how equity-oriented critiques of DBR were aligned with Adorno’s concerns that “concrete and positive suggestions for social change” were unlikely to succeed due to backlash and their tendency to be coopted. This provides a clear path to the background section on utopianism. However, I would have liked to see this thread further developed in the background section. The background subsection on CHAT (p. 7-8), consisting of two paragraphs, is particularly inadequate. I do not think we can assume a reader’s familiarity with CHAT; particularly considering the prior reference to how Vygotsky’s revolutionary potential has been blunted, it would be helpful to head the authors’ construal of CHAT.
In excerpt 1:

The fundamental issue is one of relationship between this work and prior work. This paper is focused on theoretical and methodological development; it must rely on the analysis in prior work, which there isn’t time to reiterate here in full. However, the excerpts here ask the reader to do too much trusting, and do not address issues which come up for a reader who has not read the referenced works.

Not just what

What does it mean that Jaana was selected to lead the pilot? This feels reminiscent to what happens in diversity and equity committees in the US–needs infrastructural analysis.

What was the “radical edge of the pilot” (p. 15) which was blunted? The racial utopian project was never clearly articulated.
positionality. What was Author 1’s role as a leader, organizer, and participant in the excerpts, and how were they accounted for theoretically and methodologically? Summarizing statements such as, “the research group was left with too few resources to assist Jaana” (p. 15-16) imply that the

Positionality is addressed in the final paragraph of the case study (p. 16), but these questions are not answered. Rather, positionality is addressed primarily in terms of how the research group gained access.

why was Jaana’s interiority taken as the primary evidence of intervention into everyday racism?

Uniting the case studies

Bringing together examples from disparate contexts could be a real strength for this paper, but …

More focus on the components, relations, and routines (Penuel’s (2018) “infrastructuring”) of the case studies would help.

Perhaps the paper could include a methods section contextualizing the case studies and explaining how their findings would be synthesized into a broader argument.

Too much detail and too little.

– If the examples are all utopian racial projects, I’m surprised there isn’t any conceptualization of race in the background…

Or at least more clearly conceptualize the kind of social change imagined.

The third case study was strongly focused on infrastructure, but less so on the relationship between the project goals and the infrastructure.

#2: Again, what is the radical project? “Funds of knowledge” has itself been domesticated along with multiculturalism (I’m thinking, for example, of Paris’s “loving critique” (2012) of culturally-relevant pedagogy or Ladson-Billings’s (1998) reflection on multiculturalism)

Minor issues

some trouble understanding arguments due to opaque prose and integration of dense theoretical concepts not sufficiently unpacked or previously discussed, such as hegemony, praxis, and identity construction (e.g. first paragraph of p. 21)

unfinished sentence spanning pages 23 and 24.

These claims are reiterated in the discussion: the study’s contributions are described as “we continue the recently established fruitful dialogue between DBR and CHAT” and “specifying an argumentative grammar (Kelly, 2004) for a utopian methodology as a form of DBR.” Neither was discussed in the case studies.

Later claims made in the discussion are unfounded and, to me, doubtful. For example:

“In DBR studies, such as those examined in this paper, a utopian goal serves as the ’null hypothesis’” (p. 29) The statement that “None of the three cases began as a self-consciously utopian methodological study” (p. 29) undercuts (on my reading) the significance of the case studies of examples of DBR oriented to utopian goals.

No conclusion at all.