ISLS 2018 Reviewer Notes

ISLS 2018 Reviewer Notes What exactly is meant by computational thinking?

This study explores affordances for visualizing metacognition in the design of computer-based learning environments. The experimental condition was adding a travel diary to an online game for learning history, which showed artifacts of users’ past progress through the game. A strong conceptual framework is presented, with clear research questions asking about the extent to which students would look at the travel diary, and the extent to which interacting with the diary would have an effect on students’ self-regulatory metacognition (measured by playing the game longer).  The study’s methods are quantitative, drawing data from server logs. No significant results are reported. 

 While the study pulls together an interesting theoretical premise, the lack of any qualitative grounding makes the authors’ interpretation of server log data less persuasive. Most centrally, I am not convinced that  the amount of time students spend voluntarily playing a game is a valid measure of metacognitive self-regulation. There are several other areas in which an extension of this study to include qualitative data showing the practices students engage in around the new software features would be beneficial. For example, the authors argue that “when embedding all pedagogical properties inside an instructional software, overlapping tasks and multiple layers of information should probably be avoided.” However, there are plenty of examples in which adding complexity to a task makes it more engaging and easier to dig into. 

PAPERS Barr, V., & Stephenson, C. (2011). Bringing computational thinking to K-12: what is Involved and what is the role of the computer science education community?. Acm Inroads, 2(1), 48-54.

This paper applies a critical literacy perspective to computation as a literacy. The study’s method is interviews with nine participants before and after a 45-minute session playing a programming game. While the author frames an interesting and important question, the study as presented lacks the precise conceptual framing, methods, and evidence that would be required to produce sound findings.

The paper’s conceptual framework draws heavily on the work of Paulo Freire. The author uses Freirian though to raise interesting questions, but they need to be more specifically connected to the study’s methods and discussion. This is evident in the structure of the paper; there are numerous long block quotations from Freire (one quotation is used twice) which are not sufficiently leveraged to develop interpretive approaches (such as qualitative coding systems) or in analyzing results. Too often, the block quotation is given as self-evident justification for the meaning of a participant’s utterance.

The study’s methods are not sufficiently developed to be able to yield solid results. Putting aside the question of whether meaningful growth in literacy practices could be detected with pre/post interviews and 45 minutes of gameplay, the author’s interpretation of the interviews is not methodical enough to justify his or her findings. Without a coding scheme or theoretically-motivated approach to analyzing the interviews, how can the reader know whether the author is presenting quotations selectively or interpreting their significance in a reasonable manner?

Shanahan 2017 metal-levels (meta-levels) a diary were tokens with knowledge related content were received (were -> where)