Cognizant Program Officer Comments

Dear Dr. Wang, Thank you for submitting a proposal to the CSforAll program. As indicated in an e-mail you received from the Division of Research on Learning in Formal and Informal Settings Division Director or Deputy Division Director, this proposal could not be funded this year by the CSforAll program. I realize this is disappointing news. I hope you find the comments of reviewers helpful as you consider next steps in pursuing your project. Three or more reviewers read your proposal and then served on a panel to consider your proposal along with other proposals. The individual reviews and panel summary provide critiques of the strengths and limitations of your proposal as perceived by the reviewers in terms of the two NSF Merit Review Criteria: Intellectual Merit and Broader Impacts, and the Solicitation Specific Review Criteria associated with the program. The individual reviews generally add depth to the panel summary. Please note, however, that the individual reviewers’ comments and the panel summary do not necessarily reflect NSF’s policy or position. The following comments represent my summary of the major strengths and weaknesses of this proposal in terms of intellectual merit and broader impacts. The proposed work has presented a forward-thinking and research-driven approach for promoting AI literacy among young children. The techno-ethnography method is well aligned with the proposed research goal and questions. However, the current proposal lacks a solid description of the characteristics, assets, and needs of disadvantages students to be served and examined, lending a lack of clarity on what relational or situated learning tasks will be culturally or age appropriate. The current data analysis plan reads generic and lacks critical details on how the proposed thematic analysis will be conducted systematically or rigorously. The immediate impact of the proposed work on the target participants is also limited. If you are interested in submitting a revision of this proposal to a future competition, please note that a declined proposal may only be resubmitted after it has undergone substantial revision and clearly addresses the major comments or concerns cited in the previous NSF review. Resubmitted proposals are review de novo. Recommendations for funding consider both the quality of other proposals received and the availability of program funds, and there is no guarantee that a revised proposal would be funded. We appreciate the effort you invested to develop your proposal and thank you for your interest in the CSforAll program. Should you have inquiries regarding this or a future proposal submission, please contact me at fke@nsf.gov. Thank you for your interest in the CSforAll program. Sincerely, Fengfeng Ke Program Officer EDU/DRL October 29, 2024

Panel Summary

I. Project Overview The proposed project aims to explore the situated and relational AI experiences of young children aged 5-8 in three schools. It focuses on fostering AI literacy through a care-infused sociocultural framework that integrates sociocultural theory with an ethic of care. This approach prioritizes relationships, agentive interactions, and the goals of disadvantaged individuals. The project will involve a techno-ethnographic study, partnering with three elementary schools—one in an underserved community, one in an affluent community, and one in a rural community. Participants will be recruited to co-design AI learning activities that reflect children’s everyday AI encounters, along with the mediating practices of caregivers, educators, and peers.
II. Intellectual Merit All panelists agreed that the proposal presents an innovative framework that combines co-design and techno-ethnography with qualitative techniques. Despite lacking a detailed explanation of data analysis methods, the staggered data collection and multiple points of research outputs were described as promising. The research team is highly capable, with notable strengths in addressing disadvantaged groups and inviting children into the design process. However, the definition and focus on disadvantaged groups are vague, which is a significant weakness. One panelist noted that the proposal provides a strong rationale and justification, focusing on young students. The evaluation section needs more details on the analysis methods and types of questions for evaluation. Overall, the project has merit, but clarity on data analysis and disadvantaged groups needs improvement.
III. Broader Impacts All panelists agreed that the project has strong potential for positive societal impact through the release of materials and feeding findings back into the community. It aims to broaden participation in computing and has a forward-looking approach. While the immediate impact on students may be somewhat limited, the theoretical contributions could be substantial. One panelist emphasized that the dissemination plan is robust, targeting educators and the general audience through publications and a dedicated website. The use of an app is particularly attractive to participants and reduces their burden of participation. Another panelist noted that the project has the potential to generate themes for future work, extending its impact beyond the immediate community to theoretical foundations.
V. Additional Solicitation-Specific Criteria All panelists highlighted strengths in co-design, making materials accessible in necessary languages, and supporting co-design sessions. However, the proposal lacks specificity in defining the disadvantaged groups it aims to serve. Data Management Plan: The Data Management Plan is adequate. Post-Doc and Graduate Students Mentoring Plan: The post-doc/grad student mentoring plan is adequate, with inclusion in publication a strength. Prior NSF Support (if applicable): The panel noted strong prior support, lending confidence in the team.
V. Summary Statement This proposal demonstrates notable strengths in its intellectual merit and broader impacts, with a strong potential to contribute to AI literacy among young children through an innovative care-infused sociocultural framework. The project’s focus on early childhood education and the integration of relationships and care-thinking in AI literacy are particularly commendable. However, the proposal has significant weaknesses in its superficial inclusion of disadvantaged populations and insufficient detail on recruitment and engagement strategies for these groups. Addressing these issues with a clearer focus on the characteristics and needs of disadvantaged populations, along with specific strategies for their engagement and support, would strengthen the proposal considerably. The panel has read this summary and agrees that it accurately represents the panel discussion.

PANEL RECOMMENDATION: Competitive

CP Analysis

lacks a solid description of the characteristics, assets, and needs of disadvantages students to be served and examined,
lending a lack of clarity on what relational or situated learning tasks will be culturally or age appropriate.
The current data analysis plan reads generic and lacks critical details on how the proposed thematic analysis will be conducted systematically or rigorously.
The immediate impact of the proposed work on the target participants is also limited.

Review 1

Rating:

Fair

Review:

Summary

In the context of the five review elements, please
evaluate the strengths and weaknesses of the proposal with respect to intellectual merit.

The proposed project aims to explore the situated and relational AI experiences of young children aged 5-8 in three schools. The project focuses on fostering AI literacy through a care-infused sociocultural framework, integrating sociocultural theory with an ethic of care to prioritize relationships, agentive interactions, and, purportedly, the goals of disadvantaged individuals. The authors will conduct a techno-ethnographic study, partnering with three elementary schools: 1 in an in underserved community, 1 in an affluent community, and 1 in a rural community. Authors will recruit participants and co-design AI learning activities that reflect children’s everyday AI encounters and the mediating practices of caregivers, educators, and peers and conduct a techno-ethnography.

STRENGTHS

The proposal presents an innovative care-infused sociocultural framework for AI literacy that integrates sociocultural theory with an ethic of care, which is a novel perspective in the field of early childhood AI literacy. The authors have also identified a range of gaps in understanding young children’s AI experiences and the role of a broader network of adults and peers in fostering AI literacy. This author appreciated the challenges to more teleological approaches to AI education, through the authors’ more holistic and equitable framework. Further, the techno-ethnographic methodology is an interesting approach for capturing situated and relational aspects of children’s AI experiences. This reviewer particularly appreciated the co-design approach with children, educators, and caregivers. The research team is composed of interdisciplinary experts with strong backgrounds in AI, early childhood education, child-computer interaction, and ethnographic research.

WEAKNESSES

The proposal could benefit from a more detailed explanation of the data analysis plan, specifically how the qualitative data from the techno-ethnographic study will be systematically analyzed to draw out themes and insights related to AI literacy. In addition, while the proposal highlights the involvement of community partners, additional details on how these partnerships will be maintained and how community feedback will be integrated throughout the project’s lifecycle would enhance the overall feasibility and sustainability of the initiative.

A major weakness with this proposal is its superficial inclusion of disadvantaged groups. The proposal appears to treat disadvantaged individuals as an afterthought, rather than a central focus. These individuals and groups are mentioned only 5 times in the entire project description. No description or definition of “disadvantaged” is provided. To be sure, this proposal could remove the term “disadvantaged individuals and groups” entirely and not be impacted in any significant way.

In the context of the five review elements, please
evaluate the strengths and weaknesses of the proposal with respect to broader impacts.

STRENGTHS

The proposed activity has potential to benefit society in its approach to enhancing AI literacy among young children. The focus on early childhood education could promote future engagement in STEM fields. The approach is also creative and original with its care-infused sociocultural framework. The emphasis on relationships and care-thinking in AI literacy is a novel concept and potentially transformative. The diversity of sites–reminiscent of maximum variation sampling–is also a strength in that it could lend insights into how children in diverse communities respond to the care-infused approach (although research consistently shows that affluent, predominantly white children tend to benefit most). The development of equitable AI educational resources and curricula that align with care-centered practices is commendable, although just how equitable and inclusive these resources might be is unclear, and the authors do not include a plan for evaluating this aspect of the project.

WEAKNESSES

A major weakness is the lack of any meaningful or specific focus on underrepresented groups. The proposal is remarkably vague about which specific underrepresented or underserved populations it aims to benefit. While it mentions disadvantaged backgrounds, it does not clearly define or operationalize this term beyond economic disadvantage (in Table 1 on pp. 11). This lack of specificity tremendously undermines the project’s alignment with the solicitation’s emphasis on broadening participation in computing among traditionally underrepresented groups. Who are these groups in this population? Further, the authors do not appear to have considered how they will distribute their findings, particularly to the disadvantaged individuals and groups who will participate. No clear plan for distributing findings to participants and their families is provided. However, ensuring that the community benefits from the research outcomes is a key aspect of broader impacts, and the absence of such a plan is a notable weakness. Finally, the proposal lacks specific strategies for addressing or accommodating the particular needs of participants from disadvantaged backgrounds. It does not detail how these individuals will be supported throughout the project to ensure their meaningful engagement and benefit from the AI literacy activities.

Please evaluate the strengths and
weaknesses of the proposal with respect to any additional solicitation-specific review criteria, if
applicable

DOES THE PROPOSAL IDENTIFY THE CHARACTERISTICS AND NEEDS OF THE INTENDED POPULATION(S) TO BE SERVED?

STRENGTHS

The proposal identifies three distinct communities—an underserved city, an affluent suburb, and a rural town. The authors appear to have the intention to understand and address the diverse needs and characteristics of these populations. Selecting a sample from these varied communities could allow for a more holistic understanding of how AI literacy can be developed across various sociocultural contexts. The focus on early childhood education is also a strength. By targeting young children (ages 5-8, specifically), the proposal aims to meet the developmental and educational needs of this age group. This focus acknowledges the critical importance of early childhood education in shaping long-term academic and social outcomes, particularly as relates to STEM. Related to this is the authors’ emphasis on relationships and care. Their care framework is particularly relevant for young children.

WEAKNESSES

Unfortunately, as mentioned before, the proposal does not clearly define or operationalize the characteristics of those disadvantaged individuals and groups it aims to serve. While it mentions economically disadvantaged individuals in Table 1, it does not provide detailed information on other dimensions of disadvantage, such as race, ethnicity, or disability status, nor does it explicitly explain and describe how it will explicitly address the needs of these disadvantaged populations.

DOES THE PROPOSAL INCLUDE SPECIFIC PLANS OR STRATEGIES FOR ADDRESSING OR ACCOMMODATING THE PARTICULAR NEEDS OF PARTICIPANTS OF THE INTENDED POPULATION(S)?

STRENGTHS

The proposal includes plans for an iterative co-design process involving children, teachers, and caregivers, which is a major strength. This co-design approach could ensure that learning activities are developed with input from those who will use them, potentially making the activities more relevant and effective. This would be particularly effective when working with disadvantaged groups, but the authors do not make this connection explicitly. This reveiwer also found that the use of techno-ethnographic methods to document children’s AI experiences in real-world settings could be a strong strategy for understanding and addressing their needs. This approach could allow for a more nuanced and holistic examination of how children from diverse backgrounds interact with AI technologies in situ. Further, the proposal mentions providing training for teachers, which will be crucial for successful project implementation. More detail about teacher training would have been helpful, as it only appears that teachers will be trained to facilitate co-design, yet there appear to be other areas of teacher training needs with this project.

WEAKNESSES

The proposal lacks specific plans for how disadvantaged individuals will be intentionally recruited and meaningfully engaged in the project. It does not provide strategies for reaching out to these populations or ensuring their active participation in the co-design process. As mentioned above, the proposal also does not include plans to compensate participants. While the proposal mentions iterative co-design and ethnographic documentation, it does not provide detailed plans on how feedback from participants would be integrated into the development and refinement of the AI literacy activities, nor does it provide detail of how these refinements would be evaluated. This lack of clarity on the feedback loop weakens the proposal’s methodological rigor. In addition, the proposal does not provide specific strategies for addressing or accommodating the particular needs of participants from disadvantaged backgrounds. It lacks detailed plans on how the project will support these individuals throughout the learning process to ensure their meaningful engagement and benefit from the AI literacy activities.

Summary Statement

This proposal demonstrates notable strengths in its intellectual merit and broader impacts, with a strong potential to contribute to AI literacy among young children in general with a view towards addressing broader societal goals. Strengths include an innovative care-infused sociocultural framework and a well-qualified interdisciplinary team. The project’s focus on early childhood education and the integration of relationships and care-thinking in AI literacy are particularly interesting. However, the proposal has severe weaknesses in light of the current program solicitation. These include: a lack of specific focus on underrepresented groups, superficial inclusion of disadvantaged populations, insufficient detail on recruitment and engagement strategies, and unclear plans for participant compensation and integration of feedback. Ultimately, more attention is needed regarding the disadvantaged populations that are the focus of the CSforAll solicitation, their characteristics, and their needs.

Review 2

Rating:

Fair

Review:

Summary

In the context of the five review elements, please
evaluate the strengths and weaknesses of the proposal with respect to intellectual merit.

Strengths - I find this learning content, research and design innovative and unique, as well as needed for understanding the capacity young children have to use and evaluate AI in social-relational context with peers, mentors, and family members. With the rapidly evolving needs for AI learning, both conceptually and societally, this project investigates and disseminates highly needed perspectives.

Weakness - I wonder if there are ways to equip families to be able to hold and navigate discussions, increasing their capacity and creating efficacy and agency where there is an expressed need and interest around AI tools and knowledge. This project assumes that visible/known uses of AI technology are embedded in family’s normal lives and that members of the group are conversant with the topic. I am also concerned with the implementation of Phase 2 in homes of students from lower SES families. There seems to be a lot of assumed use of technology that is embedded in family’s home lives. Also, examples from the lessons shared focused on hidden uses of AI, where it could be easily convoluted with technology use.

In the context of the five review elements, please
evaluate the strengths and weaknesses of the proposal with respect to broader impacts.

Strengths - If this project was awarded and successful, the research team would discover and share AI literacy levels after implementation.

Weakness - I don’t know that this project would uncover authentic care-centered AI literacy practices because the deployment of activities for use in the study would hide that. I’m also not seeing a strong enough connection for teacher pedagogy in relevant, inclusive, and equity-focused strategies that would be replicable or of interest for scaling.

Please evaluate the strengths and
weaknesses of the proposal with respect to any additional solicitation-specific review criteria, if
applicable

Summary Statement

I think there is a lot of room for improvement for this proposed project. While the initial idea and investigation is interesting, I believe the design for uncovering the situational and relational AI experiences through the three phases doesn’t connect to the same research objectives.

Review 3

Rating:

Excellent

Review:

Summary

In the context of the five review elements, please
evaluate the strengths and weaknesses of the proposal with respect to intellectual merit.

STRENGTHS

• Advance Knowledge: This proposal addresses the understudied context of AI in early childhood education, and can advance knowledge about children’s relationships with AI. The focus on a care framework and mediated relationships provides a unique lens of the topic.
• Design: The methods are appropriately chosen and well-justified (co-design, ethnography, qualitative analysis). Challenges have been considered and addressed (e.g. privacy and security, staggering data collection, pre-testing the data collection website, multiple points of research output from design process and final data collection, balancing and supporting teacher effort).
• Assessment: The proposal considers assessment at multiple points, and generates research knowledge from the iterative design, deployments, and final vignettes.
• Team: The team has significant expertise in all necessary areas, and prior collaboration experience together. The identified schools have committed to participating, and sufficient stipends are listed to support teacher effort.
• Resources: The proposal will hire 4 graduate students, which seems appropriate for collecting ethnographic data from 90 children.

WEAKNESSES
none noted

PRIOR SUPPORT
• Prior Support: Prior support is extensive. Some are ongoing and have few publications. Most relate to AI learning, CS teaching, and early elementary students. None specifically address the topics in this proposal, but evidence complementary topics and methods.

In the context of the five review elements, please
evaluate the strengths and weaknesses of the proposal with respect to broader impacts.

STRENGTHS
• Significance: The proposal addresses an understudied population, and would provide significant new knowledge about emerging technologies, which can inform future educational interventions. It also generates activities, data collection platform, and public resources for educators, researchers, and parents to build on.
• Dissemination: The proposal provides multiple forms of dissemination, within research communities and practitioners, using both papers and a casebook format. The presentations within the selected schools is also worth highlighting, as it makes sure the knowledge generated by the study is returned to the communities who participated.
• Participation: The proposal engages three distinct school contexts with different demographics and needs. For an in-depth ethnographic study, 90 students is a large sample, which should help provide insights about each context despite the differences between them. The knowledge from the study can inform future educational interventions to increase STEM literacy and learning activities, and the study itself may help these 90 children talk about, understand, and build on their own AI relations.

WEAKNESSES
none noted

MENTORING
• No postdocs. Graduate mentoring plan is adequate, and specifies mentoring support for both hourly and fulltime GRAs.

DATA MANAGEMENT
• Data plan is adequate, however special care must be taken to ensure the CARE site is up to security standards.

Please evaluate the strengths and
weaknesses of the proposal with respect to any additional solicitation-specific review criteria, if
applicable

• Does the proposal identify the characteristics and needs of the intended population(s) to be served?
o The proposal identifies the population as 1) young children, ages 5-8, in three schools in New York with distinct demographics and community dynamics. It primarily considers age, languages, and resources as the key characteristics the design must be sensitive to.

• Does the proposal include specific plans or strategies for addressing or accommodating the particular needs of participants of the intended population(s)?
o The proposal specifically plans for translating research materials to the preferred languages of the participants. Co-design and a care-based ethnographic approach should also help align the activities with the needs of the students. The proposal could be more specific about other challenges of addressing three distinct contexts, and whether activities might need to be adapted or differentiated between them.

Summary Statement

• This proposal is a research strand proposal seeking to understand children’s care-based and relational understanding of AI through technoethnography. The team is qualified, and the design detailed and well-constructed. The topic is timely, important, and understudied.

Review 4

Rating:

Excellent

Review:

Summary

In the context of the five review elements, please
evaluate the strengths and weaknesses of the proposal with respect to intellectual merit.

This project is centered on examining the young children (ages 5-8). as they interact with AI. By adopting a sociocultural perspective they will focus on the role of care and care-givers as these young children interact with the AI technology around them. The grant explores the role of caregivers by involving families within the research design, They will on three different demographical areas which will provide a greater range of dimensions of diversity. Through using a techno-ethnographic method to explore critical aspects of the interactions, the researchers will begin to unpack the relationship between AI, caregiving and pedagogical approaches that are equitable for all.

Through the use of TAIL and RAIL versions of using technology, where students give voice to their experiences, the researchers will gather data that examines the relationship that students have with their lived experiences of after school care. Important for this project is the students agentic interactions within their settings - this is work that is imposed on, but rather invites students to interact with the technology.

By building a tool for data collection through the phone, the researchers seek to gather data from the caregivers as a means to determine how students are interacting with the current AI being used - Alexa and Siri are two common examples. By linking this community based technology with classroom based work, the researchers will be a rich understanding of young children’s understanding of this technology.

The research is intended to examine young children’s interaction with technology, what is an AI system and what awareness students have this technology and the role it plays.Utilizing a design based framework the researchers will seek to understand participants experiences, practices and understandings.

The data analysis plan is strong, highlighting multiple dimensions by coding for AI products utilized, children and caregiver experiences, and where possible pedagogical affordances. This will enable the generation of emergent themes. While recognizing the breadth of data gather, the amount of time required for analysis may need to be addressed. How will the researchers ensure the degree of analysis is achieved to reach strong conclusions?

In the context of the five review elements, please
evaluate the strengths and weaknesses of the proposal with respect to broader impacts.

The range of different settings, the broad range of data gathered and the generation of themes has the potential to have broader impacts that arise from this project. The innovative and important component of the research is to involve outside school caregivers. This approach enables the researchers to begin to highlight how young children as interactive agents of their own learning are able to build an understanding, and an appreciation of, the role of AI.

In recognizing the impact of different contexts as a critical component of understanding what is an equitable context for learning, will provide the basis of broader impact. This innovative project has the potential to provide some strong themes for researchers that can be applied a large range of socio-economic settings.

Please evaluate the strengths and
weaknesses of the proposal with respect to any additional solicitation-specific review criteria, if
applicable

The research team has the experience and institutional resources to ensure that the project can be successfully conducted. The evaluation plan will enable a very strong review of the project, and provide guidance for the researchers.

Summary Statement

This projects seek to adopt a sociocultural exploration of AI literacy and young children. Working with a number of different schools to ensure that broadest possible demographic diversity, the project will work with young children and their caregivers to better understand the role of care and caregivers within the development of such literacy. The research plan is centered on in school and outside of school care, and provides all students agentic opportunities to interact with, and learn from, the TAILS and RAILS processes. The data collection and analysis plan highlights the techno-enthnographic methods to be adopted and will provide a rich number of emergent themes.

Review 5

Rating:

Good

Review:

Summary

In the context of the five review elements, please
evaluate the strengths and weaknesses of the proposal with respect to intellectual merit.

- This project seeks to “address critical gaps in understanding young children’s situated and relational AI experiences and the role of adults and peers in fostering care-thinking-centered AI literacy.”
- The project advances knowledge through a techno-ethnographic method to address changing AI adoption and capture children’s perspectives on this topic.
- There are not many studies in AI on children below 8.
- A compelling rationale was provided for the study and selection of participants.
The project team has the right background and experience to complete the project.

In the context of the five review elements, please
evaluate the strengths and weaknesses of the proposal with respect to broader impacts.

-Good description of the demographic characteristics of the participating schools.
- The dissemination plan is well described including information on how to reach different groups.
- How many individuals can this potentially reach beyond the study participants?
- More details on the evaluation plan could be beneficial; what types of questions will be asked?

Please evaluate the strengths and
weaknesses of the proposal with respect to any additional solicitation-specific review criteria, if
applicable

The needs of the population were described, and approaches to address these needs were also discussed in the proposal.

Summary Statement

This project focuses on promoting AI literacy in early childhood education (ECE) in young children. The project advances knowledge through a techno-ethnographic method to address changing AI adoption and capture children’s perspectives on this topic.