Educational Data Science article

Educational Data Science article One point I’ve been wanting to make is that data science provides the possibility of transcending different layers of analysis in educational research. In particular, there seems to be a strong divide between 

Also, the possibility of 

ducational data science could have the 

Hi Dan, 

This is a great opportunity to make some important points, and the medium post pointed to a couple of important fairness in ML papers! I agree that CSS ought to define itself on its own terms, rather than just being an interstitial space demarcated by caveats. Here are a few possible directions forward:

Theory & Prior work

“[C]omputer scientists may be interested in finding the needle in the haystack…but social scientists are more commonly interested in characterizing the haystack.”

I thought this was the most important point in Wallach’s Viewpoint column, but wish she had developed it in how it plays out in practice.

Positionality I think there’s a missing layer at the end of the diagram, or maybe an arrow looping back around. CSS needs to address positionality in two senses, both Latour-flavored. First, the social practice of science; who gets to do it, how they benefit from it, how it shapes broader social life and rearranges power relationships. Second, we need to think of AI not just as tools, but as participatory social agents. This raises Asimov-style ethical issues: who’s responsible for my killer robot / fake news generator. Or even the voice interface that issues a feminine giggle and fake key-tapping sounds when you pick up? So much of the tech community has ducked these issues. 

In Rhetoric and Digital Media, Bogost contrasts McLuhan, who saw media as extensions of man, with Kittler, who saw technology as having certain autonomous operations. He also points to Flusser: “contemporary German media theory, one whose approach focuses on the workings of an increasingly automated and technical world rather than the agency of human beings who interact with technologies and media merely as tools. I think this agentic framing is the way forward. (If someone used CRISPR to design an obedient sociopath, who would be responsible for its damage?)

Wallach writes:

It is clear from the media that one of the things that terrifies people the most about machine learning is the use of black-box predictive models in social contexts, where it is possible to do more harm than good.

Doesn’t that describe all AI?

Epistemology Wallach has a great point when she writes, 

we need to know who is affected when there is a mistake, and in what way.

I just want to amplify that. We still try too hard to find the universals, and don’t give enough attention to error analysis. I’m a huge fan of Mitchell’s Unsimple Truths, which proposes “integrative pluralism” as an epistemological framework to replace the presumed universality we inherited from physics. To cast her point in learning sciences terms,  the kinds of results CSS will yield are situated: context-dependent, perspective-dependent, and often (as Sebastian wrote a few weeks ago) co-constitutive of the situations they study. 

-Chris

I think Sebastian had a couple of great points: a data science track would help get more ed researchers to code, and it might get social scientists to think critically about things built with code. I think a general lack of computational literacy in GSE is responsible for some critiques of important issues (ex: racist algorithms) that wouldn’t hold up in more technical forums. It also feels to me like AI gets so much hype that a lot of people are positioning themselves and their work as AI without substance to back it up. There’s nothing wrong with moving toward hot areas, but it would be concerning if we developed a reputation for unrigorous research. 

I think it would be important to consider how a data science track might interact with the the trajectories of PhD students. For people entering GSE PhDs without a computational background, attaining master’s level expertise in CS, Stats, MCS, or cognate fields is a big time commitment which must come as a tradeoff in theoretical depth. In my first-year review, Dan Schwartz argued pretty strongly along that I shouldn’t do a Master’s in CS for this reason. So would there be prerequisites? Do we imagine something parallel to CEPA’s rigorous econometrics core? 

Related to this, Dan McFarland had an interesting point about cross-disciplinary collaboration, the idea of pairing methods-focused MSCS (or MCS/stats, etc.) students with social science grad students. My experience in the project-based AI classes (221; 224n/s/u/w; 229) was that social science research wasn’t a high priority for many people on the CS side. I regularly posted on Piazza looking for collaborators on social-science-related course project and got very few takers. Walking around the CS 229 final projects expo (which took over the entire ground floor of the alumni center), social science was poorly represented. We should consider the incentives for technical people in these potential collaborations. Funding is obviously attractive, but I’d hope for more than a mercenary relationship in which the social science domain is just a dataset for the technical people. 

-Chris

London Festival of Learning: ICLS, AIED, Learning at Scale