Matt Reed (aka
@deandad)’s article Predictions, Probabilities, and Placebos and an ensuing twitter discussion raised some important questions about potential dangers of predictive analytics. Specifically, he links Claude Steele’s stereotype threat to predictive analytics. An ensuing twitter discussion explored the conflicting issues around using predictive analytics. Is there a danger of predictive analytics being used in unintentionally harmful ways?
Somewhere in the mid 2000’s I was at a National Center for Academic Transformation conference to learn about promising ideas for course redesign. One of the first sessions I was in raised huge alarms. It was a presentation on a math redesign and the director of the math center, a young, enthusiastic (and I am sure well- meaning) young woman proudly stated how wonderful the redesign had been and how it was improving their success rates so well. She said, and I loosely quote because it was a long time ago,
“It’s great. Now when students log into the system, they get an immediate message about their progress in the course. There is a big warning sign first thing, if they are not on track. What it is has meant is that students know when they should drop the course, and that has really improved our success rates.”
This is not to say that the institution or NCAT or anyone else has negative intentions. This was a long time ago when we, in higher education, were just beginning to grapple with what data can do and how it should be use. However, the comment has stuck with me since then.
Previous to that, when I first began teaching online in 1999, there was a debate among a number of colleagues some of whom were advocating to not allow some students to take online courses because of the poor success rates they were seeing. I have seen this same conversation repeated over the years in various iterations. The issue I had with this argument was that it focused on the students as not being successful, rather than meaningfully looking at what we could do to support students into success. It was a blunt reading of the data, in my opinion.
Both of these anecdotes bring me to my thoughts on Reed’s essay today. I think we, in higher ed, need to do a better job of thinking about how we use data. We need to be very mindful of the real harm that can be caused by not being thoughtful about how we use and convey data. That is not to say that we shouldn’t take advantage of the information and the modeling that is available– and we should be transparent about it. In a talk I heard about 2 years ago, Mark Milliron (@) makes a strong case for being thoughtful about how we talk about the data we have when we talk with students in order to avoid the creep factor. I think Reed goes further with his questions. Do we do real harm to students if we at all communicate predictive analytics as infallible? Do we unintentionally build stereotype threats in our intentions to help students?
I think there is another side to this conversation. The genie is out of the bottle when it comes to data collection and predictive analytics. It is not a question of whether or if we should be collecting data and making predictions. That is happening and impacting all our lives well beyond our institutions. I do think the conversations about using data effectively to design interventions and supports is happening and should be a central part of our work as educators.However, I think this raises the more important point of how we are giving students opportunities to understand data collection and predictive analytics and how we give them the agency to own their own journeys. We need to empower students to understand that predictive analytics are just a tool they can use to change their own paths.
I will end this with one more personal anecdote. When I learned about how Amazon uses predictive analytics to make recommendations, I immediately felt challenged– I am not my data. As a result, I intentionally started doing more random searches for things, and sometimes eschewing the recommendations. I didn’t do this in a lame attempt to disrupt the prediction, but more to own my own purchases. I appreciate those recommendations, but I also have loved finding things I never would have predicted that I would like.
So, how do we ensure that students understand and own their own journeys? How do we teach them about predictive analytics and data collection– regardless of how we use it in higher ed?
The call for action in higher ed is tri-fold:
- Ensure that we use predictive analytic data in ways that do not do harm by creating standards of ethical use;
- Emphasize data use to create successful interventions and intervention points, not to create self-fulling prophecies;
- Give students agency by ensuring that data literacy includes an understanding of predictive analytics