Monday, May 27, 2013

A Little Moonshine

I read with fascination this article on using analytics to make predictions and prescriptions regarding your child's career. Both words - prediction and prescription make me very nervous. The premise of the product IBM India's research lab is building is summed up thusly:

“As a student go through courses, every test is recorded electronically (in fact, it is done in most of the US schools). With this kind of data, we can build models of students and cluster them to categories based on some analytics. So we can say, this student is similar to this cluster of people and predict how he can perform,” 

I thought it may be interesting to apply this line to reasoning to a small cohort - the forty some kids that I grew up with taking many, many tests along the way. I have data points for about forty students and eight subjects for say about seven years - several tests at a year for all of these kids. 


Of course, the numbers will not be terribly accurate but the kids are easy enough to cluster in three broad categories - top, middle and bottom of the class in aggregate and by subject. My little group showed remarkable consistency in their test performances from middle school through high school. There was some movement between back and forth between the middle and top of the class. Not much changed at the bottom. Similarly, folks at the very top of the class remained there.

Two decades after taking our last tests seems like a good time to take stock of where we all are. The bottom of the class has done extremely well almost without exception. The middle of the class has remained there - in the middle. The top of the class has beat expectations too but in a negative way. None in that group have realized their assumed "potential" to succeed. I can't wait to find out if the predictive model they build will prove what history has for my tiny sample - that test scores are the worst way to gauge aptitude, interest, motivation or potential to succeed; personality and work ethic are the two big drivers for success that are not easily testable.


Being that this work is taking place in India where astrology has been part of the cultural fabric for thousands of years, maybe they should throw planetary positions in the mix - just for fun. In India it would be a crowd-pleaser. We like to believe that the stars and planets hold secrets and portents for our future, we generally like and trust math, test taking is in our DNA - so we have a winning combination here for believer and skeptic alike. Car insurance has been mixed with zodiac signs in the past, so why leave out education specially when the action is taking place in India ?

2 comments:

Anonymous said...

I do not agree with your argument that test scores are bad predictor of success. Yes, there are exceptions but on an average they
are good predictor of success. Test scores do measure drive, motivation, work ethic, and interest. Off course, they can't measure personality.

Most people who get SAT score > 1400 tend to do better than < 1000. Find me a successful lawyer, doctor, engineer who had a SAT score of 1000. Sure, lawyers who might be more successful might have better personality but most had high academic scores.

In some fields like arts and business, academic success does not translate well into real life. However, success in arts is very unpredictable. Success in business is driven by contacts, old money more than anything else.

Raksha Bharadia said...

I have had a similar experience. The bottom of the class have done extremely well, the middle have stayed in the middle and the top have been mostly disappointing. I have one more theory...they were at the top then because they followed rules, studied regularly and never went out of the box...the world we live in today needs thinking/acting outside the box...