- April 5, 2018
- Blog
There are problems that just cant be solved by AI or ML! (Well..not yet!!)
Folklore has it that just about 10% of the allied forces's first of the listed airmen survived World War II Nazi anti-aircraft guns. Planes that returned were riddled with bullet holes. There was compelling need to armor the planes to win. So what is the issue? Good you asked.
The catch is that if the entire plane is armored, the weight may not even let it take-off or its span of operation would be lesser considering higher fuel consumption. A military think-tank took the following data to a mathematician Abraham Wald, so that they may spot the area that needs more armoring:
There is a high probability that you will consider armoring the Fuselage!
But, Abraham Wald was clear that the engine required the max armoring. And that is because, the planes that took the larger number of bullets than "1.11" per square foot never returned!
In the absence of data about the missing planes, only a human mind (at least, as of today) can judge that the armor should go to the Engine and not the Fuselage. AI / ML cant work when there is no data!
Clearly, a data scientist generating an algorithm to solve any such problem would need to provide a data set that has both the "1"s and the "0"s - "1" being data about downed planes and "0" being data about returned planes. The problem arises when she has just one section of the data and does not realize so.
What are some data sets that have such issues:
- An Indian insurance company assessed that there are more grievous injuries at speeds less than 40 kms per hour than faster. Possibly, the victims are not even alive to be accounted for injuries in accidents at higher speeds.
- There are nearly 50 million MSMEs (Micro, Small & Medium Enterprises) in India. A government report suggest that MSME has a healthy growth rate of 10% each year. However, the data excludes about half a million of MSMEs folding up each year and when that is accounted for, the growth rate is just about 7%.
- The book 'The Outliers' has an interesting finding: all world greats have 10,000 hours of hard work - be it an Olympian, Musician, Nobel Laureates, and all similar. It provides an impression that all those that did put the effort would succeed.
- Your parents may often spot a couple of old models still working and comment that the newer models are not built to last. Possibly, a handful of the current ones will survive and you will make the same comment to your daughter.
What other data-sets do you think may have this kind of issue?
