
Measuring AI's ability to learn is difficult
Organizations looking to benefit from the artificial intelligence (AI) revolution should be cautious about putting all their eggs in one basket
Organizations looking to benefit from the artificial intelligence (AI) revolution should be cautious about putting all their eggs in one basket
By Media RelationsOrganizations looking to benefit from the artificial intelligence (AI) revolution should be cautious about putting all their eggs in one basket, a study from the University of Waterloo has found.
In a study published in Nature Machine Intelligence, Waterloo researchers found that contrary to conventional wisdom, there can be no exact method for deciding whether a given problem may be successfully solved by machine learning tools.
“We have to proceed with caution,” said Shai Ben-David, lead author of the study and a professor in Waterloo’s School of Computer Science. “There is a big trend of tools that are very successful, but nobody understands why they are successful, and nobody can provide guarantees that they will continue to be successful.
“In situations where just a yes or no answer is required, we know exactly what can or cannot be done by machine learning algorithms. However, when it comes to more general setups, we can’t distinguish learnable from un-learnable tasks.”
In the study, Ben-David and his colleagues considered a learning model called estimating the maximum (EMX), which captures many common machine learning tasks. For example, tasks like identifying the best place to locate a set of distribution facilities to optimize their accessibility for future expected consumers. The research found that no mathematical method would ever be able to tell, given a task in that model, whether an AI-based tool could handle that task or not.
“This finding comes as a surprise to the research community since it has long been believed that once a precise description of a task is provided, it can then be determined whether machine learning algorithms will be able to learn and carry out that task,” said Ben-David.
The study, Learnability can be Undecidable, was co-authored by Ben-David, Pavel Hrubeš from the Institute of Mathematics of the Academy of Sciences in the Czech Republic, Shay Morgan from the Department of Computer Science, Princeton University, Amir Shpilka, Department of Computer Science, Tel Aviv University, and Amir Yehudayoff from the Department of Mathematics, Technion-IIT.
Read more
Dr. Anita Layton and Dr. Kerstin Dautenhahn complete their C150 terms but their research impacting the future of health and technology continue
Read more
The Governor General Gold Medal awards graduate students for their remarkable academic achievements and transformative research
Read more
Six Waterloo graduates share their stories of perseverance, innovation and advocacy as they prepare to cross the stage at convocation
The University of Waterloo acknowledges that much of our work takes place on the traditional territory of the Neutral, Anishinaabeg, and Haudenosaunee peoples. Our main campus is situated on the Haldimand Tract, the land granted to the Six Nations that includes six miles on each side of the Grand River. Our active work toward reconciliation takes place across our campuses through research, learning, teaching, and community building, and is co-ordinated within the Office of Indigenous Relations.