The cutting edge of artificial intelligence research is based on a set of mathematical techniques called deep neural networks. These networks are mathematical algorithms that can learn tasks on their own by analyzing data. By looking for patterns in millions of dog photos, for example, a neural network can learn to recognize a dog. This mathematical idea dates back to the 1950s, but it remained on the fringes of academia and industry until about five years ago.
By 2013, Google, Facebook and a few other companies started to recruit the relatively few researchers who specialized in these techniques. Neural networks now help recognize faces in photos posted to Facebook, identify commands spoken into living-room digital assistants like the Amazon Echo and instantly translate foreign languages on Microsoft’s Skype phone service.
Using the same mathematical techniques, researchers are improving self-driving cars and developing hospital services that can identify illness and disease in medical scans, digital assistants that can not only recognize spoken words but understand them, automated stock-trading systems and robots that pick up objects they’ve never seen before.
With so few A.I. specialists available, big tech companies are also hiring the best and brightest of academia. In the process, they are limiting the number of professors who can teach the technology.
Uber hired 40 people from Carnegie Mellon’s groundbreaking A.I. program in 2015 to work on its self-driving-car project. Over the last several years, four of the best-known A.I. researchers in academia have left or taken leave from their professorships at Stanford University. At the University of Washington, six of 20 artificial intelligence professors are now on leave or partial leave and working for outside companies.
“There is a giant sucking sound of academics going into industry,” said Oren Etzioni, who is on leave from his position as a professor at the University of Washington to oversee the nonprofit Allen Institute for Artificial Intelligence.
Some professors are finding a way to compromise. Luke Zettlemoyer of the University of Washington turned down a position at a Google-run Seattle laboratory that he said would have paid him more than three times his current salary (about $180,000, according to public records). Instead, he chose a post at the Allen Institute that allowed him to continue teaching.
“There are plenty of faculty that do this, splitting their time in various percentages between industry and academia,” Mr. Zettlemoyer said. “The salaries are so much higher in industry, people only do this because they really care about being an academian.”
To bring in new A.I. engineers, companies like Google and Facebook are running classes that aim to teach “deep learning” and related techniques to existing employees. And nonprofits like Fast.ai and companies like Deeplearning.ai, founded by a former Stanford professor who helped create the Google Brain lab, offer online courses.
The basic concepts of deep learning are not hard to grasp, requiring little more than high-school-level math. But real expertise requires more significant math and an intuitive talent that some call “a dark art.” Specific knowledge is needed for fields like self-driving cars, robotics and health care.
In order to keep pace, smaller companies are looking for talent in unusual places. Some are hiring physicists and astronomers who have the necessary math skills. Other start-ups from the United States are looking for workers in Asia, Eastern Europe and other locations where wages are lower.
“I can’t compete with Google, and I don’t want to,” said Chris Nicholson, the chief executive and a co-founder of Skymind, a start-up in San Francisco that has hired engineers in eight countries. “So I offer very attractive salaries in countries that undervalue engineering talent.”
But the industry’s giants are doing much the same. Google, Facebook, Microsoft and others have opened A.I. labs in Toronto and Montreal, where much of this research outside the United States is being done. Google also is hiring in China, where Microsoft has long had a strong presence.
Not surprisingly, many think the talent shortage won’t be alleviated for years.
“Of course demand outweighs supply. And things are not getting better any time soon,” Yoshua Bengio, a professor at the University of Montreal and a prominent A.I. researcher, said. “It takes many years to train a Ph.D.”