Man Behind the ‘Google Brain’ Joins Chinese Search Giant Baidu
http://www.wired.com/2014/05/andrew-ng-baidu/
Andrew Ng.
Photo: Ariel Zambelich/Wired
Andrew Ng is the man who helped launch Google’s wildly ambitious effort to recreate the human brain with computer hardware and software. And now, he will oversee a similar project at Baidu, often called “the Google of China.”
Last year, in Cupertino, California, not far from Apple headquarters, Baidu quietly opened a
research outpost dedicated to “deep learning”–a subfield of artificial intelligence that seeks to vastly improve computing tasks by mimicking the way the human brain operates–and in the months since, this operation has expanded in significant ways. Today, the Chinese search giant will announce that the lab has graduated to a much larger space in Sunnyvale and that Ng, a Stanford University professor, will oversee a new Baidu artificial intelligence research group that spans this lab and an operation in China.
“Andrew is one of the intellectual leaders in machine learning, and deep learning in particular,” says Bruno Olshausen, the director of the Redwood Center for Theoretical Neuroscience at the University of California, Berkeley. “I expect he will continue to lead in this way at Baidu.”
‘Andrew is one of the intellectual leaders in machine learning’
Deep learning–something that seeks to improve everything from natural language processing to voice and image recognition–is a technology that gestated in academia for decades, driven by small group of maverick researchers, including Geoff Hinton of the University of Toronto and Yann LeCun, of NYU. But in recent years, it has quickly spread to the giants of the internet.
Ng, a disciple of Hinton and LeCun, helped launch Google’s efforts in this field, with a project called “the Google Brain,” and after Google
acquired his deep learning company, Hinton now works at least part-time at the search giant. Meanwhile, Facebook
recently hired LeCun, and many other big names are exploring this technology, including Microsoft and IBM. Even Netflix is
getting into the act.
At Baidu, Ng will run both the company’s Sunnyvale lab and an R&D center based in Beijing, which will deal in deep learning and “big data” — i.e. efforts to analyze large amounts of information. Baidu is set to invest about $300 million in this international project over the next five years.
Ng, who starts in his new job today, is stepping away from the day-to-day operations at Coursera, the online-education startup he co-founded. He will still be involved in some projects at Coursera, he says, and will remain the chairman of the board and the public face of the company. But his main focus will be on building up Baidu’s AI chops and its Silicon Valley presence. He’ll spend most of his time in Sunnyvale. “I’m really excited about the opportunity to build an international research organization from scratch,” Ng says. “I’ve been super excited about AI for a long time, and this is an opportunity for me to return to that.”
The Road Back
Since taking a leave of absence from Stanford to start Coursera in 2012, Ng had been splitting his time between running the company and doing AI research. Coursera was growing steadily, having secured another $20 million in funding in November, but Baidu’s Kai Yu, a longtime friend of Ng’s who helped found the Chinese search company’s deep learning labs, urged him to focus on artificial intelligence. “He was doing amazing things in online education, but this is not AI,” Yu says.
During his last visit from Beijing last March, Yu approached Ng about joining Baidu. The pair talked several times at a Sheraton in Palo Alto–first over a pool-side breakfast and later that same day at dinner. Yu then introduced him to two of Baidu’s vice presidents, Jing Wang and Alex Zheng. Later, Ng would fly to Beijing to meet with Baidu CEO Robin Li. Over a three-hour lunch, the two men mapped out their visions for what Baidu’s research arm might look like and the types of problems it would tackle.
The 38-year-old seems a good fit for the company. Like Li, Ng has close ties to both the U.S. and Asia, having grown up in Hong Kong and Singapore. That means he may be in a good position to help merge Baidu’s Asia and California operations well. “I am a product of both of these cultures,” Ng says. “Diversity leads to great creativity and having some of the best ideas from Beijing and Silicon Valley will allow us to innovate faster and come up with more surprising things.”
The Hunt for Autonomous AI
Under Ng’s leadership, Baidu will grow its Silicon Valley office to roughly 200 people by the end of 2015, most of whom will be deep-learning researchers and computer systems engineers. The systems geeks will focus on things like building clusters of low-cost graphical processing units–or GPUs– to crunch through the massive amounts of data that deep learning thrives on. GPUs let data scientists work through billions of calculations more quickly and cheaply than using traditional CPUs.
Google, IBM, and others have also leveraged GPUs for deep learning.
Meanwhile, Baidu’s deep-learning researchers will focus on developing algorithms that are better at learning from unlabeled data, through what’s called unsupervised learning–a concept Ng, together with Google’s Geoff Hinton, has been pushing for years. ”Andrew Ng and me believe strongly in unsupervised learning,” Hinton told WIRED during a conversation at the Google Plex last summer. “Andrew, in particular, pushed on the idea that if we could just use unsupervised learning, then we could go quite a long way.”
That’s because, right now, AI researchers have to do a lot of hand-holding when teaching computers to identify things like words and images. The true promise of AI will be realized, experts say, when computers can teach themselves–when they’re able to absorb and understand data without always being told explicitly what it is. That process, Ng says, is closer to how humans learn and represents a still under-explored avenue for improving AI’s capabilities.
Other deep-learning heavy-hitters agree. “We want to have machines that can take advantage of all of the data out there, and that requires better unsupervised learning,” says the University of Montreal’s Yoshua Bengio, whose work focuses largely on unsupervised learning. Most of the world’s data, you see, is unlabeled, and tagging all of it would be incredibly expensive. Figuring out better ways to get machines learning on their own could improve the economics of AI and lead to better applications for consumers. That’s why Ng is joining Baidu.
Man Behind the ‘Google Brain’ Joins Chinese Search Giant Baidu | Enterprise | WIRED