This researcher uses AI to add “human elements” to 3D content generation

New, human-centred, visual computing and computer graphics technologies offer innovative ways for machines to generate functional and engaging 3D content for computer-aided designs, smart homes, and in simulated learning environments for training doctors, military personnel—even autonomous vehicles.

At the forefront of these technologies is SFU computing science professor Richard Zhang. A visual computing researcher, he specializes in developing computer graphics and computational methods for understanding, processing and generating visual data.

In particular, he’s exploring how we might use machine learning and shape analysis to automatically generate 3D objects and spaces optimized for human interaction. To do this, however, the machines must first “understand” how humans interact with their environment and how objects perform their functions.

“Learning to generate is at the heart of artificial intelligence (AI), and the human element must be there,” says Zhang. He focuses on 3D data—such as furniture shapes in smart homes, industrial models for design and production, and scene environments—for virtual and augmented reality.

His first step is to develop methods and algorithms that examine how humans interact with objects to reveal their functions. For example, how a person interacts with a chair and how the chair supports the person. He and his collaborators then capture these human-object interactions to learn how to generate functional and engaging 3D content.

These 3D objects and scenes can then be applied to design and manufacturing, or to create simulated environments in which doctors, military personnel, machine operators and others can learn new skills.

Zhang is currently working with Autodesk, a leader in 3D design software, to transform 3D machine-generated models so they can be more easily manufactured using traditional fabrication methods such as computer numerical control (CNC) and 3D printing.

He is also working with Google to train deep neural networks to understand, predict and generate 3D shapes in an interpretable way. This collaboration led to a research publication that won the Best Student Paper Award at the annual Computer Vision and Pattern Recognition (CVPR) conference, which is renowned for having the highest impact among all computing science conferences.

While Zhang’s expertise is regularly sought for industry projects, his academic contributions are also significant. He has published more than 180 papers, and has been cited more than 10,000 times. And according to csrankings.org, Zhang ranks first among Canadian university researchers for publishing in the top computer graphics publications.  

Zhang also helped establish SFU’s Professional Master of Science Program in the School of Computing Science, serving as program director and developing the visual computing concentration.

Now, he has turned his keen mind to the topic of computational creativity, an emerging AI area focused on a machine’s ability to create. It’s a relatively new and unexplored area, but Zhang and his team are determined to find answers.

“Personally, this is my dream problem,” he says. “In one of our earlier works, we mimicked the evolution process where random mutation and crossover operations over 3D shapes led to potentially creative outcomes.”

“With advancements in machine learning, we are exploring means to break the barrier of traditional learning paradigms, which would allow a trained network to venture beyond the training data, gradually but surely, and hopefully lead to creativity.”

SFU’s School of Computing Science boasts one of the strongest computer vision and graphics research groups in the world, according to CS Rankings.

Simon Fraser University’s School of Computing Science is mobilizing brilliant minds to create business and societal innovation for good. For more information, visit sfu.ca/csresearch.