Yun Raymond Fu, a distinguished figure in Artificial Intelligence and Computer Science, brings a wealth of expertise to his role as COE Distinguished Professor at Northeastern University. With a rich academic background spanning Xi’an Jiaotong University and the University of Illinois at Urbana-Champaign, Fu’s contributions to AI, particularly in computer vision and machine learning, have garnered global recognition. He is a prolific author with over 500 scientific publications and numerous awards, Fu’s groundbreaking work includes the Residual Dense Network (RDN) and Residual Channel Attention Networks (RCAN), revolutionizing image restoration. Beyond academia, Fu’s entrepreneurial ventures have reshaped industries, notably through Giaran, a company acquired by Shiseido, showcasing the tangible impact of his research on global markets.
In honor of his many achievements, he has received the 2024 Edward J. McCluskey Technical Achievement Award for, “… innovative and impactful contributions to representation learning, computer vision, face, and gesture recognition.”
Building a strong research community is crucial for advancing any field. How do you foster collaboration and make connections within the IEEE community, and what role do you believe this plays in advancing technology?
IEEE is the largest technical professional association in the world. Engaged activities within IEEE community are great and most effective ways to advance research and technology development. I foster collaborations and make connections through research publications, society services, and education outreach. Publishing technical research papers in IEEE conferences and transactions is the most common way for researchers to exchange ideas, collaborate, and jointly promote technology breakthroughs. Most collaborators first met in IEEE conferences or workshops. The casual discussions on such professional occasions often stimulate new connections, novel research topics, and academic ideas. I am also enthusiastic to organize and serve conferences and workshops as Chair, Committee Members, and Reviewers, which naturally foster my collaboration with other organizers and connections with sponsors, industry, and the global IEEE community. Motivating students to stay interested in technology and science becomes a major educational challenge because they cannot see much practical use in their studies. I have been actively leading our IEEE local chapter to foster education outreach to connect local community instructors with students at different levels to participate in research projects to enable students to experience the excitement of research and technology innovation.
Honor your colleagues’ achievements. Nominate Someone for a Major Award Today!
Your Residual Dense Network (RDN) and Residual Channel Attention Networks (RCAN) are groundbreaking in image restoration. Can you share what inspired you behind the development and how you overcame any challenges throughout the process?
Our Residual Dense Network (RDN) and Residual Channel Attention Networks (RCAN) were the seminal very Deep Convolutional Neural Networks for image restoration which have been highly cited and become benchmark state-of-the-art in the computer vision community since 2018, adopted by thousands of research teams, companies, and products. Unlike most previous deep convolutional neural network models which do not make full use of the hierarchical features from the original low-resolution images, RDN and RCAN incorporate the Contiguous Memory mechanism and Residual in Residual structure to form very deep networks which can fully exploit the hierarchical features from all the convolutional layers. The major challenge behind this project was that the image restoration field has been developed for long history. The latest improvement has been incremental and bottlenecked. Our key to making this breakthrough to bring the general image restoration performance to a new level of accuracy is by critically believing feasibility and exploring previously considered impossible ideas, such as the computability of extremely deep neural networks for image superresolution or denoising. I would give credit to all my students who made significant efforts and excellent contributions to this work.
As a successful entrepreneur and founder of AI startups, can you share your insight on the challenges and opportunities of transferring academic research into commercially viable products?
Transferring academic research outcomes into commercially viable products is a real challenge. The gap between academic research and commercial market fit is prominent. Most researchers or professors in the technology field do not have enough domain knowledge for business or commercialization. I summarize three paths for this gap-filling based on my previous experiences. First, connecting with the commercialization office of the academic institution is the fastest path to get started. Different institutions may have different policies and supporting resources for technology transfer. It is a good idea to get familiar with the institutional policy and all the available resources and take advantage of these low-hanging fruits. Some institutions directly provide a transferring mechanism for commercialization and marketing research. Second, we shall consider entrepreneurship as a lifelong learning journey. Attending technology transfer training programs, such as NSF I-Corp, or VC-backed incubators and accelerators is extremely helpful for first-time entrepreneurs. These programs will provide basic education functions to help participants quickly gain enough commercialization knowledge, such as marketing research, and product development. Usually, an experienced mentor will be matched to supervise the entrepreneur, which is a valuable learning opportunity. Third, finding the right partner who complements skill sets and expertise is another way to overcome the challenges. It is always helpful for a technology entrepreneur to team up with someone with a business and product background, who could translate “technical language” using “business language” and bridge the conversation between technology founders and investors. Most early investors weigh the founding members more than the technology itself.
Your company Giaran, acquired by Shiseido, transformed the global cosmetic industry through computer vision-driven augmented facial analysis. How do you see technology continuing to shape the future of the cosmetic industry, and what lessons have you learned from this venture?
Giaran Inc. was a spin-off company from my Northeastern University SMILE lab that leverages advanced artificial intelligence technology to provide an interactive virtual experience for consumers to find and try cosmetics products. In the SMILE Lab, our team focuses primarily on research with security and public safety applications. Beauty, on the other hand, was hardly on my team’s radar initially. In the early stages of this research, we didn’t know much about fashion, makeup, and beauty. However, this potential application took shape as we were researching how to digitally remove someone’s makeup to improve facial recognition. It was then that we thought If we can remove someone’s makeup, maybe we can apply this to other domains. After conducting extensive market research with beauty industry professionals as well as consumers. We found a strong interest in the market for an efficient web tool that allows users to virtually try on cosmetics and find products that best match their face shape, skin tone, and texture. Merging into Shiseido the global leading firm will advance our beauty simulation technology and transform the consumer experience by taking personalized beauty to the next level.
With over 500 scientific publications and numerous awards, it’s evident that you’ve made several impactful contributions to the computing community, and beyond. How does it feel to see your work continuously having a significant impact, and what advice do you have for young researchers aspiring to follow a similar path?
I am always curious about unknown nature and enthusiastic about exploring discoveries from practice– the motivation that always drives me to create disruptive technology and tools from fundamental research to benefit people and help save our planet. To achieve the goal, we need curiosity, passion, diligence, and persistence. Seeing my work continuously having a significant impact in both academic and industry communities is a great self-satisfaction and inspiring. This honor means a lot to me. It recognizes my two decades of research and innovation achievements with broad impact. l suggest aspiring researchers always stay passionate and persistent in their field of study and research. Research is a high-risk and high-return work, we need to be patient and positive when facing challenges.
More About Yun Raymond Fu
Yun Raymond Fu is COE Distinguished Professor in the Department of Electrical and Computer Engineering and jointly appointed in the Khoury College of Computer Science at Northeastern University. His research in Artificial Intelligence includes computer vision, machine learning, and data mining, especially representation learning, image, and video analysis, and face and gesture recognition. He received a B.Eng. degree in information engineering and an M.Eng. degree in pattern recognition and intelligence systems from Xi’an Jiaotong University, and M.S. degree in statistics, and Ph.D. degree in electrical and computer engineering from the University of Illinois at Urbana-Champaign, respectively. He is a Member of Academia Europaea (MAE), Member of European Academy of Sciences and Arts (EASA), Fellow of National Academy of Inventors (NAI), AAAS Fellow, IEEE Fellow, AIMBE Fellow, IAPR Fellow, OSA Fellow, SPIE Fellow, AAIA Fellow, and ACM Distinguished Scientist.
He has authored over 500 scientific publications and 40. Among his 100 honors and awards, he received 7 Prestigious Young Investigator Awards from NAE, ONR, ARO, IEEE, INNS, UIUC, Grainger Foundation; 12 Best Paper Awards from IEEE, ACM, IAPR, SPIE, SIAM; and many major Industrial Faculty Research Awards. His several high-impact and fundamental research contributions include the Residual Dense Network (RDN, CVPR2018) and Residual Channel Attention Networks (RCAN, ECCV2018), two seminal very deep convolutional neural networks for image restoration, which have been cited over 8,000 times within five years. He was also leading original research in representation learning, by recreating graph-embedded manifold learning, deep learning, and transfer learning through sparse, low-rank, and generative modeling to extract discriminative visual patterns, for face and gesture recognition, which has been 5 times First-Place Winner in international technology competitions held in CVPR, ICCV, and ACM MultiMedia.
Beyond academic research achievements, he has been a high-impact inventor and a successful serial entrepreneur. As a 4-times AI startup founder, he commercialized his research inventions and created the Northeastern University spin-out company, Giaran, which was acquired by the prestigious global cosmetic firm Shiseido in 2017. His computer vision-driven augmented facial analysis and interaction technology has transformed Shiseido’s worldwide billion-level e-commerce market; and reinvented the digital cosmetic industry business model via technology innovation.
