News

PolyU harnesses GeoAI technologies to enable sustainable urban development

Geospatial artificial intelligence (GeoAI) is an interdisciplinary field that combines geospatial science and artificial intelligence (AI). The Hong Kong Polytechnic University (PolyU) is harnessing innovative GeoAI technologies to provide ground-breaking solutions for some of the environmental and social challenges facing the world today, in various fields including transportation, urban and public safety, planning, climate change and natural disasters. Prof. Qihao WENG, Chair Professor of Geomatics and Artificial Intelligence of the Department of Land Surveying and Geo-Informatics, and Global STEM Professor, established the PolyU Research Centre for Artificial Intelligence in Geomatics (RCAIG), to focus on the development of original and innovative AI methodologies and technologies for geomatics and their applications in urban areas, with the goal of it becoming a global R&D hub in GeoAI. Prof. Weng has recently been honoured with the 2024 American Association of Geographers (AAG) Wilbanks Prize for Transformational Research in Geography and the 2024 AAG Remote Sensing Specialty Group Lifetime Achievement Honor Awardfor his ground-breaking contributions in geography. One direction for the Research Centre is investigation into human-environment interactions in urbanisation by utilising geospatial analytics, GeoAI and big data methods. Another direction is to create various data products of global urban areas using Earth Observations (EO) and to provide EO-based urban data services. As RCAIG Principal Investigator and Director, Prof. Weng said, “Earth observation is important as a guiding compass for understanding changes in the environment and society. Our research focuses on diverse fields including geospatial big data and AI, remote sensing, ground-based sensors, navigation and positioning, surveying and geodesy, laser scanning, and photogrammetry. These technologies play a crucial role in addressing and resolving key environmental and social challenges.” In particular, GeoAI has revolutionised building monitoring by utilising thousands of learnable parameters. An illustration of this is its ability to automatically learn and identify general patterns of buildings such as colour and shape. This technology is crucially applied to detect disaster-damaged buildings, retrieve building height, identify structural changes and estimate building energy consumption. As a result, GeoAI has emerged as a mainstream solution for more efficient and insightful building monitoring. Environmental monitoring In the field of urbanisation monitoring, an RCAIG research team has developed an impervious surface area based urban cellular automata (CA) model that can simulate the fractional change of urban areas within each grid by utilising annual urban extent time series data obtained from satellite observations. By categorising the historical pathways of urban area growth into different levels of urbanisation, the model offers more detailed insights compared to traditional, binary, CA models. This demonstrates its great potential in supporting sustainable development. Research conducted by Ms Wanru HE, an RCAIG doctoral research assistant, and her team was reported in the paper “Modeling gridded urban fractional change using the temporal context information in the urban cellular automata model”, and has been published in Cities. Their model effectively captures the dynamics of urban sprawl with significantly improved computational efficiency and performance, and will help enable the modelling of urban growth at regional and even global level, under diverse future urbanisation scenarios. GeoAI for traffic management In the area of smart traffic management, to enhance the efficiency of ride-hailing platforms and achieve intelligent management of their services, the RCAIG research team has developed a multi-agent order matching and vehicle repositioning approach. This innovative technology focuses on coordinating the supply and demand of ride-hailing services, ultimately aiming to improve their overall efficiency. Their approach provides a ground-breaking solution to tackle two critical aspects necessary for efficient ride-hailing services. Firstly, it addresses order matching by efficiently assigning orders to available vehicles. Secondly, it incorporates proactive vehicle repositioning, strategically deploying idle vehicles to regions with potentially high demand. Based on multi-agent deep reinforcement learning, this innovation solves the complex planning issues in transportation and offers a new perspective on a long-term spatio-temporal planning problem. The research conducted by Ms Mingyue XU, another RCAIG researcher, and her team, was reported in the paper “Multi-agent reinforcement learning to unify order-matching and vehicle-repositioning in ride-hailing services”, and has been published in the International Journal of Geographical Information Science. The study achieved outstanding results, including reduced passenger rejection rates and driver idle time. About RCAIG With a focus on GeoAI, RCAIG is dedicated to conducting research in diverse fields, including urban building and energy, urban safety and security, environmental monitoring and conservation, urban resilience and public health. This aligns with the 11th United Nations Sustainable Development Goal, which aims to create inclusive, safe, resilient and sustainable cities and human settlements. Website: https://rcaig.com/ Learn more about Prof. Qihao Weng’s research focus in the video: https://polyu.me/3Vt7yih

PolyU researcher recognised and awarded for excellence in colour and imaging technologies

The Hong Kong Polytechnic University (PolyU) has developed a series of colour management technologies that provide vision-based solutions for lighting, imaging and metaverse systems. These cutting-edge technologies have been adopted by social media, imaging systems and manufacturers of light-emitting diode (LED) lighting products, smartphones and drones. Prof. Tommy Minchen WEI, Professor of the Department of Building Environment and Energy Engineering and Director of the Colour Imaging and Metaverse Research Centre at PolyU, has spearheaded research on the fundamental of colour science and the development of advanced algorithms and solutions for various imaging systems. He has been bestowed with the Hong Kong Engineering Science and Technology (HKEST) Award by the Hong Kong Academy of Engineering Sciences (HKAES) for his contributions in colour and imaging science.  Prof. WEI said, “This achievement has further supported my ongoing pursuit of interdisciplinary research excellence in colour, with the aim of sharping a visually better world for all.” Translating research outcomes with a significant community impact is highly fulfilling. Prof. WEI added, “As a scientist, my aspiration is to develop new algorithms based on human visual mechanisms that enhance colour appearance and calibrate perception variances across imaging systems. These systems encompass displays and cameras used in virtual reality, augmented reality, mixed reality, and the metaverse.”  One notable achievement is the development of a six-channel RGBACL (Red, Green, Blue, Amber, Cyan and Lime) lighting module and control algorithm, which enhances the colour presentation of LED lighting systems. The algorithm for tuning the colour appearance of human face skin allows quick adjustment when captured by different cameras. This technology has been used by a famous manufacturer of cinema lighting fixtures and in the production of several Hollywood movies and TV series. Prof. WEI also developed various solutions for capturing and displaying colours on video and image shooting equipment. These technologies encompass the implementation of artificial intelligence (AI) technology to detect human faces in the frame and optimise the colour appearance of skin tone. Other innovations, such as chips with AI-powered white balance algorithm built-in ensure more faithful colours in photos and videos taken by digital cameras under different environment. These technologies have been adopted by world-class smartphone and XR device manufacturers. Prof. WEI said, “My goal is to empower creative freedom by enabling creators to deliver the precise colours they intend for users to perceive. It is a tremendous honour to witness my research being translated and endorsed by world-class social media platforms, imaging systems, and manufacturers of digital devices worldwide.”

PolyU food scientist’s research on fungus-based fat replacer rolls out for low-fat dessert production at hotel

The Research Institute for Future Food (RiFood) at the Hong Kong Polytechnic University (PolyU) has developed a groundbreaking fat replacer called AkkMore™. This innovative formula has been used in the production of low-fat food products, including low-fat ice cream with a fat content of 3%. AkkMore™ is a fungus and plant-based supplement targeting obesity or prediabetes. This innovative formula was developed by a research team led by Dr Gail Jinhui Chang, Research Assistant Professor in the Department of Food Science and Nutrition at PolyU and a project investigator at RiFood.  Produced from natural fungal sources, AkkMore™ has been tested for its effectiveness in providing numerous health benefits, which include preventing obesity and other metabolic diseases, enhancing gut health, modulating the immune response, and reducing anxiety. It was awarded a silver medal at the 2022 Special Edition of the Geneva International Exhibition of Inventions. Dr. CHANG, said “Our research team has completed three rounds of animal trials on AkkMore™. The results show that the formula can effectively improve metabolism and aid in weight management. Moving forward, we are focusing on exploring applications of AkkMore™ in the development of healthy food.”  The research team is currently developing “Cream Mate”, an AkkMore™-based cream substitute. Cream Mate enables cream products to contain less fat, have the ability to be frozen for a long time, and extend shelf-life while maintaining sensory appeal comparable to traditional cream. The use of Cream Mate helps reduce calorie intake and the consumption of dairy products in desserts, as well as minimise food waste due to expiration. This contributes to making food production more sustainable and economical. In addition, RiFood is collaborating with Hotel ICON on the use of Cream Mate in the preparation of reduced-fat desserts that are served on the hotel’s regular menus, starting from 1 May.  

PolyU researchers introduce biomineralisation as sustainable strategy against microbial corrosion in marine concrete

Microbially induced corrosion (MIC) is a major issue in marine environments, leading to structural damage such as cracking in concrete infrastructure. This corrosion poses a persistent challenge, significantly reducing the lifespan of marine structures and resulting in substantial economic losses. In response to the need for an effective solution to combat marine corrosion in concrete, The Hong Kong Polytechnic University (PolyU) researchers have developed a biomineralisation approach to protect marine concrete from MIC. Prof. Xiang-dong LI, Dean of the Faculty of Construction and Environment, Director of the Research Institute for Sustainable Urban Development, Chair Professor of Environmental Science and Technology, and Ko Jan Ming Professor in Sustainable Urban Development at PolyU, has led research that successfully introduced a novel biomineralisation strategy which effectively isolates marine concrete from MIC, thereby contributing to the realisation of sustainable coastal structures. MIC in concrete usually occurs in harsh environments with the presence of corrosive microorganisms in, for example, sewage structures, wastewater treatment plants, and marine structures. The formation of a biomineralised film on concrete surfaces is typically considered to be the most effective anticorrosion mechanism as it can provide a barrier to inhibit corrosion build-up. Prof.Li said, “Our biomineralisation technique serves as an environmentally friendly coating method for controlling concrete corrosion, with minimal impact on the overall biofilm communities. Also, it utilises carbon dioxide to produce mineral precipitates, enhancing the durability of concrete structures. This process not only reduces the carbon footprint and energy consumption of marine infrastructure throughout its lifespan, but also makes a valuable contribution to carbon neutrality and sustainability.” The study has demonstrated that the biomineralisation treatment effectively prevents corrosion by reducing the total and relative abundance of sulfate-reducing bacteria (SRB). SRB is a type of anaerobic bacteria and can produce hydrogen sulfide, which is corrosive and can, in turn, lead to material deterioration. The biomineralised film acts as a protective layer, controlling sulfate diffusion and isolating the concrete from corrosive SRB communities. This protective mechanism significantly extends the lifespan of concrete structures and has no negative impact on the native marine microbial communities. Prof. Li added, “If the biomineralised film remains intact, repainting the concrete structures is unnecessary. The utilisation of a single coating treatment eliminates the need for multiple treatments, further minimising the cost and carbon footprint.” This biomineralisation strategy has strong potential for applications in corrosive environments, such as in marine and sewage settings, and water-cooling utilities, where concrete corrosion is induced by corrosive microorganisms. A paper reporting the research, “Biomineralisation to prevent microbially induced corrosion on concrete for sustainable marine infrastructure”, has been published in Environmental Science & Technology. The study employed a combination of chemical and mechanical property measurements of concrete, in conjunction with an analysis of the microbial community of biofilms, to evaluate the effectiveness of biomineralisation techniques in inhibiting corrosion of marine concrete. These assessments aimed to enhance understanding of MIC development. The results contribute to the development of new techniques for inhibiting corrosion to achieve sustainable marine concrete structures. Considering that the type of colonised surface also affects the treatment effect of biomineralisation, the effectiveness of biomineralisation will be further investigated for different types of concrete to expand its potential application.  In addition, functional prediction can be used in future studies to obtain a mechanistic understanding of the possible metabolic capability of microbial action on concrete corrosion. This understanding would be beneficial for uncovering the mystery of the effect of SRB on the lifespan of marine concrete structures.

PolyU drives sustainable development through scientific research supported by the Environment and Conservation Fund

Ten projects of the Hong Kong Polytechnic University (PolyU) have been funded by the Environmental and Conservation Fund (ECF), with a total awarded amount of HK$5.396 million. PolyU scholars excel in innovative and applied research, making a significant contribution to goals of carbon neutrality and sustainable development for environmental protection in Hong Kong.  The ten awarded research projects cover a variety of topics, including waste management and recycling, air and water quality protection, climate change, energy conservation and carbon reduction. These projects demonstrate PolyU's research expertise in environmental protection across diverse disciplines such as in construction and building, engineering and applied sciences.  The research topics initiate innovative solutions and technologies, such as a mobile sensor platform and computer vision-based technology for street air quality monitoring, unmanned ground and aerial vehicles for beach refuse collection, and the development of a photocatalytic system to reduce carbon footprint. Some of the awarded projects aim to address climate change by focusing on the development of cooling films for the next generation of data centre and analysing the environmental impact of tyre wear emission.  The ECF was set up by the government of HKSAR to provide funding support for educational, research and other projects and activities in relation to environmental improvement and conservation of local environment since 1994. Details of the 10 ECF funded projects:   Principal Investigator Project Title Funded Amount Faculty of Construction and Environment Dr HOU Huiying, Assistant Professor of Department of Building Environment and Energy Engineering Dynamic Monitor and Analysis of Street Air Quality in High-density Urban Environments in Hong Kong Utilizing Mobile Sensor Platform and Computer Vision-based Technology HK$499,600 Dr SONG Dingbao, Research Assistant Professor of Department of Civil and Environmental Engineering Turning Incinerated Sewage Sludge Ash into Engineering Fill Using Microbial Induced Calcium Carbonate Precipitation Method Incorporating CO2 Sequestration HK$500,000 Dr SHEN Peiliang, Research Assistant Professor of Department of Civil and Environmental Engineering Development of Concrete Slurry Waste-derived Carbonation Coating to Improve the Durability of Concrete Structures HK$500,000 Dr WANG Meng, Research Assistant Professor of Department of Civil and Environmental Engineering Advance Our Understanding of the Environmental Impact of Tyre Wear Emission in a Changing Climate HK$900,000 Faculty of Engineering Dr CHAN Kit Ying, Research Assistant Professor of Department of Aeronautical and Aviation Engineering Multifunctional Cooling Films for Energy-efficient Passive Cooling of Hong Kong's Next-generation Data Centre HK$498,600 Dr XU Gangyan, Assistant Professor of Department of Aeronautical and Aviation Engineering Autonomous Sandy Beach Refuse Collection System with Unmanned Ground and Aerial Vehicles HK$498,000 Dr REN Jingzheng, Associate Professor of Department of Industrial and Systems Engineering Turning Sewage Sludge into Green Hydrogen and E-fuel with Carbon-neutrality Goals in Hong Kong: Innovative Process Design, Optimization, Economic Viability and Environmental Sustainability Assessment HK$500,000 Faculty of Science Dr LO Tsz Woon Benedict, Assistant Professor of Department of Applied Biology and Chemical Technology Developing a Photocatalytic System to Reduce Hong Kong's Carbon Footprint: Converting CO2 Emissions into Useful Two-carbon Chemicals HK$500,000 Prof WU Jianyong, Research Professor of Department of Food Science and Nutrition Development of Phase Change Nano-emulsion Based Spectral Splitting PV/T System for the Full Solar Energy Usage HK$500,000 School of Fashion and Textiles Dr KANG Zhanxiao, Research Assistant Professor of School of Fashion and Textiles Textile-based Evaporative Air Grid to Enhance the Efficiency of Air Conditioners Utilizing Condensed Water HK$500,000

PolyU and Jiangyin High-tech Zone in Jiangsu Province signed agreement on technology and research collaboration

Professor Christopher Chao, Vice President (Research and Innovation) of The Hong Kong Polytechnic University (PolyU), led professors from different departments of PolyU to Jiangyin Hi-tech Zone in Jiangsu Province for technology exchange activities in late March. With the support of the Department of Science and Technology of Jiangsu and the Jiangsu Productivity Promotion Center, PolyU and Jiangyin High-tech Zone signed a MoU for technology and research collaboration.  This MoU will further expand the areas of cooperation between Jiangsu and Hong Kong. By enhancing the supply of technological innovation resources, Jiangyin government will support PolyU's innovation projects to be developed and applied in Jiangyin High-tech Zone. They will establish a connection between the technological needs of the enterprises and the research outcomes of PolyU to solve the technical problems faced by these enterprises. Additionally, they will jointly organise technology exchange activities and explore the establishment of technology transfer and R&D platforms, such as joint laboratories, etc. This cooperation enables PolyU and Jiangyin Hi-tech Zone to further explore technological innovation and industrial transformation, injecting new vitality into Jiangsu-Hong Kong cooperation. After the signing ceremony, researchers from different departments of PolyU introduced the latest scientific and technological achievements to representatives of enterprises in Jiangyin District. The discussions covered technology areas such as ultra-precision machining, electric vehicles, microelectronics, biomedicine, new materials, etc. The enterprises showed great interest in these advanced technologies and are looking forward to research collaboration with professors. The delegation has also visited high-tech enterprises in the Jiangyin area, including Exhibition Hall of CITIC Pacific Special Steel Group, Jiangsu Changjiang Electronics Tech, and the Jiangyin Intelligent Manufacturing Innovation Institute to learn about the history and development of different industries while interacting with corporate R&D personnel. In the past few years, PolyU has promoted cooperation in research and education between Jiangsu and Hong Kong which has achieved fruitful results. This exchange activity is a continuation of the cooperation agreement signed by PolyU and the Jiangsu Productivity Promotion Center at the "The 2nd Jiangsu Conference for Industry-University-Research Cooperation and Exchange" last year. The agreement proposed the joint establishment of a technology innovation transfer center to support technology transfer, knowledge exchange activities, and delegations. In mid-April, PolyU will organise a technology exchange event with Jiangsu enterprises. It is anticipated to strengthen communication and understanding between university researchers and enterprises, fostering research collaboration and promote technology transfer. It will also serve as a platform for PolyU researchers to cooperate with enterprises.   

Visit by Delegation of CHINT Group

A delegation led by Mr. NAN Cunhui, Chairman of CHINT Group, visited The Hong Kong Polytechnic University (PolyU) on April 2 to exchange views on strengthening cooperation in technology and innovation between the two sides. The delegation was warmly welcomed by Prof. Jin-Guang TENG, President of PolyU, Prof. Christopher CHAO, Vice President (Research and Innovation) of PolyU and Dr Laura LO, Associate Vice President (Institutional Advancement) of PolyU, and exchanged views with the guests during the lunch meeting at Hotel Icon.  After the meeting, Prof. CHAO led the delegation to visit the PolyU campus and related laboratories. The delegation visited the Research Institute for Artificial Intelligence of Things (RIAIoT), Advanced Materials and Electronics Laboratory, and State Key Laboratory of Ultra-precision Machining Technology (SKL-UPMT).  

PolyU partners with Yangjiang Offshore Wind Energy Laboratory to establish joint research centre to drive development of offshore wind power technology

The Hong Kong Polytechnic University (PolyU) and Yangjiang Offshore Wind Energy Laboratory (the Laboratory) signed an agreement to jointly establish the PolyU-Yangjiang Laboratory Joint Research Centre for Offshore Wind Power (JRC). The JRC will be operated and managed jointly by the PolyU Research Institute for Land and Space (RILS) and the Laboratory, signifying an increasingly close research collaboration between the two organisations in offshore wind power technology. Under the new partnership, PolyU and Yangjiang researchers will share resources and laboratory facilities, and join force in research projects on new technologies and materials for offshore wind power applications, as well as translating the research outcomes into practical solutions for industry. The JRC signing ceremony was held on the PolyU campus and attended by a delegation from Yangjiang in Guangdong province together with PolyU representatives. Distinguished guests included Prof. Christopher CHAO, PolyU Vice President (Research and Innovation); Mr LIU Dewei, Vice Mayor of Yangjiang Municipal People’s Government; Prof. DONG Cheng, PolyU Associate Vice President (Mainland Research Advancement); Prof. CHEN Qingyan, Director of the PolyU Academy for Interdisciplinary Research (PAIR); Prof. DING Xiaoli, Director of RILS; Mr MO Jiaqiang, Deputy Director of Yangjiang Science and Technology Bureau; Mr LIN Tao, Deputy Director of Yangjiang Development and Reform Bureau, and Prof. ZHU Ronghua, Director of Yangjiang Offshore Wind Energy Laboratory. In his welcoming remarks, Prof. Christopher Chao said, “Wind power has emerged as a pivotal strategy for many countries to achieve carbon neutrality. The collaboration between PolyU and Yangjiang Offshore Wind Energy Laboratory not only propels the advancement of offshore wind power knowhow, but also fortifies a mutual engagement in research, talent development and knowledge exchange. The JRC will amalgamate the research capabilities and expertise of both organisations to address major scientific challenges and surmount technical barriers in the energy field. This, in turn, will support the development of a world-class research and innovation powerhouse for wind power industry and clean energy.” The Electrical and Mechanical Services Department of the HKSAR Government has estimated that wind power could satisfy approximately one-third of Hong Kong’s electricity demand if the City’s offshore wind energy reaches 10.7 billion kilowatts. The construction of offshore wind turbines often involves technical challenges in engineering design and construction. The collaboration between PolyU and the Laboratory will facilitate interdisciplinary research and the development of practical engineering solutions to benefit Hong Kong, the Nation, and the world in the construction of offshore wind farms. The Laboratory brings together top experts from mainland universities, including Zhejiang University, Sun Yat-sen University, Shanghai Jiao Tong University, South China University of Technology, Nanjing University of Aeronautics and Astronautics, Guangdong Ocean University and Hebei University of Engineering, to conduct research on fundamental science, as well as applied and core technology in the field of offshore wind power. Established in 2021, RILS is one of PAIR’s 18 constituent research institutes and research centres with over 50 PolyU scholars from different disciplines specialising in interdisciplinary research and knowledge transfer across five major areas: land reclamation; innovative solutions for land and space development; environmental treatment and impact; land economics and planning; and land analysis and management. Offshore wind power is one of the areas which RILS actively pursues. In November 2023, RILS delegates visited wind energy-related laboratories in Yangjiang, including the Yangjiang Offshore Wind Energy Laboratory and met with representatives of the Laboratory for potential research collaboration opportunities.

PolyU and Ganzhou sign strategic collaborative framework agreements to deepen research collaboration between the two regions

The Hong Kong Polytechnic University (PolyU) has partnered with Ganzhou Municipal Government and Xingguo County Government in Jiangxi province to capitalise on their strengthens and promote the high-quality development of the textile industry in Ganzhou and Hong Kong. Prof. Christopher CHAO, PolyU Vice President (Research and Innovation), led a delegation to participate in the “1st China (Ganzhou) Graphene cum Textile and Apparel and New Materials Industry Forum” on 19 March in Xingguo County, Ganzhou. At the event, Prof. Chao signed strategic collaborative framework agreement separately with Mr DOU Liangtan, Member of the Standing Committee of the Communist Party of China (CPC) Ganzhou Municipal Committee and Vice Mayor of Ganzhou, and Mr LIU Zhanghong, Deputy Secretary of Xingguo County Party Committee and County Mayor of Xingguo, for the joint establishment of the PolyU-Xingguo Textiles Technology and Innovation Research Institute (the Institute) to promote in-depth and effective industry-university-research collaboration. The Institute will support the innovative development of the textile industry in Ganzhou and its counties, while creating further opportunities for PolyU in terms of education, research and translation of research into innovative outcomes in the field of textiles. Prof. Christopher Chao said, “By putting efforts into brand building, technology promotion, professional training and talent development, PolyU will make contributions to the advancement of the textile industry and market in Ganzhou and Xingguo County. The solicitation of business and projects will also facilitate investment and further collaborative opportunities. We will both fully leverage our respective strengths and join forces to create a new chapter in the textile industry.” PolyU possesses strong educational and research capabilities in the textiles field, which align with the development needs of Ganzhou’s textile industry in the areas of talent cultivation, research, industrial transformation and upgrading. The collaboration between PolyU, Ganzhou and Xingguo County and the establishment of the Institute will leverage the University’s expertise in talent, technology and research platforms, to support the enduring growth of local textile enterprises. The forum themed on “New Weave of New Graphene” gathered experts from universities, enterprises and industry in the fields of graphene and textiles from all over the country, to discuss research and industrial applications of graphene (also known as the King of New Materials). Dr Shou Dahua, Assistant Professor and Limin Endowed Young Scholar in Advanced Textiles Technologies of PolyU School of Fashion and Textiles introduced his research achievements in textiles and new materials at the Forum. On the same day, the country’s first Graphene Technology Museum was inaugurated in Xingguo County, to serve as a national hub for graphene innovation to translate research outcomes into new productive forces.

PolyU research reveals significant effects of instructors onscreen during video classes in aiding student learning

Online learning has become “the new normal” of education since COVID-19 severely disrupted face-to-face teaching activities. Researchers from The Hong Kong Polytechnic University (PolyU) have conducted a study to analyse whether and how the instructor’s presence in online video lectures affected student learning and learning outcomes. The results reveal that students are more motivated to perform socio-emotional and cognitive processing when an instructor, human or animated, is present onscreen, hence facilitating more effective learning. The study findings have been published in the journal Proceedings of the National Academy of Sciences. Previous studies have indicated that socio-emotional cues, such as human facial expressions and gestures, help students understand and stay focused on learning content. Led by Prof. Ping LI, Dean of the Faculty of Humanities and Sin Wai Kin Foundation Professor in Humanities and Technology at PolyU, the research team studied how learners respond to virtual learning with different types and levels of interaction by examining multimodal data of students’ learning performance, brain activity and eye movement, as well as the correlation between these measurements. Eighty-one PolyU students participated in the experiment where some of them watched video lectures with a human instructor accompanying the lecture slides, some watched the same video lectures with an animated instructor, and still some watched the lectures with no onscreen instructor and only the lecture slides. This was followed by a set of assessments of how effectively they had learned. Compared with the no-instructor group, students with an onscreen instructor performed significantly better in the post-course assessments, while the exact embodiment of the instructor—real vs animated—did not affect the overall scores. The results provide strong and important evidence that the instructor’s image, human or animated, improves educational outcomes in the virtual setting. The researchers examined the impact of instructor’s image on learning through a combination of functional magnetic resonance imaging (fMRI) and eye-movement tracking of the students as they watched the lectures. While students with an instructor performed better, the eye-tracking results, surprisingly, showed that the human instructor may actually distract the students from the slides, as more time was spent by students looking at the slides when the instructor was absent or was an animation. A deeper analysis of the eye-tracking data resolved this contradiction. Crucially, the correlation of eye movements—the extent to which the students shifted their gaze in unison—was higher in instructor-present groups than in the no-instructor group, and better-performing students also displayed more correlated eye movements than the lower performers. This suggests that although an instructor’s image may distract students from the slides, it is also more likely in guiding them to pay attention to the appropriate parts of the onscreen content. In other words, students with an instructor tend to focus on the same places, while those without an instructor are more random in their focus. The fMRI results, which identified the specific brain regions students used most, aligned with the eye-tracking data. Just as their eye movements were synchronised, so did the students with an instructor show greater synchrony in the activity of brain regions crucial for learning, including regions involved in working memory and mentalising. This alignment can be attributed to the higher level of cognitive and socio-emotional processing motivated by the onscreen instructor that served as a social cue. Under this condition, learners follow the visual content in the video more closely, allocate attention more proactively and ultimately learn better. As the data suggest that an onscreen instructor’s image entails both socio-emotional benefits and attentional distraction unrelated to learning, the researchers further propose a trade-off hypothesis suggesting learning outcome depends on whether the benefits can outweigh the costs brought by the distraction. While the trade-off also relies on a learner’s ability to leverage socio-emotional processing and attention control for learning, it explains the individual difference in student learning under the same virtual setting. Prof. Li remarked, “Although the pandemic has subsided, online learning through the use of multimedia instructional videos continues to shape education. Our findings suggest that an onscreen instructor—even an animated one—can make up for some deficits of the online learning setting, where socio-emotional cues are less salient and cognitive engagement is harder to sustain. This contributes to the development of an evidence-based instructional design for multimedia learning, thereby enhancing student’s learning experience and outcome.”