bullet Technology & Innovation
Artificial intelligence improves shopping experience Artificial intelligence improves shopping experience

PolyU-developed technologies are revolutionising how customers shop for clothing online.

Prof. Calvin Wong 

Prof. Calvin Wong

As e-commerce flourishes globally, more and more people are shopping for clothes online. However, despite the convenience of online shopping, not only do customers experience difficulty in searching for the exact fashion design or style they have in mind, they also often encounter problems of ill-fitted clothes. In view of this, researchers from PolyU’s Institute of Textiles and Clothing (ITC) set out to meet customers’ needs and enhance their online shopping experience.

Bridging between fashion and artificial intelligence

ITC has collaborated with the Alibaba Group’s Vision and Beauty Team to create the first FashionAI Dataset to systematically analyse and label fashion images based on garments’ attributes and key points. This integration of machine learning and fashion knowledge has produced an unprecedented level of AI fashion understanding.

The current search technology for online fashion images is limited because it uses the whole image to search for similar images. Customers cannot search based on particular fashion attributes – the basic design elements of a garment – which greatly limits the extent to which their shopping experience can be customised. A fashion image dataset is therefore needed to include professional fashion knowledge and to fulfil the requirements of deep learning so that computers can be trained to recognise fashion attributes in images. However, the wide variety of fashion attributes makes attribute recognition complicated. A systemic classification system is thus essential to accurately label fashion attributes.

President Prof. Timothy W. Tong (first from left) experiencing the new technologies at the Fashion AI Concept
     Store. On his side are Alibaba Group Vice President Mr Zhuang Zhuo-ran, Deputy President and Provost
     Prof. Philip Chan and Institute of Textiles and Clothing Prof. Calvin Wong. 

President Prof. Timothy W. Tong (first from left) experiencing the new technologies at the Fashion AI Concept Store. On his side are Alibaba Group Vice President Mr Zhuang Zhuo-ran, Deputy President and Provost Prof. Philip Chan and Institute of Textiles and Clothing Prof. Calvin Wong.

A PolyU research team led by Prof. Calvin Wong of ITC worked with Alibaba to develop the FashionAI Dataset. It can detect both the key points of a garment, such as the neckline, cuff and waistline, and the garment’s fashion attributes, such as sleeve length, collar type and skirt style. This enables the computer to efficiently and effectively understand fashion images. This revolutionary dataset will improve the accuracy of online fashion image searching, enhance the effectiveness of cross-selling and up-selling, create innovative buying experiences for customers and facilitate the customisation of online shopping platforms.

A FashionAI Concept Store was set up on PolyU’s campus earlier to demonstrate this integration of AI and fashion retail. Visitors to the store experienced the benefits brought by the new technologies in selecting clothes and viewing personalised mix-and-match options.

3D human modelling technology

Dr Tracy Mok (left) and Dr Zhu Shuaiyin 

Dr Tracy Mok (left) and Dr Zhu Shuaiyin

Another PolyU team also developed an innovative technology for the fashion industry. ITC researcher Dr Tracy Mok and PhD graduate Dr Zhu Shuaiyin have integrated cutting-edge computer graphic and vision technology to develop an efficient new method for customised modelling individuals of all sizes and shapes.

Conventional methods for automatic shape modelling have limitations, as they use expensive and bulky scanners. In contrast, the newly developed 3D human modelling technology can accurately create a 3-dimensional figure from merely two full body photographs taken on mobile phones and extract over 50 size measurements, including the bust, waist, hip, thigh, knee, calf and neck, plus arm length and shoulder slope. These measurements are precise enough to fulfil the industry requirements for fashion design and manufacturing, and can be created in only 5–10 seconds.

This efficient and accurate modelling method enables customers to virtually “try on” garments before purchasing apparels online, helping them to better evaluate items and to choose the correct sizes. Thus, in addition to enhancing customers’ online shopping experience, this innovation will stimulate growth in online fashion shopping.

With support from the Shanghai Technology Entrepreneurship Foundation - PolyU China Entrepreneurship Fund, Dr Zhu established a start-up in Shenzhen, TOZI, to commercialise and further develop this 3D human modelling technology.

TOZI has collaborated with Industry 4.0 factories to launch a brand-new customer-to-manufacturer service that enables end users to order tailor-made clothing anywhere, anytime, in just a few minutes. Using their smartphones, customers measure themselves and order tailor-made shirts with the fabric, collar, pocket, cuffs and front plackets of their choice.

2D under-the-cloth contours prediction based on 2D photograph Digital reconstruction of 3D body shape 

2D under-the-cloth contours prediction based on 2D photograph Digital reconstruction of 3D body shape