Promoting integration of fashion and artificial intelligence

PolyU and Alibaba collaborate to establish the first-of-its-kind “FashionAI Dataset”, fostering the application of artificial intelligence (AI) in the fashion industry. (From left): Prof. Calvin Wong, Cheng Yik Hung Professor in Fashion, Associate Head of ITC; Prof. Wong Wing-tak, Dean of Faculty of Applied Science and Textiles; Menglei Jia, Senior Staff Engineer, Vision and Beauty, Alibaba Group

The Institute of Textiles and Clothing (ITC) of The Hong Kong Polytechnic University (PolyU) is collaborating with the Vision and Beauty Team at Alibaba Group, which specialises in vision intelligence and applications, to establish the first-of-its-kind “FashionAI Dataset” for systematic analysis and labelling of fashion images based on “fashion attributes” (fashion characteristics) and “key points” of an apparel.

By integrating fashion knowledge and machine learning formulation, the establishment of the Dataset will enable machine to better understand fashion, bringing a new horizon to the fashion retail industry through the application of AI.

“Transforming fashion knowledge into determination of fashion related attributes and fashion item categorisation of the fashion image database is a very complicated and challenging task, while it is the most fundamental task in deep learning applications. ITC is pleased to collaborate with Alibaba to address the needs of fashion retailers and consumers,” said Professor Calvin Wong, Cheng Yik Hung Professor in Fashion and Associate Head of ITC.

 Menglei Jia, Senior Staff Engineer at the Vision and Beauty team at Alibaba commented: “There is a huge potential for AI applications in the fashion industry. In order for AI to understand fashion, which could be very subjective, we need to turn fashion knowledge and experience into language that machine can understand.” 

Jia added, “We hope to work with academics and the industry alike to explore the wider applications of AI in scenarios including fashion mix-and-match, assisting design and shopping guide, with the aim to bring new values to the fashion industry. The traditional fashion sector should embrace the new retail practice, and we hope FashionAI can be a bridge that connects AI with fashion.”

Challenges presented by data on fashion image 

Current fashion image searching technology used on online platforms is based on the whole fashion image to search the exact or other similar images. However if a customer is interested in some particular fashion attributes of a fashion image and wants to search other fashion items with these attributes, the current searching technology cannot meet the needs of the customer. This greatly limits the potential development and applications for offering more customised shopping experience. 

From artificial intelligence (AI) research perspective, this limitation of the current image searching technology is caused by the absence of available fashion image dataset constructed with both fashion professional knowledge and fulfils the requirement of deep learning, i.e. the current technology is unable to train a machine to accurately understand and recognise the fashion attributes of each fashion image.   

World’s first dataset addressing the needs of fashion retailers

Fostering the application of AI in the fashion industry, a PolyU research team led by Professor Wong, worked closely with Alibaba to develop “FashionAI Dataset” to solve two fundamental problems of the deep learning algorithm: “apparel key points detection” and “attribute recognition”. 

Key points (e.g. neckline, cuff, waistline) and fashion attributes (e.g. sleeve length, collar type, skirt style) build the foundation for machine learning in understanding fashion images. The establishment of key points and fashion attribute database enables the computer to effectively and efficiently understand the fashion image which is fundamental for deep learning and recognition algorithms.

The accuracy of key points detection is determined by several factors such as the dimension and shape of the apparel, distance and angle of shooting, or even how the apparel is displayed or the model is posing in a photo. These factors can lead to poor key points detection and result in an inaccurate analysis of fashion images by the computer. Accurate key points detection can therefore improve the performance of deep learning algorithms.

Fashion attributes are the basic design elements of an apparel, and their combination determines the product category and styles of a fashion item. With the wide variety of fashion attributes, attribute recognition is a complicated process.  A systemic classification of fashion attributes is essential to accurately label fashion attributes, facilitating research on deep learning and algorithm design for fashion image searching, navigating tagging and mix-and-match ideas, etc. 

The revolutionary Dataset can greatly facilitate understanding fashion images and related algorithm design, and developing machine learning. It would help improve the accuracy of online fashion image searching, enhance effectiveness of cross-selling and up-selling, create innovative buying experience and facilitate customisation of online shopping platforms.

  World-first events: Artificial Intelligent on Fashion & Textile (AIFT) Conference and FashionAI Global Challenge 

With an aim of bringing a new horizon to the fashion retail industry through the application of AI and encouraging knowledge exchange among practitioners, PolyU and Alibaba will host two world-first events: AIFT Conference and FashionAI Global Challenge.

AIFT Conference is a first-of-its-kind academic conference to bring together researchers, engineers and practitioners to share their insights on the most updated development and applications of AI and fashion. AIFT Conference will be held on 3-6 July at PolyU. This event will become an annual activity for academic exchange and networking with like-minded individuals who are redefining the world of AI and fashion, and advancing AI research in fashion and textile. For more information, please visit https://www.polyu.edu.hk/itc/aift2018/index.html.

FashionAI Challenge invites worldwide AI researchers and developers to solve two imminent issues on the application of AI in fashion with over 400,000 images with high-quality annotations from Alibaba ecommerce platforms. The competition offers a prize pool of RMB 1.34 million. The FashionAI Global Challenge 2018 is open to the public, and the competition will be held from 1 February to 21 April. Visit http://fashionai.alibaba.com/ to learn more about the competition.