摘要 在本文中，我们提出了一个实用的服装推荐系统，在指定一件服装（上身或者下身服装）的情况下，该系统能自动的推荐出和指定服装相搭配的服装。然而，这是一个具有挑战性的问题因为服装种类多种多样。服装是人们外观最重要的组成信息。在我们日常生活中，人们在各种各样的场合下需要穿的得体并且美观以突显自己的自信、礼貌以及社会地位。但是如何穿着以及如何搭配服装并不是一件容易的事情。为了解决这个问题，我们提出了一个四路的卷积神经网络结构，其中一个双路网络采用siamese网络结构，训练样本是上下身服装的匹配或者不匹配对，另外一个双路网络用于学习输入图像的风格特征。本文所提出的框架可以把服装图像特征转换成两个隐式空间中，这两个隐式空间分别称为匹配空间和风格空间。在训练完这两个双路网络之后，我们使用特征距离融合的方法来融合从匹配和风格空间提取的特征。为了得到一个优化模型以及为了验证本文所提出的方法，我们扩充了一个大型的服装数据库称为“How to wear beautifully?”，简称H2WB。通过在H2WB数据集上做实验，说明了本文所提出的距离融合以及服装搭配推荐都有比较高的效率。
In this paper, we present a practical system to automatically suggest the most pairing clothing items, given the reference clothing (upper-body or low-body). However this has being a challenge, due to having varieties of clothing categories. Clothing is one of the most informative cues for human appearance. In our daily life, people need to wear properly and beautifully to show their confidence, politeness and social status in various occasion. But, it is not easy to decide to decide on what and how to wear at the same time to match with the selected clothes. To address this problem, we propose a quadruple network architecture, where one dual network adopts Siamese convolution neural network architecture. Training examples are pairs of upper-body and low-body clothing items that are either compatible or incompatible. Another dual convolution neural network is used to learn clothing style features of the input image. This framework allows learning a feature transformation from the images of clothing items to two latent spaces, which we called compatible space and style space. After training the two dual networks, we use a distance fusion method to fuse the features extracted from the compatible and style dual networks. To acquire an optimized model and verify our proposed method, we expand a large clothing dataset called "How to Wear Beautifully" (H2WB). Experiments on H2WB dataset demonstrated that our learning model are effective with feature distance fusion and clothing item pairing recommendation.
This work was supported by the National Natural Science Foundation of China under Grant Nos. 61379106, 61379082 and 61227802, the Natural Science Foundation of Shandong Province of China under Grant Nos. ZR2013FM036 and ZR2015FM011, the Major Program of National Natural Science Foundation of China under Grant No. 41631073, and the Qingdao National Laboratory for Marine Science and Technology of China under Grant No. QNLM2016ORP0401.
About author: Wen-Guang Chen is a professor in the Department of Computer Science and Technology, Tsinghua University, Beijing, where he has been teaching since 2003. He received his B.S. and Ph.D. degrees both in computer science from Tsinghua University in 1995 and 2000 respectively. His research interest is in parallel and distributed computing. He is a CCF distinguished member and a CCF distinguished speaker, and an ACM member and the vice chair of ACM China Council. He has served in program committees of a variety of major conferences in the parallel and distributed computing area, including PLDI, PPoPP, SC, CGO, CCGrid, IPDPS, APSys, and ICPP. He is the editor-in-chief of Communication of ACM China Edition.
Yu-Jie Liu, Yong-Biao Gao, Ling-Yan Bian, Wen-Ya Wang, Zong-Min Li.怎么样穿的更美观?服装搭配推荐[J] Journal of Computer Science and Technology , 2018,V33(3): 522-530
Yu-Jie Liu, Yong-Biao Gao, Ling-Yan Bian, Wen-Ya Wang, Zong-Min Li.How to wear beautiful? Clothing pair recommendation[J] Journal of Computer Science and Technology, 2018,V33(3): 522-530
 Kiapour M H, Han X, Lazebnik S, Berg A C. Where to buy it:Matching street clothing photos in online shops. In Proc. IEEE International Conference on Computer Vision, Dec. 2015, pp.3343-3351. Li Z, Li Y, Gao Y, Liu Y. Fast cross-scenario clothing retrieval based on indexing deep features. In Proc. PacificRim Conference on Advances in Multimedia Information Processing, Sept. 2016, pp.107-118. Lu H, Xu C, Liu G, Song Z, Liu S, Yan S. Street-to-shop:Cross-scenario clothing retrieval via parts alignment and auxiliary set. In Proc. IEEE Conference on Computer Vision and Pattern Recognition, June 2012, pp.1335-1336. Chen Q, Huang J, Feris R, Brown L M, Dong J, Yan S. Deep domain adaptation for describing people based on fine-grained clothing attributes. In Proc. IEEE Conference on Computer Vision and Pattern Recognition, June 2015, pp.5315-5324. Liu Z, Luo P, Qiu S, Wang X, Tang X. DeepFashion:Powering robust clothes recognition and retrieval with rich annotations. In Proc. IEEE International Conference on Computer Vision and Pattern Recognition, June 2016, pp.1096-1104. Zhang X, Jia J, Gao K, Zhang Y, Zhang D, Li J, Tian Q. Trip outfits advisor:Location-oriented clothing recommendation. IEEE Transactions on Multimedia, 2017, 19(11):2533-2544. Liu S, Nguyen T V, Feng J, Wang M, Yan S. Hi, magic closet, tell me what to wear! In Proc. ACM International Conference on Multimedia, October 2012, pp.1333-1334. Liu Y, Gao Y, Feng S, Li Z. Weather-to-garment:Weatheroriented clothing recommendation. In Proc. IEEE International Conference on Multimedia and Expo, June 2017, pp.181-186. Sha D, Wang D, Zhou X, Feng S, Zhang Y, Yu G. An Approach for Clothing Recommendation Based on Multiple Image Attributes. Springer International Publishing, 2016. Cheng M M, Hou Q B, Zhang S H et al. Intelligent visual media processing:When graphics meets vision. Journal of Computer Science and Technology, 2017, 32(1):110-121. Simo-Serra E, Fidler S, Moreno-Noguer F, Urtasun R. Neuroaesthetics in fashion:Modeling the perception of fashionability. In Proc. IEEE Conference on Computer Vision and Pattern Recognition, June 2015, pp.869-877. Jagadeesh V, Piramuthu R, Bhardwaj A, Di W, Sundaresan N. Large scale visual recommendations from street fashion images. In Proc. ACM SIGKDD international conference on Knowledge discovery and data mining, August 2014, pp.1925-1934. Yamaguchi K, Berg T L, Ortiz L E. Chic or social:Visual popularity analysis in online fashion networks. In Proc. ACM International Conference on Multimedia, Nov. 2014, pp.773-776. Zheng Z H, Zhang H T, Zhang F L, Mu T J. Imagebased clothes changing system. Computational Visual Media, 2017, 3(4):337-347. Ren S, He K, Girshick R, Sun J. Faster R-CNN:Towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6):1137-1149. Di W, Wah C, Bhardwaj A, Piramuthu R, Sundaresan N. Style finder:Fine-grained clothing style detection and retrieval. In Proc. IEEE International Conference on Computer Vision and Pattern Recognition Workshops, June 2013, pp.8-13. Chen K T, Chen K, Cong P, Hsu W H, Luo J. Who are the devils wearing prada in New York city? In Proc. ACM International Conference on Multimedia, August 2015, pp.177-180. Yamaguchi K, Kiapour M H, Ortiz L E, Berg T L. Parsing clothing in fashion photographs. In Proc. IEEE International Conference on Computer Vision and Pattern Recognition, June 2012, pp.3570-3577. Garcia N, Vogiatzis G. Dress like a star:Retrieving fashion products from videos. In Proc. IEEE International Conference on Computer Vision, October 2017. Cheng Z Q, Wu X, Liu Y, Hua X S. Video2Shop:Exact matching clothes in videos to online shopping images. In Proc. IEEE Conference on Computer Vision and Pattern Recognition, June 2017, pp.4169-4177. Kalantidis Y, Kennedy L, Li L J. Getting the look:Clothing recognition and segmentation for automatic product suggestions in everyday photos. In Proc. International Conference on Multimedia Retrieval, April 2013, pp.105-112. Shen E, Lieberman H, Lam F. What am I gonna wear?:Scenario-oriented recommendation. In Proc. International Conference on Intelligent User Interfaces, January 2007, pp.365-368. Zhao Y, Araki K. A recommendation system for a fashion coordination based on user's information. Ieice Technical Report Life Intelligence and Office Information Systems, 2012, 111:13-18. Hu Y, Yi X, Davis L S. Collaborative fashion recommendation:A functional tensor factorization approach. In Proc. ACM International Conference on Multimedia, October 2015, pp.129-138. Veit A, Kovacs B, Bell S, Mcauley J. Learning visual clothing style with heterogeneous dyadic co-occurrences. In Proc. IEEE International Conference on Computer Vision, Dec. 2015, pp.4642-4650. Song X, Feng F, Liu J, Li Z, Nie L, Ma J. Neurostylist:Neural compatibility modeling for clothing matching. In Proc. ACM International Conference on Multimedia, October 2017, pp.753-761. Melekhov I, Kannala J, Rahtu E. Siamese network features for image matching. In Proc. International Conference on Pattern Recognition, April 2017, pp.378-383. Hidayati S C, You C W, Cheng W H, Hua K L. Learning and recognition of clothing genres from full-body images. IEEE Transactions on Cybernetics, 2017, PP(99):1-13. Liu Q, Wu S, Wang L. DeepStyle:Learning user preferences for visual recommendation. In Proc. the International ACM SIGIR Conference, June 2017, pp.841-844. Jia Y, Shelhamer E, Donahue J et al. Caffe:Convolutional architecture for fast feature embedding. In Proc. ACM International Conference on Multimedia, June 2014, pp.675-678. Krizhevsky A, Sutskever I, Hinton G E. Imagenet classification with deep convolutional neural Networks. In Proc. International Conference on Neural Information Processing Systems, January 2012, pp.1097-1105. Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Huang Z, Karpathy A, Khosla A, Bernstein M. ImageNet large scale visual recognition challenge. International Journal of Computer Vision, 2015, 115(3):211-252. Yang Y, Ramanan D. Articulated pose estimation with exible mixtures-of-parts. In Proc. IEEE International Conference on Computer Vision and Pattern Recognition, June 2011, pp.1385-1392. Liang X, Lin L, Yang W, Luo P, Huang J, Yan S. Clothes coparsing via joint image segmentation and labeling with application to clothing retrieval. IEEE Transactions on Multimedia, 2016, 18(6):1175-1186. Hao P Y, Xia Y, Li X X, Kamata S I, Chen S Y. Discriminative histogram intersection metric learning and its applications. Journal of Computer Science and Technology, 2017, 32(3):507-519. Rubner Y, Tomasi C, Guibas L J. The earth mover's distance as a metric for image retrieval. International Journal of Computer Vision, 2000, 40(2):99-121. Siddiquie B, Feris R S, Davis L S. Image ranking and retrieval based on multi-attribute queries. In Proc. IEEE International Conference on Computer Vision and Pattern Recognition, June 2011, pp.801-808.
Copyright 2010 by Journal of Computer Science and Technology