Shop The Look: Building a Large Scale Visual Shopping System at Pinterest
Authors:
Raymond Shiau,
Hao-Yu Wu,
Eric Kim,
Yue Li Du,
Anqi Guo,
Zhiyuan Zhang,
Eileen Li,
Kunlong Gu,
Charles Rosenberg,
Andrew Zhai
Abstract:
As online content becomes ever more visual, the demand for searching by visual queries grows correspondingly stronger. Shop The Look is an online shopping discovery service at Pinterest, leveraging visual search to enable users to find and buy products within an image. In this work, we provide a holistic view of how we built Shop The Look, a shopping oriented visual search system, along with lesso…
▽ More
As online content becomes ever more visual, the demand for searching by visual queries grows correspondingly stronger. Shop The Look is an online shopping discovery service at Pinterest, leveraging visual search to enable users to find and buy products within an image. In this work, we provide a holistic view of how we built Shop The Look, a shopping oriented visual search system, along with lessons learned from addressing shopping needs. We discuss topics including core technology across object detection and visual embeddings, serving infrastructure for realtime inference, and data labeling methodology for training/evaluation data collection and human evaluation. The user-facing impacts of our system design choices are measured through offline evaluations, human relevance judgements, and online A/B experiments. The collective improvements amount to cumulative relative gains of over 160% in end-to-end human relevance judgements and over 80% in engagement. Shop The Look is deployed in production at Pinterest.
△ Less
Submitted 18 June, 2020;
originally announced June 2020.
Related Pins at Pinterest: The Evolution of a Real-World Recommender System
Authors:
David C. Liu,
Stephanie Rogers,
Raymond Shiau,
Dmitry Kislyuk,
Kevin C. Ma,
Zhigang Zhong,
Jenny Liu,
Yushi Jing
Abstract:
Related Pins is the Web-scale recommender system that powers over 40% of user engagement on Pinterest. This paper is a longitudinal study of three years of its development, exploring the evolution of the system and its components from prototypes to present state. Each component was originally built with many constraints on engineering effort and computational resources, so we prioritized the simpl…
▽ More
Related Pins is the Web-scale recommender system that powers over 40% of user engagement on Pinterest. This paper is a longitudinal study of three years of its development, exploring the evolution of the system and its components from prototypes to present state. Each component was originally built with many constraints on engineering effort and computational resources, so we prioritized the simplest and highest-leverage solutions. We show how organic growth led to a complex system and how we managed this complexity. Many challenges arose while building this system, such as avoiding feedback loops, evaluating performance, activating content, and eliminating legacy heuristics. Finally, we offer suggestions for tackling these challenges when engineering Web-scale recommender systems.
△ Less
Submitted 25 February, 2017;
originally announced February 2017.