eBay Innovates with Computer Vision and Image Search Technologies

eBay Inc. Staff

A picture is worth a thousand words.  That’s a well-known concept at eBay, where innovative new computer vision and image search applications are taking shape. From image-centric work going on at eBay Research Labs to new features found in eBay’s mobile applications, powerful new ways for consumers to shop and research purchases are emerging.

Ecommerce is flourishing, but there are still huge opportunities for technologies that allow consumers to see and visually interact with products they are interested in. It's also true that users on all types of devices, including many mobile devices, are looking for these technologies.

PhiSix Fits
With that in mind, eBay recently announced its acquisition of PhiSix, a computer graphics company that creates 3D models of clothing from photos, pattern files and other sources and simulates the behavior of the garments.  PhiSix’s technology has exciting possibilities – both online and offline – as it allows consumers to see how clothes fit, look and move in different environments without actually having to try them on.

The end result of PhiSix’s technology is that consumers can experience merchandise in a more efficient and impactful way, which can help drive sales for retailers and create a compelling experience for shoppers. The technology may also help minimize returns of products such as clothing, since users will be able to better evaluate potential purchases before buying.

“We actually have opportunities to create new in-store experiences with the PhiSix technology, too,” said Rob Veres, Senior Director of Innovation and New Ventures. “Imagine you’re in a store and have found a shirt you like, but your size isn’t in stock. With PhiSix’s technology you could figure out in the store what the right sized shirt would look like on you.  Likewise, you might use the technology to investigate pants that might go with a given shirt.”

Shop the Look
Some of eBay’s next-generation computer vision applications are already in the hands of consumers. For example, the eBay Fashion app for iPhone includes Shop The Look – a color and pattern matching discovery tool that helps users find fashion from photos.

To shop with Shop The Look, a user can tap the “Shop The Look” button on the home screen and take a photo of something inspiring or upload a photo from an existing iPhone photo library. eBay’s computer vision technology does the magic to then populate items similar in color palette and pattern. The feature expects only a picture, without the user having to type anything, and is a very popular in the eBay fashion app.

eBay Research Labs
There is also interesting computer vision work going on at eBay Research Labs, including innovative fashion applications. This includes large scale visual search as well as fine-grained classification such as style recognition.

“Our goal is to use cutting-edge computer vision algorithms to very efficiently match users with fashions and accessories that will work for them,” said Robinson Piramuthu, Senior Research Scientist on eBay's Vision Team.  “One interesting technology is to match fashion items as a collection. We can surface appropriate fashion choices intelligently. We might, say, recommend a black satin off-shoulder top, a minaudiere and a pair of pumps to go well with a houndstooth pencil skirt. ”

As 2014 progresses, eBay will deliver a number of new features and applications leveraging computer vision technologies for fashion. They will help consumers shop smarter, from any device, any time.