As part of an ambitious three-year effort to replatform and modernize our backend infrastructure, eBay is announcing our own custom-designed servers, built by eBay, for eBay. We plan to make eBay’s servers available to the public via open source in the fourth quarter of this year.
Building servers specialized for the complex demands of our marketplace took intense research, testing across our many applications and partnerships with key hardware vendors. These servers cater to eBay’s specific needs, enable an improved customer experience, create greater efficiencies and reduce our dependence on third parties.
With 175 million active users and over 1.1 billion live listings on eBay, the infrastructure needed to manage our global marketplace is massive. While managing traffic and moving huge amounts of data are complex endeavors in themselves, the infrastructure that runs and maintains the platform is also a critical enabler of any future shopping experiences.
The replatforming of our backend infrastructure included designing our own hardware and AI-engine, decentralizing our cluster of data centers, moving to an edge computing architecture and leveraging the latest in open source technologies. Now, midway through our journey, we are already seeing meaningful results that offer greater predictability, more control and needed flexibility.
Tackling the Infrastructure Transformation
eBay operates in a complex environment at massive scale and when thinking about replatforming, we have to be mindful of the amount of data that we handle. eBay processes 300 billion data queries each day, and our data footprint is more than 500 petabytes. To put it into context, 500 petabytes is the equivalent of one trillion songs, 2.5 million hours of movies and enough to backup the American Library of Congress more than 300 times.
As we undertook this evolution within that significant scale, our priorities were keeping the site running healthy and secure for years, and enabling opportunities to incorporate and support new technologies. We looked at the entire technology stack—physical and logical layers—because all the layers are intertwined in some way. The stack is like connective tissue, you cannot isolate one of the layers; you must advance them together. To make a meaningful impact, the transformation should be cohesive and orchestrated from end-to-end. We systematically went through each layer of our technology stack and examined efficiency, capability and the opportunity to improve existing solutions.
It’s worth noting that most of the time when companies undergo a replatforming effort like this, they need to increase their budget and it often takes several years. However, eBay has been able to replatform on an aggressive timeline without incremental cost and, in fact, we have reinvested savings back into the business.
Evolution at Each Layer of the Technology Stack
In the physical foundation, we are using a Point of Presence (PoP) strategy, decentralizing the cluster of data centers we have in the United States. Through this edge computing approach, we can create a faster, more consistent user experience, saving 600-800 milliseconds of load time. We are deploying online services and data closer to our users, enabling dynamic and static caching capabilities, decreasing latency and improving the experience.
In the data layer we are also creating more customized models. Using open source technologies, we built NuData, a fault tolerant, geo-distributed object and data store. Long term, this will allow eBay to distribute data geographically to improve our customers’ experience, offer resiliency to our services and provide data isolation solutions for countries that require them.
We are using servers and hardware that we designed, reducing our dependence on third parties. This allows us to more specifically cater to eBay’s needs, offering greater predictability to our business, enabling more control and promoting flexibility. It took us nine months to build our prototype and to deploy our custom hardware. With this shift, we are homogenizing our infrastructure, leading to significant development and operational efficiencies.
With the transformation that we have made and the massive data that funnels through eBay, we used open source to build an in-house AI engine that is highly shareable across all of our teams and aims to increase productivity, collaboration and training. It is allowing our data scientists and engineers to experiment, build products and experiences for customers, and leverage AI at scale. Our AI engine has already accelerated the production of new features, such as computer vision, Image Search and social sharing—allowing teams to reduce development time from weeks to hours.
The Open Source Philosophy
True to our core as a beacon of opportunity and empowerment, our approach to technology is centered on an open source model. Open source is fueling our transformation, with technologies like Kubernetes, Envoy Proxy, MongoDB and Apache Kafka, propelling eBay to new performance levels for our customers. In a sense, open source gives us control of our destiny, offering greater capabilities to customize for eBay’s needs.
We also have the opportunity to give back by sharing eBay innovations and technology experiences with the broader engineering community through open source. Developers and communities who leverage our tools improve upon what we are building and help us create better experiences overall.
And we’re incorporating the open source philosophy not only into hardware but also into our approach to software development by bolstering our API offerings and investing in our developer program as a cornerstone of eBay’s future.
Building the Right Culture
The key to successfully replatforming our infrastructure on an ambitious three-year timeline is, of course, our people. Building the right culture and creating the best atmosphere requires deliberate and delicate work. With the right culture in place, the technology and innovation will follow.
With this in mind, we’ve organized our team efficiently to be able to respond quickly and to be flexible as priorities change. And we promote a learning culture within our team as a way to eradicate risk averse behaviors and embolden creativity. Even more, we believe it’s important to over-index on transparency to promote a strong, collaborative and effective working environment.
Our people and culture are the drivers of this tremendous effort, with many thought leaders showing up as change agents across multiple disciplines to facilitate this transformation on an aggressive schedule. We are only halfway through our journey, and I’m proud of the strides our team has accomplished, the culture we have created, the strategy we are executing and how we have never lost sight of our customers.
By replatforming our infrastructure, we are able to take advantage of a number of new technologies. This further improves efficiency, flexibility and scalability. Our work today positions eBay to deploy new experiences at scale for years to come.