viralamo

Menu
  • Technology
  • Science
  • Money
  • Culturs
  • Trending
  • Video

Subscribe To Our Website To Receive The Last Stories

Join Us Now For Free
Home
Technology
How continuous product testing tripled Microsoft Bing’s U.S. market share (VB Live)
Technology

How continuous product testing tripled Microsoft Bing’s U.S. market share (VB Live)

24/12/2019

Presented by Optimizely


Experimentation is vital in turning out flawless products — but it can go terribly wrong in a single iteration. Learn how to choose the right metrics and avoid wasting time, opportunities, and your project testing budget when you join this VB Live event!

Access on demand for free right here.


In product development, teams often feel like they’re wandering around in the dark, trying to figure out what users really want and whether their product is going to deliver for them, says Jon Noronha, director of product management at Optimizely.

“What many product teams often feel is this real lack of confidence, because we don’t know, until the very end of this process, how our users will react to the things we build,” Noronha says. “We can sink months or even years into something that might have a negative impact.”

Take, for example, his first-hand experience as a product manager at Microsoft in the wake of the disastrous Windows Vista waterfall launch. Microsoft spent years and years developing — mostly in secret, mostly internally, without a lot of external feedback. Once it got into users’ hands and problems emerged, a PR nightmare ensued and the company’s stock dropped drastically.

It was a wake-up call for the company. It realized that this waterfall approach to development, where everything is done in sequential stages and feedback only comes at the end, just couldn’t work in this modern software development era. Especially because other companies in Silicon Valley were pioneering a new way of working, one that was a lot more agile, a lot more gradual and iterative. That was exemplified most of all by Google, which was deploying code every single day and A/B testing everything.

That’s why so many product development teams have started to change how they operate, moving away from a linear process of design, build, launch, and pray, to a more gradual, iterative process where they’re constantly experimenting on ideas in small pieces and tweaking them as they go based on the feedback that they get.

For every single thing you do in product development, how can you actually quantify the effect that it has? How can you validate if a feature does what you said it would? How can you see what kind of adoption it gets? Does it change some core business metrics for you, like retention and engagement of your audience? And then using that information to feed back into the beginning of the cycle, and develop new ideas or refine older ones.

“When you start to embrace this way of doing product development, it changes your mindset,” Noronha says. “We don’t just have a product road map and a couple of experiment ideas on the side. Soon we start to think of every idea we have for improving our products, whether we’re a designer or an engineer or a product manager. It just becomes another hypothesis. They’re all on equal footing, ready to be tested.”

Now, this describes a kind of ideal, this end state of a culture of experimentation. But the truth is that for most companies it’s quite a journey to get there, to climb this mountain of experimentation culture, and go step by step from that launch-pray mindset to the more cyclical experimentation mindset. It is not an easy thing to do, Noronha warns. It requires some real culture change at a lot of companies, in how you work.

There’s no silver bullet, he says, pointing to his work on Microsoft’s Bing product. In 2011, Bing was managing to lose almost $500,000 per hour, which reflects how hard it is to build something as complex and big as a search engine. Thousands of people, thousands of machines and servers were all working toward this goal, but until that search engine was really good and high quality and worked for users, it wasn’t getting to benefit.

By 2015, however, Bing had actually quietly become quite profitable, to the tune of over a billion dollars a quarter — a pretty drastic turnaround.

“There was no magic silver bullet in that process, or some single thing we changed that suddenly made the company profitable or grew our market share,” he explains. “Instead, it was a gradual process of continuous experimentation, where we were constantly trying out new ideas and trying to build a culture that allowed hundreds of teams to all make those incremental improvements and measure the impact of every change they made.”

The usage of A/B testing and these techniques was directly responsible for tripling Bing’s market share in the U.S. and the rest of the world. Experimentation steadily increased, from just a handful of experiments running every week to hundreds of experiments running weekly on all different parts of the service. The user experience, the search algorithm, the advertising, the mobile apps, everything was grounds for experimentation, continuously. It meant that every product team had multiple experiments in play at any given time.

“It really showed me the power that A/B testing could have, but what it also showed me was some of the challenges,” he says. “We had big advances, but also big setbacks and big challenges in embracing this mindset of experimentation, especially in a company that didn’t always have a mindset that was suited to that constant experimentation.”

To learn how companies succeed and where they get stuck when they implement a culture of experimentation, the most common pitfalls and challenges a company can face, and how to accelerate your own progress through this journey, access this free VB Live event on demand now!


Access on demand for free right here.


You’ll learn about:

  • The most common mistakes product teams make when running experiments
  • Which metrics correlate best with your business’s success
  • Strategies to scale experimentation across multiple teams and squads
  • How the world’s top technology companies are able to experiment on all product decisions

Speakers:

  • Jon Noronha, Director of Product Management at Optimizely

Source link

Share
Tweet
Pinterest
Linkedin
Stumble
Google+
Email
Prev Article
Next Article

Related Articles

Move over Slack — Space is a new project management platform for developers – TechCrunch
While file sharing, time tracking, email integration, Gantt charts and …

Move over Slack — Space is a new project management platform for developers – TechCrunch

DeepMind’s MEMO AI solves novel reasoning tasks with less compute
Reinforcement learning, which spurs AI to complete goals using rewards …

Google Brain and DeepMind researchers attack reinforcement learning efficiency

Leave a Reply Cancel reply

Find us on Facebook

Related Posts

  • Obama-era tech advisors list potential challenges for the White House’s AI principles
    Obama-era tech advisors list potential challenges for …
    12/01/2020
  • Check Point researchers uncovered Alexa flaw that exposed personal information and speech histories
    Check Point researchers uncovered Alexa flaw that …
    13/08/2020
  • Beat Saber is now an Oculus studio after Facebook acquisition
    Plant-based milk substitute market gets frothy with$225 …
    15/01/2020
  • PlayStation 5 has the best launch lineup of this century
    PlayStation 5 has the best launch lineup …
    12/11/2020
  • Buildbox owner AppOnboard hires Riot Games leader Mike Seavers as CEO
    Buildbox owner AppOnboard hires Riot Games leader …
    10/07/2020

Popular Posts

  • Mega says it can’t decrypt your files. New POC exploit shows otherwise
    Mega says it can’t decrypt your files. …
    21/06/2022 0
  • Top 10 Mysteries, Cold Cases & Puzzles …
    23/05/2022 0
  • Ransomware attack on Planned Parenthood steals data of 400,000 patients
    Why it’s hard to sanction ransomware groups
    23/05/2022 0
  • 10 Most Common Mutations in Humans – …
    24/05/2022 0
  • 10 Signs You Might Have Already Been …
    24/05/2022 0

viralamo

Pages

  • Contact Us
  • Privacy Policy
Copyright © 2022 viralamo
Theme by MyThemeShop.com

Ad Blocker Detected

Our website is made possible by displaying online advertisements to our visitors. Please consider supporting us by disabling your ad blocker.

Refresh