“The old principle of tinkering with an instinctively designed car has long since been superseded by systematic testing of every major component and structure – both before and after the car is fully built and ready to race.”

—“Understanding F1 Racing,” Formula1.com

While speed is the most obvious shared goal between racing and software development teams, you might be surprised at the striking similarities between how these teams achieve that goal. Each relies on top talent, funding and innovation to reach the speeds at which their competition cannot, but there’s another oft-ignored component that’s quickly becoming one of the greatest differentiators between first place and everyone else: testing.

The way F1 teams test their cars has gone through a transformation just as monumental as what we’ve seen in the software world over the last decade. The ability to continually test every major component and structure of a race car or piece of software is crucial, but the ability to quickly respond to the data that comes out of those tests is just as important. It’s continuous testing before, during and after the rubber meets the road—or the software hits the marketplace or app store—that determines the winners.

The Evolution of Testing

In the aforementioned Formula1.com article, an all-too-familiar scenario to those in the software world is described:

“By midway through the first decade of the 21st century, a typical Formula 1 testing programme had become a major exercise in both manpower and logistics…spiralling costs, which invariably resulted in development work that was invisible and meaningless to fans …”

Replace “Formula 1” with “software” and the above excerpt is just as factual for those who don’t know the struggles of yesterday’s approaches to software quality, and just as painful for those who personally experienced them. Manual processes, insufficient test automation technologies, shared unrealistic environments, lack of trust between teams and out-of-control costs are constraints of the past for today’s top-performing teams, but they remain for many still relying on legacy testing approaches and technologies.

What Continuous Testing Looks Like Today

Some of the racing enthusiasts from Tricentis’ leadership team were invited earlier this year to meet the Haas F1 racing team during their 2018 preseason testing at Circuit de Barcelona-Catalunya. We witnessed firsthand the remarkable transformation that Haas and F1 racing as a whole have embraced. Track testing is where, as Formula1.com says, “the steady evolution that happens to all Formula One cars during the course of their life begins, a constant improvement of tiny details and setup.” This constant evolution and iterative, rapid innovation is exactly the goal of high-performing software organizations today.

After spending the day at the track, we noted the following approaches and methodologies that we see in innovative continuous testing and DevOps practices today:

• F1 cars are continuously delivering data and feedback to the organization.
• The cars have millions of data points and thousands of sensors, providing actionable data.
• These data points and sensors are grouped into core processes, and isolated by key questions.
• Test runs are done with the cars in as realistic environments as possible.
• The speed at which that data can be received, interpreted—and most importantly—acted upon, is the difference between high-performing and low-performing teams.
• The tiniest changes to the car make enormous differences. Nobody is showing up in an entirely different car, they’re making minute changes.
• These cars cannot fail on race day or the race is lost.

Today’s high-performing organizations are treating the delivery of software in the exact same manner as the world’s leading racing teams. Innovative technologies are being put into each stage of the delivery life cycle that provide invaluable feedback that helps enterprises better understand risks to their business. Being able to rapidly analyze—and act on—this amount of data has become imperative for avoiding disruption. Testing software in environments that are as realistic and as close to production as possible is mandatory, and those environments need to be available when they’re needed most. These needs have driven the adoption of what’s known as continuous testing—an evolution of traditional test automation that has proven unable to support software’s increased complexity, and expectations around the pace of development and delivery.

What Questions Are F1 Racing and Software Enterprises Asking?

While each have their own unique challenges, there are many similar questions that need answers—in real-time, and at any development, testing, or release stage—that the racing and software industries share.

• How fast are we today and how fast will we be tomorrow?
• How fast is our competition, and how fast will they become?
• What are we measuring today that’s impacting the business?
• How quickly can we introduce change?
• What data are we using to validate that change?
• What’s the risk of not changing?
• How low can we get our mean-time-to-repair?

The fastest time to market—or time to the finish line—is how you win the race. Your developers or engineers don’t just build the fastest car, or the most popular app to today’s standards and then call it a day. They’re constantly looking at what speed, UX, functionality or accessibility will be needed to be the best tomorrow, and then iteratively adding those improvements as quickly as possible. They don’t stop engineering, ever. And testing shouldn’t be performed any differently. Each new feature, condition, region, device, persona, currency, language and OS that your app needs to offer, integrate with, accept or run on is going to need continuous testing before, during and after that app’s release.

Whether you’re responsible for the quality and performance of a billion-dollar race car or the applications for a billion-dollar enterprise, speed and quality are your path to the winner’s circle. Continuous testing ensures that you remain there.

— Noel Wurst