Vibe Coding Testing: How to Test AI-Built Web Apps

Tue, 08 Sep 2026 06:56:15 +0000

A web app that once took weeks to build can now take days, sometimes hours.

Describe a screen to Cursor, Claude Code, Lovable, Bolt, or another AI coding tool. Ask for authentication. Add a dashboard. Connect an API. Fix a few errors through prompts.

Soon you have something that looks surprisingly close to a real product.

Then comes the awkward question:

Does it actually work?

That is where vibe coding starts creating an interesting problem for QA.

Vibe coding changed development speed, not software risk

Andrej Karpathy coined the term vibe coding in 2025 for a style of development where you largely describe what you want, let AI generate the implementation, run it, and continue prompting rather than working directly with the code.

Martin Fowler makes an important distinction: true vibe coding often means building without paying much attention to the generated source code.

That can be perfectly reasonable for prototypes.

Production applications are different.

Authentication still has to work.

Permissions still have to work.

Forms still need validation.

A checkout still needs to handle failed payments.

A user still needs to be able to reset a forgotten password.

AI has changed how quickly those features get created, but it has not removed the need to verify them.

Developers already don’t fully trust AI output

This is not a theoretical concern.

The 2025 Stack Overflow Developer Survey found that 84% of respondents were already using or planning to use AI tools in software development.

But only 33% said they trusted the accuracy of AI output, while 46% actively distrusted it.

Even more interesting, the most common frustration was not completely broken code.

66% complained about AI solutions being “almost right, but not quite.”

That description should sound familiar to anyone in QA.

Almost correct is often harder to detect than obviously broken.

A registration page that crashes immediately gets fixed.

A registration page that works except when an existing email address is entered may survive much longer.

The happy path is becoming dangerously convincing

Consider a founder building a SaaS application with AI.

They prompt:

Create a signup page with email and password, then redirect the user to the dashboard.

AI generates it.

They test:

Enter name
Enter email
Enter password
Click Sign Up
Dashboard opens

Everything works.

Feature done.

But QA would immediately start asking different questions.

What happens with an existing email?

What happens if the API returns 500?

Can an unverified user access the dashboard URL directly?

What happens if the password contains unusual characters?

Can the form be submitted twice?

Does logout actually invalidate the session?

Can User A access User B’s data by changing an ID in the URL?

The difference is important.

Building software asks: “Can I make this flow work?”

Testing asks: “In how many realistic ways can this flow stop working?”

AI is becoming extremely good at answering the first question.

Write on Medium
We still need the second.

Faster coding can move the bottleneck somewhere else

Google’s 2025 DORA research found that about 90% of surveyed technology professionals were using AI at work, with more than 80% reporting productivity gains.

But DORA also found that AI adoption continued to have a negative relationship with software delivery stability. Their explanation is particularly relevant: increasing development speed can expose weaknesses downstream when teams do not have strong automated testing and fast feedback loops.

This may become one of the biggest changes AI brings to QA.

Imagine development previously produced five meaningful changes per week.

Now the same team produces fifteen.

If testing capacity stays the same, the problem has not disappeared.

It has moved.

Before AI

Development → Development → Development → Testing

With AI

Development
Development
Development  →  Testing bottleneck
Development
Development

Generating code faster is useful only if teams can verify it fast enough.

Don’t test the AI. Test the user journey.

There is a temptation to create a completely new testing methodology for AI-generated applications.

Most teams probably don’t need one.

If AI generates a new checkout flow, QA does not need to determine whether every line was written by a human or an LLM.

The customer certainly does not care.

The useful questions remain:

Can I add a product?
Can I update the quantity?
Can I apply a discount?
Can I complete payment?
What happens when payment fails?
Does the order appear afterward?

This is why end-to-end testing becomes particularly valuable for AI-built web applications.

It validates the behaviour users actually depend on rather than making assumptions about how the underlying code was produced.

One practical rule for vibe-coded applications

Before shipping an AI-built feature, identify its critical user journeys.

For a simple SaaS application, that might be:

Register
Log in
Create the main business object
Edit it
Delete it
Verify permissions
Log out
Log back in and confirm the data is still correct

Run those flows every time AI makes a meaningful change.

Not because AI-generated code is automatically bad.

Because changing software quickly increases the number of opportunities to accidentally break something that already worked.

That is exactly what regression testing is supposed to protect against.

Where QAlity fits

This is one area where I think no-code UI automation becomes particularly interesting.

The person building an application with AI may not want to stop and write hundreds of lines of Selenium or Playwright code just to automate the critical flows.

But relying on manual clicking after every AI-generated change does not scale either.

With QAlity, the idea is to let teams create and run UI tests around real user journeys without having to build and maintain a traditional automation framework.

AI can help developers produce software faster.

Testing tools need to help teams verify it at a similar pace.

Otherwise we have simply made one half of the software-development lifecycle much faster than the other.

Adding a feature is easy. Hiding the complexity behind it is much harder

Mon, 31 Aug 2026 06:21:43 +0000

One thing I have learned while building QAlity:

Adding a feature is easy. Hiding the complexity behind it is much harder.

Test automation has a lot of moving parts.

Locators. Waits. Assertions. Test data. Browser behavior. Execution history. Reusable flows. Failure analysis.

It’s tempting to expose all of that because technically, every option can be useful.

But then you end up recreating the same complexity that made automation difficult in the first place.

So lately, I have been asking a different question when reviewing QAlity:

“Does the user actually need to understand this?”

If the answer is no, the product should probably handle it.

I think this is especially important for developer tools.

Advanced software does not have to look advanced.

Sometimes the best feature is the configuration screen you never had to build.

Explore QAlity - https://qality.dev

Everyone’s online. But not everyone’s in focus.

Wed, 19 Aug 2026 05:47:14 +0000

Everyone’s online. But not everyone’s in focus.

Software teams are constantly shipping new features, fixing bugs, and preparing for the next release.

But with every release comes the same repetitive testing:

The product changes, but many of these tests stay the same.

And when your team spends hours repeating the same tests, there’s less time for what really matters — finding new issues, improving the product, and building better experiences.

That’s where QAlity comes in.

QAlity helps teams automate repetitive tests so they can spend less time on busywork and more time focused on quality.

Less repetition.
Fewer distractions.
More impact.

Focus on what matters. Automate the rest.

👉 Get started at qality.dev

#QAlity #TestAutomation #SoftwareTesting #QA #QualityEngineering

Coder Legion - Recent posts tagged qality