Coder Legion - Recent posts tagged technical-seo

SEO for 2027: A Developer Workflow for AI Search Visibility and Trust

Fri, 30 Oct 2026 19:21:01 +0000

A page can be technically accessible, appear in a search result and still fail to help someone choose a service. An AI answer can mention a business and still describe it incorrectly.

Those are different problems. They need different tests.

I am Jarno Saarimies, founder of AEOvara in Lappeenranta, Finland. I work on SEO and AI search visibility, and I also run a photography business. That combination gives me a useful question to bring to a technical discussion: can a customer actually understand and use what we have built?

For a 2027 planning discussion, I would start with a testable workflow rather than an algorithm prediction. The recommendations below are engineering choices, not promises of higher rankings.

Start with documented requirements

Google's documentation on AI features and websites says that a supporting page in AI Overviews or AI Mode must be indexed and eligible for a search snippet. It does not specify additional technical requirements for those features.

The same guidance recommends accessible textual content, internal links, crawler access and structured data consistent with the visible page. These are eligibility and implementation considerations, not guarantees of selection.

That distinction matters when someone proposes a new AI visibility feature. Ask which documented requirement it satisfies and what observable result would count as success.

Separate discovery from answer quality

I would organise the work around four questions.

Can the page be retrieved? Can its service information be understood? Does an answer represent the business correctly? Can a visitor complete the intended task?

For a service page, my suggested review would cover:

The response status and the canonical URL.
Applicable indexing and snippet directives.
Crawler restrictions at the application, CDN and robots.txt layers.
The important service information in the rendered page.
Internal links that lead to the page.
Agreement between visible content and structured data.
The contact or booking flow on a phone.

Record what was inspected and what was observed. A successful response from one HTTP client does not establish what Google indexed. For Google-specific diagnosis, inspect the page in Search Console as well.

I would avoid collapsing all of this into one pass or fail score. A form validation error and an incorrect service description require different fixes.

Make the service page a useful specification

Consider a hypothetical company booking portraits for ten employees.

The useful questions are specific. Can the photographer work at the office? What is included? How are travel costs handled? When are the files delivered? What information is needed to request a quote?

A developer can turn those questions into acceptance criteria. For example, someone unfamiliar with the business should be able to identify the service area and submit a valid enquiry without asking the owner to explain the page.

This is where I would bring the service provider into the review. Developers can make information accessible, but the owner has to supply accurate scope, prices and limitations.

My proposed content model would include a service name, audience, location, inclusions, exclusions, pricing explanation, process and next action. It is a review template, not a search engine requirement.

Measure four different AI answer outcomes

For AEOvara's visibility work, I use four categories to keep the questions separate: Mention, Recommendation, Citation and Narrative Accuracy.

Here is a proposed operational definition for a test log:

Category	What to record
Mention	Is the target business named?
Recommendation	Does the answer explicitly suggest choosing or considering it?
Citation	Does the answer link to a page on the target domain as supporting evidence?
Narrative Accuracy	Are the factual claims about the business correct?

These are my measurement definitions, not metrics supplied by Google. A different team can choose different definitions, but it should write them down before scoring answers.

Fun fact from that scoring model: an answer can pass the first three categories and fail the fourth. It could name and recommend a company, cite its website, and still invent a service the company does not offer. That is a hypothetical example, but it shows why a citation is not an accuracy check.

For each test, I would retain the exact question, platform, date, language, relevant location and response. Record whether browsing was enabled and any known account or conversation context. Repeat the questions and report the number of observations alongside the results.

Keeping the prompt unchanged improves comparability. It does not control every variable in a changing search or language model system. I would therefore treat a small test as a sample of behaviour, not a market share estimate or evidence that one site change caused the result.

Treat trust as evidence a reader can inspect

Google's people-first content guidance says E-E-A-T is not itself a specific ranking factor. It also encourages accurate authorship information and clarity about how content was produced.

My practical interpretation is to make claims auditable. Name the author, explain the relevant experience, link the primary source and describe the test method. If an example is hypothetical, label it. If a result comes from your own small sample, say so.

A byline cannot substitute for evidence. Neither can a large schema block. For this article, the Google-specific requirements are linked above; the review process and scoring definitions are my recommendations.

Ship one change you can evaluate

For the next iteration, I would choose one important service page, record its current state and fix a concrete problem. That might be missing service information, inaccessible content or a broken enquiry step.

Keep a dated change log. Compare relevant search observations with actual enquiries, and note other changes that could affect the result. A before and after comparison is useful for investigation, but it is not automatically a controlled experiment.

This is how I would approach planning for 2027: documented requirements, explicit definitions and changes small enough to inspect. No forecast about the next algorithm update is needed to start.

About the author

Jarno Saarimies is the founder of AEOvara, based in Lappeenranta, Finland. His work focuses on technical SEO, content clarity and visibility in AI-powered search. He also runs the photography business Kuvaajankulma.

A related article written in Finnish is available at https://aeovara.fi/hakukoneoptimointi-2027/. It covers the broader 2027 planning topic, including predictions. This developer-focused article stands on its own.

What is the hardest part to validate in your projects: crawler access, the accuracy of an AI answer, or whether search visibility leads to a useful enquiry? I would be especially interested in a concrete test that changed your implementation, including one that disproved your original assumption.

We ran SEO audits on 226 homepages. Here's what kept showing up.

Mon, 26 Oct 2026 04:17:13 +0000

RankAgent is the SEO tool we built. For this analysis, we used its free homepage audit. It checks page titles and descriptions, headings, image descriptions, link previews, mobile basics, security, and some performance signals. Each result covers one homepage. The rest of the site and its Google traffic sit outside this analysis.

TL;DR: The 226 homepages averaged 75.3 out of 100. Security (96.2) and mobile (88.5) scored well; headings (54.0) and meta tags (64.2) lagged. Fifty-eight percent skipped heading levels, and 51% triggered the image-description check. The percentages tell us where to look; we didn't measure visits lost.

From 4 June to 3 August 2026, RankAgent completed 244 audits. We kept the first completed audit for each homepage host, leaving 226 unique homepages. Some sites were selected for our outreach and others came through the free audit. This was a non-random group, so the findings describe these pages rather than websites in general.

These are RankAgent's category scores, based on our own checks and weights. They make a useful worklist; they are not Google ranking factors.

Average category scores in the 226-homepage sample. Source: RankAgent audits, 4 June–3 August 2026.

Fifty-seven of the 226 homepages had at least one issue our tool labels critical. That label helps sort a worklist; it isn't a prediction of traffic or revenue impact.

Heading structure kept breaking

On 58% of these homepages, heading levels skipped a step. A section might jump from an H2 to an H4. Another 37 pages, or 16%, had no H1 at all. About 21% had more than one.

This is easy to miss in a browser. Text can be large and bold without being marked as a heading, and a page can look orderly while its HTML outline wanders around. The W3C's accessibility guidance recommends a meaningful hierarchy because people using assistive technology can navigate by headings.

Google says heading order does not matter to Google Search. I would review these flags for clarity and accessibility. Our audits did not test whether heading order changed rankings or AI citations.

Images need a human second look

Our image-description check flagged 51% of the homepages. It fires when 20% or more of the page's image elements have no nonblank alt text. Inspect those images individually. A decorative image should often have an empty alt value, as the W3C explains. A product photo, chart, or image carrying information deserves a useful text alternative.

Another 42% hit our "many images" flag, which means the HTML contained at least 30 image elements. This check counts tags, not bytes or load time. We measure speed before calling any of those pages slow.

Share of the 226 homepages with each flag. These checks identify places to review; they are not measurements of traffic lost.

One problem can repeat across a site

A separate full-site report made that clear. In an 89-page crawl, our audit flagged missing H1s on 88 pages and missing descriptions on 60. Those counts suggest a shared template or page-generation rule. We'd check that first, before editing pages one at a time. This site is an example from a separate audit; its counts are not part of the 226-homepage percentages above.

The metadata wasn't always complete

Meta tags were the second-lowest category in our scoring system. On 36% of the homepages, at least one of the three tags we check for a link preview (og:title, og:description, or og:image) was missing. On 22%, our checker found neither JSON-LD nor Microdata.

Paste a URL into a messaging app to check the actual link preview. Structured data is worth adding where it accurately describes a page and makes it eligible for a relevant search feature. Neither flag tells us how many visits a site missed.

Google builds title links from several sources and may create snippets from the page text, even when a meta description is present. Our audit shows what a site supplied in its HTML. Search Console and actual search results show what happened afterward.

We built RankAgent, so there is an obvious interest in these findings. We audited only homepages in the 226-site sample. We did not measure rankings, organic traffic, conversions, or mentions in AI answers for those sites. Our thresholds and severity labels are our own.

What I would do with an audit now

I'd start with the page itself. Read the visible main headline and the browser title as if you've never heard of the business. Can you tell what it offers? Open the heading outline and check whether the sections make sense in order. Review the images that carry information, leaving decorative ones alone. Paste the URL into a messaging app to see its link preview. If the page has dozens of images, measure its loading speed before deciding it needs performance work.

Read the full 226-site study and methodology or run the same free homepage audit on your own site.

Cloudflare's AI Bot Controls, and the Setting That Can Block Googlebot

Fri, 23 Oct 2026 04:01:47 +0000

If Cloudflare sits in front of your site, it may already be choosing which AI bots get through. New domains have been asked about AI crawlers at sign-up since July 2025, and from September 15, 2026 newly added domains start with training and agent bots blocked on pages with ads. This guide explains each feature and the one setting that can quietly block search crawlers.

I lay out the timeline, from AI Audit in 2024 to separate Search, Agent and Training policies on every plan in July 2026. Then the trap: since September 2026, any setting that blocks Training also blocks crawlers Cloudflare classifies as both search and training, and its documentation names Googlebot, Applebot and Bingbot as examples. If you depend on Google and Bing, don't block Training on all pages; block training-only crawlers individually instead, and verify with Search Console's live test.

I walk through AI Crawl Control's tabs and what the Free plan can and can't detect, Cloudflare's managed robots.txt and Content Signals block, Pay Per Crawl with its HTTP 402 response (still a closed beta), and how verified bots and Web Bot Auth work.

There's a step-by-step Free-plan setup for a small business that wants to be found, plus limits: only proxied hostnames are covered, classifications can change, and allowing bots doesn't by itself earn citations.

Key takeaways

New Cloudflare domains may block AI bots by default
Blocking Training can also block Googlebot and Bingbot
Block training-only crawlers individually instead
Check the robots.txt Cloudflare actually serves
Confirm with Search Console and Bing live URL tests

Related on BuiltToWinWeb: what my own Cloudflare defaults were doing

Read the full guide on BuiltToWinWeb: How Cloudflare handles AI crawlers

A summary by Jacob Campbell of an article first published on built2winweb.com on October 7, 2026.

noindex vs robots.txt: Which to Use and the Mistake to Avoid

Fri, 23 Oct 2026 04:00:51 +0000

noindex is for keeping a page out of results; a robots.txt Disallow is for stopping crawlers from requesting URLs. The two aren't interchangeable. A disallowed URL can still appear in Google without a description, and a noindex tag on a disallowed page is never seen, so using both keeps pages in Google. That combination is the most common indexing mistake I see on small-business sites.

The guide explains the difference between discovery, crawling and indexing, and what each control does at which step, based on Google's documentation and the RFC 9309 standard. If you want something private, neither is right; it belongs behind a login or HTTP authentication.

I walk through fixing the double-blocked page (remove the Disallow, keep the noindex, let it be recrawled), the difference between the meta robots tag and the X-Robots-Tag header for PDFs and other files, and two things that don't work long term: a Noindex line in robots.txt (ignored since September 2019) and the Removals tool (temporary).

For AI, robots.txt is the main documented control for AI crawlers, while Google's own AI features follow the same noindex and snippet rules as Search, and Google-Extended controls Gemini use without affecting Search. I finish with common cases like staging sites and a decision table matching each goal to the right control.

Key takeaways

robots.txt controls crawling; noindex controls indexing
A noindex on a disallowed page is never seen
Use X-Robots-Tag for PDFs and non-HTML files
Protect staging sites with authentication, not robots.txt
Noindex in robots.txt has been ignored since 2019

Related on BuiltToWinWeb: canonical tags explained

Read the full guide on BuiltToWinWeb: Noindex vs robots.txt, and when to use each

A summary by Jacob Campbell of an article first published on built2winweb.com on October 7, 2026.

Canonical Tags Explained: Fixing Duplicate URLs for Google and AI

Sun, 18 Oct 2026 07:09:38 +0000

A canonical tag tells search engines which address is the primary one when identical content lives at several URLs. Google handles it as a hint rather than a rule, and duplicates are rarely deliberate: protocol and www variants, trailing slashes, tracking and filter parameters, print views and now Markdown versions of pages for AI agents. This guide explains how canonicals really work.

I start with why it matters: choosing which URL appears in results, consolidating link signals, keeping reporting clean and saving crawl time. Duplicate content isn't a penalty; the cost is dilution and loss of control. Then the four ways to declare a canonical, from strongest to weakest: a permanent redirect, rel=canonical in the head or an HTTP Link header (useful for PDFs and Markdown twins), and sitemap inclusion.

Google weighs several signals and may override you, so I explain how to check the result in URL Inspection by comparing the user-declared and Google-selected canonicals. There's a list of where duplicates come from with the right fix for each, noting that many are better solved with a redirect, and that paginated pages should keep their own canonicals.

The mistakes section covers canonicals pointing at noindexed, redirected or missing pages, everything canonicalised to the homepage, tags pushed into the body, conflicting canonicals and accidental cross-domain ones. I also cover multilingual sites and a closing checklist.

Key takeaways

A canonical is a strong hint, not a command
Redirect duplicates that don't need to exist
Use a Link header for PDFs and Markdown versions
Compare declared and Google-selected canonicals in URL Inspection
Never point a canonical at a noindexed, redirected or missing page

Related on BuiltToWinWeb: noindex vs robots.txt

Read the full guide on BuiltToWinWeb: Canonical tags explained

A summary by Jacob Campbell of an article first published on built2winweb.com on October 7, 2026.

The Files AI Agents Look For: llms.txt, Agent Cards and .well-known

Fri, 16 Oct 2026 03:58:11 +0000

Search engines learned about websites from robots.txt and sitemap.xml. AI agents need more, because they don't just read pages; they take actions. Between 2024 and 2026 a set of new files grew up that describe a site to agents, list its capabilities and explain how to connect. A few are formal standards, some are proposals with real adoption and some are experiments, and this guide says which is which.

At the root there's robots.txt for permissions, the sitemap for completeness, llms.txt and llms-full.txt for a curated summary and full text, and agents.txt, a newer and less settled convention I'd treat as a readable pointer file rather than a contract.

Under /.well-known/, reserved for site metadata by RFC 8615, the entries that matter are the A2A agent card, an MCP server card, OAuth protected-resource metadata (RFC 9728), authorisation server metadata, an API catalogue (RFC 9727) and security.txt (RFC 9116). I include a minimal agent card and a warning: a card that promises skills your endpoint doesn't have is worse than no card.

Discovery also works from any page through rel=alternate links and Link headers. I finish with the order I'd add everything in, starting with robots.txt and sitemaps and ending with agent cards and an API catalogue.

Key takeaways

Agents need discovery files beyond robots.txt and sitemaps
Know which files are standards and which are proposals
The /.well-known/ path holds agent cards, OAuth metadata and security.txt
Keep agent cards accurate and minimal
Add discovery links in page heads and HTTP headers too

Related on BuiltToWinWeb: the AI agent-readiness checklist

Read the full guide on BuiltToWinWeb: Every agent discovery file explained

A summary by Jacob Campbell of an article first published on built2winweb.com on October 5, 2026.

Two thirds of what Google's AI Overviews cite comes from the top 30% of a page.

Tue, 29 Sep 2026 23:46:22 +0000

We checked 600 real searches and traced about 4,300 cited passages back to the exact spot on the page they came from.

67% came from the top 30% of the page in Google AI Overviews, and 60% in Google AI Mode.

Kevin Indig's research found the same pattern for ChatGPT, but weaker: 44% of its citations come from the top 30%.

Takeaway: don't bury the lead. Put the answers you want AI to cite in the top 30% of your pages.

We're repeating these searches on a regular basis to see how much this changes. Follow along for the results and more of what we're finding about how to get cited by AI.

How to Audit Your Website for Broken Links and Technical Errors

Fri, 25 Sep 2026 17:01:13 +0000

A website can look completely functional while carrying technical problems that users, search engines, and security systems encounter every day. Broken links, missing pages, failed redirects, inaccessible resources, indexing errors, insecure content, and malformed HTML can remain hidden until someone performs a proper technical audit.

Regular auditing is therefore part of basic website maintenance. It is also relevant to security, compliance, and site management because technical failures can affect access to content, expose outdated resources, interfere with security controls, and create poor experiences for visitors.

The good news is that a technical audit does not require guessing. Website owners can combine crawling tools, browser developer tools, server information, Google Search Console, and standards-based validators to identify and correct many common problems.

Start With a Complete Crawl
The first step is to crawl the website rather than checking pages manually one at a time.

A crawler follows links across a site and records information about the URLs it encounters. Depending on the software, it can identify broken internal links, redirect chains, missing pages, duplicate URLs, response codes, canonical information, missing metadata, and other technical conditions.

For a useful audit, begin with the site's primary domain and allow the crawler to discover as many publicly accessible pages as possible. Review both HTML links and important resources such as images, scripts, stylesheets, documents, and downloadable files.

The objective is not simply to produce a large error report. It is to determine which errors represent genuine problems that require action.

Identify Broken Links and 404 Errors
A broken link sends visitors to a resource that cannot be retrieved as intended. One of the most common responses is HTTP 404, which means the server cannot find the requested resource. Common causes include deleted pages, incorrect URLs, renamed content, and links containing typing errors.

Start by separating internal and external broken links.

Internal broken links point from your website to another URL on the same website. These normally deserve priority because you control both the source page and destination.

External links lead to other websites. They can become invalid when another publisher removes, renames, or relocates a page. These links should also be reviewed, particularly when they support important claims, references, products, or instructions.

Do not automatically treat every 404 reported by Google as a technical emergency. Google states that 404 responses do not generally harm a site's search performance when the URLs genuinely should not exist. If content has been permanently removed and there is no suitable replacement, returning a 404 or 410 can be appropriate. If content has moved to a new URL, Google recommends using a redirect to the new location.

Check Redirects Carefully
Redirects are necessary when URLs change, but they should be audited rather than installed without review.

A 301 response indicates that a resource has been permanently moved to another URL. Browsers follow the redirect, and search engines can attribute the original URL's signals to the destination.

During an audit, look for:

Redirects pointing to another redirect.
Redirect chains involving several URLs.
Redirect loops.
Redirects sending visitors to irrelevant pages.
Old HTTP URLs that should resolve to HTTPS.
Links that still point to obsolete URLs when a current destination is available.
A redirect chain can often be simplified by changing an old link so it points directly to the final destination. This reduces unnecessary requests and makes the site's URL structure easier to maintain.

Avoid redirecting every missing URL to the homepage merely to eliminate 404 reports. Google specifically notes that redirecting a nonexistent page to another page, such as the homepage, instead of returning an appropriate 404 or 410 can create problems.

Use Google Search Console to Find Problems Google Sees
A crawler provides one perspective. Google Search Console provides another.

The Page Indexing report shows pages Google knows about and identifies indexing problems encountered during crawling. The URL Inspection tool can then be used to examine individual URLs and determine whether Google can access and index them.

Pay particular attention to:

Not found, 404 URLs.
Server errors.
Redirect problems.
Pages blocked from crawling.
Pages marked with noindex.
Soft 404 responses.
Canonicalization issues.
Important pages that are not indexed.
A soft 404 deserves special attention. This occurs when a page appears to tell visitors that content is unavailable but does not return an actual 404 response. Google recommends returning a proper 404 for genuinely missing pages.

Search Console should not be treated as a list where every warning must be eliminated. Some URLs are intentionally excluded from indexing, and Google explicitly notes that not every non-indexed URL represents a problem. Focus on important pages and investigate unexpected conditions.

Inspect HTTPS and Mixed Content
A technical audit should also examine the security of resource loading.

An HTTPS page can still contain links to HTTP resources. This is known as mixed content. Depending on the resource, browsers can upgrade the request to HTTPS or block it. Scripts, stylesheets, frames, and other active resources are particularly important because insecure loading can undermine the protection provided by HTTPS.

Check your pages for HTTP references involving:

JavaScript files.
CSS files.
Images.
Fonts.
Embedded frames.
APIs.
Downloads.
Third-party resources.
Open the browser's developer console while testing important pages. Browser warnings can identify resources that are being blocked or upgraded because they are being requested insecurely.

The preferred solution is to serve website resources through HTTPS. Where appropriate, security controls such as Content Security Policy can also help control which resources browsers are permitted to load. CSP is primarily a defensive mechanism that can restrict resource loading and reduce certain classes of security risk.

SSL and TLS management tools can assist with certificate and configuration monitoring, but they should complement the technical audit rather than replace it.

Test HTML and Technical Markup
Not every technical error produces a broken link.

Malformed HTML can create rendering problems, accessibility issues, and unexpected browser behavior. The W3C Markup Validation Service can validate HTML and related markup against applicable standards.

Run validation on representative pages, particularly:

Homepage.
Main navigation pages.
Landing pages.
Blog or article templates.
Product or service pages.
Forms.
Custom error pages.
Do not assume that a validation warning automatically means visitors are experiencing a visible problem. Validation is one diagnostic tool within a larger audit.

The important question is whether the reported issue affects page rendering, functionality, accessibility, maintainability, or another measurable requirement.

Examine Server and Browser Errors
HTTP status codes provide valuable information about what happens when a browser or crawler requests a resource.

The major response classes are:

2xx, successful responses.
3xx, redirects.
4xx, client-side request errors.
5xx, server-side errors.
A concentration of 5xx errors deserves immediate investigation because these responses indicate that the server is unable to successfully handle requests.

Check server logs where available. Look for recurring failures, application errors, database connection problems, timeout conditions, and requests for resources that no longer exist.

Browser developer tools can provide another layer of evidence. The Console can expose JavaScript errors, CSP violations, mixed-content warnings, and other client-side failures. The Network panel can show failed requests, response codes, redirects, and resource-loading problems.

This is particularly useful when a page appears normal visually but one of its interactive functions does not work.

Review Forms, Navigation, and Important User Journeys
A crawler can discover many technical errors, but it cannot always determine whether a website actually works for a visitor.

Manually test the most important journeys.

For example, test:

Homepage to primary navigation.
Navigation to important landing pages.
Search functionality.
Contact forms.
Account or login processes.
Checkout or payment pages, where applicable.
Downloads.
Mobile navigation.
Password-reset processes.
Error pages.
Test these journeys on both desktop and mobile devices where practical.

Pay attention to buttons that do nothing, forms that fail silently, links that open the wrong destination, downloads that return errors, and pages that require resources that are being blocked.

Build a Repair Priority List
An audit can produce hundreds of findings. Fixing them in arbitrary order wastes time.

Prioritize issues according to their practical impact.

Start with failures that prevent visitors from reaching important pages or completing essential actions. Next, address security-related problems, server errors, broken internal links, incorrect redirects, and indexing problems affecting important content.

Then handle lower-impact issues such as isolated external links, minor markup warnings, and obsolete references that do not affect important user journeys.

Keep a record of each issue, its source URL, destination URL, response code, recommended action, date discovered, and date resolved. This creates a maintenance history that can be reviewed after major site changes.

Turn Auditing Into Routine Maintenance
A website audit should not be a one-time exercise.

New content creates new links. Website redesigns change URLs. Plugins and extensions can modify resources. Third-party services can remove content. Developers can introduce new code. Security configuration can also affect how browsers load resources.

Google recommends checking indexing information regularly and after significant site changes.

A practical maintenance routine can include automated crawling, periodic Search Console reviews, SSL and TLS monitoring, browser testing, server-log analysis, and markup validation.

Security plugins, SSL providers, monitoring platforms, compliance SaaS products, and website management tools can help automate parts of this process. Their value depends on what they actually monitor and whether their alerts lead to actionable information.

The core principle remains simple: do not wait for visitors to discover technical failures. Crawl the site, verify the reported problems, investigate the underlying cause, fix the appropriate URLs or resources, and test the affected pages again.

A clean technical foundation supports reliable navigation, secure resource delivery, accurate indexing, and a better experience for the people using the website.

One Schema Bug, 39 Pages: Why a 100 Health Score Still Had 66 Open Issues

Thu, 17 Sep 2026 01:05:18 +0000

A crawler reported 101 error URLs on a 199 page site. They traced back to three causes: one wrong property in a shared schema generator, redirect rules living outside the repository, and a live page missing from the sitemap. Here is what the score actually counts, what it ignores, and why you verify findings against the build instead of the source.

An Ahrefs Health Score of 100 looks like a finished job. On one site, the score rose from 64 to 100 after 101 errors were reduced to zero across 199 internal pages. Yet the same crawl still reported 66 open issues. That apparent contradiction reveals what the score actually measures, what it leaves out, and why a site audit should be treated as a diagnostic tool rather than a checklist.

The score went from 64 to 100

In the first crawl, 101 of the site's 199 internal URLs contained at least one error. After the confirmed problems were corrected, the next crawl returned zero internal URLs with errors and raised the Health Score by 36 points, from 64 to 100.

Measure	First crawl	After fixes	Change
Ahrefs Health Score	64	100	+36
Internal URLs with at least one error	101	0	101 fewer
Internal URLs crawled	199	199	no change

The improvement was real, but the score alone made the result appear more complete than it was. The same report still contained seven warnings and 59 notices. Those 66 open issues did not prevent the site from receiving a perfect Health Score because Ahrefs does not include them in that calculation.

What an Ahrefs Health Score of 100 actually means

Ahrefs calculates its Health Score using the percentage of crawled internal URLs that contain no errors. If every internal URL passes without an error, the site receives a score of 100. Warnings and notices may still appear in the report, but they do not lower the score. A score of 100 therefore means that one crawler found no errors on the internal pages it reached during that crawl. It does not mean the site is finished, ranks well, attracts traffic, converts visitors, or appears in AI-generated recommendations. It is a useful measure of technical condition, but it is not a complete measure of website performance.

A perfect score can still have 66 open issues

The completed crawl displayed a Health Score of 100 alongside seven warnings and 59 notices. Six pages still had structured data that did not pass Google's rich-results validation. All of these findings appeared in the same report, but none counted against the score because Ahrefs classified them below the error level.

Finding type	Count	Counts toward the Health Score
Errors	0	Yes
Warnings	7	No
Notices	59	No
Total open issues	66

That does not mean every warning or notice represents a problem that must be fixed. Some findings reflect recommendations, crawler limitations, or conditions that require further testing. However, it does mean that the score should never be read without the issue list beside it. The 100 shows that the crawler found no errors. The remaining 66 items show that there was still work to evaluate.

The three problems hiding behind 101 errors

The 101 figure represented internal URLs with at least one error, not 101 unrelated defects. Much of the report could be traced back to three underlying problems:

One schema mistake repeated across 39 pages. The pages applied addressRegion and addressCountry to a City object, even though those properties belong to PostalAddress. Because the same schema generator served every location page, one mistake was reproduced 39 times.
Six URLs had been broken for two months. Four cross-posted articles and two retired case studies returned 404 errors. Their redirects were managed through the hosting console rather than the repository, allowing the live rules and the saved configuration to drift apart.
One live page was missing from the sitemap. A Spanish-language case study returned a successful 200 response when visited directly, but it was absent from the translated-path map. As a result, it never appeared in the sitemap and had no matching hreflang reference.

One schema mistake repeated across 39 pages

The site applied addressRegion and addressCountry to a City object, even though those properties belong to PostalAddress. The correct way to describe the geographic relationship was through containedInPlace, which the site was already using elsewhere in its structured data. Because the same schema generator served every location page, one incorrect implementation produced validation errors across 39 pages. Correcting the shared generator fixed all of them at once.

Structured data property	What the site did	What is correct
`addressRegion`	Applied to a City object	Belongs on PostalAddress
`addressCountry`	Applied to a City object	Belongs on PostalAddress
`containedInPlace`	Already used elsewhere in the site's structured data	The correct way to describe the geographic relationship
Pages affected	39	All repaired by correcting one shared generator

Redirect rules that drifted outside the repository

Four cross-posted articles and two retired case studies had returned 404 errors for two months because their redirects were managed through the hosting console rather than the normal deployment process. The repository contained four rules that had never been applied, while the live console contained three working rules missing from the repository. Replacing the live configuration with the saved version would have repaired the six broken URLs but deleted three valid rules, including one used to serve images.

Where the rules lived	Rules present	Status
Repository	4	Never applied to the live site
Hosting console	3	Working, but missing from the repository
Affected URLs	6	Returning 404 for two months

If part of a site's behavior lives outside its repository, it is invisible to code review, invisible to deployment, and free to drift quietly out of sync.

A live page missing from its own sitemap

One Spanish-language case study was live and working, but missing from the translated-path map.

What visitors saw

Anyone with the direct link could open the page normally. It returned a successful 200 response and appeared to work like any other page on the site.

What search engines saw

The page was absent from the XML sitemap and had no matching hreflang reference connecting it to the English version. The page existed, but the site was not properly telling search engines where to find it or how it related to the rest of the content.

Signal	Status
Direct visit to the URL	200 response, page loads normally
Present in the translated-path map	No
Listed in the XML sitemap	No
hreflang reference to the English version	No

Publishing a page and making it discoverable are two separate jobs. The first can succeed while the second fails silently.

Audit tools report clues, not conclusions

During an earlier crawl of the same site, Ahrefs reported five broken images and two broken scripts. Each of the site's 41 images and 32 scripts was then tested directly, and every file returned a successful 200 response. Reports of multiple H1 elements and duplicate title tags could not be reproduced either. This does not make the audit useless, but it does show why every finding should be treated as a lead to investigate rather than proof of a defect. Confirm the problem on the live site before changing anything.

What the crawler reported	What was tested	What was found
5 broken images	All 41 images on the site	Every file returned 200
2 broken scripts	All 32 scripts on the site	Every file returned 200
Multiple H1 elements	The live pages	Could not be reproduced
Duplicate title tags	The live pages	Could not be reproduced

Check the website visitors receive, not just the source code

Reviewing the project files is not enough because templates, build tools, and scripts can change what is ultimately published. After the schema correction, all 391 JSON-LD blocks in the compiled website were parsed to confirm that none still contained the invalid Place properties. Searching the original source files could have missed generated markup and produced a clean result while the live output remained wrong. The final website, not the code used to create it, is what visitors, search engines, and AI crawlers actually receive.

Fix the generator, not every affected page

The 39 affected location pages did not need 39 separate edits. They were all produced by the same schema generator, so correcting three shared values in one file repaired every page at once. When the same defect appears repeatedly, the first question should be what those pages share. Editing each output individually may clear the current report, but fixing the generator removes the underlying cause and prevents the same problem from returning on the next page created.

Sixteen issues disappeared and the score did not move

After the schema correction was published, a new crawl confirmed the result:

Measure	Before	After	Change
Health Score	100	100	no change
Schema validation notices	16	0	16 fewer
Total notices	75	59	16 fewer
Total open issues	81	66	15 fewer

Sixteen confirmed issues disappeared, yet the Health Score remained exactly the same. The total issue count fell by 15 rather than 16 because the new crawl added one unrelated warning. The result demonstrates the limitation of the score using the site's own data: meaningful technical improvements can be measured, confirmed, and completely invisible to the headline number.

What a score of 100 is actually worth

A Health Score of 100 means that 199 internal pages returned no errors to one crawler during one crawl. That is useful. It confirms that the most severe technical problems have been removed and gives the site a cleaner foundation. It does not prove that the site is finished, visible, competitive, or producing results.

The real value of reaching 100 is not the number itself. It is reaching the point where technical errors can no longer serve as the easy explanation for everything that is not working. From there, attention can move to the harder questions: whether people can find the site, whether the content earns trust, whether search engines understand it, and whether visitors become customers. A perfect audit score is not the finish line. It is where the easy technical excuses end.

Audit your website

If you want to see what a crawler finds on your own website, start with DeepAudit AI. It is free, requires no signup, and runs approximately 100 checks across nine categories, including technical SEO, performance, accessibility, structured data, security, and AI discoverability. It renders the website in a browser, can crawl the sitemap, and provides a report with the affected code so you can see where each finding came from.

Treat its results the same way every audit result should be treated: as a starting point for investigation, not a list to follow blindly. Confirm what is real, fix the underlying cause, and measure the website again after the change.

First published on the Axion Deep Digital blog: An Ahrefs Health Score of 100 Can Still Mean 66 Open Issues

Google Rankings Aren’t the Whole Story Anymore: SEO in the Age of AI Search

Wed, 09 Sep 2026 08:03:47 +0000

🚀 Google Rankings Aren’t the Whole Story Anymore: SEO in the Age of AI Search

You rank on Google. Great. But are people actually finding you?

For years, SEO had a familiar goal:

Get higher on Google.

Page 10 → Page 3 → Page 1 → Position #1 🎯

But search is changing.

Today, people discover information through traditional search engines, AI-generated answers, AI Overviews, conversational search, and other discovery experiences.

And that creates an important question:

Is a Google ranking alone enough to understand your real search visibility?

Probably not.

Welcome to SEO in the age of AI search.

🤔 A Quick SEO Challenge

Imagine this:

Your website ranks #3 on Google for an important keyword.

Sounds great.

But then you notice:

📈 Impressions are increasing
🖱️ Clicks are staying flat
🤖 AI-generated answers appear prominently
👀 Competitors appear across more search experiences
📉 Your organic traffic barely changes

Is ranking #3 still a success?

Yes — but it doesn't tell the whole story.

Modern SEO requires us to look beyond a single position number.

🔍 The Search Journey Is Changing

Traditional Search

User searches
↓
Google results
↓
User clicks a website
↓
User finds the answer

AI-Assisted Search

User asks a question
↓
Search engine / AI system
↓
Generated answer or summary
↓
Sources, recommendations and links
↓
Maybe a website click

Notice the important word?

Maybe.

Users can sometimes get the information they need directly from the search experience.

That means the way we measure SEO success needs to evolve too.

🧠 Ask a Better SEO Question

Instead of asking only:

“What position am I ranking?”

Start asking:

“How visible is my website or brand wherever my audience is searching?”

That's a much bigger question.

And it changes what we need to monitor.

📊 5 SEO Signals Worth Watching

1️⃣ Keyword Rankings

Rankings still matter.

But don't look only at today's position.

Look at the movement.

Day	Position
Monday	#18
Tuesday	#15
Wednesday	#11
Thursday	#8
Friday	#6 🚀

The trend tells you much more than simply seeing:

Current position: #6

2️⃣ Search Impressions

A good ranking doesn't automatically mean strong visibility.

Search impressions help answer:

How often is my website actually appearing in search?

For example:

Metric	Last Month	This Month
Average Position	8.1	5.4
Impressions	12,400	18,900
Clicks	820	790

Interesting, isn't it?

Your ranking improved. ✅

Your impressions increased. ✅

But your clicks dropped. ⚠️

That's something worth investigating.

3️⃣ Click-Through Rate

Now consider two pages:

Page	Position	Impressions	CTR
Page A	#2	20,000	1.8%
Page B	#5	12,000	5.7%

Which page is performing better?

There isn't always a simple answer.

CTR can be influenced by:

Search intent
Page title
Search snippet
Brand recognition
Competing results
SERP features
AI-generated answers

So remember:

Ranking ≠ Complete Performance

4️⃣ Backlinks & Authority

Search visibility isn't only about keywords.

Quality backlinks can help search engines understand that other websites consider your content useful or authoritative.

Ask yourself:

Who is linking to my website?
Are those links relevant?
Are important backlinks disappearing?
Which pages attract the most links?
Are competitors earning links that I'm missing?

5️⃣ Technical SEO Health

Great content can still struggle if your website has technical problems.

Keep an eye on:

🔎 Crawlability
📑 Indexing
🔗 Broken links
🏷️ Metadata
🧱 Page structure
📱 Mobile usability
⚡ Performance
🔁 Canonical issues

Your content can't perform well if search engines struggle to access or understand it.

🎯 30-Second SEO Health Check

Can you answer these questions about your website right now?

[ ] What are my highest-ranking keywords?
[ ] Which keywords improved this week?
[ ] Which keywords dropped?
[ ] Which pages receive the most impressions?
[ ] Which pages have poor CTR?
[ ] Are there technical SEO problems?
[ ] Which websites link to me?
[ ] Does performance change by country?
[ ] Am I checking more than one search engine?
[ ] Can I see important SEO changes without jumping between several tools?

What's your score?

8–10 → 🟢 Strong visibility monitoring

5–7 → 🟡 Good start, but there may be gaps

0–4 → 🔴 Your website may be telling you a story you're not currently seeing.

🌍 Google Isn't the Entire Search Ecosystem

Google is hugely important.

But your audience doesn't necessarily exist in only one search environment.

Depending on the country and audience, discovery can happen through:

Google
Bing
DuckDuckGo
Yahoo
Baidu
Yandex
AI-powered search experiences

This is why looking at:

Keyword + Country + Search Engine + Performance

can provide a better picture than looking at one global ranking number.

🤖 And Then Came AI Search

This is where things get particularly interesting.

Search engines are increasingly trying to answer questions, rather than simply return a list of links.

Someone might search:

“How can I improve technical SEO for a small business website?”

Traditionally, the user would choose a result and visit the website.

Now, they may first encounter an AI-generated answer or summary.

That means websites are increasingly competing for more than:

Position #1

They're competing for:

👀 Visibility
🧠 Authority
🎯 Relevance
🤝 Trust
💬 Mentions
📚 Useful content
🖱️ Clicks

💡 So What Should Website Owners Do?

Don't abandon traditional SEO.

The fundamentals still matter.

✅ Create genuinely useful content

Answer the question the visitor actually has.

Not what you think Google wants.

✅ Maintain strong technical SEO

Make it easy for search engines to:

Crawl → Understand → Index → Rank

your content.

✅ Build topical authority

Don't publish 50 unrelated articles simply because keywords have search volume.

Build useful depth around subjects connected to your business.

✅ Earn trustworthy signals

Relevant backlinks, references and brand mentions can strengthen your presence.

✅ Monitor what changes

SEO isn't:

Optimize → Done ❌

It's:

Measure → Understand → Improve → Monitor → Repeat 🔄

🛠️ Why We Built DevnDespro SEO

While working with websites, we kept seeing the same challenge:

SEO information is scattered across different places.

Rankings in one place.

Search Console somewhere else.

Backlinks somewhere else.

Technical issues somewhere else.

Reports somewhere else.

That led us to build DevnDespro SEO around a simple idea:

Make everyday SEO monitoring easier to access and understand.

With DevnDespro SEO, we're bringing important SEO workflows closer together.

🔎 Keyword Monitoring

Track the keywords that actually matter to your website.

📈 Ranking Insights

See where your keywords stand and monitor changes over time.

🌍 Country-Based Tracking

Search visibility isn't identical everywhere.

Track rankings with location in mind.

🔍 Multiple Search Engines

Go beyond a Google-only view of SEO.

📊 Google Search Console Insights

Understand impressions, clicks and search performance.

🔗 Backlink Information

Keep an eye on the websites connecting to yours.

🩺 SEO / Website Auditing

Identify problems that could be holding your website back.

📑 Reports

Bring important SEO information together in a clearer format.

💻 At Your Desk. 📱 On the Move.

SEO problems don't politely wait until you're sitting in front of your laptop.

Maybe you published a new page yesterday.

Maybe an important keyword suddenly moved.

Maybe you're travelling and simply want to check your project.

That's why DevnDespro SEO is available on both Web and Android.

💻 DevnDespro SEO Web

Use the web application when you want a broader view of your SEO projects, rankings, backlinks, reports and website insights.

👉 Explore DevnDespro SEO

https://www.devndespro.com/

📱 DevnDespro SEO for Android

Want to check SEO without opening your laptop?

Take your SEO monitoring with you.

👉 Get DevnDespro SEO on Google Play

https://play.google.com/store/apps/details?id=com.devndespro.seo

🧪 Try This With Your Own Website

Choose five keywords that matter to your business.

Fill in this simple table:

Keyword	Current Rank	Country	Search Engine	Landing Page
Keyword 1	?	?	?	?
Keyword 2	?	?	?	?
Keyword 3	?	?	?	?
Keyword 4	?	?	?	?
Keyword 5	?	?	?	?

Check them again next week.

But don't only ask:

“Did I reach #1?”

Ask:

“What changed — and why?”

That's where useful SEO analysis begins.

🔮 The Future Isn't “SEO vs AI”

It's tempting to frame the conversation as:

Traditional SEO is dying. AI search is replacing it.

But the reality is more interesting.

People will continue searching for:

Answers
Products
Businesses
Comparisons
Recommendations
Tutorials
Services
Trusted information

What changes is how those answers are discovered and presented.

And that means SEO isn't disappearing.

SEO is evolving.

🚀 One Question to Take Away

For years, SEO conversations started with:

“Where do I rank on Google?”

In the age of AI search, there's a bigger question:

“When someone searches for what I offer, where — and how — does my brand appear?”

That's the visibility worth understanding.

👋 Your Turn

What are you paying the most attention to right now?

Google rankings?
Organic traffic?
Click-through rate?
Backlinks?
AI-search visibility?

SEO is changing quickly.

The interesting part is learning what to measure next.

🚀 Explore DevnDespro SEO

🌐 Web Application
https://www.devndespro.com/

📱 Android Application
https://play.google.com/store/apps/details?id=com.devndespro.seo

Track. Understand. Improve. Wherever you work.

DevnDespro SEO is continuously evolving. Available features and supported data sources may change as the platform develops.

How to Tell If Your Store's SEO Work Is Actually Working

Fri, 28 Aug 2026 06:32:26 +0000

Most of us here build the store rather than run the marketing for it, so SEO tends to arrive as a ticket with no acceptance criteria attached. This is the measurement half of that problem, written for whoever has to prove the work did something.

Rankings Are a Lagging Indicator

Rank tracking is the first thing everyone reaches for and the slowest thing to move. A position change reflects work you did weeks ago, it is noisy across locations and devices, and a single keyword tells you almost nothing about the pages that actually earn revenue.

Impressions in Search Console move first. When you fix a title, restructure a category page, or unblock a section of the site, impressions for that page shift within days, well before position does. If impressions stay flat after a change, the change did not register at all, and waiting longer will not fix that.

The Four Numbers Worth Tracking

Indexed page count against expected page count. If you have 800 products and Google has indexed 300, nothing else you do matters until that gap closes. It is usually a crawl or duplication problem rather than a content one.

Impressions by page type. Split product pages, category pages and content pages apart. They fail for different reasons, and an aggregate number hides which one is broken.

Click through rate at a fixed position. If a page holds position 4 and its CTR is half what position 4 normally returns, the problem is the title and description, not the ranking.

Revenue per organic session by landing page. This is the one that decides where the next sprint goes. A page with modest traffic and high purchase intent beats a high traffic page that only attracts researchers.

Why the First Six Months Look Like Failure

The uncomfortable part is that all four of these can be improving while the traffic graph stays flat. Indexation fills in first, then impressions, then positions, then clicks. Most stores need 6 to 12 months of consistent work before that chain finishes and organic traffic becomes meaningful.

That lag is why so many teams abandon the work and go back to paying $2 to $8 per click for the same visitors. It is also why the ones who stay end up with a channel competitors cannot simply buy their way into. For the full picture of what feeds that chain, from keyword intent through the technical foundations, this ecommerce SEO guide covers it end to end.

Measure the leading indicators and the six month wait becomes legible instead of scary. You can tell the difference between work that is not landing and work that has not surfaced yet, and those two need completely different responses.

The Complete Technical SEO Checklist for Next.js Developers (2026 Guide)

Sun, 09 Aug 2026 05:53:15 +0000

You can write the best blog post in your niche, ship a beautifully designed landing page, and still get zero organic traffic — because Google never properly crawled it, never understood what it was, or ranked a slower competitor above it instead. That's the frustrating part of SEO nobody warns you about: content quality is only half the battle. The other half is technical SEO — the plumbing that decides whether search engines can find, understand, and trust your pages in the first place.

The good news is that technical SEO isn't a mystery box. It's a checklist. Once you know what to check, most of it takes an afternoon to fix in a Next.js app — no marketing degree required.

By the end of this guide, you'll have a complete, practical checklist covering crawlability, metadata, structured data, Core Web Vitals, and content quality — with real Next.js 16 App Router code for each item.

Why Technical SEO Is Different From "Writing Good Content"

Content SEO is about what you say. Technical SEO is about whether search engines can even reach what you said. A page can have perfect content and still fail because:

It's blocked in robots.txt by accident
It has no canonical URL, so Google indexes three duplicate versions of it
It loads so slowly that Google deprioritizes it in rankings
It has no structured data, so it never qualifies for rich results

None of these are content problems — they're implementation problems. And implementation problems are exactly what a developer is best positioned to fix.

Why Technical SEO Matters for AI Search Too

In 2026, "search" doesn't just mean the Google results page anymore. Tools like ChatGPT, Gemini, Claude, and Perplexity all crawl and cite web pages when answering questions — and the exact same fundamentals that help Google rank you are what let these AI systems find, parse, and trust your content in the first place.

Structured data gives AI systems an unambiguous, machine-readable summary of what a page is — an article, a product, a FAQ — instead of forcing them to guess from raw HTML.
Semantic HTML (real <h1>–<h3> headings, <article>, <nav>, proper list elements instead of div soup) makes it far easier for any crawler, AI or otherwise, to understand a page's structure and pull an accurate answer out of it.
Crawlable pages — no accidental noindex, no content trapped behind client-side-only rendering with nothing in the initial HTML — matter just as much to an AI crawler as to Googlebot. If a bot can't fetch it, it can't cite it.

The practical takeaway: none of this requires a separate "AI SEO" strategy. Everything in this checklist already does double duty — it's the same groundwork that gets a page indexed by Google, ranked well, and eligible to be cited by an AI assistant.

1. Crawlability & Indexing

Before Google can rank your page, it has to find it and be allowed to index it. This is the foundation — get it wrong and nothing else on this list matters.

Add a `robots.txt`

robots.txt tells search engine crawlers which parts of your site they're allowed to visit. In Next.js, you don't write raw text — you export a config from a special file.

// app/robots.ts
import { MetadataRoute } from "next";

export default function robots(): MetadataRoute.Robots {
  return {
    rules: {
      userAgent: "*",
      allow: "/",
      disallow: ["/admin", "/api"], // block anything that shouldn't be crawled
    },
    sitemap: "https://yourdomain.com/sitemap.xml",
  };
}

What's happening here: Next.js automatically generates a working /robots.txt route from this file. allow: "/" lets crawlers visit everything by default, while disallow blocks specific paths like admin dashboards or internal API routes that have no business being indexed.

⚠️ Common Mistake: Accidentally leaving disallow: "/" in a robots.txt after moving from staging to production. This single line blocks your entire site from every search engine — always double-check this file right after deploying.

Add a `sitemap.xml`

A sitemap is a map of every URL on your site that you want indexed. It doesn't force Google to index anything, but it makes discovery much faster — especially for new pages that have no other links pointing to them yet.

// app/sitemap.ts
import { MetadataRoute } from "next";
import { getAllPosts } from "@/lib/posts";

export default function sitemap(): MetadataRoute.Sitemap {
  const baseUrl = "https://yourdomain.com";

  const posts = getAllPosts().map((post) => ({
    url: `${baseUrl}/blog/${post.slug}`,
    lastModified: new Date(post.updatedAt || post.publishedAt),
    changeFrequency: "weekly" as const,
    priority: 0.8,
  }));

  const staticRoutes = ["", "/about", "/blog", "/contact"].map((route) => ({
    url: `${baseUrl}${route}`,
    lastModified: new Date(),
    changeFrequency: "monthly" as const,
    priority: route === "" ? 1 : 0.6,
  }));

  return [...staticRoutes, ...posts];
}

What's happening here: this generates every blog post URL dynamically from your data source, plus a handful of static pages, and outputs them at /sitemap.xml automatically. lastModified helps Google know when to re-crawl a page; priority is a soft hint about relative importance, not a guarantee.

💡 Tip: Submit your sitemap URL directly inside Google Search Console → Sitemaps after deploying. It won't guarantee faster indexing, but it removes any ambiguity about where your sitemap lives.

Check Your Robots Meta Tag

Even if robots.txt allows crawling, an individual page can still say "don't index me" via a meta tag. This matters most for things like thank-you pages, internal search results, or draft content.

// app/some-page/page.tsx
import type { Metadata } from "next";

export const metadata: Metadata = {
  robots: {
    index: false, // don't show this page in search results
    follow: true, // but still crawl the links on it
  },
};

⚠️ Common Mistake: Setting index: false on your root layout "just to be safe" during development, then forgetting to remove it before launch. This is one of the most common reasons a brand-new site gets zero organic traffic — always verify your homepage's robots meta in production.

Verify With Google Search Console

Search Console is free and non-negotiable. It tells you, directly from Google, which pages are indexed, which are excluded (and why), and whether your sitemap was actually read. Set it up early — indexing issues are invisible until you check.

The single most-used feature here is the URL Inspection Tool. Paste any URL from your site into it and Google tells you exactly whether that page is indexed, when it was last crawled, and — if it's not indexed — why not. After a major change to a page (new metadata, fixed structured data, a rewritten intro), use it to request reindexing instead of waiting for Google to notice on its own.

💡 Tip: URL Inspection is also the fastest way to debug "why isn't my new blog post showing up in search" — it'll tell you plainly if it's still queued, blocked, or already indexed but just not ranking yet (which is a different problem).

Don't Forget Bing Webmaster Tools

Google isn't the only search engine worth indexing on. Bing powers Bing search itself, plus Yahoo, DuckDuckGo (partially), and is the underlying index behind Microsoft Copilot's web results — meaningful traffic that's easy to pick up for almost no extra work.

Sign up at Bing Webmaster Tools and verify your site
Submit the exact same sitemap.xml you already built for Google — no separate file needed
Bing also offers a URL Inspection-style tool and a URL Submission option for faster indexing of new pages

💡 Tip: Bing Webmaster Tools lets you import your verified site directly from Google Search Console in a couple of clicks, which skips most of the manual verification steps entirely.

2. Metadata: Titles, Descriptions & Canonical URLs

Metadata doesn't change how your page looks — it changes how search engines describe and rank it, and how it appears when shared.

Related: How to Add Metadata in Next.js 16 (2026 Beginner's Guide) covers the full Metadata API in more depth if you want a deeper dive after this section.

Set a Title Template Once, Not on Every Page

// app/layout.tsx
import type { Metadata } from "next";

export const metadata: Metadata = {
  metadataBase: new URL("https://yourdomain.com"),
  title: {
    default: "YourSite — Tagline Here",
    template: "%s | YourSite",
  },
  description: "A clear, honest description of what your site does.",
};

What's happening here: metadataBase resolves every relative URL elsewhere in your metadata (like Open Graph images) into an absolute URL automatically. title.template means any page that sets title: "My Post" automatically renders as "My Post | YourSite" — you never repeat the suffix manually.

⚠️ Common Mistake: Manually typing "My Post | YourSite" as the title on individual pages while also having a template set in the root layout. This doubles the suffix into something like "My Post | YourSite | YourSite". Let the template do the work — pages should only set the unique part.

Write Unique Titles and Descriptions Per Page

// app/blog/[slug]/page.tsx
export async function generateMetadata({ params }: Props): Promise<Metadata> {
  const { slug } = await params;
  const post = getPostBySlug(slug);

  return {
    title: post.title,
    description: post.description,
    alternates: {
      canonical: `https://yourdomain.com/blog/${post.slug}`,
    },
  };
}

Length guidelines to actually stick to:

Title: ~50–60 characters (roughly 580px) before Google truncates it
Description: ~150–160 characters — treat it as ad copy for your own page, not a summary

Canonical URLs

A canonical tag tells Google "this is the one true version of this page" when the same content is reachable through multiple URLs — with or without a trailing slash, with query params, or duplicated across categories.

export const metadata: Metadata = {
  alternates: {
    canonical: "https://yourdomain.com/blog/my-post",
  },
};

💡 Tip: Next.js already infers a canonical URL from your route and metadataBase in most cases. Set this explicitly only when you have genuine duplicate content — like a product reachable at two different category paths.

3. Structured Data (JSON-LD)

Structured data doesn't change what a page looks like to a human — it tells search engines and AI assistants explicitly what kind of content a page is: an article, a product, an FAQ, a business. This is what unlocks rich results like star ratings, breadcrumbs, and FAQ dropdowns directly in search listings.

// app/blog/[slug]/page.tsx
import Script from "next/script";

<Script
  id="blog-post-jsonld"
  type="application/ld+json"
  dangerouslySetInnerHTML={{
    __html: JSON.stringify({
      "@context": "https://schema.org",
      "@type": "BlogPosting",
      headline: post.title,
      description: post.description,
      datePublished: post.publishedAt,
      dateModified: post.updatedAt || post.publishedAt,
      author: { "@type": "Person", name: "Your Name" },
      publisher: {
        "@type": "Organization",
        name: "YourSite",
        url: "https://yourdomain.com",
      },
    }),
  }}
/>

What's happening here: this injects a <script type="application/ld+json"> block with zero visible UI impact. Google's crawler parses it separately from your rendered content to understand the page's structure.

⚠️ Common Mistake: Assuming valid JSON-LD automatically means eligible for rich results. Google layers its own required-field requirements on top of the schema.org spec — an Article missing an author, for instance, is valid JSON-LD that Google will still ignore for rich-result purposes. Always test with Google's Rich Results Test before assuming it's working.

Common types worth adding depending on your page:

Page type	Schema type
Blog post	`Article` / `BlogPosting`
FAQ section	`FAQPage`
Product page	`Product`
Local business	`LocalBusiness`
Software/SaaS tool	`SoftwareApplication`
Nav trail	`BreadcrumbList`

💡 Save time: use the JSON-LD Generator to generate any of these schema types with a live eligibility checklist scored against Google's actual Rich Results requirements, instead of hand-writing and second-guessing the JSON.

4. Open Graph & Twitter Cards

This is technically a "social sharing" feature, but it directly affects click-through rate from search and social — and CTR is a real ranking signal.

export const metadata: Metadata = {
  openGraph: {
    title: post.title,
    description: post.description,
    url: `https://yourdomain.com/blog/${post.slug}`,
    siteName: "YourSite",
    type: "article",
    images: [{ url: "/og-image.png", width: 1200, height: 630 }],
  },
  twitter: {
    card: "summary_large_image",
    title: post.title,
    description: post.description,
  },
};

💡 Tip: 1200×630px is the sweet spot for Open Graph images — smaller images get stretched and look blurry on most platforms. Reuse the same image for both openGraph and twitter unless you have a specific reason not to.

Not sure how Open Graph and Twitter Cards actually differ, or whether you need both? Open Graph vs Twitter Cards: What's the Difference? breaks down the exact syntax differences and when each one is read. And if you'd rather generate all of this instead of hand-writing it, the Meta Tag Generator outputs SEO tags, Open Graph, Twitter Cards, and JSON-LD from one set of inputs.

5. Core Web Vitals & Performance

Google explicitly uses Core Web Vitals as a ranking signal. These aren't vague "make it fast" advice — they're three specific, measurable metrics.

Metric	What it measures	Good threshold
LCP (Largest Contentful Paint)	How fast the main content loads	Under 2.5s
INP (Interaction to Next Paint)	How responsive the page feels to input	Under 200ms
CLS (Cumulative Layout Shift)	How much the layout jumps around while loading	Under 0.1

The Next.js-Specific Fixes That Matter Most

// Always use next/image instead of <img>
import Image from "next/image";

<Image
  src="/hero.jpg"
  alt="Descriptive alt text" // required — also helps SEO and accessibility
  width={1200}
  height={630}
  priority // only for above-the-fold images
/>

Why this matters: next/image automatically serves resized, modern-format (WebP/AVIF) images and reserves layout space before the image loads — directly improving both LCP and CLS. priority should only go on the one or two images visible without scrolling; overusing it defeats lazy loading for everything else.

// Load fonts through next/font to avoid layout shift
import { Inter } from "next/font/google";

const inter = Inter({ subsets: ["latin"], display: "swap" });

Why this matters: next/font self-hosts Google Fonts at build time, eliminating the render-blocking network request to Google's font CDN and preventing the "flash of invisible/unstyled text" that causes layout shift.

⚠️ Common Mistake: Reserving no space for ads, embeds, or dynamically loaded content. Every element that pops in after initial render without a reserved size pushes content around — that's exactly what CLS measures.

Run Lighthouse (built into Chrome DevTools) for quick local lab data, but don't stop there — Google PageSpeed Insights runs the same Lighthouse audit and layers in real-user field data (from the Chrome User Experience Report) when enough traffic exists for your URL. That field data is what actually matters, since lab data and field data can disagree, and Google ranks based on the field data your real visitors generate — not your local Lighthouse score. Cross-check both there and in Search Console's Core Web Vitals report.

6. Mobile-Friendliness

Google indexes the mobile version of your site first (mobile-first indexing), not desktop. If your mobile layout is broken, that's the version Google evaluates for ranking — even for desktop searches.

Quick checklist:

Responsive layout at all breakpoints (test with Tailwind's sm:/md:/lg: prefixes actually rendering correctly)
Tap targets at least 44×44px
No horizontal scrolling
Text readable without zooming (16px minimum for body text)

💡 Tip: Test with Chrome DevTools' device toolbar set to an actual mid-range device profile, not just "responsive" mode dragged to a guess-width.

7. URL Structure & Internal Linking

Keep URLs Short, Readable, and Keyword-Relevant

Good:  /blog/nextjs-metadata-guide
Bad:   /blog/post?id=48291&ref=xyz

Next.js's file-based routing makes this almost automatic — just name your route folders and slugs sensibly.

Internal links help Google understand which pages are related and pass authority between them. They also keep users (and crawlers) moving deeper into your site instead of bouncing after one page.

<p>
  Once auth is set up, check out our{" "}
  <Link href="/blog/stripe-checkout-guide">Stripe Checkout guide</Link> to
  add billing next.
</p>

⚠️ Common Mistake: Building tool pages or landing pages with almost no surrounding prose or internal links. Thin, isolated pages are exactly what tools like Google AdSense and Search Console's coverage report flag as low-value content — a page with substantial explanatory text and links to related pages reads as more legitimate to both crawlers and ad review systems.

8. HTTPS & Basic Security

This one's usually already handled if you're on Vercel, Netlify, or similar — but confirm it:

Every page loads over https://, with no mixed-content warnings (images/scripts loaded over plain http://)
www vs non-www redirects to one canonical version, not both serving live content
No broken SSL certificate warnings

HTTPS has been a confirmed Google ranking signal since 2014 — it's small, but it's free, so there's no reason to skip it.

9. Content Quality & Thin Content

Technical SEO gets your page crawled and indexed. It does not make a page worth ranking if there's nothing substantial on it. This matters especially for:

Tool pages (calculators, generators) — pair the actual tool with real explanatory prose about what it does and why it matters, not just a bare form
Category/index pages — a list of links with zero context reads as thin to both search engines and human reviewers
Duplicate boilerplate — if ten pages share 90% identical copy with one variable swapped, consolidate or genuinely differentiate them

💡 Tip: A useful gut check: if you deleted the interactive/dynamic part of a page, would there still be a paragraph or two of genuinely useful content left? If not, that page is a thin-content risk.

The Full Checklist at a Glance

[ ] robots.txt exists and isn't accidentally blocking the whole site
[ ] sitemap.xml exists and is submitted in Search Console
[ ] No unintended noindex on pages that should rank
[ ] Site verified and sitemap submitted in Bing Webmaster Tools too
[ ] Every page has a unique title (50–60 chars) and description (150–160 chars)
[ ] Canonical URLs set where duplicate content is possible
[ ] JSON-LD structured data added for articles, FAQs, products, or business info
[ ] Open Graph + Twitter Card tags on every shareable page
[ ] Images use next/image with descriptive alt text
[ ] Fonts loaded through next/font
[ ] Core Web Vitals checked in Search Console (not just Lighthouse locally)
[ ] Mobile layout tested on a real device profile
[ ] URLs are short, readable, and keyword-relevant
[ ] Related pages link to each other with real anchor context
[ ] Site is fully on HTTPS with no mixed content
[ ] No thin, boilerplate-only, or near-duplicate pages

Frequently Asked Questions

How long does it take for technical SEO changes to affect rankings?

Indexing changes (like fixing robots.txt) can show up in days. Ranking changes from Core Web Vitals or content improvements typically take weeks, since Google needs to re-crawl and re-evaluate the page relative to competitors.

Do I need a sitemap if my site is small?

Technically no — Google can discover a handful of linked pages without one. But it costs almost nothing to add and removes any doubt about discovery, so there's no real reason to skip it even on small sites.

Is Core Web Vitals a huge ranking factor?

It's one signal among many, and generally a tiebreaker rather than a dominant factor — strong content with mediocre performance usually still outranks weak content with perfect performance. That said, "mediocre" has a floor; genuinely poor Core Web Vitals can hold a page back.

Can I check all of this without Google Search Console?

Not fully. Lighthouse and third-party tools give you lab data, but Search Console gives you real user field data and actual indexing status — which is what Google uses for ranking decisions. It's free and essential, not optional.

Does technical SEO matter if I'm not writing much content yet?

Yes — technical SEO is largely a one-time setup cost (metadata patterns, sitemap, structured data templates) that pays off on every future page automatically. It's far easier to build it into your app's foundation now than to retrofit dozens of pages later.

Wrapping Up

Technical SEO isn't a mysterious marketing skill — it's a checklist of implementation details that a developer is uniquely positioned to get right: correct metadata exports, a real sitemap, valid structured data, and a page that actually loads fast. Get the foundation right once in your layout.tsx, sitemap, and robots files, and every page you ship afterward inherits it for free.

From here, a natural next step is auditing your existing pages against the checklist above one by one, starting with your highest-traffic pages first — small technical fixes there tend to move the needle fastest.

Useful Resources

How to Add Metadata in Next.js 16 (2026 Beginner's Guide)

Fri, 07 Aug 2026 04:36:07 +0000

You've built your Next.js app, it looks great, it works great — and then you share the link on Twitter or Slack and it shows up as a bare, ugly URL with no title, no image, nothing. Or worse, Google indexes your page with a title that just says "Home."

That's what happens when metadata is an afterthought. The good news: Next.js 16's App Router has a built-in, type-safe way to handle all of this — no next/head, no manual <meta> tag juggling. In this guide, you'll learn exactly how metadata works in the App Router, how to set it per-page, how to make it dynamic for things like blog posts, and the mistakes that trip up almost everyone the first time.

Why Metadata Matters

Metadata is the stuff that doesn't show up on your page, but controls how it's represented everywhere else:

The tab title and description in search results
The preview card when someone shares your link on X, LinkedIn, or Discord
Whether search engines are even allowed to index the page

Skip it, and your app might work perfectly for people already on it, while being invisible or unappealing to everyone who hasn't found it yet.

The Old Way vs the App Router Way

If you've used the Pages Router before, you probably reached for next/head and manually wrote <meta> tags inside a <Head> component. The App Router replaces this entirely with a Metadata API — you export a plain object (or a function) from your page or layout file, and Next.js generates all the right tags for you at build or request time, including merging metadata from parent layouts.

💡 Tip: You never import anything for basic metadata — no next/head, no <Head> component. Just export metadata (or generateMetadata) from the file.

Step 1: Static Metadata on a Page

For a page with fixed content — an About page, a Contact page — export a metadata object directly:

// app/about/page.tsx
import type { Metadata } from "next";

export const metadata: Metadata = {
  title: "About Us",
  description: "Learn more about our team and what we build.",
};

export default function AboutPage() {
  return <h1>About Us</h1>;
}

What's happening here: Next.js scans your app directory at build time, finds this exported metadata object, and injects the corresponding <title> and <meta name="description"> tags into the <head> automatically. No manual DOM manipulation, no client-side JavaScript needed.

Step 2: Metadata in the Root Layout (Site-Wide Defaults)

Your root layout.tsx is the place to set defaults that apply across the whole site — things like your site name and a fallback description.

// app/layout.tsx
import type { Metadata } from "next";

export const metadata: Metadata = {
  metadataBase: new URL("https://example.com"),
  title: {
    default: "DevStacked",
    template: "%s | DevStacked",
  },
  description: "Practical Next.js, React, and Supabase tutorials.",
};

export default function RootLayout({
  children,
}: {
  children: React.ReactNode;
}) {
  return (
    <html lang="en">
      <body>{children}</body>
    </html>
  );
}

Understanding title.template: this is one of the most useful (and most missed) features. Once you set a template in the root layout, any page that sets its own title gets automatically wrapped in that pattern.

So if a blog post page sets:

export const metadata: Metadata = {
  title: "Stripe Checkout Guide",
};

The actual rendered <title> becomes "Stripe Checkout Guide | DevStacked" — you never have to type "| DevStacked" on every single page.

⚠️ Common Mistake: Setting a full title like "Stripe Checkout Guide | DevStacked" on every page manually. If you've set a template in the root layout, this doubles up or looks inconsistent. Let the template do the work — just set the page-specific part.

What Does `metadataBase` Do?

Notice the metadataBase field above. Any relative URL you use elsewhere in your metadata — an openGraph.images path, a canonical URL — gets resolved against this base automatically. Without it, Next.js falls back to http://localhost:3000 in development and can log a warning in production, or worse, ship a broken relative URL to a social platform that needs an absolute one.

export const metadata: Metadata = {
  metadataBase: new URL("https://devstacked.tech"),
  openGraph: {
    images: ["/og/cover.png"], // resolves to https://devstacked.tech/og/cover.png
  },
};

Set it once, in the root layout, and every nested page and layout can use short relative paths instead of repeating the full domain everywhere.

Metadata Inheritance

Metadata in the App Router isn't isolated per file — it's merged down the layout tree. A field set in app/layout.tsx applies to every page unless a more specific layout.tsx or page.tsx overrides it. So app/blog/layout.tsx could add a shared openGraph.siteName, and each app/blog/[slug]/page.tsx only needs to set the fields that actually change per post, like title and description.

The merge is shallow per key, not a deep object merge — if a page sets its own openGraph object, that whole object replaces the parent's openGraph, it doesn't merge field-by-field inside it. Keep that in mind if a nested page's Open Graph tags seem to be missing fields you set higher up.

Example:

// layout.tsx

openGraph: {
  siteName: "DevStacked",
  images: [...]
}

// page.tsx
openGraph: {
  title: "Stripe Guide"
}

Result? People often expect:

siteName
+
images
+
title

But actually

openGraph
↓
entire object replaced

Step 3: Dynamic Metadata with `generateMetadata`

Static objects work fine when you already know the content. But what about a blog post page at /blog/[slug] — the title and description depend on which post is being viewed, and that's only known at request time. For this, Next.js gives you generateMetadata, an async function that runs before the page renders.

// app/blog/[slug]/page.tsx
import type { Metadata } from "next";
import { getPostBySlug } from "@/lib/posts";
import { notFound } from "next/navigation";

interface Props {
  params: Promise<{ slug: string }>;
}

export async function generateMetadata({
  params,
}: Props): Promise<Metadata> {
  const { slug } = await params;
  const post = getPostBySlug(slug);

  if (!post) {
    notFound();
  }

  return {
    title: post.title,
    description: post.description,
  };
}

export default async function BlogPostPage({ params }: Props) {
  const { slug } = await params;
  const post = getPostBySlug(slug);

  return <article>{/* ... */}</article>;
}

Why this matters: generateMetadata runs on the server, separately from your page component, and Next.js waits for it to resolve before streaming the <head> to the browser — so search engines and social crawlers always see the correct, post-specific title and description, not a generic fallback.

⚠️ Common Mistake: Forgetting params is a Promise in Next.js 16. Just like inside the page component itself, you must await params before reading slug — reading it directly (params.slug) will throw a type error or return undefined.

Step 4: Open Graph Tags (Rich Link Previews)

Open Graph tags control how your link looks when shared on LinkedIn, Slack, Discord, or Facebook — the title, description, and preview image shown in the card.

export const metadata: Metadata = {
  title: "Stripe Checkout Guide",
  description: "Learn how to integrate Stripe Checkout with Next.js 16.",
  openGraph: {
    title: "Stripe Checkout Guide",
    description: "Learn how to integrate Stripe Checkout with Next.js 16.",
    url: "https://example.com/blog/stripe-checkout-guide",
    siteName: "DevStacked",
    images: [
      {
        url: "https://example.com/og/stripe-checkout.png",
        width: 1200,
        height: 630,
        alt: "Stripe Checkout Guide",
      },
    ],
    type: "article",
  },
};

Why a separate openGraph object instead of reusing title/description? Because you often want slightly different copy for a social preview card versus a search result snippet — a social card benefits from being punchier, while a search description often needs to match user search intent more literally. Next.js lets you diverge, but falls back to the top-level title/description if you don't specify openGraph values.

💡 Tip: The recommended Open Graph image size is 1200×630px. Anything smaller gets stretched and looks blurry on most platforms.

Step 5: Twitter Card Tags

X (formerly Twitter) can technically fall back to your Open Graph tags, but adding explicit Twitter Card metadata gives you more reliable control:

export const metadata: Metadata = {
  twitter: {
    card: "summary_large_image",
    title: "Stripe Checkout Guide",
    description: "Learn how to integrate Stripe Checkout with Next.js 16.",
    images: ["https://example.com/og/stripe-checkout.png"],
  },
};

card: "summary_large_image" is what you want almost every time — it renders a large, prominent image above the title and description, instead of a small square thumbnail next to the text.

Step 6: Generating Open Graph Images Dynamically

Instead of designing a static OG image for every single blog post, Next.js lets you generate one on the fly using next/og. Drop a special file named opengraph-image.tsx right next to your page.tsx:

// app/blog/[slug]/opengraph-image.tsx
import { ImageResponse } from "next/og";

export const runtime = "edge";
export const alt = "Blog Post Preview";
export const size = { width: 1200, height: 630 };
export const contentType = "image/png";

export default async function Image({
  params,
}: {
  params: Promise<{ slug: string }>;
}) {
  const { slug } = await params;
  const title = slug.replace(/-/g, " ");

  return new ImageResponse(
    (
      <div
        style={{
          height: "100%",
          width: "100%",
          display: "flex",
          alignItems: "center",
          justifyContent: "center",
          backgroundColor: "white",
          fontSize: 64,
          fontWeight: 700,
          textTransform: "capitalize",
        }}
      >
        {title}
      </div>
    ),
    { ...size }
  );
}

What's happening here: Next.js automatically detects this file, renders the JSX into a PNG using the Edge Runtime, and serves it at a route matching the page — then automatically wires it up as the og:image for that page. No manual linking required.

💡 Tip: runtime = "edge" matters here — image generation is CPU-light but benefits from edge deployment for fast, globally distributed rendering.

Step 7: Canonical URLs and Robots

Two smaller but important fields:

export const metadata: Metadata = {
  alternates: {
    canonical: "https://example.com/blog/stripe-checkout-guide",
  },
  robots: {
    index: true,
    follow: true,
  },
};

Canonical URLs tell search engines "this is the one true version of this page" — useful if the same content is reachable through multiple URLs (with/without trailing slash, query params, etc.), so you don't get penalized for duplicate content.

💡 Tip: Next.js already auto-generates a canonical URL for a page based on its route and your metadataBase, so in many straightforward cases you don't have to set alternates.canonical at all. Setting a custom value matters when you have genuine duplicate content — the same product reachable at two different URLs, paginated pages that should all canonicalize back to page 1, or a query-string variant of a page that shouldn't be indexed separately.

Robots controls whether search engines should index the page at all. Set index: false for things like admin pages, thank-you pages, or draft content you don't want showing up in search results.

Step 8: File-Based Metadata

Not all metadata comes from a metadata object or generateMetadata function. Next.js also supports file-based metadata — special file names inside your app folder that Next.js automatically detects and turns into the correct tags, no export required:

File	What it generates
`favicon.ico`	The browser tab icon
`icon.tsx` / `icon.png`	App icons (including dynamically generated ones via `next/og`)
`apple-icon.png`	Icon used on iOS home screens
`opengraph-image.tsx` / `.png`	The `og:image` tag, as shown in Step 6
`twitter-image.tsx` / `.png`	The Twitter Card image, if different from Open Graph
`sitemap.ts`	Your `sitemap.xml`
`robots.ts`	Your `robots.txt`

Just like opengraph-image.tsx, you can drop any of these next to a page.tsx to scope them to that route, or in the root app folder for site-wide defaults. Next.js wires up the correct <link> or <meta> tag automatically — you never manually reference these files anywhere else in your code.

Frequently Asked Questions

Do I need `generateMetadata` for every page?

No — only for pages where the metadata depends on data you don't know until request time (a slug, a database lookup, etc.). Static pages like About or Contact should just export a plain metadata object; it's simpler and doesn't need to run on every request.

What happens if I set metadata in both the layout and the page?

Next.js merges them, with the page's metadata taking priority for overlapping fields. Non-overlapping fields (like a description only set in the layout) still apply. This is what lets title.template work — the layout sets the pattern, the page sets the specific title.

Can I use `generateMetadata` and `generateStaticParams` together?

Yes, and you should for statically generated dynamic routes. generateStaticParams tells Next.js which slugs exist at build time; generateMetadata runs for each of those and produces the right metadata per page — both commonly pull from the same data source.

Most likely the image URL isn't absolute, or the platform cached an old preview. Always use a full https:// URL in openGraph.images, and if you're testing, use each platform's own debugger tool (like Facebook's Sharing Debugger) to force a re-scrape.

Does metadata affect Core Web Vitals or performance?

No — metadata is just <head> tags, not render-blocking scripts or styles. generateMetadata running on the server also doesn't block your page's own data fetching, since Next.js can run them in parallel.

Why isn't Google showing my description?

Google often rewrites meta descriptions if it thinks another page excerpt better matches the user's search query.

What's the ideal title length?

Aim for roughly 50–60 characters (or about 580–600 pixels) so titles are less likely to be truncated in search results.

Wrapping Up

Next.js 16's Metadata API takes what used to be manual, error-prone <head> management and turns it into a small, typed export at the top of your file. Static pages get a plain object, dynamic pages get generateMetadata, and layouts let you set sensible defaults once instead of repeating them everywhere.

From here, a natural next step is wiring up JSON-LD structured data for rich search results, or building out per-post dynamic OG images across your whole blog if you haven't already.

💡 Save time: Use our Meta Tag Generator to instantly generate a complete Metadata object, Open Graph tags, Twitter Cards, and JSON-LD for any page — fill in a page's title, description, URL, and image once, and it outputs both raw HTML <meta> tags and a ready-to-paste Next.js Metadata object.

Useful Resources

Continue Learning

Open Graph vs Twitter Cards: What's the Difference? (Nextjs 16)

Thu, 06 Aug 2026 04:15:34 +0000

You share a link on Slack, and it shows up as a beautiful card with a title, description, and image. You share a link on X, and it shows up... as bare text. Same website, two totally different results. What's going on?

The short answer: those preview cards aren't magic, and they're not one single system. They're powered by two separate sets of meta tags — Open Graph and Twitter Cards — and if you only add one of them, some platforms simply won't know what to show.

By the end of this guide, you'll know exactly what each one does, how they're different, when you actually need both, and how to add them correctly in a Next.js 16 App Router project.

Why This Confuses Almost Everyone

Here's the thing that trips people up: Open Graph and Twitter Cards look almost identical. Both control a title, a description, and an image. Both live in your page's <head>. Both were designed for the exact same job — making a shared link look good.

So why do we need two of them?

Because they were built by two different companies, at two different times, for two different platforms — and neither one fully committed to just using the other's format.

Open Graph vs Twitter Cards: The Real Differences

	Open Graph	Twitter Cards
Created by	Facebook	Twitter (now X)
Attribute used	`property`	`name`
Prefix	`og:`	`twitter:`
Used by	Facebook, LinkedIn, Slack, Discord, WhatsApp, and most others	X (Twitter)
Controls layout?	Not really — mostly fixed	Yes, via `twitter:card`
Required if you already have OG tags?	—	Technically no, X can fall back to OG — but see below

Does X Really Fall Back to Open Graph?

Yes, X can fall back to Open Graph tags for several fields, but relying on the fallback means you can't explicitly control twitter:card, so providing Twitter Card tags is still recommended.

⚠️ Common Mistake: Assuming "X reads Open Graph tags anyway, so I don't need Twitter tags." Technically true for basic fields, but you'll usually get a worse-looking card. If X previews matter for your traffic, add both.

What Is Open Graph?

Open Graph (often shortened to "OG") is a protocol created by Facebook in 2010. It defines a set of <meta> tags that tell any platform reading your page: "here's the title, description, image, and type of content this page represents."

<meta property="og:title" content="Stripe Checkout Guide" />
<meta property="og:description" content="Learn how to integrate Stripe Checkout with Next.js 16." />
<meta property="og:image" content="https://example.com/og-image.png" />
<meta property="og:url" content="https://example.com/blog/stripe-checkout-guide" />
<meta property="og:type" content="article" />

💡 Tip: Notice the tags use property, not name, for the attribute. That's an Open Graph convention — it's part of the original protocol spec, not a typo you'll see in other meta tags.

Open Graph became the industry default. Facebook, LinkedIn, Slack, Discord, WhatsApp, and dozens of other platforms all read Open Graph tags to build their link preview cards. If you only add one type of meta tag to your site, this is the one to add — it covers the widest range of platforms.

What Are Twitter Cards?

Twitter Cards (still called that even though the platform is now X) are a separate set of meta tags created by Twitter, also released around 2010–2012, specifically for how links preview inside tweets.

<meta name="twitter:card" content="summary_large_image" />
<meta name="twitter:title" content="Stripe Checkout Guide" />
<meta name="twitter:description" content="Learn how to integrate Stripe Checkout with Next.js 16." />
<meta name="twitter:image" content="https://example.com/twitter-image.png" />

💡 Tip: Unlike Open Graph, Twitter Card tags use the standard name attribute — this is one of the small syntax differences that catches people copy-pasting between the two.

The one genuinely unique piece here is twitter:card, which controls the layout of the preview:

Value	What it does
`summary`	Small square image, text beside it
`summary_large_image`	Large image above the title and description (the one you want almost every time)
`player`	Embeds video/audio directly in the tweet
`app`	Links to a mobile app store listing

Adding Both in Next.js 16

Next.js's App Router Metadata API handles both tag types through one typed object — no manual <meta> tags needed.

// app/blog/[slug]/page.tsx
import type { Metadata } from "next";

export const metadata: Metadata = {
  title: "Stripe Checkout Guide",
  description: "Learn how to integrate Stripe Checkout with Next.js 16.",

  openGraph: {
    title: "Stripe Checkout Guide",
    description: "Learn how to integrate Stripe Checkout with Next.js 16.",
    url: "https://example.com/blog/stripe-checkout-guide",
    siteName: "DevStacked",
    type: "article",
    images: [
      {
        url: "https://example.com/og-image.png",
        width: 1200,
        height: 630,
        alt: "Stripe Checkout Guide",
      },
    ],
  },

  twitter: {
    card: "summary_large_image",
    title: "Stripe Checkout Guide",
    description: "Learn how to integrate Stripe Checkout with Next.js 16.",
    images: ["https://example.com/og-image.png"],
  },
};

What's happening here: the openGraph object generates every og:* tag, and the twitter object generates every twitter:* tag — Next.js writes the correct property or name attributes for you under the hood. You're writing one object; Next.js is producing two separate tag sets.

💡 Tip: You can reuse the same image for both. There's no rule requiring separate images — most sites use one 1200×630px image for both og:image and twitter:image.

Can I Skip the Twitter Object Entirely?

If you set openGraph but skip twitter, Next.js does not automatically duplicate your OG tags into Twitter tags — it just omits twitter:* tags from the page entirely, relying on X's own (imperfect) fallback behavior described above.

// This works, but X's preview may not use the large-image layout
export const metadata: Metadata = {
  openGraph: {
    title: "My Post",
    images: ["/og-image.png"],
  },
  // no twitter object — X falls back to og: tags, but loses twitter:card
};

If you want a guaranteed summary_large_image layout on X, always set the twitter object explicitly.

Image Size: One Size Fits (Almost) Both

Both formats recommend the same dimensions, which is why one image usually works everywhere:

Recommended size: 1200×630px
Aspect ratio: roughly 1.91:1
Format: PNG or JPEG, keep it under ~5MB

⚠️ Common Mistake: Using a square logo as your og/twitter image. It gets stretched or cropped awkwardly on most platforms — build a proper wide banner image instead, even a simple one with your title text on a solid background.

Frequently Asked Questions

If I only have time to add one, which should I pick?

Open Graph. It covers Facebook, LinkedIn, Slack, Discord, WhatsApp, and (partially) X. Twitter Cards only improve the experience on one platform — X — but that improvement (the large image layout) is significant enough to add both if you can.

Does Google use Open Graph or Twitter Cards for search results?

Neither, directly. Google Search mostly relies on your page's <title>, meta description, and structured data (JSON-LD) — not OG or Twitter tags. OG/Twitter tags are purely for social link-preview cards, not search rankings or snippets.

Why isn't my preview updating after I changed the image?

Almost every platform caches link previews aggressively. Facebook has a Sharing Debugger to force a re-scrape, and X's card validator works similarly — always test with these tools instead of just re-sharing the link and hoping.

Do I need `og:type` set to something other than "website"?

Only if you want to be precise. article is common for blog posts, product for e-commerce pages, video.other for video content. Most platforms treat website as a safe default and won't visibly change your card's appearance.

Can I test my tags without actually posting them?

Yes — use Facebook's Sharing Debugger for Open Graph and Twitter's Card Validator for Twitter Cards. Both let you paste a URL and preview exactly how the card will render before you share it anywhere.

Wrapping Up

Open Graph and Twitter Cards solve the same problem — making your links look good when shared — but they're two separate protocols from two separate companies, with slightly different syntax and slightly different platform support. The safest move is simple: always set both openGraph and twitter in your Next.js metadata, reuse one well-sized image for both, and test with each platform's debugger before you trust it in production.

From here, a natural next step is wiring up JSON-LD structured data for richer search results, or generating dynamic OG images per page with next/og if you haven't already.

If you're new to metadata, first read How to Add Metadata in Next.js 16 (2026 Beginner's Guide).

💡 Save time: use the Meta Tag Generator to generate both Open Graph and Twitter Card tags — plus canonical URLs and JSON-LD — from one set of inputs, in both raw HTML and Next.js Metadata API format.

Useful Resources

Continue Learning

AI Answers Name a Few Brands. Is Yours One?

Fri, 17 Jul 2026 19:22:02 +0000

More people start their research by asking an AI than by typing into a search box. And the AI does not hand back ten blue links. It answers in a paragraph, names two or three tools, and moves on.

If your brand is one of the names, you win the buyer before they ever reach a search result. If it is not, you are invisible, and the worst part is you cannot see it happening. Nothing shows up in your analytics. There is no impression count for "times an AI recommended someone else."

That gap is what AI search visibility measures. Not "do I rank on Google," but "when a buyer asks an AI in my category, does it say my name."

Why one prompt tells you nothing
The instinct is to open ChatGPT, ask "what are the best tools for X," and see if you show up. Do not trust that answer. It is one roll of the dice.

Two things make a single check meaningless.

The engines barely agree. One 2026 study found only about 11 percent of the domains ChatGPT cites overlap with the ones Perplexity cites. Showing up in one says almost nothing about the others.
The engines behave differently. An analysis of 2.4 million answers found Claude names brands in about 97 percent of its responses, while Perplexity does so in under half. Same question, completely different odds of any brand getting named at all.
So real visibility is not a lucky reply. It is a measurement across many prompts and several engines, counted.

The three questions a real check answers
Do I appear at all? For the prompts a buyer would actually type, does your brand surface anywhere.
How do I compare? Out of all the brands named in your category, what share of the mentions are yours. This is your AI share of voice.
Where am I missing? The specific prompts and engines where a rival gets named and you do not. That list is your content and PR to-do.
Answer those three and you have gone from a vague worry to a scoreboard.

The free way to do it
You can run this by hand today, no tools.

Write 5 to 10 prompts a real buyer would ask. Not "is [your brand] good," that one is cheating. Ask the neutral category questions, like "what is the best affordable SEO tool for a solo founder." Run each one across ChatGPT, Claude, and Perplexity in fresh sessions. Tally who gets named.

That gives you an honest baseline. The catch is it is slow, and it does not repeat cleanly. Do it again next month and the wording drift and session randomness make the two runs hard to compare. Which is a problem, because the whole point is the trend.

Doing it inside the agent
This is the job I moved into my AI agent so it repeats without the manual grind.

I use ContextBolt SEO. Two tools cover it. ai_visibility checks whether a brand surfaces for a set of prompts. ai_share_of_voice measures your mention percentage against named competitors. I ask in plain language, the agent runs the checks, and because it lives in the same chat, I go straight from the numbers into "so which prompts am I losing, and what would I write to fix it." It is $35 a month flat, and it keeps the history, so month two is actually comparable to month one.

The honest framing I hold it to. AI visibility is a sample, not a census. It is estimates from a batch of prompts, not a reading of every answer every engine gives. The same tool can wobble week to week because the engines are not deterministic. So I watch the trend over months, never a single snapshot. Trend beats the snapshot every single time.

The takeaway
Search used to end with a ranked list you could go read. More and more it ends with a sentence that names a few brands and stops. Whether you are in that sentence is now a real metric, and right now only around 14 percent of marketers track it, even though a lot more say AI search is a 2026 priority.

The ones checking will know which prompts they are losing and go fix them. Everyone else will keep guessing why the pipeline is quiet. Pick a handful of buyer prompts, run them across a few engines, and count. Whether you use a tool or a spreadsheet, start counting before your competitors do.

If you want the longer version with the full workflow and the honest limits, I wrote up AI search visibility here.

Tools Are Easy. Outcomes Are Hard. How I Build an SEO Stack That Actually Moves Numbers

Fri, 03 Jul 2026 01:26:15 +0000

Buying tools is the easy part. Wiring them into a system that produces rankings, traffic, and booked revenue is the hard part, and it's an engineering problem more than a marketing one. This is how I think about the stack, layer by layer: research, indexing data, technical audits, crawls, Core Web Vitals, content, lead ops, distribution, and closing the loop on what actually converts. Real numbers and a couple of code snippets included.

I build software in the SEO and lead-gen space, so I spend a lot of time looking at how people try to grow a website. And the pattern is almost always the same. Somebody buys Ahrefs, installs a WordPress plugin, opens fourteen browser tabs, exports three spreadsheets, and then wonders why the needle isn't moving.

The tools are fine. The tools are great, honestly. That's the problem. The tools are so good and so easy to acquire that people mistake owning them for having a system. You end up with copy-paste, manual posting, missed follow-ups, and a graveyard of broken workflows that nobody wired together. Tabs are not a strategy.

If you write code for a living, you already know this feeling. Picking a library is easy. Shipping an outcome with it is hard. SEO is the same, and looking at it like an engineer instead of a marketer is the unlock.

The trap: a shelf of tools is not a pipeline

Some numbers to set the scene, because they're genuinely wild.

Scott Brinker's martech landscape counted 15,384 tools in 2025, up about 9% year over year and roughly 100x the 150 tools that existed in 2011. So the category isn't just big, it's compounding.

And we barely use what we buy. Gartner's marketing survey found teams using only about 33% of their martech stack's capabilities in 2023, recovering to roughly 49% by 2025. Sit with it either way: half of the software you pay for never gets touched.

None of that is a tooling shortage. Nobody's problem is "I need a fifteen-thousand-and-first tool." The problem is that the tools don't talk to each other, the workflow between them is manual, and there's no pipeline connecting "I published a thing" to "a customer paid me."

So instead of shopping, build the pipeline. Here's the stack I actually reach for, bottom to top, and what each layer is really doing.

Layer 1: Keyword and backlink research (decide what to build for)

Skipping research is like writing a feature before reading the ticket. The job isn't "find high-volume keywords." It's the intersection of what people search, what you can credibly rank for, and what actually makes you money. A 40,000-searches-a-month term you have no authority to win, and that converts nothing, is worse than a boring 90-a-month term a ready-to-buy customer types in.

Backlink research is the reconnaissance half: who links to the sites already winning your queries, and whether there's a realistic path to earning similar links. You're mapping the competitive graph before you commit engineering effort.

Output of this layer: a short, ranked list of pages you're going to build or fix, each tied to a real query and a real conversion. That list is your backlog.

Layer 2: Search performance and indexing data (your production telemetry)

If you ship code without logs and metrics, you're flying blind. Same with a website. Google Search Console is the closest thing you get to production telemetry straight from the search engine, and it's free.

Two reports do most of the work:

Search Performance: Clicks, Impressions, CTR, and average Position, sliceable by query, page, country, and device. Your "what am I actually ranking for, and where" dashboard.
Page Indexing (the old Coverage report): which URLs are indexed, and for the ones that aren't, why. Reasons like "Crawled - currently not indexed" and "Excluded by noindex tag" are the error log of your site's relationship with Google.

Two limits will bite you if you build on top of it. Performance data is a rolling 16-month window, so export it yourself if you care about year-over-year. And the UI caps exports at 1,000 rows, which is nothing for a real site. Get past the cap with the Search Analytics API (up to 50,000 rows/day per property per search type) or the Bulk Data Export to BigQuery, which isn't row-limited. If you can write a for loop, you can pull your own data at real scale instead of squinting at the dashboard.

Don't eyeball GSC once a month. Pipe it somewhere you can query it.

Layer 3: Technical SEO audits (lint your site)

A technical audit is a linter for your site's crawlability. It isn't one official Google checklist, it's an industry-consensus set of categories, but the categories are stable:

Indexability and directives: robots.txt, meta robots, canonical tags, HTTP status codes
XML sitemaps: present, valid, submitted, and actually fresh
Structured data: valid JSON-LD, eligible for rich results
Mobile and responsive: you're being indexed mobile-first whether you like it or not
HTTPS and security: TLS, no mixed content
Site speed and Core Web Vitals: covered below, it's big enough for its own layer
Internal linking and architecture: crawl depth, orphan pages, sane anchor text
Rendering: does the rendered page match what you think you shipped

One honest caveat so you don't over-index on the wrong thing: HTTPS is a (minor) ranking signal, but specific security headers like CSP and HSTS are security best practice, not ranking factors. Do them because they're correct, not because you think they'll bump you up a spot.

Layer 4: Deep site crawls (understand crawl to render to index)

Here's the part most developers get wrong, and it matters more every year because we ship more JavaScript.

Google's pipeline is crawl, then render, then index. Every page that returns a 200 gets queued for rendering. Google runs a headless Chromium, executes your JavaScript, and indexes the rendered HTML, not just the raw source that came off the server.

That rendering step sits in a queue. Google's own docs say a page "may stay on this queue for a few seconds, but it can take longer." This is the reality behind the old "two-wave indexing" phrase people used to throw around. Google doesn't frame it as two fixed waves anymore, but the deferred-rendering queue is real and documented.

Two practical consequences:

Raw HTML and rendered DOM can differ, and the difference is what gets indexed. If your content only exists after a client-side fetch, Google has to render to see it, and rendering is deferred. Server-render or statically generate the content that matters.
Do not put noindex in the raw HTML of a page you want indexed and plan to "fix with JavaScript later." If Google sees noindex in the initial response, it can skip rendering entirely and never run your JS. You can't un-noindex a page client-side. I've watched this silently nuke real pages.

A deep crawl (Screaming Frog, Sitebulb, or your own script) is how you catch orphan pages, redirect chains, canonical conflicts, and raw-versus-rendered gaps before Google does.

Layer 5: Core Web Vitals (performance is a feature)

Core Web Vitals is where "SEO" and "you are a software engineer" fully overlap. There are three, all measured at the 75th percentile of real user page loads:

Metric	Measures	"Good" threshold
LCP (Largest Contentful Paint)	Loading	≤ 2.5 s
INP (Interaction to Next Paint)	Responsiveness	≤ 200 ms
CLS (Cumulative Layout Shift)	Visual stability	≤ 0.1

Note the middle one. INP replaced FID (First Input Delay) as a Core Web Vital on March 12, 2024. FID only measured the delay of the first interaction. INP measures the latency of all interactions on the page, which is a much harder and more honest bar. If your mental model still says "FID," update it.

Why bother past the ranking nudge? Because speed is a conversion feature independent of Google. Google and Deloitte's "Milliseconds Make Millions" study (2020, 30M+ sessions) found that a 0.1-second improvement in mobile load time lifted retail conversions by 8.4% and average order value by 9.2%. You'd chase an 8% conversion lift in a sprint. This is that, hiding in your bundle size.

You can measure it without guessing:

# Field data from the Chrome UX Report via PageSpeed Insights API
curl "https://www.googleapis.com/pagespeedonline/v5/runPagespeed?url=https://example.com&category=performance&strategy=mobile"

Lab tools (Lighthouse) tell you what could be slow. Field data (CrUX, the loadingExperience block in that response) tells you what real users actually feel. Optimize against the field data.

Layer 6: Content optimization (make the page legible to machines)

Once the page is fast and crawlable, make it legible, to both readers and parsers. On the machine side, that means structured data.

JSON-LD is Google's recommended format for structured data: a script block that describes the entities on your page using schema.org vocabulary. It's decoupled from your markup, so it's easy to maintain and hard to accidentally break with a CSS change.

<script type="application/ld+json">
{
  "@context": "https://schema.org",
  "@type": "LocalBusiness",
  "name": "Example Co",
  "address": {
    "@type": "PostalAddress",
    "streetAddress": "123 Main St",
    "addressLocality": "Denver",
    "addressRegion": "CO",
    "postalCode": "80202"
  },
  "telephone": "+1-555-0100",
  "openingHoursSpecification": [{
    "@type": "OpeningHoursSpecification",
    "dayOfWeek": ["Monday","Tuesday","Wednesday","Thursday","Friday"],
    "opens": "09:00",
    "closes": "17:00"
  }]
}
</script>

Match the schema to the content. Use a specific LocalBusiness subtype (Restaurant, HealthClub, and so on) for a physical business, Organization when there's no storefront, and Article / BlogPosting for content like this post. Valid structured data makes you eligible for rich results (review stars, FAQs, breadcrumbs, local packs). It's widely believed to help AI answer engines cite you too, and that's plausible, but I'll be straight with you: Google doesn't document it as a guaranteed mechanism, and structured data never guarantees a rich result even when it's valid. Do it because it's cheap and correct, not because it's magic.

And it isn't only Googlebot reading this anymore. AI answer engines and LLM-backed search pull from the same clean, server-rendered content and structured markup you'd build for a rich result. The work didn't change; the audience for it did. A page that's easy for a crawler to parse is easy for a model to quote. That's a reason to hold the line on the fundamentals, not a new workstream to bolt on.

Layer 7: Lead operations (a contact form is a production endpoint)

This is the layer people bolt on last and it's the one that actually touches revenue. If you drive traffic and capture leads but nobody responds, you've built a system that carefully collects dropped requests.

The numbers on this are brutal and old enough to be settled. From the Harvard Business Review study "The Short Life of Online Sales Leads" (2011, audit of 2,241 US companies):

Firms that made contact within an hour were nearly 7x more likely to qualify a lead than those who waited just one more hour, and more than 60x more likely than those who waited a full day.
The average first-response time across those companies was 42 hours.
23% never responded at all.

Earlier MIT / InsideSales research (2007, vendor-commissioned, so grain of salt) put even sharper numbers on the first few minutes, with contact odds dropping up to 100x between a 5-minute and a 30-minute response. The exact figure is arguable; the direction isn't.

Think of an inbound lead as an HTTP request with an aggressive client-side timeout. Respond in milliseconds (well, minutes) or the connection drops. As an engineer this is the most tractable layer in the whole stack: it's automation, routing, and instant acknowledgement. You already know how to make a system respond fast. Point that skill here.

Layer 8: Content distribution (shipping isn't publishing)

Writing the post is git commit. Hitting publish just merges it to main. It isn't deploy, and it definitely isn't "users are running it in production."

The maxim I keep coming back to, popularized by Derek Halpern, is to spend 20% of your time creating content and 80% distributing it. Rule of thumb from one practitioner, not a study, and plenty of people argue it's closer to 50/50. The point survives the quibbling: most people pour everything into creation and nothing into distribution, then blame the content.

Distribution is the deploy step, so systematize it: syndicate the piece, link it from relevant answers, put it in front of people who'd actually use it. (One gotcha for anyone cross-posting to a community like this: set rel="canonical" on the syndicated copy back to the original on your own domain, so you get the ranking credit instead of splitting it between two URLs.)

Layer 9: Measure what converts (close the loop)

Every layer above is theater if you can't answer one question: which of this produced revenue?

Rankings are a proxy. Traffic is a proxy. Even leads are a proxy. This is the feedback loop, and a pipeline without one isn't a pipeline, it's a conveyor belt pointed at a wall. Trace it end to end, from query, to landing page, to lead, to booked revenue, so you know which keyword and which page paid for itself. Connect that loop and the whole stack stops being a pile of activities and becomes a system you can profile and tune like any other.

It's also the layer that kills tool sprawl on its own. Once you can see what converts, the 33% of your stack that touches nothing measurable becomes obvious, and easy to cut.

The point

Tools are easy. You can assemble every tool named in this article in an afternoon and a credit card. Outcomes are hard because outcomes come from the system, the wiring between the layers, the automation, and the feedback loop from revenue back to your backlog.

So stop shopping for the missing tool. You don't have a tool problem. Build the pipeline instead. Drawn out, the whole thing is one loop:

   Research (Ahrefs)
        │
        ▼
   Backlog  ◄──────────────────────────┐
   (ranked pages, each tied to a query) │
        │                               │
        ▼                               │
   Build & write the page               │
        │                               │
        ▼                               │
   Technical audit + deep crawl         │
   (DeepAudit AI, Screaming Frog)       │
        │                               │
        ▼                               │
   Core Web Vitals pass                 │
   (PageSpeed Insights / CrUX)          │
        │                               │
        ▼                               │
   Publish                              │
        │                               │
        ▼                               │
   Distribute (Made4Founders)           │
        │                               │
        ▼                               │
   Measure what converts (GA4)          │
        │                               │
        ▼                               │
   Lead ops, fast response (Site2CRM)   │
        │                               │
        ▼                               │
   Revenue ─────────────────────────────┘
   (which query and page paid off feeds the next backlog)

Every arrow is a place the work flows or leaks. The tools just sit on the boxes; the value is in the arrows, and it only compounds once that bottom one, revenue back to backlog, is connected.

That's the stack. The tools are just the parts. The system is the job.

Mitigating Dynamic Search Spam: Resolving GA4 Data Corruption and Server-Side Crawl Loops

Sun, 28 Jun 2026 11:48:12 +0000

The formatting issue shown in your screenshot occurs because the Markdown editor
interpreted leading spaces or tabs (indentations) as a command to generate a
dark code block.

To fix this, here is the complete, clean text of your article. Every line is
formatted to start exactly at the left margin (with zero leading spaces or tabs)
so it will render correctly on the website:

Mitigating Dynamic Search Spam: Resolving GA4 Data Corruption and Server-Side Crawl Loops

Dynamic search parameters (like /search?q=) are highly susceptible to
programmatic scraping and crawl-loop attacks. When automated scraper
networks—often routed through high-bandwidth data centers—flood these entry
points, the server-side impact is severe:

Analytics Distortion: Spikes of thousands of concurrent hits drive sitewide
bounce rates to 98%+, dropping average engagement times to near-zero.
Layout Engine Failure: High-velocity queries can cause legacy dynamic
variables to break, recording HTML syntax errors in your analytics page
reports.
Crawl Budget Exhaustion: Search engine spiders get caught in infinite search
parameter loops, ignoring high-value canonical routes.

Here is a multi-layered technical roadmap to isolate, challenge, and block
dynamic search spam at the theme, DNS, and crawl-routing levels.

Resolving Theme-Level Layout Errors (Blogger XML)

If your Google Analytics 4 (GA4) page title reports are corrupted with syntax
errors like "Search results for ", your theme layout engine is failing to parse
empty or rapid-fire bot queries.

Locate your title conditional block in your theme's HTML and swap out the
deprecated global variable for the modern layout engine counterpart:

Deprecated (Avoid):

<b:elseif cond='data:view.isSearch'/>
Search results for <data:blog.searchQuery/> | <data:blog.title/>

Modernized (Correct):

<b:elseif cond='data:view.isSearch'/>
Search results for <data:view.search.query/> | <data:blog.title/>

This prevents engine rendering failures, ensuring that legitimate search terms
are resolved cleanly while bot-triggered dynamic queries do not corrupt your
analytical titles.

Filtering Traffic at the DNS Edge (Cloudflare WAF)

Malicious scrapers completely ignore robots.txt guidelines. To block automated
traffic before it can execute your client-side GA4 scripts, configure a custom
WAF (Web Application Firewall) Rule at the zone level:

Rule Name: Block Search Spam
Expression: (http.request.uri.path contains "/search")
Action: Managed Challenge

How this works:

Cloudflare fingerprinting inspects incoming headers and behavior. Legitimate
human users executing a search will pass a silent JS verification or a brief
interactive challenge. Automated bots running headful browsers or scrapers (such
as Puppeteer or standard cURL requests) fail the challenge and are blocked at
the proxy level.

Optimizing Crawl Routing (robots.txt Specificity)

To prevent legitimate search engines from crawling low-value dynamic search
query URLs while keeping your category hub pages fully crawlable, utilize
RFC 9309 length-based specificity matching:

User-agent: *
Allow: /search/label/
Disallow: /search

The Logic:

Modern search engines evaluate rules sequentially and prioritize the longest,
most specific match.

/search/label/ (14 characters) is longer than /search (7 characters).
A bot crawling your category pages matches both, but is allowed access
because /search/label/ is more specific.
A bot hitting /search?q=test matches only /search and is blocked. This
protects your crawl budget.

Case Study: Before vs. After Mitigation

Deploying this layered approach yields the following metrics:

Real-Time Active Users: High spikes (unsecured) vs. Normal, low-level human
traffic (secured).
Bounce Rate: Artificially inflated to 96% - 99% (unsecured) vs. Healthy user
bounce rates of 30% - 60% (secured).
Average Engagement Time: Drops to near 0 seconds (unsecured) vs. Normal
human duration (secured).
Dynamic Search Access: Fully open to flooding (unsecured) vs. Protected by
Cloudflare WAF Managed Challenge (secured).
Search Engine Crawling: Risk of index bloat (unsecured) vs. High-value
content prioritized (secured).

For a complete step-by-step breakdown and additional technical configurations
(including llms.txt integration for AI search indexing), read the full case
study:

🔗 https://www.seosiri.com/2026/06/internal-search-query-spam-ga4-seo-fix.html

How to Run a Technical SEO Audit With SeekLab's Free Audit Tool

Thu, 23 Apr 2026 12:44:36 +0000

After auditing websites across SaaS, FMCG, and Web3 clients, one pattern holds: most teams either skip technical SEO checks entirely or run audits they don't know how to read. SeekLab's free audit tool was built to close both gaps — it runs in under five minutes and produces a prioritised report any team member can act on.

Here's exactly how to use it, what you'll see, and what to do next.

What the SeekLab Audit Tool Does

The SeekLab Audit Tool scans your website for technical SEO issues and surfaces them in a single structured report. It crawls up to 100 pages per scan, assigns an overall SEO score out of 100, and groups every issue into three priority levels: Critical, Warning, and Notice.

In SeekLab's own test scan, the tool completed a 100-page crawl and returned a score of 77/100 — with one Critical issue, a cluster of Warnings, and several Notice-level items spread across template and page-level problems.

How to Run Your Audit

Go to seeklab.io/audit
Enter your domain in the "Enter your domain to audit" field
Click to start the scan
Wait approximately 5 minutes for the crawl to complete
When the report status shows Success, your results are ready

No setup, no account required to start.

How to Read the Report

The report organises every issue into three tiers, so you always know what to fix first.

Critical

These are issues that directly harm crawlability, indexing, or user experience. Fix these before anything else. In the SeekLab test scan, this tier flagged one duplicate meta description.

Warning

These reduce ranking potential but won't break the site. Common examples from the SeekLab scan:

Issue	Count
Missing Alt Text	10 instances
Title Too Long	2 instances
Missing Meta Descriptions	3 pages
Missing H2 Tags	14 pages
Meta Description Too Long	8 pages

Notice

Lower-priority patterns worth tracking over time. The SeekLab scan returned notices for duplicate H1/title tags (40 instances), nofollow external links (90), and a small number of redirect and anchor text issues.

One practical rule: When Warning or Notice issues appear repeatedly across many pages, they're almost always template-level problems. Fix the template once rather than editing each page individually.

What to Do After the Report Is Ready

Three actions are available directly from the report:

Copy Link — saves the report URL so you can share it or bookmark it
Export — downloads the full report for your records or team review
Contact Us — connects you with SeekLab's team if you need help turning findings into an action plan

When to Run an Audit

Run the SeekLab audit:

After any significant website update or migration
Before launching a new content push
When traffic drops unexpectedly
Quarterly as a routine health check

Exporting the report each time gives your team a clear before/after record across audits.

The One Thing Most Teams Miss

Running the audit is easy. The common failure point is letting the report sit unread. The most valuable use of the tool is to focus the Critical tier first, then batch the Warning items by type — because grouped fixes at the template level take a fraction of the time that page-by-page edits do.

A website that gets audited and acted on outperforms one with a perfect-looking homepage every time.

Want to run your first scan? Visit seeklab.io/audit — it's free, no signup required, and results are ready in about five minutes.

For teams needing deeper analysis or help prioritising fixes, seeklab.io offers full-scope technical SEO consulting across website structure, content, and AI search visibility.

Your Website Exists. AI Doesn't Know That.

Thu, 09 Apr 2026 23:55:56 +0000

Someone just asked ChatGPT for a recommendation in your industry.

Your competitor was in the answer. You weren't.

Not because they're better. Because the AI knew what they do. It didn't know what you do.

That gap is growing every day. And it has a very simple fix.

The Problem No One Is Talking About

Google crawls your site. It reads your HTML, follows your links, indexes your pages. That system is 25 years old and it works.

AI assistants don't do that.

When Perplexity, ChatGPT, or Claude constructs an answer, it pulls from whatever structured context it can find. If your site is a JavaScript-heavy single-page app with no clear text about what you offer, where you operate, or what you charge -- the AI has nothing. It skips you. Or worse, it gets your details wrong and recommends someone else.

This is not theoretical.
It is happening right now.
Across every industry.

The Fix: One File, 10 Minutes

The solution is called llms.txt.

Think of it as robots.txt for AI. Where robots.txt tells search crawlers what to index, llms.txt tells AI assistants how to understand and recommend your business.

It's a plain Markdown file at the root of your domain. No plugins. No API. No special server config. Just text that tells AI exactly who you are.

The standard defines two files:

llms.txt is the summary. Your name, what you do, key links. Think elevator pitch in plain text.

llms-full.txt is the deep reference. Every service, pricing, process, FAQs, team, locations, and URLs. Everything an AI needs to answer any question about your business without visiting another page.

What Goes In Each File

Here's exactly what the AI needs to see. Miss any of this, and you're invisible again.

llms.txt -- Keep It Tight

# Your Business Name

> What you do, in one sentence.

- Full details: https://yourdomain.com/llms-full.txt

## Services

- **Service 1**: What it is. What makes it different.
- **Service 2**: What it is. What makes it different.

## Free Tools

- **Tool Name**: What it does. Direct URL.

## Areas Served

- City, State (headquarters)
- City, State

## Contact

- Website: https://yourdomain.com
- Email: *Emails are not allowed*

No HTML. No JavaScript. No images. Pure Markdown.

llms-full.txt -- Go Deep

This is where specificity wins. For each service, include what's included, how the process works, pricing ranges, timelines, and FAQs with real answers. For each location, include the city page URL, population, and key industries. For your team, include names, roles, and credentials.

The goal: an AI reading this file should be able to answer any question a potential customer would ask. Without visiting your site.

The single biggest mistake people make is being vague. "We offer digital solutions" tells an AI nothing. "Custom web development with React and Next.js. 95+ Lighthouse scores. Starting at $2,500." gets quoted verbatim.

Implementation

Next.js / Static Sites

Drop both files in /public:

public/
  llms.txt
  llms-full.txt
  robots.txt

Served automatically. Done.

WordPress

Upload to your root directory via FTP. Or use any plugin that serves static files from document root.

Optional: Reference in robots.txt

Sitemap: https://yourdomain.com/sitemap.xml
Llms: https://yourdomain.com/llms.txt

Not part of the formal spec yet. But it makes the file more discoverable.

Three Mistakes That KillThe Problem No One Is Talking About Your AI Visibility

Writing marketing copy instead of facts. AI assistants ignore superlatives. "Award-winning" and "industry-leading" get filtered out. Specific numbers and concrete descriptions get quoted. Write like you're filling out a form, not a brochure.

Skipping the full version. llms.txt alone is a business card. Without llms-full.txt, the AI fills in the blanks with guesses -- or with your competitor's details. The full file is where the real value lives.

Forgetting to update it. New service? New city page? Changed pricing? Both files need to reflect current state. Stale information is worse than no information, because the AI will confidently repeat it.

How to Know It's Working

Visit yourdomain.com/llms.txt in your browser. If you see plain text, it's live.

Then test it. Ask ChatGPT or Perplexity about your business. Ask about your services, your pricing, your locations. If the AI references details from your llms.txt, it's beiYour account may not be allowed to perform this action. Please refresh the page and try again.ng consumed.

This is new enough that the feedback loop is fast. Add the file today, and AI assistants can reference it within days.

The Bigger Picture

llms.txt is one piece of a broader shift. AI discoverability is becoming as important as search engine optimization was 10 years ago.

Structured data (JSON-LD) on every page tells Google's AI Overviews about your services and FAQs. City-specific landing pages let AI assistants answer location queries. FAQ schema gives AI direct question-answer pairs to pull from. Clean semantic HTML gives crawlers content they can parse without rendering JavaScript.

The businesses that build for AI discoverability now will own the answers for the next 2-3 years. The ones that wait will wonder why they keep getting skipped.

The file takes 10 minutes to write. The cost of not having it compounds every day.

Written by Joshua R. Gutierrez in collaboration with the engineering team at Axion Deep Digital. We build high-performance websites, rank them on Google, and make sure AI knows they exist.

Right now, AI is recommending someone else. Find out if you're even in the conversation. 60 seconds. No signup.

Try DeepAudit AI | SEO Scanning Tool for Free (No Signup Required):

Is Your Website Invisible to AI Search? A Developer's Checklist

Thu, 09 Apr 2026 08:52:02 +0000

I run an AI-powered test prep platform (LearnQ.ai) and a SaaS product for educational institutions (VEGA AI). When AI search engines like ChatGPT, Perplexity, and Google AI Overviews started becoming real traffic sources, I went back to basics and audited both sites for AI crawlability.
What I found surprised me. Several things that were perfectly fine for Google were actively blocking AI crawlers. This checklist is what I now run on every site before doing any AI search optimization work.

1. Check Your robots.txt for AI Crawler Blocks

Most developers set up robots.txt for Google and Bing and forget about it. AI crawlers use different user-agent strings.

Open your robots.txt file and check for blocks on these agents:

GPTBot (OpenAI)
ClaudeBot (Anthropic)
PerplexityBot
GoogleOther (used for AI training and features)
CCBot (Common Crawl, used by many LLM training pipelines)

A blanket Disallow: / on User-agent: * blocks all of them. If you added aggressive bot protection after a traffic spike or a security scare, there is a good chance you blocked AI crawlers along with the bad bots.

Fix: Add explicit Allow rules for AI crawlers you want to permit, or remove overly broad Disallow rules.

2. Create and Publish an llms.txt File

llms.txt is an emerging standard (think robots.txt but designed for LLMs). It sits at yourdomain.com/llms.txt and tells AI systems what your site is about, which pages are most important, and how the content should be used.

It is not yet universally supported, but ChatGPT, Perplexity, and several AI agents are beginning to respect it. Early adoption gives you an edge.

A basic llms.txt looks like this:

 llms.txt for proaisearch.com

> Pro AI Search is India's first AI search optimization resource.

 Key Pages
- [Home](https://proaisearch.com/)
- [GEO Guide](https://proaisearch.com/generative-engine-optimization/)
- [AEO Guide](https://proaisearch.com/answer-engine-optimization/)

Keep it simple. List your most important pages with clean descriptions.

3. Audit Your Bot Protection Settings

Cloudflare, Wordfence, Sucuri, and similar tools are aggressive by default. Check these specific settings:

Cloudflare: Under Security > Bots, check if "Block AI Scrapers and Crawlers" is enabled. This is a relatively new toggle that blocks GPTBot, ClaudeBot, and others by default when switched on.

Wordfence: Check the rate limiting rules. If you set aggressive thresholds after a brute force attack, legitimate AI crawlers hitting multiple pages during a crawl session can get auto-blocked.

Sucuri: Check the "Block All Bots" option under Security > Hardening. It does exactly what it says.

Review your server access logs if you have them. Look for 403 or 429 responses against GPTBot or ClaudeBot. Those are your AI crawlers getting turned away at the door.

4. Validate Your Schema Markup

AI engines do not just crawl text. They parse structured data to understand entities, relationships, and content type. Missing or broken schema is a silent visibility killer.

Check for these schema types depending on your site:

Organization schema on your homepage (name, URL, logo, social
profiles)
Article or BlogPosting schema on content pages
FAQPage schema on pages with Q&A content
BreadcrumbList schema for site structure signals
Person schema for author pages (critical for E-E-A-T in AI citations)

Use Google's Rich Results Test and Schema.org's validator. Fix any errors before doing anything else. AI engines rely heavily on structured data to decide whether to cite your content.

5. Check Page Rendering: Is Your Content Actually in the HTML?

If your site is JavaScript-heavy and content renders client-side, AI crawlers may be seeing blank pages. Most AI crawlers do not execute JavaScript the way Googlebot does.

Quick test: Disable JavaScript in your browser and load your key pages. If the main content disappears, AI crawlers likely see the same empty page.

Fix options:

Implement server-side rendering (SSR) or static site generation (SSG)
Use pre-rendering services for specific pages
At minimum, ensure critical content (headings, body text, key facts)
is in the initial HTML response

6. Check Canonical Tags and Indexability

This one catches people off guard. If you have:

<meta name="robots" content="noindex"> on important pages
Canonical tags pointing to a different URL
Pages blocked in robots.txt that you actually want crawled

...AI search engines will either ignore those pages or attribute the content to the wrong URL.

Run a crawl with Screaming Frog or Sitebulb. Filter for noindex pages and mismatched canonicals. Fix anything that should be crawlable and citable.

Running This Checklist

I go through these six checks in this order on any new site before touching content or link building. Technical access comes first. There is no point optimizing content that AI crawlers cannot reach in the first place.

For WordPress sites specifically, WPCode is useful for adding llms.txt content via a snippet rather than manually creating a file on the server. That is how I manage it on proaisearch.com.

If you find issues, fix robots.txt and bot protection first since those are the most common blockers. Then llms.txt, then schema, then rendering.

If you want to go deeper on any of these, I cover the full AI search optimization framework at proaisearch.com.

Amit Kumar is the founder of Pro AI Search and Growth Manager at LearnQ.ai.

Our Free AI SEO Tool Scanned 500+ Sites

Tue, 17 Mar 2026 19:07:06 +0000

Axion Deep Digital | DeepAudit AI Insights

We are Axion Deep Digital. We build systems that see what most people
overlook.

Six months ago, we released DeepAudit AI. On the surface, it is a free
SEO audit tool. Underneath, it is something more precise. It shows you
how your site is actually perceived, not how you think it looks.

Since then, hundreds of websites have passed through it. Patterns
emerged quickly. Not just technical flaws, but behavioral blind spots.

This is what we discovered.

Why This Exists

Most SEO tools give you numbers. Scores. Warnings.

They create the illusion of clarity while hiding the real problem.

They do not render your site the way Google does. They do not understand
modern frameworks. They do not tell you what actually changes outcomes.

They point at issues without resolving the tension behind them.

That gap is where most businesses quietly lose visibility.

We built DeepAudit AI to remove that gap. Not to report problems, but to
expose them in a way that forces action.

What We Saw Across 500 Plus Sites

The invisible first impression problem

Nearly 80 percent of sites either have no meta description or something
meaningless.

This is not just a missing tag. It is a missed moment.

Search is the first interaction a user has with your brand. If that
moment is generic, forgettable, or unclear, the decision is already made
before they click.

Most sites never realize they are losing before the interaction even
begins.

Accessibility is treated like an afterthought

Over half of the sites fail basic accessibility checks.

But this is not about compliance. It is about friction.

Every missing label, every poor contrast choice, every broken navigation
path adds cognitive load. Users do not analyze it. They feel it. And
they leave.

Google measures those behaviors. Quietly.

Untapped visibility signals everywhere

More than 70 percent of sites have no structured data.

This is one of the few places where you can directly shape how you
appear in search. Rich results. Enhanced listings. More space. More
attention.

Most businesses leave it empty.

Performance is silently killing trust

The average performance score sits in the 40s.

Users do not think in milliseconds. They feel delay as hesitation. As
doubt.

A slow site does not just rank worse. It feels less credible.

Security is invisible until it matters

The majority of sites are missing critical security headers.

Users may not see this. Search engines do.

Trust is being evaluated in the background, even when you are not aware
of it.

Structure defines discoverability

Only a small fraction of sites have clean internal linking.

If your pages are not connected, they do not exist in a meaningful way.
Not to users. Not to crawlers.

Visibility is not just about content. It is about pathways.

What We Built Instead

DeepAudit AI does not simulate a crawl. It becomes one.

It loads your site in a real browser. It waits. It observes. It extracts
what actually exists after everything renders.

Then it does something most tools avoid.

It tells you exactly what to do next.

Not suggestions. Not vague guidance.

Actual rewritten meta tags. Actual code snippets. Actual priorities.

Because knowing is not the problem. Acting is.

The Real Difference

Most tools measure.

This one interprets.

It bridges the gap between data and decision by removing ambiguity.

When you run a scan, you are not just getting a report. You are seeing
how your site performs in the only context that matters.

Perception.

Try It

DeepAudit AI is free. No signup. No friction.

Run your site through it and look at the results the way your users and
search engines do.

You might realize the problem was never what you thought it was.