Coder Legion - Recent posts tagged query

Find Keyword Cannibalization in Search Console | FloxoLab

Wed, 07 Oct 2026 15:50:00 +0000

Keyword cannibalization is an SEO diagnosis, not a Search Console status or a named Google penalty. The practical problem occurs when two or more pages on your site serve the same search intent and make it unclear which page should be the primary result. The symptom can be impressions divided across URLs, the visible URL changing between periods, or a weaker page receiving the clicks that the stronger page was built to earn.

Multiple URLs for one query can also be completely normal. A product page and a buying guide may both be relevant. A broad query can support several intents. Google also clusters duplicate URLs and assigns most Performance data to the canonical URL. Diagnose the relationship before deleting, redirecting, or rewriting anything.

Use this decision table before calling it cannibalization

What the pages do	Is it a problem?	Likely action
Same query, clearly different intent	Usually no	Keep both and make each purpose unmistakable
Same query and same intent, one page is much stronger	Likely	Merge useful material and redirect the weaker URL
Near-duplicate URLs created by parameters or variants	Canonical issue	Align redirects, canonical tags, internal links, and sitemap signals
Two useful pages with blurred boundaries	Possible	Reposition titles, headings, sections, and internal anchors
One page ranks for many related queries	No	That is normal query coverage, not cannibalization

Step 1: start with queries that matter

Open Search Console → Performance → Search results. Set a useful date range, usually at least three months for a small site, and enable Clicks, Impressions, CTR, and Average position. Choose queries with business value and enough impressions to compare. A query with three impressions is not a stable basis for restructuring two pages.

Search Console supports exact query filters, containing filters, and RE2 regular expressions. Start with one exact query when possible. Use a regex only when several close variations express the same intent.

Optional regex example: in the Search Console query filter, choose Custom (regex) and enter ^(seo audit price|seo audit cost|cost of seo audit)$. This matches only those three complete queries. The vertical bars mean "or", while ^ and $ mark the beginning and end of the query.

Do not combine every keyword containing "SEO" and call the resulting pages competitors. Query families need a shared user task, not merely one repeated word. The Search Console pre-audit is a useful first pass for separating commercial terms from filler before you spend time on overlap.

Step 2: open the Pages tab for one filtered query

In the Queries table, click the query. Search Console applies it as a filter. Now open the Pages tab. This is Google's documented workflow for seeing which URLs from your site were shown for that query.

Export the table or copy the relevant rows into a worksheet with these columns:

URL	Clicks	Impressions	CTR	Position	Intended job
/seo-audit	18	320	5.6%	8.4	Buy a fixed audit
/blog/seo-audit-checklist	7	260	2.7%	10.1	Learn what an audit checks

Those two example rows are not automatically a problem. The service and guide have different jobs. The next step is to test whether the distinction is visible and whether performance is stable.

Step 3: remove false positives

Different intent

Search the query yourself in the target market and classify what the result page rewards: product pages, service pages, definitions, comparisons, videos, local results, or a mix. Then read both of your pages. If one helps a person buy and the other helps them learn, both may deserve visibility even when the query overlaps.

Canonical aggregation

Search Console assigns most Performance data to a page's canonical URL, including clicks that may have happened on a duplicate. Inspect each URL in URL Inspection before assuming the table exposes every variant independently. If two paths are duplicates, solve the canonical setup rather than editing their copy as if they were separate articles.

One-off or seasonal noise

A URL can appear briefly because a page changed, an event created temporary demand, or Google tested another result. Compare equivalent periods and look for a repeatable pattern. One short spike is not enough evidence to merge pages.

Incomplete Search Console rows

Search Console omits anonymized queries for privacy and stores or shows only the most important rows. Query filters can also change totals because anonymized queries drop out. Treat the report as strong first-party evidence, not a complete log of every impression. Sites that need exhaustive analysis can use Search Console bulk data export, but most small sites can diagnose the important queries in the interface.

Step 4: look for a harmful pattern over time

With the query filter still active, compare the last period with the preceding equivalent period. The Pages table can show which URL gained or lost clicks and impressions. For a closer view, add an exact Page filter for one candidate and use the Dates tab, then repeat for the other URL.

Stronger evidence of genuine cannibalization includes several of these together:

The pages answer the same question for the same audience and funnel stage.
Neither page has a clear internal-link or title signal establishing its role.
The URL receiving most impressions changes repeatedly while total query clicks stay flat or fall.
The weaker or less relevant page replaces the page you want users to land on.
Both pages are thin because useful material has been divided between them.
External links and internal anchors point to both pages for the same concept.

Use clicks and impressions as the main outcome measures. Average position is useful as a direction, but Search Console's position calculations depend on aggregation and the topmost result. Do not redesign a content cluster because one average moved from 9.3 to 10.1.

Step 5: choose the smallest correct fix

Keep both and clarify their boundaries

Keep both pages when they serve distinct tasks. Rewrite titles, H1s, opening answers, and major sections so the difference is obvious. A commercial page can target booking, pricing, scope, and fit. An informational guide can target process, checks, examples, and preparation.

Link the guide to the service when a reader is ready to act, and link the service to the guide when a buyer needs evidence. Use descriptive anchors that name the destination's role. Google recommends crawlable internal links and says anchor text helps people and Google understand pages.

Merge and permanently redirect

Merge when both pages solve the same problem and one URL is the clear keeper. Move any unique, useful material into the stronger page. Then use a server-side 301 or 308 redirect from the retired URL, update internal links, remove the old URL from the sitemap, and keep the destination self-canonical.

Choose the keeper using evidence: existing clicks, relevant links, clearer URL, stronger content, conversion value, and the URL already used across your site. Do not default to the newer page or the page with the exact-match slug.

For the wider audit decision, use the old SEO page lifecycle guide to compare updating, merging, redirecting, noindexing, keeping, and deleting against page value and replacement intent.

Consolidate duplicate variants with canonical signals

Use canonicalization when the URLs are duplicate or very similar variants that must remain accessible. Google describes redirects and rel="canonical" as strong canonical signals, while sitemap inclusion is weaker. Point internal links to the preferred URL and keep the signals consistent.

Do not canonicalize a useful article to a service page merely because both mention the same keyword. Canonicalization is for duplicate or very similar content, not for choosing a winner between pages with different primary content.

Reposition one page around a different job

Sometimes both pages deserve to exist but one has drifted into the other's territory. Give it a distinct primary query and reader outcome, remove duplicated sections, add missing original evidence, and change the internal anchors pointing to it. This is a real content change, not a title swap over the same body.

Noindex or remove only when that is the intended outcome

Use noindex for a page that should remain available to users but should not appear in search. Remove an obsolete page with no useful replacement. Neither action is a routine fix for two valuable pages. If one URL is already excluded, the Crawled - currently not indexed guide helps decide whether to improve, consolidate, noindex, or leave it alone.

Step 6: validate after Google recrawls the change

Record the implementation date and exact change. Inspect the keeper and any redirect in URL Inspection, update the sitemap if needed, and allow Google to recrawl. Then compare equivalent periods after the query has accumulated enough impressions to be meaningful.

A successful outcome is not "only one row exists". The outcome should match the decision:

Keep both: each URL earns impressions for its intended query family and sends visitors to the right experience.
Merge: the keeper gains or retains the combined useful traffic, and internal links no longer point to the retired URL.
Canonicalize: URL Inspection and Performance data increasingly reflect the preferred canonical.
Reposition: each page develops a clearer query set rather than trading the same query back and forth.

There is no guaranteed timeline or ranking improvement. Search behavior, competitors, seasonality, and other site changes can move the same metrics. Google recommends mapping observed changes to the timing of your work and comparing similar periods. Keep the annotation and judge the trend, not one day's position.

A compact cannibalization worksheet

Field	What to record
Query or query family	Exact filter or documented regex
Candidate URLs	Pages shown under that filtered query
Intent	Task, audience, and funnel stage for each URL
Evidence	Clicks, impressions, CTR, position, and period comparison
Canonical state	User-declared and Google-selected canonical where available
Decision	Keep, merge, canonicalize, reposition, noindex, or remove
Implementation	Content, redirect, canonical, sitemap, and internal-link changes
Validation	Recrawl state and equivalent-period performance trend

Prevent overlap when planning a small topic cluster

The workflow above diagnoses overlap after pages have impressions. During earlier discovery, the AI keyword research workflow turns customer language and tool evidence into query families without deciding that every phrase needs a page. A small site can prevent much of the remaining overlap before publishing by giving every proposed URL a distinct job. The topic cluster planning guide provides the page contract, query-family map, link directions, and collision test for that earlier decision. Topic clusters do not need a large number of spokes. They need a clear parent topic, useful page boundaries, and links that help readers move between the overview and the specific task.

Before creating a cluster page, record its primary query family, search intent, reader outcome, page type, and closest existing URL. Then use this test:

Proposed page	Best decision	Reason
Same task and expected answer as an existing page	Update the existing URL	A new spoke would divide the same useful material
Distinct task, failure, comparison, or decision	Create a separate page	The reader needs a materially different answer
Narrow variation with too little substance for a full guide	Add a section to the parent page	A thin spoke creates maintenance and overlap without enough reader value
Useful page with a clear role but weak discovery	Keep it and improve internal links	The problem is navigation and context, not duplicate intent

A pillar should provide the broad orientation and route readers to deeper tasks. Each spoke should solve one narrower problem in enough depth to justify its own URL. Link the pillar to the spoke and the spoke back to the most useful parent or related guide with concise, descriptive anchor text. Do not manufacture pages to reach a cluster quota, and do not rely on a hub card or sitemap as the only path to an important guide.

After publishing, repeat the Search Console query-and-pages check from this guide. If two cluster pages begin earning impressions for the same query, compare intent before changing anything. Shared vocabulary is normal. The warning sign is two pages doing the same job for the same reader.

Practical questions

Can two pages rank for the same keyword without a problem?

Yes. The pages may serve different intents or both may be relevant to a broad query. Investigate when they do the same job, switch unpredictably, split useful content, or send visitors to the wrong destination.

Should I add a canonical from the weaker page to the stronger page?

Only when the pages are duplicate or very similar variants. If both contain distinct useful content, merge and redirect, or keep both and clarify their roles. A canonical is not a substitute for an editorial decision.

Does duplicate content cause a penalty?

Google says some duplicate content is normal and is not a spam-policy violation. The practical risks are poor user experience, confusing measurement, diluted internal signals, and Google selecting a different canonical than you prefer.

AI Keyword Research: Cluster and Prioritize Ideas

Thu, 24 Sep 2026 15:50:00 +0000

Direct answer: use AI keyword research for expansion, normalization, classification, and cluster hypotheses. Do not use it as the source of search volume, ranking difficulty, current search intent, or business value. The reliable output is a short, evidence-backed queue of query families. It is not a list of hundreds of phrases ready to become pages.

This workflow stops at research prioritization. The topic cluster planning guide makes the later URL and internal-link decisions. If two live pages already appear for the same query family, use the Search Console cannibalization workflow instead of treating the problem as fresh keyword research.

What AI can and cannot do in keyword research

Job	Good use of AI	Evidence still required
Expand language	Suggest wording, modifiers, questions, and adjacent jobs from supplied seeds	Customer language, Search Console queries, sales notes, and support records
Clean a list	Normalize spelling, flag duplicates, and preserve a source reference	A human check that meaningful distinctions were not erased
Propose clusters	Group phrases by the reader's task and explain each proposed boundary	Current result-page review and a collision test before assigning URLs
Classify intent	Suggest informational, comparison, transactional, local, or navigational labels	Manual review because one phrase can support several situations
Estimate demand	Prepare candidates for tool checks	Search Console, Keyword Planner, Trends, and real customer frequency
Prioritize	Apply a scoring rule consistently to evidence you provide	Business fit, service capacity, proof, and the next useful action

Never ask the model to fill missing metrics. A plausible search-volume, CPC, trend, or difficulty number is still fabricated if it did not come from a named data source. Keep missing values blank and record what must be checked.

Step 1: begin with customer language, not an AI blank page

Collect the words people already use when a real problem is present. Useful sources include Search Console queries, sales-call summaries, contact forms, support tickets, internal site search, reviews, proposals, and objections recorded by staff. A small list of genuine questions is a stronger seed than a generic request for every keyword in an industry.

Remove names, email addresses, phone numbers, account details, confidential pricing, and customer-specific context before sending notes to an external AI service. Keep a neutral source label such as sales-call-07 or support-theme-refunds so every suggestion can be traced without exposing a person.

Start a research sheet with these fields:

Raw phrase: the wording as received, including awkward language.
Source: Search Console, customer call, ticket, review, or staff observation.
Offering and audience: the service or product and the person making the decision.
Problem stage: learning, comparing, buying, implementing, or troubleshooting.
Location or constraint: market, platform, budget, company size, or deadline when relevant.
Available proof: experience, screenshot, example, primary source, or measured result.

Step 2: expand candidates with a constrained prompt

Give the model the seed table and ask for controlled variations. Require a reason for every row and forbid invented metrics. The prompt below is deliberately narrower than “find keywords for my business.”

You are expanding a keyword research seed list.

Use only the business context and raw customer phrases below.
For each useful candidate, return:
- candidate_phrase
- source_seed
- reader_job
- likely_stage
- modifier_type
- ambiguity_or_missing_context
- evidence_needed

Rules:
1. Preserve meaningful customer wording.
2. Do not invent search volume, CPC, trend, competition, or rankings.
3. Do not turn simple synonyms or singular/plural variants into separate ideas.
4. Include a candidate only when it represents a real question, decision, or task.
5. Mark weak extrapolations as hypothesis.

Business context:
[paste a short factual description]

Sanitized seed rows:
[paste the seed table]

Review the result before continuing. Remove phrases that do not describe a useful reader job, promises the business cannot support, and variations created only by swapping adjectives. Keep the original source beside every surviving row.

Step 3: normalize and cluster by task

Clean obvious duplicates without destroying intent. Lowercase and trim whitespace for comparison, standardize spelling, and flag singular or plural variants. Preserve the original phrase in a separate column. Location, audience, platform, and problem-stage modifiers can change the job, so do not strip them automatically.

Then ask AI to propose clusters. A cluster is a shared reader task, not a bag of phrases containing the same noun.

Group these candidates by the reader's primary job.

For every proposed cluster, return:
- cluster_name
- one-sentence reader_job
- included candidate IDs
- phrases that look similar but must stay separate
- boundary_reason
- confidence: high, medium, or low

Do not merge candidates when they differ by:
- audience or decision maker
- problem stage
- local versus non-local need
- comparison versus implementation task
- troubleshooting versus general education

If the evidence is insufficient, place the candidate in "manual review".
Do not decide how many URLs to publish.

The final instruction matters. Query grouping happens before page architecture. One cluster may become a section, an existing-page update, a new guide, a service-page addition, or nothing at all.

Step 4: validate every cluster with named evidence

Source	Best question it answers	Limit to record
Customer and sales records	Do real prospects use this language or need this decision?	Small samples reflect the customers you already reach
Google Search Console	Which queries already produce impressions or clicks for this site and page?	Anonymous queries are omitted and the table exposes only top rows
Google Ads Keyword Planner	Is there approximate demand in a selected location and period?	Volumes are rounded, close variants are grouped, and advertiser competition is not SEO difficulty
Google Trends	Is relative interest rising, falling, seasonal, or regionally different?	Values are normalized from 0 to 100 and do not show absolute search volume
Current search results	What jobs and result types does the wording currently surface?	Results vary by location, time, language, and personalization context
Business evidence	Can this site give a useful, specific, supportable answer?	A keyword can have demand and still be a poor fit for the business

Search Console: first-party demand already touching your site

Start with the Performance report when the site already has data. Filter a relevant page, inspect its queries, and use a regular expression only when a well-defined phrase family needs grouping. Google omits anonymized queries for privacy and limits the table to top rows, so a visible query list is not the complete universe of searches. Bulk export provides the most complete query dataset when the property has enough data and the workflow justifies it.

Existing impressions are evidence of eligibility, not proof that a new URL is needed. Record the current page beside the query family. That single field helps prevent a research idea from competing with a page that already performs the job.

Keyword Planner: approximate market demand, not an organic forecast

Set the correct location, language, network, and date range. Google's average monthly searches include close variants and are rounded. Forecasts also depend on bid, budget, seasonality, and ad quality. Use the output as directional market evidence. Do not copy advertiser competition into a column called SEO difficulty, and do not present an ad forecast as expected organic traffic.

Google Trends: direction and seasonality, not volume

Compare candidates over the same geography and period. A Trends value of 100 marks the peak relative interest within that comparison, not one hundred searches. Low-volume phrases can display as zero. Compare a literal search term when wording matters and a topic when you need a broader concept across related terms or languages.

Manual result and evidence check

Open the live results for representative phrases and record what kind of task dominates: definition, step-by-step help, product category, comparison, local provider, tool, or troubleshooting. Do not imitate the top pages mechanically. The purpose is to test the cluster boundary and find what evidence a useful answer would need.

Step 5: score business fit before volume

Use a small scoring model that a second person can reproduce. The following ten-point model works well for a small content queue:

Business fit, 0 to 3: no meaningful connection, adjacent, relevant, or directly connected to a service and qualified next step.
Evidence, 0 to 3: unsupported, thin, adequate, or strong first-hand and primary-source support.
Task clarity, 0 to 2: ambiguous, partly defined, or one clear reader job distinct from existing pages.
Actionability, 0 to 2: passive information, a useful decision, or a practical next action the reader can complete.

Query family	Business fit	Evidence	Task clarity	Actionability	Total	Decision
AI keyword research for a small service business	3	3	2	2	10	Prioritize
Free keyword list generator	1	1	1	1	4	Hold or answer briefly
Best SEO tool	2	2	0	1	5	Narrow the task first
Guaranteed number-one ranking keywords	0	0	1	0	1	Reject the premise

Search demand can break a tie after the usefulness gates pass. It should not rescue a weak business fit, missing evidence, or a duplicate reader job. If a paid research platform would save time, the hands-on SE Ranking review shows what its keyword and competitor research can and cannot replace. A tool subscription does not remove the need for this evidence model.

Step 6: let AI apply the rule, not make the decision

Give the model only verified evidence and the scoring rubric. Require it to cite input fields and leave unsupported scores blank.

Score the validated query families using this rubric:
- business_fit: 0-3
- evidence_strength: 0-3
- task_clarity: 0-2
- actionability: 0-2

Use only the supplied evidence columns.
For each score, cite the exact input field that supports it.
If a field is missing, return "needs evidence" instead of guessing.
Flag any family already served by an existing URL.
Return a prioritized queue, a hold list, and a rejection list.
Do not recommend new URLs or page titles.

Review the ranking manually. A business constraint can override the total. For example, a high-scoring service topic should wait if the company cannot deliver that service, prove its claims, or handle the expected lead type.

The research handoff

The completed research sheet should contain one row per query family, not one row per wording variation:

representative query family and included variations;
reader job, audience, stage, and relevant location;
original source and sanitized customer language;
Search Console, Planner, Trends, and manual result evidence;
business fit, available proof, score, and uncertainty;
existing URL that may already serve the job;
status: prioritize, hold, reject, or investigate.

Stop here. The next step is the content map, where the prioritized families are tested against existing URLs and given distinct page jobs. This separation reduces cannibalization because research does not silently create one page per keyword. If one family reveals a repeatable set of real entities rather than ordinary editorial topics, use the programmatic SEO quality-gate workflow to test the dataset before generating pages.

Common failure modes

Starting from a generic AI prompt. The output reflects common web language, not evidence from your buyers.
Treating every variation as a separate opportunity. Singular, plural, and close wording often belong to one task.
Clustering by shared words. Similar vocabulary can hide different audiences, stages, and decisions.
Inventing metrics. AI cannot know your current Search Console data or live tool settings unless you provide them.
Sorting only by volume. A broad phrase can attract the wrong audience and consume evidence the business does not have.
Ignoring the existing URL column. A promising phrase may belong in an update, not a new article.
Uploading raw customer records. Sanitize notes and keep only the context needed for classification.
Publishing the whole queue. A prioritized hypothesis still needs a page contract, sources, and a collision check.

Decision checklist

Every candidate traces back to customer, search, market, or business evidence.
The model is forbidden from inventing volume, competition, trend, or rankings.
Clusters represent reader jobs, not repeated words.
Search Console query limits and anonymized data are acknowledged.
Keyword Planner settings and close-variant grouping are recorded.
Google Trends is used for relative interest, not absolute demand.
Business fit and available proof are scored before search volume.
Every family records an existing URL that might already serve it.
The output stops at prioritize, hold, reject, or investigate.
URL architecture and live cannibalization remain separate workflows.

Sources checked

Tool behavior and reporting limits were verified from official Google documentation on August 2, 2026.

MySQL Indexing: Slash Query Times by 96%

Wed, 08 Jul 2026 07:45:45 +0000

?>

MySQL Indexing: Slash Query Times by Up to 96%

November 2025 14 min read

I’m Jacob Campbell, and the single biggest backend speed win I see is also the most overlooked: indexing. Without the right index, MySQL reads every row in a table to answer a query; with one, it jumps straight to the rows it needs. The difference is routinely a query dropping from a few hundred milliseconds to single digits — in real terms I’ve taken a 340 ms query down to 8 ms. Here’s how indexes work and how to find the ones you’re missing.

SET GLOBAL slow_query_log = 1;
SET GLOBAL long_query_time = 0.5;  -- log queries slower than 0.5 seconds
SET GLOBAL slow_query_log_file = '/var/log/mysql/slow.log';

EXPLAIN SELECT * FROM products WHERE category_id = 5 AND price < 100;

CREATE INDEX idx_user_id ON orders (user_id);

-- Slow: 2 seconds, scans 50,000 rows
SELECT * FROM products WHERE category_id = 5 AND price < 100;

-- Add composite index (order matters!)
CREATE INDEX idx_category_price ON products (category_id, price);

-- Now: 30ms, scans 127 rows

-- Query only needs id and name
SELECT id, name FROM products WHERE category_id = 5;

-- Covering index
CREATE INDEX idx_category_id_name ON products (category_id, name);

-- EXPLAIN will show "Using index" in Extra column

SELECT * FROM products 
WHERE category_id = 12 
  AND brand_id IN (3,7,9) 
  AND price BETWEEN 50 AND 200 
ORDER BY price LIMIT 24;

CREATE INDEX idx_category_brand_price ON products (category_id, brand_id, price);

-- If you frequently query postmeta by meta_key and meta_value
ALTER TABLE wp_postmeta ADD INDEX idx_meta_key_value (meta_key, meta_value(100));

CREATE INDEX idx_city_price_beds_type ON properties (city_id, price, bedrooms, property_type);

SELECT * FROM sys.schema_unused_indexes;
SELECT * FROM sys.schema_redundant_indexes;

Prometheus Scalability: High Cardinality and How to Fix It

Sat, 25 Apr 2026 02:25:50 +0000

Prometheus has become the de facto standard for metrics collection in cloud-native environments. Its pull-based model, powerful query language, and deep Kubernetes integration make it an obvious choice for platform teams. But as organizations scale — more services, more replicas, more labels — Prometheus starts showing cracks. Queries slow down, memory usage balloons, and what was once a reliable monitoring backbone becomes an operational liability. This article examines exactly why that happens and what you can do about it, from quick tactical fixes to full architectural overhauls.

The Cardinality Problem: Why It Kills Prometheus

Cardinality is the single most important concept to understand when troubleshooting Prometheus scalability. In the context of time series databases, cardinality refers to the total number of unique label combinations that exist across all your metrics. Every unique combination creates a distinct time series, and Prometheus must store, index, and query each of them independently.

Consider a simple HTTP request counter: http_requests_total. If you label it with method (GET, POST, PUT, DELETE), status_code (200, 201, 400, 404, 500, 503), and endpoint (50 distinct API paths), you already have 4 × 6 × 50 = 1,200 time series from a single metric. Now add a customer_id label with 10,000 distinct values. You have just created 12 million time series from one counter.

This is the cardinality explosion pattern, and it is the most common cause of Prometheus degradation in production. The problem is compounded by labels that have unbounded or high-entropy values:

User IDs or session tokens embedded in labels
Request IDs or trace IDs (effectively infinite cardinality)
Pod names without proper aggregation, especially in autoscaling environments
Free-form error messages or SQL query strings
IP addresses, particularly in environments with high churn

The relationship between cardinality and resource consumption is not linear — it is roughly proportional but carries significant overhead per series in memory indexing structures. Prometheus stores its head block (the most recent data) entirely in memory. Each time series in the head block requires approximately 3–4 KB of RAM for the series itself plus index entries. A Prometheus instance with 1 million active time series will typically consume 4–6 GB of RAM just for the head block, before accounting for query processing overhead.

Memory Explosion Patterns and Real Symptoms

Memory issues in Prometheus rarely announce themselves cleanly. Instead, they manifest through a cascade of symptoms that are easy to misdiagnose. Understanding the failure modes helps you identify the root cause faster and apply the right remedy.

The Head Block Growth Pattern

Prometheus keeps a two-hour window of data in memory as the head block before compacting it to disk. If your series count grows continuously — which happens when pod churn creates new series faster than old ones expire — the head block never shrinks. You can monitor this directly with prometheus_tsdb_head_series and prometheus_tsdb_head_chunks. A healthy instance shows this number plateauing. A cardinality problem shows it growing monotonically until OOM.

Query Timeout Cascades

As series count grows, even well-written PromQL queries that worked fine at 100k series become unbearably slow at 1M. Grafana dashboards start timing out, alert evaluation lags behind schedule, and Alertmanager begins receiving delayed or duplicated firing alerts. The prometheus_rule_evaluation_duration_seconds metric is a reliable early warning — when p99 evaluation time for your recording rules exceeds your evaluation interval, you have a problem.

Scrape Failures Under Memory Pressure

When Prometheus is under heavy memory pressure, its Go garbage collector starts spending more time collecting, which introduces latency into the scrape loop. Scrapes begin timing out, causing gaps in your data. This creates a deceptive situation where you have gaps in metrics precisely when your system is under stress — exactly when you need monitoring most. Watch up metric drops and prometheus_target_scrapes_exceeded_sample_limit_total for these patterns.

Compaction Pressure

High cardinality also stresses the TSDB compaction process. Prometheus compacts head block data into persistent blocks every two hours. With millions of series, compaction can take tens of seconds to minutes, during which write performance degrades. prometheus_tsdb_compaction_duration_seconds rising above 30 seconds is a warning sign. Compaction failures leave orphaned blocks on disk, gradually consuming storage and potentially corrupting the TSDB if left unaddressed.

Short-Term Fixes: Tactical Remediation

When you are dealing with a Prometheus instance under active stress, you need immediate relief before you can implement architectural changes. These techniques can be applied quickly and provide meaningful headroom while longer-term solutions are planned.

Recording Rules: Pre-Computing Aggregations

Recording rules are the most underutilized tool in the Prometheus toolbox. They allow you to pre-compute expensive PromQL expressions and store the results as new time series. The key benefit for scalability is that you can aggregate away high-cardinality dimensions, dramatically reducing the number of series that dashboards and alerts need to query at runtime.

Consider an example where you have per-pod HTTP request rates with labels for pod, namespace, service, method, and status_code. Your dashboards mostly need service-level aggregations, not per-pod breakdowns. A recording rule can produce that aggregation once per evaluation interval:

groups:
  - name: http_aggregations
    interval: 30s
    rules:
      - record: job:http_requests_total:rate5m
        expr: |
          sum by (job, namespace, method, status_code) (
            rate(http_requests_total[5m])
          )

      - record: job:http_request_duration_seconds:p99_5m
        expr: |
          histogram_quantile(0.99,
            sum by (job, namespace, le) (
              rate(http_request_duration_seconds_bucket[5m])
            )
          )

      - record: namespace:http_requests_total:rate5m
        expr: |
          sum by (namespace, status_code) (
            rate(http_requests_total[5m])
          )

Notice that the pod label is dropped in all three rules. If you had 500 pods, you have just reduced the cardinality of these series by a factor of 500. Dashboards querying job:http_requests_total:rate5m instead of computing rate(http_requests_total[5m]) on the fly will return results orders of magnitude faster.

The naming convention level:metric:operations is the Prometheus community standard. Following it consistently makes recording rules self-documenting and helps teams understand the aggregation level at a glance.

Metric Dropping via Relabeling

Relabeling gives you surgical control over what metrics Prometheus actually ingests. There are two stages where relabeling applies: relabel_configs (applied before scraping, based on target metadata) and metric_relabel_configs (applied after scraping, based on scraped metric names and labels). For cardinality control, metric_relabel_configs is your primary tool.

Dropping entire metric families that you do not use is the most impactful change you can make. Many exporters emit dozens of metrics that are irrelevant for most use cases:

scrape_configs:
  - job_name: kubernetes-pods
    metric_relabel_configs:
      # Drop metrics we never query
      - source_labels: [__name__]
        regex: 'go_gc_.*|go_memstats_.*|process_.*'
        action: drop

      # Drop high-cardinality label values while keeping the metric
      - source_labels: [__name__, pod]
        regex: 'http_requests_total;.*'
        target_label: pod
        replacement: ''

      # Drop entire time series based on label combinations
      - source_labels: [__name__, le]
        regex: 'http_request_duration_seconds_bucket;(\+Inf|100|250|500)'
        action: keep

      # Replace high-cardinality endpoint paths with normalized versions
      - source_labels: [endpoint]
        regex: '/api/v1/users/[0-9]+'
        target_label: endpoint
        replacement: '/api/v1/users/:id'

Be careful with metric_relabel_configs — they are applied per scraped sample, so computationally expensive regex patterns across high-frequency scrapes can add CPU overhead. Test regex patterns and prefer anchored, non-backtracking expressions.

Cardinality Limits as a Safety Net

Prometheus 2.x introduced per-scrape sample limits as a defensive mechanism. These do not solve cardinality problems but prevent a single misbehaving exporter from taking down your entire Prometheus instance:

global:
  # Global limit across all scrapes
  sample_limit: 0  # 0 = no limit

scrape_configs:
  - job_name: application-pods
    # Reject scrapes that return more than 50k samples
    sample_limit: 50000

    # Limit unique label sets per scrape
    label_limit: 64

    # Limit label name and value lengths
    label_name_length_limit: 256
    label_value_length_limit: 1024

    kubernetes_sd_configs:
      - role: pod

When a scrape exceeds sample_limit, Prometheus rejects the entire scrape and marks the target as having failed. This is a hard circuit breaker, not a graceful degradation — the target’s up metric goes to 0. Set limits conservatively above your expected maximum to avoid false positives, and alert on prometheus_target_scrapes_exceeded_sample_limit_total > 0.

Architectural Solutions: Federation and Remote Write

Once you have exhausted tactical optimizations or when your scale genuinely exceeds what a single Prometheus instance can handle, architectural changes become necessary. Prometheus offers two built-in mechanisms for scaling horizontally: federation and remote_write.

Federation: Hierarchical Scraping

Prometheus federation allows one Prometheus instance to scrape aggregated metrics from other Prometheus instances via the /federate endpoint. In a typical setup, leaf-level Prometheus instances collect raw metrics from targets, while a global Prometheus instance federates pre-aggregated recording rule results from the leaves.

# Global Prometheus configuration federating from regional instances
scrape_configs:
  - job_name: federate-eu-west
    scrape_interval: 15s
    honor_labels: true
    metrics_path: /federate
    params:
      match[]:
        # Only federate pre-aggregated recording rule metrics
        - '{__name__=~"job:.*"}'
        - '{__name__=~"namespace:.*"}'
        - '{__name__=~"cluster:.*"}'
        # Federate key infrastructure alerts
        - 'up{job="kubernetes-apiservers"}'
    static_configs:
      - targets:
          - prometheus-eu-west.monitoring.svc:9090
          - prometheus-us-east.monitoring.svc:9090
          - prometheus-ap-south.monitoring.svc:9090

Federation works well for multi-region global dashboards and cross-cluster alerting on aggregated signals. Its limitations are significant, though: the /federate endpoint is a point-in-time snapshot, so you cannot run range queries against federated data effectively. It also creates a single point of failure at the global layer and does not provide true long-term storage. For those requirements, remote_write is the better path.

Remote Write: Streaming to Durable Storage

Remote write allows Prometheus to stream all ingested samples to an external storage backend in real time. The external backend handles long-term retention, multi-tenancy, and global query federation. Prometheus itself becomes a stateless collection agent that maintains only a short local retention window for resilience against network outages.

remote_write:
  - url: https://thanos-receive.monitoring.svc:19291/api/v1/receive
    # Authentication for the remote endpoint
    basic_auth:
      username: prometheus
      password_file: /etc/prometheus/secrets/remote-write-password

    # Tune the write queue for throughput vs. latency
    queue_config:
      # Number of shards (parallel write connections)
      max_shards: 200
      min_shards: 1
      # Samples to batch before flushing
      max_samples_per_send: 500
      # Time to wait before flushing an incomplete batch
      batch_send_deadline: 5s
      # In-memory buffer capacity per shard
      capacity: 2500
      # How long to retry failed writes
      min_backoff: 30ms
      max_backoff: 5s

    # Metadata configuration
    metadata_config:
      send: true
      send_interval: 1m

    # Filter what gets remote-written (reduce egress)
    write_relabel_configs:
      - source_labels: [__name__]
        regex: 'go_gc_.*|go_memstats_.*'
        action: drop

The queue_config tuning is critical and frequently misunderstood. Each shard maintains its own connection to the remote endpoint and its own in-memory queue. Increasing max_shards increases parallelism and throughput but also increases memory consumption and load on the remote endpoint. The right values depend heavily on your sample ingestion rate and network latency to the remote endpoint. Monitor prometheus_remote_storage_queue_highest_sent_timestamp_seconds versus prometheus_remote_storage_highest_timestamp_in_seconds — the lag between them tells you how far behind your remote write queue is.

Long-Term Solutions: Thanos vs Grafana Mimir vs VictoriaMetrics

For production systems that need long-term storage, global query capability, high availability, and genuine horizontal scalability, purpose-built solutions are the right answer. Three projects dominate this space: Thanos, Grafana Mimir, and VictoriaMetrics. They share similar goals but differ significantly in architecture, operational complexity, and trade-offs.

Criterion	Thanos	Grafana Mimir	VictoriaMetrics
Architecture	Sidecar + object store; modular components	Fully distributed; Cortex-derived microservices	Single binary or cluster mode
Storage backend	Any S3-compatible object store	Any S3-compatible object store	Own TSDB format on local or object store
PromQL compatibility	Full PromQL; own query engine	Full PromQL; Mimir-specific extensions	MetricsQL (PromQL superset)
Operational complexity	Medium — multiple components, each simple	High — many microservices with complex config	Low — minimal components, simple config
Ingest scalability	Scales via Thanos Receive fan-out	Horizontally scalable distributors + ingesters	Excellent; handles millions of samples/sec per node
Query performance	Good; Store Gateway caches object store data	Good; query sharding and caching built in	Excellent; highly optimized query engine
Multi-tenancy	Limited; tenant isolation via external labels	Native; per-tenant limits and isolation	Enterprise only; basic in cluster mode
Deduplication	Built-in; replica dedup at query time	Built-in; ingest-time and query-time dedup	Built-in; dedup with downsampling
Downsampling	Yes; Thanos Compactor handles it	Yes; configurable per tenant	Yes; automatic with vmbackupmanager
License	Apache 2.0 (fully open source)	AGPL-3.0 (open source) + enterprise tier	Apache 2.0 (community); proprietary enterprise
Best fit	Teams already running Prometheus wanting minimal disruption	Large orgs needing multi-tenant SaaS-grade monitoring	Teams prioritizing simplicity and raw performance

Thanos: The Incremental Path

Thanos integrates with existing Prometheus deployments through a sidecar process that runs alongside each Prometheus pod. The sidecar uploads completed TSDB blocks to object storage (S3, GCS, Azure Blob) and exposes a gRPC Store API that Thanos Query uses to federate queries across all Prometheus instances plus historical data in the object store. This makes Thanos the lowest-friction path for teams with existing Prometheus infrastructure.

Thanos Receive is an alternative ingest path that accepts remote_write directly, which is useful when you want to decouple Prometheus instances from the query layer or implement active-active HA without relying on Prometheus replication. Thanos Compactor handles block compaction and downsampling on the object store, creating 5-minute and 1-hour resolution downsamples automatically for efficient long-range queries.

Grafana Mimir: Enterprise-Grade Multi-Tenancy

Mimir is a fork of Cortex, rewritten by Grafana Labs to address operational complexity issues in Cortex’s architecture. It follows the same microservices pattern — Distributor, Ingester, Querier, Query Frontend, Store Gateway, Compactor, Ruler — but with significantly improved defaults and a monolithic deployment mode that simplifies small-scale deployments. Mimir’s headline feature is native multi-tenan

OpenClaw Database Designer Skill: Comprehensive Guide

Sat, 21 Mar 2026 14:15:43 +0000

OpenClaw Database Designer Skill: Comprehensive Guide

The OpenClaw Database Designer skill is a POWERFUL Tier database design tool that provides expert-level analysis, optimization, and migration capabilities for modern database systems. This comprehensive skill combines theoretical principles with practical tools to help architects and developers create scalable, performant, and maintainable database schemas.

Core Competencies

Schema Design & Analysis

The skill offers advanced schema design capabilities including:

Normalization Analysis: Automated detection of normalization levels from 1NF through BCNF
Denormalization Strategy: Smart recommendations for performance optimization
Data Type Optimization: Identification of inappropriate types and size issues
Constraint Analysis: Missing foreign keys, unique constraints, and null checks
Naming Convention Validation: Consistent table and column naming patterns
ERD Generation: Automatic Mermaid diagram creation from DDL

Index Optimization

Advanced index analysis and optimization features include:

Index Gap Analysis: Identification of missing indexes on foreign keys and query patterns
Composite Index Strategy: Optimal column ordering for multi-column indexes
Index Redundancy Detection: Elimination of overlapping and unused indexes
Performance Impact Modeling: Selectivity estimation and query cost analysis
Index Type Selection: B-tree, hash, partial, covering, and specialized indexes

Migration Management

The skill provides comprehensive migration capabilities:

Zero-Downtime Migrations: Expand-contract pattern implementation
Schema Evolution: Safe column additions, deletions, and type changes
Data Migration Scripts: Automated data transformation and validation
Rollback Strategy: Complete reversal capabilities with validation
Execution Planning: Ordered migration steps with dependency resolution

Database Design Principles

The skill incorporates industry best practices for database design:

Schema Design Best Practices

Use meaningful names: Clear, consistent naming conventions
Choose appropriate data types: Right-sized columns for storage efficiency
Define proper constraints: Foreign keys, check constraints, unique indexes
Consider future growth: Plan for scale from the beginning
Document relationships: Clear foreign key relationships and business rules

Performance Optimization

Index strategically: Cover common query patterns without over-indexing
Monitor query performance: Regular analysis of slow queries
Partition large tables: Improve query performance and maintenance
Use appropriate isolation levels: Balance consistency with performance
Implement connection pooling: Efficient resource utilization

Security Considerations

Principle of least privilege: Grant minimal necessary permissions
Encrypt sensitive data: At rest and in transit
Audit access patterns: Monitor and log database access
Validate inputs: Prevent SQL injection attacks
Regular security updates: Keep database software current

Practical Applications

The Database Designer skill is invaluable for:

Database architects designing new systems
Developers optimizing existing schemas
DevOps engineers managing database migrations
Database administrators monitoring performance
Development teams ensuring best practices

Benefits

Using this skill provides numerous advantages:

Improved database performance through optimized schemas
Reduced maintenance costs with automated analysis
Enhanced scalability through proper design principles
Increased reliability with constraint validation
Better documentation through automatic ERD generation
Streamlined migration processes with zero-downtime capabilities

Conclusion

Effective database design requires balancing multiple competing concerns: performance, scalability, maintainability, and business requirements. The OpenClaw Database Designer skill provides the tools and knowledge to make informed decisions throughout the database lifecycle, from initial schema design through production optimization and evolution.

The included tools automate common analysis and optimization tasks, while the comprehensive guides provide the theoretical foundation for making sound architectural decisions. Whether building a new system or optimizing an existing one, these resources provide expert-level guidance for creating robust, scalable database solutions.

By leveraging this skill, organizations can ensure their database designs are not only functional but also optimized for performance, scalability, and maintainability, ultimately leading to more successful and sustainable database systems.

Skill can be found at: https://github.com/openclaw/skills/tree/main/skills/alirezarezvani/database-designer/SKILL.md

The post OpenClaw Database Designer Skill: Comprehensive Guide first appeared on Insight Ginie.

Mastering CSS Container Queries for Adaptive Layouts

Tue, 18 Nov 2025 21:15:41 +0000

The next step in responsive design: container queries.

For many years, responsive web design relied on media queries. You define some parameters, and the layout changes depending on the screen size. Ideal for adapting designs for phones, tablets, and desktops. But what if your components need to adapt to the real space they are in, rather than just the screen size?

Enter container queries: a CSS feature that lets you style components based on the size of their parent container. It’s great for creating reusable, adaptable layouts that work seamlessly across different parts of a page.

For example:

A Card component might look great in a wide sidebar but needs adjustments for a narrower one.
A Profile block might stack its elements vertically in a small space but align them horizontally when there’s more room.
With container queries, your components don’t need to care about the entire screen anymore. They just care about their own box.

How do container queries work?

To use container queries, the first step is to tell CSS which parent element should act as a “container.” You do this with the container-type property. Here’s an example:

/* Define a container */
.parent {
  container-type: inline-size;
}

What this does is let the .parent element report its inline size (basically, its width) to its child elements. Once that’s set up, you can use the @container` rule inside any child elements to adjust their styles dynamically.

.child {
  /* Default styles go here */
}

@container (min-width: 40rem) {
  .child {
    /* Apply these styles when the container's width is at least 40rem */
  }
}

It’s kind of like saying, “Hey, if my box is big enough, let’s give this component a different look.”

Real-world example

Imagine you’ve got a ProfileCard component, and it needs to display a profile picture, username, and bio.

In a small container, you might want to stack everything inside the card:

But in a larger container, you’d want a more horizontal layout inside the card:

Here’s the html I used for the card:

<section class="container">
  <div class="card">
    <div class="card__avatar">
      <img src="..." alt="Avatar" width="120" height="120" />
    </div>
    <div class="card__info">...</div>
    <div class="card__actions">...</div>
  </div>
</section>

With container queries, you can define a flexible design for this. Just mark the ProfileCard’s parent as a container, and then tweak the layout using something like this:

.card {
  display: grid;
  grid-template-columns: 1fr;
  grid-template-areas:
    'avatar'
    'infos'
    'actions';
}
.card__avatar {
  grid-area: avatar;
}
.card__info {
  grid-area: infos;
}
.card__actions {
  grid-area: actions;
}

@container (min-width: 18rem) {
  .card {
    grid-template-columns: 120px 1fr;
    grid-template-areas:
      'avatar infos'
      'actions actions';
  }
}

The card will adapt to the size of its parent container, making it a lot more flexible and reusable.

Naming containers

By default, container queries target the nearest parent with a container-type set. But what if a component is nested inside multiple containers, and you want to target a specific one? That’s where container names come in.

You can give a container a custom name using the container-name property. Then, when writing your @container rules, you can refer directly to that name.

Here’s how it works:

.sidebar {
  container-type: inline-size;
  container-name: sidebar;
}

Now that this container is named, you can write a query like this:

@container sidebar (min-width: 30rem) {
  .widget {
    /* Styles specific to when the sidebar is wide enough */
  }
}

This is super helpful when your layout has multiple containers, like a card inside a sidebar inside a grid. Instead of relying on CSS to guess which container you mean, you’re telling it exactly where to look.

Quick tip: You can also set both the type and name in one line like this:

.container {
  container: sidebar / inline-size;
}

The format is: container: <name> / <type>;.

This makes your CSS more predictable and your components easier to manage, especially in more complex layouts.

New units for container queries

Container queries also introduce some exciting new units that make it easier to build responsive, context-aware designs. These units are specifically designed to work with the size of a container, not the viewport. Here are the key ones to know:

cqw (container width) - Represents the container’s width.

.child {
  width: 50cqw; /* 50% of the container's width */
}

cqh (container height) - Represents the container’s height.

.child {
  height: 20cqh; /* 20% of the container's height */
}

cqi (container inline size) - Based on the container’s inline size (width in LTR or height in top-to-bottom layouts).

.child {
  margin-left: 5cqi; /* 5% of the container’s inline size */
}

cqb (container block size) - Based on the container’s block size (height in most layouts).

.child {
  padding-bottom: 2cqb; /* 2% of the container's block size */
}

cqmin (minimum container size) - Based on the smaller of the container’s width or height.

.child {
  font-size: 2cqmin; /* 2% of the container's smallest dimension */
}

cqmax (maximum container size) - Based on the larger of the container’s width or height.

.child {
  font-size: 3cqmax; /* 3% of the container's largest dimension */
}

These units give you flexibility to create designs that react to the container’s actual size, making your components more adaptable to different contexts.

What about browser support?

Good news, container queries are here, and they’re supported by most modern browsers! As of today, support is solid for about 93,6% of users. That said, for browsers that don’t support them (looking at you, legacy setups), it’s smart to design with progressive enhancement in mind.

Here’s the strategy:

Start with a solid set of default styles that look good no matter what.
Layer container-query rules on top for enhanced layouts when supported.

Ready to Learn More?

If you’re curious to dig even deeper, Josh W. Comeau’s article, A friendly introduction to container queries, is an incredible resource with more examples and detailed explanations. Definitely worth a read!