How to Scrape Threads with Node.js: Profiles, Posts, Replies, and an Engagement Tracker (2026)

Wed, 21 Oct 2026 09:05:56 +0000

Threads is an awkward target. The web app is a React client from Meta, most of the interesting data loads through internal GraphQL calls, and a logged-out visitor sees only a slice of what a logged-in one does. Still, a lot of public data ships inside the first HTML response, and for a modest job like "track likes and replies on one account's recent posts" that slice is enough.

This tutorial builds a small threads scraper in Node.js 18+ with nothing but native fetch. It reads a public profile, pulls its recent posts with like, reply, repost, and quote counts, and appends a timestamped snapshot to a JSON Lines file every time it runs. At the end there's a short report script that shows which posts gained the most engagement between runs.

I'll also be upfront about where this breaks, because on Threads it breaks often.

What a logged-out visitor can reach

Short version, as of writing:

Profile pages (https://www.threads.com/@handle): name, bio, follower count, verification, and the first batch of recent posts.
Single post pages (https://www.threads.com/@handle/post/<code>): the post itself and usually a first batch of replies.
Search, feeds, follower lists, "load more": these either show a login prompt or depend on GraphQL calls that need session tokens. Treat them as off-limits for a no-login scraper.

Threads moved its main domain from threads.net to threads.com in 2025. Old threads.net links redirect, so let fetch follow redirects and normalize URLs to threads.com in your own storage so the same post doesn't get two keys.

Where the data actually lives

If you open the page source of a profile, you won't find the posts as readable HTML. What you will find are many <script type="application/json"> tags, several marked with a data-sjs attribute. These are server-side preloaded data for Meta's Relay client, and some of them contain the same post objects the app renders: code, taken_at, like_count, caption.text, and a text_post_app_info object with direct_reply_count, repost_count, and quote_count.

The structure is deeply nested and changes without notice, so don't hard-code a path like result.data.mediaData.edges[0].... A sturdier approach is to parse every JSON script and walk the whole tree looking for anything that looks like a post. That costs a few milliseconds and survives most reshuffles.

Setup

mkdir threads-scraper && cd threads-scraper
npm init -y
npm pkg set type=module

No dependencies. Node 18 or newer gives you global fetch, and everything else comes from the standard library.

Step 1: Fetch a page politely

// fetchPage.js
const UA =
  "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 " +
  "(KHTML, like Gecko) Chrome/129.0 Safari/537.36";

const sleep = (ms) => new Promise((r) => setTimeout(r, ms));

export async function fetchPage(url, { retries = 3 } = {}) {
  for (let attempt = 0; attempt <= retries; attempt++) {
    const res = await fetch(url, {
      headers: {
        "user-agent": UA,
        "accept": "text/html,application/xhtml+xml",
        "accept-language": "en-US,en;q=0.9",
      },
      redirect: "follow",
    });

    if (res.ok) {
      const html = await res.text();
      // A login redirect sometimes comes back as 200 with a tiny shell page.
      if (res.url.includes("/login") || html.length < 20_000) {
        throw new Error(`Looks like a login wall for ${url} (final URL ${res.url})`);
      }
      return html;
    }

    if (res.status === 429 || res.status >= 500) {
      const wait = 2 ** attempt * 5_000 + Math.random() * 2_000;
      console.warn(`HTTP ${res.status} on ${url}, retrying in ${Math.round(wait / 1000)}s`);
      await sleep(wait);
      continue;
    }
    throw new Error(`HTTP ${res.status} for ${url}`);
  }
  throw new Error(`Gave up on ${url} after ${retries + 1} attempts`);
}

The 20,000-character threshold is a heuristic, not a documented rule. A real profile page is far larger than that, while the shell you get when Meta wants you to sign in is much smaller. Tune it if you see false positives.

Step 2: Pull the embedded JSON out of the HTML

// extract.js
const SCRIPT_RE = /<script type="application\/json"[^>]*>([\s\S]*?)<\/script>/g;

export function jsonBlobs(html) {
  const blobs = [];
  for (const m of html.matchAll(SCRIPT_RE)) {
    try {
      blobs.push(JSON.parse(m[1]));
    } catch {
      // Some blobs are not valid standalone JSON; skip them.
    }
  }
  return blobs;
}

// Walk any JSON tree and collect objects that look like Threads posts.
export function findPosts(root) {
  const found = new Map();
  const stack = [root];
  while (stack.length) {
    const node = stack.pop();
    if (!node || typeof node !== "object") continue;

    if (
      typeof node.code === "string" &&
      typeof node.taken_at === "number" &&
      "like_count" in node &&
      node.user?.username
    ) {
      found.set(node.code, node);
    }
    for (const v of Array.isArray(node) ? node : Object.values(node)) {
      if (v && typeof v === "object") stack.push(v);
    }
  }
  return [...found.values()];
}

// Same idea for the profile header.
export function findProfile(root, handle) {
  const stack = [root];
  while (stack.length) {
    const node = stack.pop();
    if (!node || typeof node !== "object") continue;
    if (node.username === handle && "follower_count" in node) return node;
    for (const v of Array.isArray(node) ? node : Object.values(node)) {
      if (v && typeof v === "object") stack.push(v);
    }
  }
  return null;
}

The Map keyed by code matters: the same post often appears in more than one blob, and you don't want to count it twice. Using an explicit stack instead of recursion avoids blowing the call stack on Meta's very deep payloads.

Step 3: Normalize a post

Raw post objects carry dozens of fields you don't need. Flatten them into something you'd actually store:

// normalize.js
export function normalizePost(p) {
  const info = p.text_post_app_info ?? {};
  return {
    code: p.code,
    url: `https://www.threads.com/@${p.user.username}/post/${p.code}`,
    author: p.user.username,
    postedAt: new Date(p.taken_at * 1000).toISOString(),
    text: p.caption?.text ?? "",
    likes: p.like_count ?? null,
    replies: info.direct_reply_count ?? null,
    reposts: info.repost_count ?? null,
    quotes: info.quote_count ?? null,
    isReply: Boolean(info.is_reply),
  };
}

Two caveats. First, authors can hide like counts, so like_count may be missing or zero even on a popular post; store null rather than guessing. Second, the profile page mixes in the author's own continuation posts (a multi-part thread shows up as a post plus replies to itself). isReply lets you filter those out if you only want top-level posts.

Step 4: The threads scraper core (a profile)

// profile.js
import { fetchPage } from "./fetchPage.js";
import { jsonBlobs, findPosts, findProfile } from "./extract.js";
import { normalizePost } from "./normalize.js";

export async function scrapeProfile(handle) {
  handle = handle.replace(/^@/, "");
  const html = await fetchPage(`https://www.threads.com/@${handle}`);
  const blobs = jsonBlobs(html);

  let profile = null;
  const posts = new Map();
  for (const blob of blobs) {
    profile ??= findProfile(blob, handle);
    for (const p of findPosts(blob)) {
      if (p.user.username === handle) posts.set(p.code, normalizePost(p));
    }
  }

  return {
    handle,
    followers: profile?.follower_count ?? null,
    verified: profile?.is_verified ?? null,
    bio: profile?.biography ?? null,
    posts: [...posts.values()].sort((a, b) => b.postedAt.localeCompare(a.postedAt)),
  };
}

if (import.meta.url === `file://${process.argv[1]}`) {
  const data = await scrapeProfile(process.argv[2] ?? "zuck");
  console.log(data.followers, data.posts.length);
  console.table(data.posts.slice(0, 5).map(({ code, likes, replies, reposts }) => ({ code, likes, replies, reposts })));
}

Run it with node profile.js someuser. Expect a first page of posts, roughly what you'd see before scrolling. If you get zero posts but a valid profile, Meta has probably changed where the posts are preloaded, or this request was served a stripped page. Save the HTML to disk and search it for a post code you know exists; that tells you quickly which case you're in.

Step 5: Replies on a single post

The post permalink page works the same way. The main post plus the first batch of replies are usually in the preloaded JSON:

// post.js
import { fetchPage } from "./fetchPage.js";
import { jsonBlobs, findPosts } from "./extract.js";
import { normalizePost } from "./normalize.js";

export async function scrapePost(url) {
  const clean = url.replace("threads.net", "threads.com").split(/[?#]/)[0];
  const code = clean.match(/\/post\/([^/]+)/)?.[1];
  if (!code) throw new Error(`Not a Threads post URL: ${url}`);

  const all = new Map();
  for (const blob of jsonBlobs(await fetchPage(clean))) {
    for (const p of findPosts(blob)) all.set(p.code, normalizePost(p));
  }
  const post = all.get(code) ?? null;
  all.delete(code);
  return { post, replies: [...all.values()] };
}

"Replies" here means everything else on the page that looks like a post. On some pages that includes the parent post (if you opened a reply) or related posts. If you need strict parent/child relationships, inspect the surrounding thread_items structure in the blob rather than relying on the flat list.

Why not just call the GraphQL API?

You'll see older tutorials that POST to https://www.threads.com/api/graphql with a doc_id and an lsd token copied from the page. It can work for a while, but plan for three problems:

doc_id churn. Each query is identified by a numeric document ID that changes when Meta ships a new client build. Hard-coded IDs go stale, sometimes within weeks.
Token coupling. The lsd token and related headers are tied to the page you loaded. You end up scraping the HTML anyway just to call GraphQL.
Pagination is where login walls live. "Load more posts" and deep reply threads are exactly the calls that tend to need a session. Using a logged-in account for scraping risks that account, and it's clearly against Meta's terms.

For a no-login threads scraper, the embedded JSON is the more stable source, even though you only get the first page.

Rate limits, IP blocks, and legal notes

Meta doesn't publish limits for the public web pages. From what people commonly report, a residential IP fetching a few pages a minute is fine, while datacenter IPs get challenged or redirected to login much sooner. I'd stick to these rules:

One request every few seconds, with random jitter, and back off on 429.
Cache HTML during development so you're not hitting Threads every time you tweak a parser.
Watch for soft blocks. A 200 with a login shell is the usual symptom, not a clean 403.

On the legal side: Meta's terms prohibit automated collection without permission, and courts in different countries have treated public-data scraping differently. Only collect public data, don't collect more personal data than your use case needs, respect GDPR if you store data about EU users, and talk to a lawyer before building anything commercial. This post is a tutorial, not legal advice.

The official Threads API

Meta has an official Threads API (graph.threads.net). It's the right tool when you're working with your own account: publishing, reading your posts and their insights, managing replies. Meta has since added keyword search and some public profile lookups, but they require app review, specific permissions, and an access token tied to a Threads user, and they come with their own rate limits. Check the current docs before relying on them, because the scope has been changing.

What it won't give you is an easy "give me any public account's posts" call with no app review. That gap is why people write scrapers in the first place.

Project: a Threads engagement tracker

Now the practical bit. This script snapshots a few accounts and appends one line per post per run to snapshots.jsonl. JSON Lines keeps it dependency-free; swap in SQLite (better-sqlite3, or node:sqlite on recent Node versions) when the file gets large.

// track.js
import { appendFile } from "node:fs/promises";
import { scrapeProfile } from "./profile.js";

const HANDLES = process.argv.slice(2);
if (!HANDLES.length) {
  console.error("usage: node track.js handle1 handle2 ...");
  process.exit(1);
}
const sleep = (ms) => new Promise((r) => setTimeout(r, ms));

const runAt = new Date().toISOString();
for (const handle of HANDLES) {
  try {
    const { followers, posts } = await scrapeProfile(handle);
    const lines = posts
      .filter((p) => !p.isReply)
      .map((p) => JSON.stringify({ runAt, handle, followers, ...p }));
    if (lines.length) await appendFile("snapshots.jsonl", lines.join("\n") + "\n");
    console.log(`${handle}: ${lines.length} posts, ${followers ?? "?"} followers`);
  } catch (err) {
    console.error(`${handle}: ${err.message}`);
  }
  await sleep(4_000 + Math.random() * 3_000);
}

Schedule it with cron every few hours:

0 */4 * * * cd /path/to/threads-scraper && /usr/bin/node track.js handle1 handle2 >> track.log 2>&1

And a report that compares each post's first and latest snapshot:

// report.js
import { readFile } from "node:fs/promises";

const rows = (await readFile("snapshots.jsonl", "utf8"))
  .split("\n")
  .filter(Boolean)
  .map((l) => JSON.parse(l));

const byPost = new Map();
for (const r of rows) {
  const s = byPost.get(r.code) ?? { first: r, last: r };
  if (r.runAt < s.first.runAt) s.first = r;
  if (r.runAt > s.last.runAt) s.last = r;
  byPost.set(r.code, s);
}

const delta = (a, b) => (a == null || b == null ? null : b - a);
const report = [...byPost.values()]
  .map(({ first, last }) => ({
    post: last.url,
    snapshots: `${first.runAt.slice(0, 16)} -> ${last.runAt.slice(0, 16)}`,
    likes: last.likes,
    likeGain: delta(first.likes, last.likes),
    replyGain: delta(first.replies, last.replies),
    repostGain: delta(first.reposts, last.reposts),
  }))
  .sort((a, b) => (b.likeGain ?? -1) - (a.likeGain ?? -1));

console.table(report.slice(0, 15));

One thing to keep in mind when reading this report: because you only see the first page of posts, older posts drop out of the snapshots once newer ones push them off. A post that stops appearing hasn't stopped growing; you just can't see it anymore. If you need to keep following specific posts, feed their URLs through scrapePost() on each run instead.

The managed alternative

If maintaining the parser becomes a chore, or you need more than the first page from datacenter IPs, ScrapeAtlas* has Threads endpoints that return the same post objects (like_count, taken_at, text_post_app_info.direct_reply_count, and so on). Authentication is an x-api-key header:

const atlas = (path, params) =>
  fetch(`https://api.scrapeatlas.com/v1/threads/${path}?${new URLSearchParams(params)}`, {
    headers: { "x-api-key": process.env.SCRAPEATLAS_API_KEY },
  }).then((r) => r.json());

const profile = await atlas("profile", { handle: "zuck" });
console.log(profile.username, profile.follower_count, profile.is_verified);

const feed = await atlas("user/posts", { handle: "zuck", since: "2026-10-01" });
console.log(feed.posts.map((p) => [p.url, p.like_count, p.text_post_app_info.direct_reply_count]));
console.log(feed.coverage); // e.g. { complete: false, warnings: ["posts_first_page_only"], ... }

const thread = await atlas("post", { url: "https://www.threads.com/@zuck/post/<code>" });
console.log(thread.post.like_count, thread.comments.length);

To be clear about the limits: user/posts is also first page only, which the response tells you in coverage.warnings, so the tracker's blind spot is the same. post takes include_comments=false if you only want the post, and every endpoint accepts trim=true for a compact projection. There are also user/replies, post/comments, search (keyword search with recent, start_date, and end_date), search/users, and search/tags. Search is the part you can't do logged-out on your own.

Wrapping up

Read the preloaded JSON, not the DOM. Walk every application/json script and match on post shape, not a fixed path.
Dedupe by code and normalize threads.net to threads.com.
Expect the first page only without logging in, and design your tracker around that.
Skip GraphQL doc_id calls unless you're ready to re-capture them every time Meta ships a new build.
Go slow and cache, and treat a small 200 response as a block.
For your own account, use the official Threads API.

If the extractor stops finding posts for you, leave a comment with what the page source looks like now and I'll update the post.

*I build it.

Coder Legion - Recent posts tagged threads