TikTok Scraper in Python: Profiles, Videos and the Walls You'll Hit (2026)

Mon, 12 Oct 2026 10:59:19 +0000

"How do I scrape TikTok?" usually means one of three things: follower counts for a list of creators, the latest videos from an account with their views and likes, or tracking how those numbers change over time. This guide builds that in plain Python, shows what works without a browser, and explains where the DIY approach stops working.

What you can get without logging in

TikTok's web app is a React app, but the first HTML response for a profile page (https://www.tiktok.com/@handle) contains a big JSON blob in a script tag:

<script id="__UNIVERSAL_DATA_FOR_REHYDRATION__" type="application/json">{...}</script>

Inside it, __DEFAULT_SCOPE__["webapp.user-detail"]["userInfo"] holds the profile (user) and its counters (stats). That's public data, and you can read it with requests and the standard json module. You don't need BeautifulSoup.

The video list is different. The grid loads from an internal endpoint (/api/post/item_list/) that needs signed query parameters (msToken, X-Bogus, X-Gnarly, and so on) produced by obfuscated JavaScript. The signing changes often, so we'll treat that part as a known limit and come back to it later.

Setup

python -m venv .venv && source .venv/bin/activate
pip install requests

Step 1: Fetch a profile

import json
import re
import time
import requests

SESSION = requests.Session()
SESSION.headers.update({
    "User-Agent": (
        "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 "
        "(KHTML, like Gecko) Chrome/129.0 Safari/537.36"
    ),
    "Accept-Language": "en-US,en;q=0.9",
})

DATA_RE = re.compile(
    r'<script id="__UNIVERSAL_DATA_FOR_REHYDRATION__"[^>]*>(.*?)</script>',
    re.S,
)

class ProfileError(Exception):
    pass

def fetch_profile(handle: str) -> dict:
    handle = handle.lstrip("@")
    resp = SESSION.get(f"https://www.tiktok.com/@{handle}", timeout=20)
    resp.raise_for_status()

    match = DATA_RE.search(resp.text)
    if not match:
        raise ProfileError("rehydration blob missing (blocked or layout changed)")

    data = json.loads(match.group(1))
    detail = data.get("__DEFAULT_SCOPE__", {}).get("webapp.user-detail", {})
    if detail.get("statusCode", 0) != 0:
        raise ProfileError(f"TikTok statusCode {detail.get('statusCode')}")

    info = detail["userInfo"]
    user, stats = info["user"], info["stats"]
    return {
        "handle": user["uniqueId"],
        "id": user["id"],
        "sec_uid": user["secUid"],
        "nickname": user.get("nickname"),
        "verified": user.get("verified", False),
        "private": user.get("privateAccount", False),
        "bio": user.get("signature", ""),
        "followers": stats.get("followerCount"),
        "following": stats.get("followingCount"),
        "likes": stats.get("heartCount") or stats.get("heart"),
        "videos": stats.get("videoCount"),
    }

if __name__ == "__main__":
    print(fetch_profile("@tiktok"))

A few details that will save you some debugging:

Check statusCode, not just the HTTP status. TikTok returns HTTP 200 for missing or banned accounts. The blob then carries a non-zero statusCode (10221 or 10202 are common) and no userInfo.
Keep sec_uid. Handles can change, but the numeric id and secUid don't. Use them as your primary key.
Don't parse the visible "1.2M". The JSON holds exact integers, so use those.

Step 2: Store snapshots in SQLite

A single scrape is a fact. A series of scrapes shows a trend. Store every run as a row, so growth is just a SELECT:

import sqlite3
from datetime import datetime, timezone

def init_db(path="tiktok.db"):
    db = sqlite3.connect(path)
    db.execute("""
        CREATE TABLE IF NOT EXISTS profile_snapshots (
            user_id   TEXT NOT NULL,
            handle    TEXT NOT NULL,
            taken_at  TEXT NOT NULL,
            followers INTEGER,
            likes     INTEGER,
            videos    INTEGER,
            PRIMARY KEY (user_id, taken_at)
        )
    """)
    return db

def save_snapshot(db, p):
    db.execute(
        "INSERT OR IGNORE INTO profile_snapshots VALUES (?,?,?,?,?,?)",
        (p["id"], p["handle"],
         datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:00Z"),
         p["followers"], p["likes"], p["videos"]),
    )
    db.commit()

The taken_at value is rounded to the hour, so if a cron job accidentally runs twice in an hour, INSERT OR IGNORE drops the duplicate.

Step 3: Loop over many creators politely

import random

def run(handles, db):
    for h in handles:
        for attempt in range(3):
            try:
                save_snapshot(db, fetch_profile(h))
                print("ok", h)
                break
            except ProfileError as e:
                print("skip", h, e)
                break
            except requests.RequestException as e:
                wait = 2 ** attempt * 5
                print("retry", h, e, f"in {wait}s")
                time.sleep(wait)
        time.sleep(random.uniform(3, 7))

if __name__ == "__main__":
    db = init_db()
    run(["tiktok", "nasa", "duolingo"], db)

Then the growth query looks like this:

SELECT handle,
       MAX(followers) - MIN(followers) AS gained,
       COUNT(*) AS samples
FROM profile_snapshots
WHERE taken_at >= strftime('%Y-%m-%dT00:00Z', 'now', '-7 days')
GROUP BY handle
ORDER BY gained DESC;

The walls you'll hit

This is where most TikTok tutorials stop being honest. Here's what actually happens once the script runs on a schedule:

Datacenter IPs get a different page. From a cloud VM you will often get a page with no rehydration blob, a "verify" screen, or a redirect. It works from a home connection and fails on a server. Residential proxies help, but they cost money and you have to rotate them.
The blob moves. In the last few years the data has lived in SIGI_STATE, then in __UNIVERSAL_DATA_FOR_REHYDRATION__, with different key paths each time. Wrap the parsing in a single function (as above) and alert when it raises, so a layout change doesn't silently produce empty rows.
Videos need signatures. The profile HTML sometimes includes a few items, but often none. Full pagination goes through item_list with signed parameters. The DIY options are running a headless browser (Playwright) and capturing the XHR responses while scrolling, or reverse-engineering the signer. Both work, both are slow or fragile, and both break when TikTok ships a new bundle.
Region matters. The same handle can return different counts or videos depending on the exit country. Pin your proxy region if you compare numbers over time.
Rate limits are soft. You won't get a clean 429. Instead, responses quietly degrade to empty blobs. Track the empty-response rate as a health metric.
Respect the data. Only collect public data, store what you actually need, and check TikTok's terms and your local privacy rules (GDPR applies to creator data in the EU).

When DIY stops being worth it

If you only need follower counts for a few dozen accounts, the script above, a cheap VPS, and maybe a proxy are enough. Once you need full video history with views, likes and comments, the signing problem eats your time.

That's the point where I switch to an API. For example, ScrapeAtlas* has a /v1/tiktok/profile/videos endpoint that takes a handle, a sort order (latest, popular, oldest), an optional since date, and a max_cursor for pagination. It returns the same itemList shape TikTok uses internally:

import os
import requests

API = "https://api.scrapeatlas.com/v1/tiktok/profile/videos"
HEADERS = {"x-api-key": os.environ["SCRAPEATLAS_API_KEY"]}

def all_videos(handle, since=None):
    params = {"handle": handle, "sort_by": "latest"}
    if since:
        params["since"] = since          # ISO 8601 or Unix seconds
    while True:
        r = requests.get(API, headers=HEADERS, params=params, timeout=60)
        r.raise_for_status()
        page = r.json()
        for v in page.get("itemList", []):
            s = v.get("stats", {})
            yield {
                "id": v["id"],
                "created": v["createTime"],
                "desc": v.get("desc", ""),
                "views": s.get("playCount"),
                "likes": s.get("diggCount"),
                "comments": s.get("commentCount"),
                "shares": s.get("shareCount"),
            }
        if not page.get("hasMore"):
            break
        params["max_cursor"] = page["cursor"]

for v in all_videos("nasa", since="2026-09-01"):
    print(v)

Since the fields match TikTok's own stats object, the SQLite schema above only needs a video_snapshots table next to profile_snapshots, and the rest of your pipeline stays the same.

Wrap-up

Profiles: requests plus the __UNIVERSAL_DATA_FOR_REHYDRATION__ blob, keyed on id/secUid, with statusCode checks.
Store hourly snapshots in SQLite and compute growth with SQL.
Expect IP blocking, layout changes and signed video endpoints. Decide early whether you'll maintain a browser or signer, or hand that part off.

If you've found a more stable way to get the video list, let me know in the comments.

*I build it.

tiktok pixel tracking

Sun, 04 Oct 2026 11:25:10 +0000

Here is a text-only summary and technical breakdown designed for a post on Coder Legion.

Enterprise-Grade Adobe Tags Extension for TikTok Pixel & Server-Side Events API

Modern digital analytics and ad tracking face massive conversion data loss due to ad-blockers, browser privacy features like Safari ITP, and network instability. Relying solely on standard client-side tracking tags leads to incomplete dataset reporting, broken attribution models, and inefficient marketing spend.

To solve this challenge, we built an open-source extension for Adobe Experience Platform (AEP) Tags (formerly Launch) that merges client-side pixel tracking with server-side conversion delivery.

Key Architectural Pillars

Dual-Track Execution: Captures events directly in the browser via TikTok's client-side library while simultaneously firing a parallel server-side payload to TikTok's Business Events API endpoint using asynchronous network calls.
Cross-Channel Event Deduplication: Every event invocation automatically resolves or generates a shared RFC 4122 v4 UUID event identifier. This matching identifier is attached to both the client-side call and the server-side payload, allowing TikTok’s processing engine to merge redundant hits seamlessly and avoid double-counting conversions.
In-Browser PII Normalization & Hashing: Privacy regulations and API specifications require that personally identifiable information (PII) such as email addresses, phone numbers, and external user IDs are hashed before transmission. The extension normalizes user input (stripping whitespace and forcing lowercase) and executes SHA-256 hashing directly inside the client engine using the browser's native Web Crypto API, backed by an optimized JavaScript fallback for legacy environments.
Single Page Application (SPA) Resilience: Modern JavaScript frameworks do not reload the page on route changes. The extension monkey-patches the browser's native History API routes (pushState and replaceState) and attaches popstate listeners to automatically capture virtual pageview events without requiring manual custom code triggers for every page navigation.
Fault-Tolerant Execution Pipeline: Unhandled tracking exceptions should never halt other analytics or tag management rule executions. All client-side calls are wrapped in robust error handling, script loading is bound to a strict 5-second timeout, and server-side API requests enforce a 10-second request timeout to guarantee non-blocking promise resolution.

Project Package Structure Overview

The extension is structured according to the official Adobe Extension Development Kit (SDK) specifications:

Manifest Configuration: Maps all underlying JavaScript modules, action definitions, and user interface views for global configuration and rule building.
Core Action Libraries: Separate execution files handling script injection, standard event processing, custom event parsing, user identification calls, and manual SPA pageview triggers.
Data Element Resolvers: Individual interface files allowing tag managers to map values dynamically from their existing Adobe data layer.
Utility Modules: Lightweight functions handling client-side SHA-256 hashing, UUID v4 generation, SPA URL listeners, and console/badge debugging helpers.
User Interface Views: HTML and JavaScript extension views leveraging Adobe's Extension Bridge script to capture configuration settings and provide data element pickers.
Automated Test Suites: Comprehensive unit testing using Jest to validate data resolution, PII hashing correctness, API payload formatting, and timeout fallbacks.

How You Can Contribute & Support

Maintaining enterprise tracking software requires continuous updates to support changing privacy standards, browser updates, and API version upgrades.

We are inviting developers, martech engineers, and analytics specialists to participate in the project. You can help by testing the codebase in your tag management sandboxes, contributing bug fixes or feature requests on GitHub, or supporting the project through GitHub Sponsors to help fund our upcoming Adobe Exchange Marketplace certification and server-side Edge Network implementation.

Coder Legion - Recent posts tagged tiktok