API documentation

Search stored homepage HTML by text, boolean expression or regex. Get up to 256 verified source matches per page.

One homepage per hostname, including subdomains. Matches describe captured source; confirm current installations separately. Authenticate with a Bearer key from your account page.

curl -G "https://www.searchwebcode.com/api/v1/search" \
  -H "Authorization: Bearer $SWC_KEY" \
  --data-urlencode 'q="wp-content/plugins/woocommerce"'

Authentication

Send your key as a Bearer token on every request. Create and revoke keys on your account page. A key spends your balance, so keep it like a password. We store only a hash of it, so a lost key can't be recovered: revoke it and make a new one.

Authorization: Bearer swc_live_xxxxxxxxxxxxxxxxxxxxxxxx

Search

GETPOST /api/v1/search

Send parameters in the query string (GET) or a JSON body (POST, handy for long regex). One endpoint handles every kind of query.

Parameters

Matching rules

• A quoted phrase, or any literal with punctuation in it (wp-content/plugins, ca-pub-2380747294665477), matches wherever that byte sequence occurs in the page source.
• A bare single word, letters, digits, - and _ only, quoted or not, matches pages where it appears as a whole word (case-folded), not inside a longer one: tagmanager does not match googletagmanager, and 2380747294665477 does not match ca-pub-2380747294665477. Such responses carry match_mode: "token". For the substring reading use a regex, q=tagmanager&regex=true, or include the punctuation that pins it, ca-pub-2380747294665477.
• Boolean keywords are the uppercase AND, OR (or |) and NOT (or a leading -). Lowercase and / or / not are ordinary words: terms and conditions searches for those three words.
• Inside quotes only \" and \\ are escapes. Any other backslash is part of the literal, so "https:\/\/cdn.example.com" finds the JSON-escaped form exactly as written.

Response

{
  "query": "\"wp-content/plugins/woocommerce\"",
  "count_kind": "estimated",
  "total": 4185023,
  "estimated_matches": 4185023,
  "candidate_total": 4185023,
  "verified_hits": 96,
  "candidates_examined": 256,
  "verify_cap": 256,
  "limit_applied": 100,
  "scan_capped": true,
  "next_offset": 256,
  "match_mode": "substring",
  "elapsed_ms": 231,
  "results": [
    {
      "domain": "example.com",
      "url": "https://example.com/",
      "tld": "com",
      "rank": 48210,
      "occurrences": 3,
      "snippet": { "text": "...href='/wp-content/plugins/woocommerce/...", "match_start": 8, "match_end": 38 }
    }
  ],
  "charged_cents": 5,
  "balance_cents": 1495
}

Reading the count

The engine verifies matches against real page source, so it tells you exactly how sure it is:

candidates_examined is how many candidate pages this request actually read against their source; verify_cap is the per-request budget (256), which a request stops short of once its page is full or the candidates run out. limit_applied is the page size the request honoured: your limit, clamped to verify_cap, and 0 under counts_only, which returns no page. match_mode says how the query was matched: token for a bare word under the whole-word rule above, substring for a phrase or punctuated literal matched wherever its bytes occur, regex for regex=true. prefilter_note explains a regex that had no literal for the index to narrow on.

A counts_only response verifies nothing, so its scan_capped and next_offset mean something different from a search: scan_capped: true with next_offset equal to your own offset says the count is a candidate ceiling over a set no request has scanned yet, not a slice of one. Do not page on it; run the search. Two counts are exact even so: a query the index answers without verification, and an empty candidate set, which is count_kind:"exact" with total: 0 and never charges. A -site: exclusion is applied while verifying pages, so a count over a set the index could not narrow still includes those domains; when that happens the prefilter_note says so.

Boolean queries also return a terms array with each arm's candidate count, which is the quickest way to spot the one term that made a query broad.

Pagination and export

Results arrive in pages of up to 256 verified source matches. Pass next_offset back as offset while it is present. A credit covers the first request and 25 further requests; each further block uses another credit. See billing before paging large sets.

offset=0     ->  { ..., "next_offset": 256 }
offset=256   ->  { ..., "next_offset": 512 }
offset=...   ->  { ... }              # no next_offset means done

For anything beyond a few thousand rows there are two shortcuts: GET /api/v1/export pages for you inside one streamed request (up to 100,000 rows), and POST /api/v1/exports runs a quoted export within your chosen row and spending limits and emails the file.

Billing

Search credits. One credit covers a query and 25 additional page or scan requests during a sliding 24-hour session. Further blocks use another credit. Current plans share this block across the website and API. Proven empty results and engine failures are refunded. Viewing stored source is free.

Pay as you go and overages. Search credits cost 5¢ from prepaid credit. Research and Pro use their included credits first, then prepaid credit. We do not automatically charge your card for overages. New-account top-ups start at $10; prepaid credit does not expire. charged_cents shows a wallet debit, so 0 can mean an included subscription credit was used. balance_cents is your prepaid balance.

Request limits. The default daily search-request caps are 500 for Research and 3,000 for Pro, shared by the website and API. They reset at midnight UTC. Account limits can differ. A 429 response identifies a throughput or daily limit; follow Retry-After.

counts_only=true uses no search credit, but it is a request for rate-limit purposes. A count followed by a search is two requests. Request caps limit traffic; monthly credits and export rows determine included usage. A 402 response means the included search credits and available prepaid balance cannot cover another credit.

Legacy plans and eligibility have separate purchased terms. Check the selected plan in your account and the pricing returned by GET /api/v1/exports before estimating a bulk export.

Streaming export: search-credit billing

GET /api/v1/export streams verified source matches as CSV or JSON Lines. It accepts the search parameters plus rows (default 10,000, maximum 100,000), offset, format=jsonl and snippets=0. Each internal page is a search request: current plans use one search credit per block of 26 requests, and pages count toward request limits. This endpoint does not use the monthly bulk-export row allowance; use export jobs for that allowance.

The stream can stop at its row limit, billing or request limit, or time budget. JSON Lines ends with complete, stopped_by and next_offset; use those fields to resume. A CSV caller must check whether the intended result set was fully delivered.

curl -G "https://www.searchwebcode.com/api/v1/export" \
  -H "Authorization: Bearer $SWC_KEY" \
  --data-urlencode 'q="wp-content/plugins/woocommerce/"' \
  --data-urlencode 'rows=50000' \
  --data-urlencode 'format=jsonl' -o woocommerce.jsonl

Examples

Boolean: sites using ServiceTitan or Housecall Pro booking

curl -G "https://www.searchwebcode.com/api/v1/search" \
  -H "Authorization: Bearer $SWC_KEY" \
  --data-urlencode 'q="scheduler.servicetitan.com/book/" OR "book.housecallpro.com/book/"'

Regex: reverse Google Analytics lookup

curl -G "https://www.searchwebcode.com/api/v1/search" \
  -H "Authorization: Bearer $SWC_KEY" \
  --data-urlencode 'q=UA-\d{4,10}-\d' --data-urlencode 'regex=true'

Export jobs: monthly rows and approved overages

curl -X POST "https://www.searchwebcode.com/api/v1/exports" \
  -H "Authorization: Bearer swc_live_…" -H "Content-Type: application/json" \
  -d '{"op":"quote","q":"wp-content","mode":"fast","max_rows":100000,"max_cost_cents":200}'
# Review pricing. Repeat without "op" to confirm.
# Read progress: GET /api/v1/exports?id=…

For a larger result set within your chosen row limit, create an export job: POST /api/v1/exports with a JSON body. The job runs on our side at the engine's own pace, writes a CSV to storage, and emails you a download link that stays valid for seven days; GET /api/v1/exports?id=… shows its progress and carries the same link once the file is ready. The worker pages through the index without using search credits or the search endpoint’s daily request allowance. Current plans use monthly export rows first, then quoted prepaid credit within your approved spending ceiling.

curl -X POST "https://www.searchwebcode.com/api/v1/exports" \
  -H "Authorization: Bearer swc_live_…" -H "Content-Type: application/json" \
  -d '{"op":"quote","q":"wp-content/plugins/woocommerce/","max_rows":2000000,"max_cost_cents":4000}'

# → 200: review maximum_charge_cents and within_spend_limit.
# Included rows can reduce the quoted charge below the $40 ceiling.
# After reviewing the quote, repeat the POST without "op".
# → 202 {"id": "…", "status": "queued", "mode": "fast", ...}
curl "https://www.searchwebcode.com/api/v1/exports?id=…" -H "Authorization: Bearer …"
# → {"status": "done", "rows": 1834211, "download_url": "https://…", "url_expires_at": "…"}

Body fields. q (required; the same syntax as a search), regex, ignore_case, tld, max_rows (default 1,000,000, up to 30,000,000), max_cost_cents (a whole-number spending ceiling required under current terms), and mode: fast (default) or verified. One job runs per account at a time; DELETE /api/v1/exports?id=… cancels one and keeps what it collected.

Fast mode. The job first verifies sample windows of the candidate set against page source. When those windows match at 97% or better, the rest of the domains are taken straight from the index without opening each page, and the file says so (precision_sample in the status, and a note in the email). That is what turns a 25-million-row footprint from 100,000 paged requests into one job of a few minutes. When the sample matches less well, or the set is small, the job runs verified instead. Verified mode checks every row against page source, exactly like paging, and takes about 1.3 seconds per 256 rows.

Columns. domain, url, tld, rank. In fast mode the url is the site's homepage and rank is left empty for rows that were not fetched. Files above a million rows are gzipped.

Count a query, then request a page

# free: how big is it?
curl -G ".../api/v1/search" -H "Authorization: Bearer $SWC_KEY" \
  --data-urlencode 'q=.myshopify.com' --data-urlencode 'counts_only=true'

# then request results (search-credit rules apply; follow next_offset)
curl -G ".../api/v1/search" -H "Authorization: Bearer $SWC_KEY" \
  --data-urlencode 'q=.myshopify.com' --data-urlencode 'limit=256'

Errors

A broad query that runs past the scan window is not an error. It returns 200 with count_kind:"partial" and a next_offset so you can keep going.

Create an API key