Shopify Scraper

Scrape any Shopify store into clean JSON or CSV. Give it a domain such as allbirds.com and it returns the whole product catalog with every variant: SKU, barcode, price, compare-at ("was") price and discount, availability, option values (size, colour, ...), weight, images, tags, vendor and product type. It also lists collections and the products in each one, runs the store's own search, looks up single products with live per-variant stock (and stock counts where the store publishes them), and reads store details such as currency, country, ship-to countries and product counts. A built-in price and stock monitor compares catalog snapshots and reports new and removed products, price drops and increases, restocks and sell-outs. It's built for competitor price monitoring, product research, catalog and feed exports, and market analysis, and ships as a CLI and a Python library.

Powered by ScrapeUnblocker. Every request goes through the ScrapeUnblocker getPageSource API, so you don't have to handle blocks, rate limits or proxies yourself.

shopify-scraper store allbirds.com tentree.com -f table
shopify-scraper products allbirds.com --variants -f csv -o allbirds_variants.csv
shopify-scraper products tentree.com --on-sale --min-discount 30 --sort discount -n 20 -f table
shopify-scraper products allbirds.com --collection mens --in-stock --max-price 120 -f table
shopify-scraper collections brooklinen.com --query sheets -f table
shopify-scraper product https://www.allbirds.com/products/mens-tree-runners-mist --variants -f table
shopify-scraper search tentree.com hoodie -f table
shopify-scraper monitor tentree.com --only price_drop,back_in_stock -f table

Features

Full catalog (products): every published product of a store, 250 per request, with every variant. Each product has its ID, handle, title, vendor, product type, tags, options with their values, images, plain-text and HTML description, created/published/updated timestamps and URL, plus summaries: lowest and highest price, compare-at price, on-sale flag and discount %, in-stock flag and how many variants are in stock. Each variant has its ID, title, SKU, option values, price, compare-at price, discount, availability, weight in grams, shipping/tax flags, image and a direct ?variant= URL.
One store or many: products allbirds.com rothys.com tentree.com collects several stores in one run. Every row carries its store and currency, so the output compares directly.
Filters: vendor, product type, tags, keywords, price range, in stock or sold out, on sale, minimum discount, and created/published/updated since a date. Without a sort, paging stops as soon as --limit products matched.
Sorting: by price (either way), discount, newest, oldest, last updated, title or vendor.
Variant rows (--variants): one flat row per variant with up to three optionN_name/optionN_value columns, ready for a spreadsheet or a price-tracking database.
Collections (collections, products --collection): every collection with its handle, title, description, product count, image and dates, and the products of any one collection.
Single products (product): a product URL (or handle) with live per-variant availability, barcodes (GTIN/UPC/EAN when the store fills them in) and, on stores that publish them, stock counts per variant.
Search (search): the store's own predictive search, the results its search box shows (up to 10 products per query).
Store details (store): name, primary domain, myshopify domain, currency, money format, country/province/city, the countries it ships to, and the number of published products and collections.
Price and stock monitor (monitor): fetches the catalog, compares it with the snapshot from the last run and reports new_product, removed_product, price_drop, price_increase, back_in_stock, out_of_stock, new_variant and removed_variant, then saves the new snapshot. Identical changes across sizes are grouped into one line. Filter by change type or a minimum price change in percent. diff compares any two saved exports offline.
Robust: retries with backoff on timeouts and transient errors, handles JSON whether it comes back raw or as the page a browser shows, removes duplicates across pages, and raises a clear NotShopifyError for sites without a public Shopify catalog.
Output as JSON, JSON Lines, CSV or a terminal table; typed dataclasses in Python.

How it works

Shopify's standard online-store storefront publishes its catalog as JSON at fixed paths:

Path	What the scraper reads from it
`/products.json?limit=250&page=N`	the catalog, every variant included
`/collections.json`, `/collections/<handle>/products.json`	collections and their products
`/products/<handle>.js`	one product with live variant availability, barcodes and (on some stores) stock counts
`/meta.json`	store details: name, primary domain, currency, country, counts
`/search/suggest.json`	the store's predictive search

The scraper fetches these through ScrapeUnblocker's getPageSource endpoint, one request per page of 250 products, and stops at the first short page. Before the first catalog request it reads the store details once to learn the store's currency (stamped on every product) and its primary domain, so allbirds.com is fetched and linked as www.allbirds.com. Pass --currency USD (or none) to skip that request.

Shopify has no server-side filters on these endpoints, so filters run on each page as it arrives. The monitor stores the products as a JSON snapshot and compares variant prices and availability by ID. Shopify IDs are unique across all stores, so snapshots line up even if you change how you write the domain.

Install

git clone https://github.com/ScrapeUnblocker/shopify-scraper.git
cd shopify-scraper
python -m pip install .

Get an API key at scrapeunblocker.com and expose it as SCRAPEUNBLOCKER_KEY (see .env.example):

export SCRAPEUNBLOCKER_KEY=your_key_here        # macOS / Linux
set SCRAPEUNBLOCKER_KEY=your_key_here           # Windows cmd
$env:SCRAPEUNBLOCKER_KEY = "your_key_here"      # PowerShell

The only runtime dependency is the official scrapeunblocker SDK (pip install -r requirements.txt works too).

CLI usage

Every command takes -f json|jsonl|csv|table and -o FILE. Progress (8 products (15 variants), 2 requests) goes to stderr, so stdout stays clean for pipes. shopify-scraper --help and shopify-scraper <command> --help list every option.

Store details

$ shopify-scraper store allbirds.com tentree.com deathwishcoffee.com -f table
name               domain                   currency  country  products  collections  ships to
Allbirds           www.allbirds.com         USD       US       397       1346         US
Tentree            www.tentree.com          USD       US       1842      293          143 countries
Death Wish Coffee  www.deathwishcoffee.com  USD       US       153       126          224 countries

Products

$ shopify-scraper products deathwishcoffee.com --published-since 2026-10-01 --sort newest -n 8 -f table
vendor                title                                   type         price  was    off%  in stock  variants  handle
Silipint              Boos Cruise Silicone Cup                Accessories  8.00                1/1       1         boos-cruise-silicone-cup
Death Wish Coffee C~  Dark Spirits Chocolate Rum Ball Coffee  Coffee       29.99               3/3       3         dark-spirits-chocolate-rumball
Klean Kanteen         Dark Spirits 8oz Klean Kanteen          Drinkware    25.00               1/1       1         dark-spirits-8oz-klean-kanteen
Death Wish Coffee     Dark Spirits Klean Kanteen Bundle       Bundle       49.99  54.99  9.1   1/1       1         dark-spirits-klean-kanteen-bundle
Planet Apparel        Dark Spirits Long Sleeve                Apparel      35.00               4/6       6         dark-spirits-long-sleeve
Planet Apparel        Fired Up for Fall Matchbook             Accessories  8.00                1/1       1         fired-up-for-fall-matchbook
Death Wish Coffee     Chaos and Coffee Pet Tag Set            Accessories  10.00               1/1       1         chaos-and-coffee-pet-tag-set
Planet Apparel        Hell Hound Pet Bandana                  Accessories  10.00               1/1       1         hell-hound-bandana

$ shopify-scraper products tentree.com --collection hoodies --in-stock --max-price 70 -n 5 -f table
vendor   title                  type    price  was     off%  in stock  variants  handle
tentree  Abstract Ridge Hoodie  Womens  52.80  88.00   40    2/5       5         abstract-ridge-hoodie-forest-river-gree~
tentree  Howson Hoodie          Mens    58.80  98.00   40    3/5       5         howson-hoodie-meteorite-black-agave-gre~
tentree  Whistler 1/4 Zip       Mens    64.80  108.00  40    2/5       5         whistler-1-4-zip-midnight-blue-elkwood
tentree  Emerald Edge Hoodie    Mens    52.80  88.00   40    3/5       5         emerald-edge-hoodie-agave-green-darkest~
tentree  Whistler 1/4 Zip       Mens    54.00  108.00  50    4/5       5         whistler-1-4-zip-khaki-jasper

Option	Meaning
`-c, --collection HANDLE`	only the products of one collection (handles come from `collections`)
`--vendor`, `--type`, `--tag`	exact vendor / product type (repeat or comma-separate), required tags
`-q, --query WORDS`	words that must appear in the title, handle, vendor, type or tags
`--min-price`, `--max-price`	bounds on the lowest variant price
`--in-stock` / `--sold-out`	at least one variant available / none available
`--on-sale`, `--min-discount PCT`	below the compare-at price / at least this many percent off
`--created-since`, `--published-since`, `--updated-since DATE`	`YYYY-MM-DD`, in the store's time zone
`--sort`	`price`, `price-desc`, `discount`, `newest`, `oldest`, `updated`, `title`, `vendor`
`-n, --limit N`, `--max-pages N`	stop after N products (per store) / N pages of 250
`--variants`	one row per variant instead of one per product
`--currency`	`auto` (default: the store's currency), `none`, or a code to stamp, e.g. `EUR`

Single products

$ shopify-scraper product https://www.allbirds.com/products/mens-tree-runners-mist --variants -f table
product_title                          variant_title  sku         price  compare_at_price  discount_pct  available  inventory_quantity
Men's Tree Runner - Mist (White Sole)  8              TR3MMST080  100                                    yes        9
Men's Tree Runner - Mist (White Sole)  9              TR3MMST090  100                                    yes        106
Men's Tree Runner - Mist (White Sole)  10             TR3MMST100  100                                    no         0
Men's Tree Runner - Mist (White Sole)  11             TR3MMST110  100                                    no         0
Men's Tree Runner - Mist (White Sole)  12             TR3MMST120  100                                    no         0

Several URLs work in one call. A bare handle needs the store: shopify-scraper product mens-tree-runners-mist --store allbirds.com.

Collections and search

$ shopify-scraper collections brooklinen.com --query sheets -n 4 -f table
handle                    title                   products  updated
sheets-bedding            All Bedding             135       2026-10-25
all-sheets-2-0            All Sheets 2.0          198       2026-10-25
brushed-flannel-sheets-1  Brushed Flannel Sheets  7         2026-10-25
classic-percale-sheets-1  Classic Percale Sheets  19        2026-10-25

$ shopify-scraper search tentree.com hoodie -n 4 -f table
handle                                    title               vendor   price  available
m-juniper-zip-hoodie                      Juniper Zip Hoodie  tentree  98.00  yes
mens-treefleece-reynard-hoodie-meteorit~  Reynard Hoodie      tentree  88.00  yes
sasquatch-hoodie-crimson-heather-dog-wa~  Sasquatch Hoodie    tentree  52.80  yes
juniper-hoodie-pine-green-heather-white~  Juniper Hoodie      tentree  88.00  yes

Price and stock monitor

shopify-scraper monitor tentree.com                     # first run: saves snapshots/tentree.com.json
shopify-scraper monitor tentree.com -f table            # later runs: what changed since
shopify-scraper monitor tentree.com --only price_drop,back_in_stock --min-change 10 -f json -o changes.json
shopify-scraper diff monday.json friday.json -f table   # any two `products -f json` exports, no requests

Run it from cron or a scheduled CI job. Here is the output format, from a run against a snapshot that was edited by hand so that every kind of change shows up:

$ shopify-scraper monitor deathwishcoffee.com -f table
change           product                                 variants             old     new     %
new_product      Fired Up for Fall Matchbook                                          8
removed_product  Old Gone Product                                             8
price_drop       Dark Spirits Chocolate Rum Ball Coffee  Whole Bean / 1 bag   34.99   29.99   -14.3
price_drop       Dark Spirits Chocolate Rum Ball Coffee  Whole Bean / 2 bags  64.98   59.98   -7.7
price_drop       Dark Spirits Chocolate Rum Ball Coffee  Whole Bean / 5 bags  154.95  149.95  -3.2
back_in_stock    Boos Cruise Silicone Cup                Default Title        no      yes

The monitor refuses to overwrite a snapshot with an empty catalog, so a store that is briefly down doesn't show up as "everything removed" on the next run.

Library usage

from shopify_scraper import ProductFilter, ShopifyScraper

shop = ShopifyScraper()  # reads SCRAPEUNBLOCKER_KEY; or ShopifyScraper(api_key="...")

# the whole catalog
products = shop.products("allbirds.com")
print(len(products), products[0].title, products[0].price, products[0].currency)

# filtered and sorted (keywords or a ProductFilter)
deals = shop.products(
    "tentree.com", on_sale=True, available=True, min_discount_pct=30, sort="discount"
)
for p in deals[:5]:
    print(f"-{p.discount_pct}%  {p.price} (was {p.compare_at_price})  {p.title}")

new_coffee = shop.products(
    "deathwishcoffee.com",
    where=ProductFilter(product_types=["Coffee"], published_since="2026-10-01"),
)

# every variant as a flat row
rows = [row for p in products for row in p.variant_rows()]

# one product, with live stock per size
runner = shop.product("https://www.allbirds.com/products/mens-tree-runners-mist")
print([(v.options.get("Size"), v.available, v.inventory_quantity) for v in runner.variants])

# collections, search, store details
mens = shop.products("allbirds.com", collection="mens")
handles = [c.handle for c in shop.collections("brooklinen.com")]
hits = shop.search("tentree.com", "hoodie")
store = shop.store("allbirds.com")  # store.currency, store.products_count, store.ships_to_countries

# price and stock monitoring
result = shop.monitor(
    "tentree.com", "snapshots/tentree.json", types=["price_drop", "back_in_stock"]
)
for change in result.changes:
    print(
        change.type, change.title, change.variants, change.old, "->", change.new, change.change_pct
    )

print(shop.requests, "requests")

products() returns a list; iter_products() yields products page by page for very large catalogs. diff_catalogs(old, new) compares any two lists of products (objects or their to_dict() form), and save_snapshot() / load_snapshot() read and write them.

Example output

One product from shopify-scraper products deathwishcoffee.com (description, images and variants shortened):

{
  "id": 192924352525,
  "handle": "dark-spirits-chocolate-rumball",
  "title": "Dark Spirits Chocolate Rum Ball Coffee",
  "vendor": "Death Wish Coffee Company",
  "product_type": "Coffee",
  "url": "https://www.deathwishcoffee.com/products/dark-spirits-chocolate-rumball",
  "store": "www.deathwishcoffee.com",
  "currency": "USD",
  "price": 29.99,
  "price_max": 149.95,
  "compare_at_price": null,
  "on_sale": false,
  "discount_pct": null,
  "available": true,
  "variant_count": 3,
  "available_variants": 3,
  "tags": ["Bagged", "Dark Spirits", "LTO Coffee", "Society of Strong Coffee"],
  "options": [
    { "name": "Type", "values": ["Whole Bean"] },
    { "name": "Size", "values": ["1 bag", "2 bags", "5 bags"] }
  ],
  "created_at": "2017-12-06T11:03:33-05:00",
  "published_at": "2026-10-21T11:33:24-04:00",
  "updated_at": "2026-10-25T18:57:31-04:00",
  "image": "https://cdn.shopify.com/s/files/1/0271/7209/files/Dark_Spirits_Chocolate_Rum_Ball_A1-_1_OE.jpg?v=1792760232",
  "images": ["https://cdn.shopify.com/s/files/1/0271/7209/files/Dark_Spirits_Chocolate_Rum_Ball_A1-_1_OE.jpg?v=1792760232", "..."],
  "description": "Awaken the spirits within.\nAged in oak rum barrels and cast into the flames to awaken the spirit within: Meet Barrel-Aged Chocolate Rum Ball. ...",
  "description_html": "<h4>Awaken the spirits within.<br>\n</h4>\n<p>Aged in oak rum barrels ...",
  "variants": [
    {
      "id": 39983629828151,
      "product_id": 192924352525,
      "title": "Whole Bean / 1 bag",
      "sku": "LWCRMB08",
      "options": { "Type": "Whole Bean", "Size": "1 bag" },
      "price": 29.99,
      "compare_at_price": null,
      "on_sale": false,
      "discount_pct": null,
      "available": true,
      "url": "https://www.deathwishcoffee.com/products/dark-spirits-chocolate-rumball?variant=39983629828151",
      "barcode": null,
      "inventory_quantity": null,
      "grams": 255,
      "requires_shipping": true,
      "taxable": true,
      "position": 1,
      "image": null,
      "created_at": "2022-10-18T12:03:44-04:00",
      "updated_at": "2026-10-25T18:57:31-04:00"
    }
  ]
}

Fields and units

Prices are numbers in the store's own currency (currency). price on a product is its lowest variant price; compare_at_price is the store's "was" price, set only when it is above the current price; discount_pct is how far below it the price is (40 = 40% off).
available is whether a variant can be bought now; on a product it means at least one variant can. inventory_quantity is only filled by product lookups, and only on stores that publish stock counts.
barcode is only filled by product lookups (the catalog endpoint doesn't include it). grams is the shipping weight.
Timestamps are ISO 8601 with the store's UTC offset, exactly as Shopify reports them. Date filters compare the date part.
A variant's options maps option names to values ({"Size": "9", "Color": "Black"}). Products without options have one variant called Default Title and empty options.
description is plain text (one line per paragraph or list item); description_html is the original HTML.

Examples

examples/export_catalog.py: export a store's whole catalog to a variant-level CSV and print a short summary (price range, stock, top product types).
examples/price_watch.py: watch several stores and print price drops, restocks and new products since the last run.
examples/sale_finder.py: find the deepest in-stock discounts across several stores and save them as JSON.

Notes

Which stores work: stores that run Shopify's standard online-store storefront, which is most Shopify stores. Stores with a custom (headless) storefront, or that turn the JSON endpoints off, have no public catalog at these paths; the scraper raises NotShopifyError for them (and for sites that aren't on Shopify at all). Password-protected stores aren't supported.
Prices are the store's base prices in its own currency. Stores selling in several currencies may show visitors converted or market-specific prices that these endpoints don't include.
Free gift-with-purchase items are often listed at 0.00 with a compare-at price, so they show up as 100% off. Add --min-price 0.01 to skip them.
Each page of up to 250 products is one request, so a 2,000-product store takes 8 requests (plus one for the store details).
Catalog data belongs to each store. Respect the store's terms, keep request volumes reasonable and reuse snapshots instead of re-fetching. This project is not affiliated with Shopify.

Project layout

src/shopify_scraper/
  __init__.py   package exports and version
  scraper.py    ShopifyScraper: fetching, retries, paging, currency lookup, monitor
  parsing.py    catalog and theme product shapes, collections, store details, search
  models.py     dataclasses for products, variants, collections, stores, search hits, changes
  filters.py    ProductFilter and sorting
  diff.py       snapshot comparison and snapshot files
  urls.py       store domains, product handles and the storefront JSON endpoints
  values.py     money, image, tag and HTML-to-text helpers
  export.py     JSON / JSONL / CSV / table output
  cli.py        the shopify-scraper command
examples/       runnable scripts (catalog CSV, price watch, sale finder)
tests/          offline tests with real, trimmed storefront responses (no API calls)

Development

python -m pip install -e ".[dev]"
make lint     # ruff check + format check
make test     # pytest, fully offline: the API client is mocked

pre-commit install runs ruff on every commit (see .pre-commit-config.yaml).

Links

ScrapeUnblocker: web scraping API that gets past blocks and CAPTCHAs
ScrapeUnblocker documentation
Python SDK

Name		Name	Last commit message	Last commit date
.github/workflows		.github/workflows
examples		examples
src/shopify_scraper		src/shopify_scraper
tests		tests
.env.example		.env.example
.gitignore		.gitignore
.pre-commit-config.yaml		.pre-commit-config.yaml
LICENSE		LICENSE
Makefile		Makefile
README.md		README.md
pyproject.toml		pyproject.toml
requirements.txt		requirements.txt

Navigation Menu

Repository navigation

About

Repository files navigation

Shopify Scraper

Features

How it works

Install

CLI usage

Store details

Products

Single products

Collections and search

Price and stock monitor

Library usage

Example output

Fields and units

Examples

Notes

Project layout

Development

Links

License

About

Releases

Packages

Contributors

Languages

Latest commit

History

Folders and files

Repository files navigation

Shopify Scraper

Features

How it works

Install

CLI usage

Store details

Products

Single products

Collections and search

Price and stock monitor

Library usage

Example output

Fields and units

Examples

Notes

Project layout

Development

Links

License

About

Topics

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages