Shopify Store Catalog & Pricing Extractor
Extract a Shopify store's product catalog with titles, prices, compare-at prices, variants, SKUs, images and stock availability from any custom or myshopify.com domain.
Input schema
Input fields, types, required flags, descriptions, defaults, and prefills come from the published definition. Prefill examples are distinct from defaults.
| Field | Type | Required | Description |
|---|---|---|---|
mode | string | required | What to scrape. `catalog` walks the public product feed, `collection` walks specific collections, `product` fetches exact product handles, `search` uses the storefront search suggestions, `collections` lists collection metadata, and `auto` detects the type of each URL in `startUrls`. |
storeUrls | array | optional | Store domains or URLs, e.g. `deathwishcoffee.com` or `https://www.allbirds.com`. Accepts custom domains and `*.myshopify.com`. A product or collection URL is reduced to its store for catalog/collections/search modes. |
startUrls | array | optional | Store, collection or product URLs. In `auto` mode each URL selects its own job: `/products/<handle>` fetches one product, `/collections/<handle>` fetches a collection, any other store URL fetches the catalog. In the other modes these URLs add stores and, for collection/product modes, their handles. |
Show all input fields
| Field | Type | Required | Description |
|---|---|---|---|
mode | string | required | What to scrape. `catalog` walks the public product feed, `collection` walks specific collections, `product` fetches exact product handles, `search` uses the storefront search suggestions, `collections` lists collection metadata, and `auto` detects the type of each URL in `startUrls`. |
storeUrls | array | optional | Store domains or URLs, e.g. `deathwishcoffee.com` or `https://www.allbirds.com`. Accepts custom domains and `*.myshopify.com`. A product or collection URL is reduced to its store for catalog/collections/search modes. |
startUrls | array | optional | Store, collection or product URLs. In `auto` mode each URL selects its own job: `/products/<handle>` fetches one product, `/collections/<handle>` fetches a collection, any other store URL fetches the catalog. In the other modes these URLs add stores and, for collection/product modes, their handles. |
collectionHandles | array | optional | Collection handles or collection URLs, e.g. `new-arrivals`, `sale`, or `https://store.com/collections/sale`. Handles are applied to every store in `storeUrls`; a full collection URL is applied to its own store. |
productHandles | array | optional | Product handles or product URLs, e.g. `dark-roast-coffee` or `https://store.com/products/dark-roast-coffee`. Handles are applied to every store in `storeUrls`; a full product URL is applied to its own store. |
searchQuery | string | optional | In-store search term. Shopify's public suggest endpoint returns at most 10 products per store and has no pagination. |
currency | string | optional | ISO 4217 code to pin prices to, e.g. `USD`, `EUR`, `GBP`. Use `store` to use the store's base currency from `/meta.json`. Pinning is important because Shopify localizes prices by IP. |
country | string | optional | Two-letter ISO country code passed as `?country=` to the storefront, e.g. `US`, `GB`, `DE`. |
vendor | string | optional | Keep products whose vendor contains this text (case-insensitive). |
productType | string | optional | Keep products whose product type contains this text (case-insensitive). |
tag | string | optional | Keep products with at least one tag containing this text (case-insensitive). |
minPrice | number | optional | Drop products whose lowest variant price is below this value. |
maxPrice | number | optional | Drop products whose lowest variant price is above this value. |
available | string | optional | Keep all products, only products with at least one purchasable variant, or only sold-out products. |
maxItems | integer | optional | Hard cap on dataset rows across all stores. When `flattenVariants` is on, each variant row counts as one. |
maxPages | integer | optional | Page cap per product/collection feed. Each page holds up to 250 items and Shopify refuses offsets above 25,000, so the effective maximum is 100. |
includeDetails | boolean | optional | Fetch `/products/{handle}.json` for each emitted product to add barcode, weight/grams and inventory fields. Costs one extra request per product and is off by default. Product mode always uses the detail endpoint. |
flattenVariants | boolean | optional | Emit one `recordType: variant` row per variant instead of one product row with nested variants. Variant and product IDs are de-duplicated. |
proxyConfiguration | object | optional | Leave empty to fetch directly. After a 403 or 429 block the actor retries through Apify's datacenter proxy (AUTO) automatically. If you set this, your configuration is used for that retry instead of AUTO; direct requests are still tried first. |
Download the published input schema
View complete JSON
{
"description": "Scrape one or many Shopify storefronts: full catalog, collection, single product, in-store search, or the collection directory.",
"properties": {
"available": {
"default": "any",
"description": "Keep all products, only products with at least one purchasable variant, or only sold-out products.",
"editor": "select",
"enum": [
"any",
"available",
"unavailable"
],
"enumTitles": [
"Any availability",
"In stock only",
"Sold out only"
],
"prefill": "any",
"title": "Availability",
"type": "string"
},
"collectionHandles": {
"default": [],
"description": "Collection handles or collection URLs, e.g. `new-arrivals`, `sale`, or `https://store.com/collections/sale`. Handles are applied to every store in `storeUrls`; a full collection URL is applied to its own store.",
"editor": "stringList",
"prefill": [],
"title": "Collection handles (mode=collection)",
"type": "array"
},
"country": {
"description": "Two-letter ISO country code passed as `?country=` to the storefront, e.g. `US`, `GB`, `DE`.",
"editor": "textfield",
"title": "Country (optional)",
"type": "string"
},
"currency": {
"default": "store",
"description": "ISO 4217 code to pin prices to, e.g. `USD`, `EUR`, `GBP`. Use `store` to use the store's base currency from `/meta.json`. Pinning is important because Shopify localizes prices by IP.",
"editor": "textfield",
"prefill": "USD",
"title": "Currency",
"type": "string"
},
"flattenVariants": {
"default": false,
"description": "Emit one `recordType: variant` row per variant instead of one product row with nested variants. Variant and product IDs are de-duplicated.",
"prefill": false,
"title": "One row per variant",
"type": "boolean"
},
"includeDetails": {
"default": false,
"description": "Fetch `/products/{handle}.json` for each emitted product to add barcode, weight/grams and inventory fields. Costs one extra request per product and is off by default. Product mode always uses the detail endpoint.",
"prefill": false,
"title": "Include barcode, weight and inventory",
"type": "boolean"
},
"maxItems": {
"default": 1000,
"description": "Hard cap on dataset rows across all stores. When `flattenVariants` is on, each variant row counts as one.",
"maximum": 50000,
"minimum": 1,
"prefill": 3,
"title": "Max records",
"type": "integer"
},
"maxPages": {
"default": 100,
"description": "Page cap per product/collection feed. Each page holds up to 250 items and Shopify refuses offsets above 25,000, so the effective maximum is 100.",
"maximum": 100,
"minimum": 1,
"prefill": 100,
"title": "Max pages per feed",
"type": "integer"
},
"maxPrice": {
"description": "Drop products whose lowest variant price is above this value.",
"maximum": 1000000,
"minimum": 0,
"title": "Maximum price",
"type": "number"
},
"minPrice": {
"description": "Drop products whose lowest variant price is below this value.",
"maximum": 1000000,
"minimum": 0,
"title": "Minimum price",
"type": "number"
},
"mode": {
"default": "catalog",
"description": "What to scrape. `catalog` walks the public product feed, `collection` walks specific collections, `product` fetches exact product handles, `search` uses the storefront search suggestions, `collections` lists collection metadata, and `auto` detects the type of each URL in `startUrls`.",
"editor": "select",
"enum": [
"catalog",
"collection",
"product",
"search",
"collections",
"auto"
],
"enumTitles": [
"Full store catalog",
"Specific collections",
"Specific products",
"In-store search",
"Collection directory",
"Auto-detect from URLs"
],
"prefill": "catalog",
"title": "Mode",
"type": "string"
},
"productHandles": {
"default": [],
"description": "Product handles or product URLs, e.g. `dark-roast-coffee` or `https://store.com/products/dark-roast-coffee`. Handles are applied to every store in `storeUrls`; a full product URL is applied to its own store.",
"editor": "stringList",
"prefill": [],
"title": "Product handles (mode=product)",
"type": "array"
},
"productType": {
"description": "Keep products whose product type contains this text (case-insensitive).",
"editor": "textfield",
"title": "Product type contains",
"type": "string"
},
"proxyConfiguration": {
"description": "Leave empty to fetch directly. After a 403 or 429 block the actor retries through Apify's datacenter proxy (AUTO) automatically. If you set this, your configuration is used for that retry instead of AUTO; direct requests are still tried first.",
"editor": "proxy",
"title": "Proxy configuration (optional)",
"type": "object"
},
"searchQuery": {
"description": "In-store search term. Shopify's public suggest endpoint returns at most 10 products per store and has no pagination.",
"editor": "textfield",
"title": "Search query (mode=search)",
"type": "string"
},
"startUrls": {
"default": [],
"description": "Store, collection or product URLs. In `auto` mode each URL selects its own job: `/products/<handle>` fetches one product, `/collections/<handle>` fetches a collection, any other store URL fetches the catalog. In the other modes these URLs add stores and, for collection/product modes, their handles.",
"editor": "stringList",
"prefill": [],
"title": "Start URLs (mixed, optional)",
"type": "array"
},
"storeUrls": {
"description": "Store domains or URLs, e.g. `deathwishcoffee.com` or `https://www.allbirds.com`. Accepts custom domains and `*.myshopify.com`. A product or collection URL is reduced to its store for catalog/collections/search modes.",
"editor": "stringList",
"prefill": [
"https://deathwishcoffee.com"
],
"title": "Shopify store URLs",
"type": "array"
},
"tag": {
"description": "Keep products with at least one tag containing this text (case-insensitive).",
"editor": "textfield",
"title": "Tag contains",
"type": "string"
},
"vendor": {
"description": "Keep products whose vendor contains this text (case-insensitive).",
"editor": "textfield",
"title": "Vendor contains",
"type": "string"
}
},
"required": [
"mode"
],
"schemaVersion": 1,
"title": "Shopify Store Catalog & Pricing Extractor Input",
"type": "object"
}
Output schema
Published result links and storage locations. This schema is not an exhaustive contract for every field in a dataset record.
| Field | Type | Required | Description |
|---|---|---|---|
items | string | optional | Dataset containing product, variant and collection records (filtered by recordType). |
runSummary | string | optional | Key-value store record named OUTPUT with per-store OK/CAPPED/PARTIAL and error statuses. |
Download the published output schema
View complete JSON
{
"actorOutputSchemaVersion": 1,
"description": "Dataset rows for products, variants or collections, plus a per-store run summary record.",
"properties": {
"items": {
"description": "Dataset containing product, variant and collection records (filtered by recordType).",
"template": "{{links.apiDefaultDatasetUrl}}/items",
"title": "Shopify records",
"type": "string"
},
"runSummary": {
"description": "Key-value store record named OUTPUT with per-store OK/CAPPED/PARTIAL and error statuses.",
"template": "{{links.apiDefaultKeyValueStoreUrl}}/keys",
"title": "Per-store run summary",
"type": "string"
}
},
"title": "Shopify Store Catalog & Pricing Extractor Output",
"type": "object"
}
Dataset schema
Published dataset views and any declared field definitions. A view can select only a subset of the returned record fields.
Download the published dataset schema
View complete JSON
{
"actorSpecification": 1,
"views": {
"collections": {
"display": {
"component": "table",
"properties": {
"handle": {
"format": "text",
"label": "Handle"
},
"productsCount": {
"format": "number",
"label": "Reported products"
},
"storeUrl": {
"format": "link",
"label": "Store"
},
"title": {
"format": "text",
"label": "Collection"
},
"url": {
"format": "link",
"label": "Collection URL"
}
}
},
"title": "Collections",
"transformation": {
"fields": [
"recordType",
"title",
"handle",
"productsCount",
"storeUrl",
"url",
"scrapedAt"
]
}
},
"detailed": {
"display": {
"component": "table",
"properties": {
"available": {
"format": "text",
"label": "Available"
},
"currency": {
"format": "text",
"label": "Currency"
},
"price": {
"format": "number",
"label": "Price"
},
"productId": {
"format": "text",
"label": "Product ID"
},
"scrapedAt": {
"format": "text",
"label": "Scraped at"
},
"sourceUrl": {
"format": "link",
"label": "Source"
},
"storeUrl": {
"format": "link",
"label": "Store"
},
"title": {
"format": "text",
"label": "Title"
},
"url": {
"format": "link",
"label": "Product URL"
}
}
},
"title": "Detailed",
"transformation": {
"fields": [
"recordType",
"productId",
"id",
"handle",
"title",
"url",
"vendor",
"productType",
"tags",
"description",
"createdAt",
"updatedAt",
"publishedAt",
"publishedScope",
"price",
"priceMin",
"priceMax",
"compareAtPrice",
"compareAtPriceMax",
"discountPercent",
"onSale",
"currency",
"available",
"variantCount",
"availableVariantCount",
"variants",
"images",
"imageDetails",
"featuredImage",
"options",
"collectionHandle",
"store",
"storeUrl",
"source",
"sourceUrl",
"scrapedAt"
]
}
},
"overview": {
"display": {
"component": "table",
"properties": {
"available": {
"format": "text",
"label": "Available"
},
"compareAtPrice": {
"format": "number",
"label": "Compare at"
},
"currency": {
"format": "text",
"label": "Currency"
},
"price": {
"format": "number",
"label": "Price"
},
"productType": {
"format": "text",
"label": "Type"
},
"storeUrl": {
"format": "link",
"label": "Store"
},
"title": {
"format": "text",
"label": "Title"
},
"url": {
"format": "link",
"label": "Product URL"
},
"vendor": {
"format": "text",
"label": "Vendor"
}
}
},
"title": "Products",
"transformation": {
"fields": [
"recordType",
"title",
"price",
"currency",
"compareAtPrice",
"available",
"vendor",
"productType",
"storeUrl",
"url"
]
}
},
"variants": {
"display": {
"component": "table",
"properties": {
"available": {
"format": "text",
"label": "Available"
},
"barcode": {
"format": "text",
"label": "Barcode"
},
"compareAtPrice": {
"format": "number",
"label": "Compare at"
},
"inventoryQuantity": {
"format": "number",
"label": "Inventory"
},
"price": {
"format": "number",
"label": "Price"
},
"productTitle": {
"format": "text",
"label": "Product"
},
"productUrl": {
"format": "link",
"label": "Product URL"
},
"sku": {
"format": "text",
"label": "SKU"
},
"title": {
"format": "text",
"label": "Variant"
}
}
},
"title": "Variants",
"transformation": {
"fields": [
"recordType",
"productTitle",
"title",
"sku",
"price",
"compareAtPrice",
"currency",
"available",
"barcode",
"inventoryQuantity",
"storeUrl",
"productUrl"
]
}
}
}
}
Key-value schema
Published key-value store definitions, when provided by this build.
This definition is not present in the default published build. See the published README for available documentation.
Published README
Read the full published documentation
Shopify Store Catalog & Pricing Extractor
Turn any public Shopify storefront into a structured product feed. Point the actor at one store or a list of stores, choose a mode, and get prices, compare-at prices, variants, stock status, images, options, collection metadata and optional barcode/weight/inventory detail.
No Shopify account, admin API key, cookie or login is required. The actor reads the same public storefront JSON that a shopper's browser receives. Proxy setup is also optional: requests go direct first, and the actor switches to Apify's datacenter proxy automatically only after a real 403 or 429 block.
Modes
| Mode | What it does | Source endpoint |
|---|---|---|
catalog |
Full published product feed for every store | /products.json?limit=250&page=N |
collection |
Products from one or more collection handles/URLs | /collections/{handle}/products.json |
product |
Exact products by handle or product URL | /products/{handle}.json |
search |
In-store product search, up to 10 results per store | /search/suggest.json |
collections |
Collection directory with titles, handles and reported counts | /collections.json?limit=250&page=N |
auto |
Mixed startUrls; each URL selects its own job |
Detected per URL |
auto treats /products/<handle> as a product job, /collections/<handle> as a collection job, /collections as a collection-directory job, and any other store URL as a catalog job.
Input
| Field | Type | Default | Notes |
|---|---|---|---|
mode |
select | catalog |
One of the six modes above. |
storeUrls |
string list | - | Store domains or full URLs; *.myshopify.com is supported. The prefill uses deathwishcoffee.com. |
startUrls |
string list | empty | Mixed store/product/collection URLs. Used mainly by auto. |
collectionHandles |
string list | empty | Handles or collection URLs for mode=collection. |
productHandles |
string list | empty | Handles or product URLs for mode=product. |
searchQuery |
string | empty | Required for mode=search. |
currency |
string | store |
ISO 4217 code such as USD or EUR; store uses /meta.json. |
country |
string | empty | Optional ISO-2 country passed as ?country=. |
vendor |
string | empty | Case-insensitive vendor substring filter. |
productType |
string | empty | Case-insensitive product-type substring filter. |
tag |
string | empty | Keep products with a tag containing this substring. |
minPrice |
number | empty | Drop products below this lowest-variant price. |
maxPrice |
number | empty | Drop products above this lowest-variant price. |
available |
select | any |
any, available or unavailable. |
maxItems |
integer | 1000 |
1-50,000 dataset rows; variant rows count individually when flattening. |
maxPages |
integer | 100 |
1-100 pages per feed. Shopify refuses offsets above 25,000. |
includeDetails |
boolean | false |
One extra /products/{handle}.json request per product for barcode, weight and inventory. |
flattenVariants |
boolean | false |
Emit one recordType: variant row per variant instead of nested product rows. |
proxyConfiguration |
object | empty | Direct requests are always tried first. After a 403/429 the actor retries via AUTO, or via this configuration if provided. |
The smallest useful run is also the daily prefill: deathwishcoffee.com, three records, no detail enrichment.
Examples
Full catalog from one store:
{
"mode": "catalog",
"storeUrls": ["https://deathwishcoffee.com"],
"currency": "USD",
"maxItems": 100
}
Collection and sale filters:
{
"mode": "collection",
"storeUrls": ["allbirds.com"],
"collectionHandles": ["mens-sale"],
"maxPrice": 100,
"available": "available",
"flattenVariants": true
}
Exact products by URL:
{
"mode": "product",
"startUrls": [
"https://deathwishcoffee.com/products/valhalla-java",
"https://deathwishcoffee.com/products/dark-roast"
],
"includeDetails": true
}
Search a set of stores:
{
"mode": "search",
"storeUrls": ["allbirds.com", "rothys.com"],
"searchQuery": "wool runner",
"maxItems": 20
}
Mixed URLs in one run:
{
"mode": "auto",
"startUrls": [
"https://allbirds.com",
"https://allbirds.com/collections/mens",
"https://allbirds.com/products/wool-runner"
]
}
Product output
Product records use recordType: "product" and contain:
productIdand legacyid,handle,title,urlvendor,productType,tagsbodyHtml(raw) anddescription(plain text)createdAt,updatedAt,publishedAt,publishedScopeprice,priceMin,priceMax,compareAtPrice,compareAtPriceMax,discountPercent,onSale,currencyavailable,variantCount,availableVariantCountvariants: variant id, title, SKU, price, compare-at price, availability, option values and namedoptionsimages(URL list),imageDetails(URL/width/height/position),featuredImageoptions: option names and their valuesstore:{url, name, currency}, plus legacystoreUrlsource,sourceUrl,scrapedAt,recordType
With includeDetails enabled, variants can also contain barcode, weight, weightUnit, grams, inventoryQuantity, inventoryPolicy, inventoryManagement, quantityRule, requiresShipping and taxable. Inventory quantities are only emitted when the store exposes inventory_management and the value is not Shopify's 999999 untracked sentinel.
When the detail endpoint omits the available flag but publishes inventory management, availability is derived from stock: positive quantity, or inventory_policy=continue, means available. Without published inventory the flag stays omitted rather than being guessed.
Variant output
With flattenVariants: true, each variant becomes its own row with recordType: "variant", plus the parent productId, productHandle, productTitle, productUrl, vendor and productType. Product and variant IDs are de-duplicated.
Collection output
mode=collections emits recordType: "collection" rows with collectionId, handle, url, title, description, productsCount (the store's reported count), timestamps, image, store, sourceUrl and scrapedAt.
Run summary
The dataset stays clean: products, variants and collections only. A separate key-value record named OUTPUT holds the run summary with each store's status, HTTP code, pages fetched, record counts, detail failures and capped flag.
Per-store statuses:
| Status | Meaning |
|---|---|
OK |
The feed was read normally. |
CAPPED |
The page cap or Shopify's 25,000-offset ceiling was reached. |
PARTIAL |
Some pages or detail requests failed after data was collected. |
PASSWORD_PROTECTED |
The store returned 401. |
FROZEN |
The store returned 402. |
BLOCKED |
The store returned 403, including after the automatic proxy retry. |
NO_JSON |
The store returned 404 or a non-JSON response. |
RATE_LIMITED |
The store returned 429 after retries. |
If every store fails, the run fails with the per-store map instead of finishing silently with an empty dataset. If some stores succeed, the run succeeds and the summary reports the failures.
Pagination notes
- 250 products per page, the Shopify maximum.
- Short pages are not the end of the feed. The walker continues until an empty page, an error,
maxPages, or the 25,000-offset ceiling. - Products are de-duplicated by store and product ID; variants by store and variant ID.
maxItemsis a global cap across all stores and jobs.
Currency
Shopify localizes prices by IP. The actor pins ?currency= on every list, collection and product request. The default store setting reads the store's base currency from /meta.json; an explicit ISO code overrides it. Each product records the currency reported by its variants when available.
FAQ
Do I need a Shopify account or API key? No. The actor only reads public storefront JSON.
Do I need a proxy? No. Direct requests are tried first. Apify's datacenter proxy is used only after a 403 or 429.
Why does search return at most 10 products per store? That is Shopify's public suggest endpoint limit. There is no pagination on it. Use catalog or collection mode for larger feeds.
Why is the product count lower than the collection page says? Collection product endpoints expose only purchasable/published products, while products_count counts more. The reported count is kept as productsCount and is not treated as ground truth.
Why are some inventory fields missing? Inventory is a merchant setting. Stores that do not publish inventory_quantity cannot be forced to reveal it, and the untracked 999999 sentinel is never emitted as stock.
What happens on password-protected stores? Their public JSON returns 401. The actor reports PASSWORD_PROTECTED and does not retry with a proxy.
Why did a large store stop at 25,000 products? Shopify rejects page offsets above 25,000 on /products.json. Split large catalogs by collection to go deeper.
Are descriptions HTML or plain text? Both are available: bodyHtml keeps the original markup and description is a plain-text version.
Is this affiliated with Shopify? No. It is an independent third-party tool that reads publicly available storefront data.
Limits
- Password-protected stores are not scrapable through the public JSON.
- Stores that disable the JSON endpoints return
NO_JSON. - Collection membership is not exposed publicly; the actor emits collection records instead of inventing per-product membership.
- Search is capped at 10 products per store.
- Inventory counts are available only where the store publishes them.
- The product feed is limited to the newest 25,000 products per store/feed.
Python quickstart
Use a schema-derived input and the Python standard library to start a run. An Apify account and token are required for execution.
Source build rfdRrnEMOeqFiOPHE. Documentation is regenerated from the default published build; private drafts are not included.