Use Google News Keyword & Full-Text Article Scraper with Python

Search Google News by keyword with real resolved article links, headlines, sources, and dates, plus optional full article text and image extraction, filterable by date, language, and publisher.

1. Choose the data you need

Review the published README for supported collection modes, examples, and limits. The recipe below uses the current published input definition.

2. Prepare a source-based input

This JSON contains published prefills and required-field defaults. Review every value for your task before running it; it is not a promise of successful collection.

{
  "articleUrls": [],
  "country": "US",
  "dateRange": "any",
  "extractFullText": false,
  "language": "en",
  "maxFullText": 100,
  "maxItems": 3,
  "maxResultsPerQuery": 3,
  "mode": "keyword",
  "newsUrls": [],
  "proxyConfiguration": {
    "useApifyProxy": false
  },
  "queries": [
    "artificial intelligence"
  ],
  "rssUrls": [],
  "topics": []
}

Download input.example.json and save your reviewed copy as input.json.

Field	Type	Required	Description
`mode`	string	required	What to scrape. Each mode reads its own list field below.
`queries`	array	optional	One or more keyword queries. Google operators such as site:, intitle: and OR are supported. Results are deduplicated across queries.
`topics`	array	optional	Topic names (WORLD, NATION, BUSINESS, TECHNOLOGY, ENTERTAINMENT, SPORTS, SCIENCE, HEALTH), TOP for top stories, Google News topic hashes/section URLs, or topic URLs. Topic feeds ignore server-side date filters; results are filtered after download.
`newsUrls`	array	optional	Google News search URLs, topic URLs, RSS search/topic URLs, or RSS/Atom feed URLs to convert into a feed.
`rssUrls`	array	optional	Any public RSS or Atom feed. Google News links inside the feed are resolved to publisher URLs automatically.
`articleUrls`	array	optional	Publisher article URLs or Google News article URLs. Every page is fetched and full text is extracted, with a per-record status when a publisher blocks or paywalls it.
`language`	string	optional	Language of the Google News edition. Combined with country in the edition token (for example US:en, DE:de, JP:ja).
`country`	string	optional	Country edition for Google News. The actor always pins the edition so results do not change with the machine's IP.
`dateRange`	string	optional	Recency window. Every item is post-filtered by its real publication time, so items newer than the window never leak. 'custom' uses startDate/endDate. Multi-day ranges can be walked day by day to collect more than one feed page.
`startDate`	string	optional	Custom range start (YYYY-MM-DD, inclusive). Also used when dateRange=custom.
`endDate`	string	optional	Custom range end (YYYY-MM-DD, inclusive). Items published after the end are dropped client-side.
`maxResultsPerQuery`	integer	optional	Maximum records taken from each Google News feed request (1-100, Google's own cap is about 100). When a date range spans several days, the actor walks day windows and can collect more than 100 for one query, up to maxItems.
`maxItems`	integer	optional	Hard cap on unique records emitted by the run (1-5000).
`extractFullText`	boolean	optional	Fetch each publisher page and extract its article body, description, author, and image. Default off for fast metadata-only runs. The articleUrls mode always extracts.
`maxFullText`	integer	optional	Maximum pages fetched for full text per run (1-500). Records beyond this cap are still returned with fullTextStatus=skipped_cap.
`proxyConfiguration`	object	optional	Optional. No proxy is needed by default; Google News RSS and publisher pages work without one. Residential proxies can make Google rate-limit more, so leave this off unless you need it.

3. Start an Actor run

Set APIFY_TOKEN in your environment. Python 3.10 or newer is sufficient; this recipe uses no third-party packages.

Execution may incur charges. Review current pricing on Apify. The request asks for a 120-second timeout and a charge limit; these settings are subject to Apify’s pricing and do not guarantee an all-inclusive spending cap.

import json
import os
import urllib.request

# Save the schema-derived example as input.json; review and edit it first.
with open("input.json", encoding="utf-8") as stream:
    actor_input = json.load(stream)

token = os.environ["APIFY_TOKEN"]
request = urllib.request.Request(
    "https://api.apify.com/v2/acts/7LV8qekDbhuvsPtdv/runs?timeout=120&maxTotalChargeUsd=0.50",
    data=json.dumps(actor_input).encode("utf-8"),
    headers={"Authorization": "Bearer " + token, "Content-Type": "application/json"},
    method="POST",
)
# This starts a potentially billable run. Do not automatically retry this POST.
with urllib.request.urlopen(request, timeout=40) as response:
    run = json.load(response)["data"]
print("Run ID:", run["id"])
print("Status:", run["status"])
# Inspect and download results in Apify Console once the run succeeds.

4. Inspect the actual results

Open the run in Apify Console and wait for a successful completion. Use the published output schema to locate results and the dataset schema to understand declared views. Missing or optional record fields must not be treated as guaranteed values.

Unique news articles with clean title, real publisher URL, source, domain, publication time, optional full text, author, image, and a machine-readable fullTextStatus. Empty fields are omitted.

Questions before you run

Which inputs are required?

The published required fields are mode

What does this guide verify?

The fields, prefills, and schema links are rendered from the published build. This is a usage recipe, not a paid live extraction test. Source availability and collection results can vary.

What if the run request times out?

Check Apify Console before retrying. The first request may already have created a run. Do not automatically repeat a run-creation POST.

Open on Apify Full Actor reference

Generated from source build bIH7aBp5p44Lvpppe and its README and schemas. Source changes regenerate this guide; no third-party editorial claims are inferred.