Use Quora Question, Answer & Relationship Dataset Builder with Python
Build a structured Quora dataset from question, answer, profile, Topic, Space, and Post URLs or search terms, with the question-answer-comment chain and relationship links kept intact.
1. Choose the data you need
Review the published README for supported collection modes, examples, and limits. The recipe below uses the current published input definition.
2. Prepare a source-based input
This JSON contains published prefills and required-field defaults. Review every value for your task before running it; it is not a promise of successful collection.
{
"dedupeMode": "canonical",
"includeAnswerStats": true,
"includeAnswerUpvoterEdges": false,
"includeAuthor": true,
"includeAuthorSnapshot": false,
"includeCommentReplies": false,
"includeComments": false,
"includeEngagementBreakdown": true,
"includeEntityGraph": false,
"includeFeedText": false,
"includeFieldEvidence": true,
"includeHtml": false,
"includeMedia": false,
"includeOutboundLinks": false,
"includePostStats": true,
"includeProfileEdges": false,
"includeProfileHistory": false,
"includeProfileStats": true,
"includeQuestion": true,
"includeQuestionAudit": false,
"includeQuestionFollowerEdges": false,
"includeQuoraPlusPreview": true,
"includeRawEvidence": false,
"includeRelatedContent": false,
"includeRelationshipCounts": false,
"includeReplyGraph": false,
"includeSensitiveRelationshipData": false,
"includeSpaceContentEdges": true,
"includeSpaceRoleEdges": false,
"includeSpaceStats": true,
"includeSpaces": false,
"includeTargetMetadata": false,
"includeTopicFeed": false,
"includeTopicStats": true,
"includeTopics": true,
"includeUnchanged": true,
"includeUnresolvedEdges": true,
"maxAnswerPages": 0,
"maxAnswersPerQuestion": 5,
"maxCommentDepth": 20,
"maxCommentsPerAnswer": 0,
"maxGraphEdges": 1000,
"maxItemsPerSource": 0,
"maxProfileHistoryPages": 0,
"maxQuestions": 10,
"maxRelatedQuestions": 0,
"maxRelationshipPages": 0,
"maxSpaceFeedBundles": 1,
"maxTopicFeedPages": 0,
"mode": "auto",
"onlyNewItems": false,
"outputFormat": "normalized",
"redactSensitiveQueryParams": true,
"scrapeDepth": "detail",
"searchDiscovery": "relatedOnly",
"searchResultsPerQuery": 0,
"startUrls": [
{
"url": "https://www.quora.com/What-is-the-meaning-of-life"
}
],
"topCommentsOnly": false
}
Download input.example.json and save your reviewed copy as input.json.
| Field | Type | Required | Description |
|---|---|---|---|
mode | string | optional | Optional dispatcher. auto detects every supported Quora route; typed modes validate startUrls against that route. Typed seed lists still work without mode. |
scrapeDepth | string | optional | Overview emits page metadata without answer/comment/feed expansion; detail enables SSR answers plus bounded GraphQL pagination and optional comments/topic feeds. |
startUrls | array | optional | Question, answer, profile, topic, Space, or search URLs. Mixed lists are auto-detected; invalid hosts become status rows instead of failing the run. |
searchQueries | array | optional | Best-effort public-index discovery only. Quora native search is login/Cloudflare-gated and robots-disallowed; related-question expansion on a question URL is the reliable discovery axis. |
questionUrls | array | optional | Explicit question-page seeds. Only question routes are accepted. |
answerUrls | array | optional | Explicit answer-page seeds. Parent question joins are emitted only from observed evidence. |
topicUrls | array | optional | Explicit Topic-page seeds for bounded visible question discovery. |
spaceUrls | array | optional | Explicit Space-page seeds for bounded visible feed discovery. |
profileUrls | array | optional | Explicit profile-page seeds. Only profile routes are accepted; emitted as normalized profile rows exactly like a profile route discovered from startUrls. |
postUrls | array | optional | Explicit standalone Post-page seeds. Only post routes are accepted; emitted with the same topic_post field family topic-discovered posts use (post_title, post_body, post_author_*). |
resultTypes | array | optional | Select normalized entities, explicit graph relationships, relationship-collection counts, status rows, and the final run manifest. |
outputFormat | string | optional | Projection after one canonical extraction path. Nested groups emitted answers and comments under explicit question/answer parents; flat keeps entity rows with parent context; edges emits explicit edge rows plus status rows. |
dedupeMode | string | optional | canonical suppresses overlapping entity rows by stable ID or canonical URL (default); none preserves every observed non-status row. The run manifest reports duplicate_records_suppressed. |
onlyNewItems | boolean | optional | Annotate canonical entity rows with new/changed/unchanged/scope_mismatch state and suppress only unchanged semantic rows when includeUnchanged is false. Requires stateStoreId; status, graph, and manifest rows remain visible. |
includeUnchanged | boolean | optional | Retain unchanged semantic entities when incremental state is enabled; output projection does not alter comparison hashes. |
stateStoreId | string | optional | Apify key-value store for canonical entity snapshots. Required when onlyNewItems is enabled; never store cookies or credentials here. |
outputFields | array | optional | Optional allowlist for compact rows, including bounded dotted paths through nested collections (for example answers.answer_text, comments.comment_text, feed.feed_text, or contributors.contributor_name where applicable). Child identity/URL fields and the actor identity, access/status, source, traversal, graph, filter, and parser-envelope fields remain protected. |
includeFieldEvidence | boolean | optional | Include field_sources and fields_present maps. Disable only when a smaller payload is required; semantic values and protected access fields remain unchanged. |
includeUnresolvedEdges | boolean | optional | Retain relationship rows whose endpoint URL was not explicitly observed, with unresolved_reason. Disabling this suppresses only those edge rows and does not imply complete relationships. |
relationshipTypes | array | optional | Explicit relationship-type allowlist for the newly ported follower/following/want-answers/upvoter/space-role relationship collections. Empty leaves the existing always-on content-membership/authorship edges untouched (question_has_topic, answer_authored_by, comment_belongs_to_answer, profile_has_activity, space_has_feed_item, etc.); this allowlist only gates the additional collections below. Sensitive follower/want-answer/upvoter types also require includeSensitiveRelationshipData and their matching collection opt-in. |
includeProfileEdges | boolean | optional | Enable explicitly labeled profile follower/following/followed-question/Space/Topic relationship collections. This does not prove a complete list. |
includeQuestionFollowerEdges | boolean | optional | Enable explicitly labeled question follower or want-answer collections. Sensitive identity rows also require the privacy gate. |
includeAnswerUpvoterEdges | boolean | optional | High-sensitivity opt-in for explicitly labeled answer upvoter identities; requires includeSensitiveRelationshipData=true. |
includeSensitiveRelationshipData | boolean | optional | Explicit privacy gate required for follower, want-answer, and upvoter identity edges. |
includeTargetMetadata | boolean | optional | Retain explicitly visible target link text and image (to_name, to_image_url) on relationship edges. URLs remain evidence; no target navigation or recursive target scraping is performed. |
includeRelationshipCounts | boolean | optional | Emit privacy-preserving relationship_collection rows with the real observed total (collection_total_text/collection_total_value) but no target identities. Requires relationship_collection in resultTypes. Observed totals are not always a server-confirmed count. |
includeSpaceRoleEdges | boolean | optional | Enable explicitly labeled Space contributor, moderator, and admin collections. |
includeSpaceContentEdges | boolean | optional | Retain bounded explicit Space post/question membership edges; this never claims a complete Space inventory. |
maxRelationshipItemsPerCollection | integer | optional | Bound explicit links emitted from each labeled relationship collection. Zero emits no relationship-collection edges and never means an empty collection. |
maxQuestions | integer | optional | Maximum question seeds or discovered question pages to process. |
maxAnswersPerQuestion | integer | optional | Maximum answer rows retained per question after SSR and GraphQL pagination. |
maxAnswers | integer | optional | Global answer-row cap applied after per-question caps. Zero means no configured global answer cap. |
includeQuestion | boolean | optional | Retain normalized question parent rows when selected in resultTypes. |
includeAuthor | boolean | optional | Retain visible author fields and profile/profile-activity rows. When false, author fields and author rows are removed from the emitted shape; no contact enrichment is performed. |
includeAuthorSnapshot | boolean | optional | Opt into bounded public profile snapshots for visible author profile links; profile access/status is separate from answer/comment success and never includes contact enrichment. |
includeTopics | boolean | optional | Retain explicit topic links and topic relationships where observed. |
includeSpaces | boolean | optional | Retain Space rows and SSR feed items. Explicit spaceUrls enable this automatically. |
includeComments | boolean | optional | Opt-in comment rows from Quora's public comment-area GraphQL. robots-sensitive; off by default. When enabled and maxCommentsPerAnswer is 0, the documented fallback is 10 per answer. |
includeEngagementBreakdown | boolean | optional | Retain explicitly labeled per-comment reaction breakdowns alongside aggregate reaction labels. Raw labels are preserved and typed companions are added only when the localized number is unambiguous. |
includeRelatedContent | boolean | optional | Retain links inside an explicitly labeled related/attached region scoped to answer cards; unlabeled navigation and neighboring regions are excluded. |
maxRelatedContent | integer | optional | Bound eligible related-content links per answer when enabled; zero emits no related-content items. |
topCommentsOnly | boolean | optional | Retain only comments with an explicit visible Top/Most helpful/Popular comments label. Unknown comments are excluded and counted; votes and position are not used as top evidence. |
commentSortOrder | string | optional | Order builder comment rows by observed order or an explicitly typed metric/timestamp. Unknown selected metrics remain in stable fallback order and are counted. |
includeReplyGraph | boolean | optional | Enable comment rows plus one-level nested reply extraction from repliesConnection. Requires includeComments. |
includeMedia | boolean | optional | Request explicitly rendered media URL fields scoped to question roots, answer/comment nodes, and Topic post cards where the page exposes them. |
includeFeedText | boolean | optional | Request bounded visible text from explicitly linked Space feed cards; page-level text and complete feed history are excluded. |
includeOutboundLinks | boolean | optional | Request visible non-Quora links scoped to question roots, answer/comment nodes, and Topic post cards; Quora links, comment permalinks, and links outside the owning scope are excluded. |
includeEntityGraph | boolean | optional | Emit explicit relationship rows even when entity_edge is not listed in resultTypes. |
maxCommentsPerAnswer | integer | optional | Maximum comment rows retained per answer when comments are enabled. Schema default 0 is kept for compatibility; comments-enabled runs treat 0 as 10. |
maxCommentDepth | integer | optional | Maximum nested reply depth when includeCommentReplies/includeReplyGraph is enabled. |
maxItems | integer | optional | Cap for normalized entity/status rows before graph and manifest rows. |
maxGraphEdges | integer | optional | Independent cap for deterministic entity_edge rows. |
maxScrollAttempts | integer | optional | Bounded lazy-render scroll budget applied during detail traversal; scroll attempts and termination state are reported. |
maxLoadMoreAttempts | integer | optional | Bound exact visible continuation-control clicks on direct question, answer, topic, and Space collection pages independently from scrolling. Search seeds fan out through their discovered child URLs and retain their own source accounting. |
includeRawEvidence | boolean | optional | Add bounded document hash and optional HTML bytes to each page-level entity row. |
includeHtml | boolean | optional | Include bounded raw-document HTML when raw evidence is enabled and bounded answer-card HTML when answer cards are extracted. |
maxHtmlBytes | integer | optional | Maximum UTF-8 bytes retained for optional raw HTML evidence. |
requestDelay | number | optional | Delay between page batches in seconds; this is not an anti-bot bypass guarantee. |
maxConcurrency | integer | optional | Maximum independent seed pages processed concurrently in deterministic batches. Each seed owns a page; discovered seeds merge in batch order. |
maxNavigationRetries | integer | optional | Bounded retries after navigation exceptions (e.g. a transient proxy-tunnel handshake failure); exponential backoff is reported in navigation_retry_delays_seconds. Defaults to 2 for reliability. |
language | string | optional | Browser language hint. Short language codes and supported regional aliases are accepted and normalize to the short code; auto keeps the actor default. This does not translate content or prove access. |
proxyConfiguration | object | optional | Optional proxy override. The actor starts with a direct connection, retries a fresh session on 403/429, then uses Apify AUTO datacenter proxies, and only falls back to RESIDENTIAL as a proven last resort. Leave this unset; a missing proxyConfiguration never stops the run. |
redactSensitiveQueryParams | boolean | optional | Redact narrow credential-like query parameter values in emitted URLs, graph endpoints, redirect chains, and raw HTML. The original document hash remains based on the captured document. |
rawEvidenceRetentionDays | integer | optional | When zero, suppress raw_html snapshots while retaining hash, size, and redaction diagnostics. Positive values permit bounded HTML for the requested run; external Apify dataset/run retention still applies. |
maxRawEvidenceBytes | integer | optional | Hard upper byte cap for returned raw HTML after the maxHtmlBytes cap. |
maxItemsPerSource | integer | optional | Maximum semantic records emitted from each direct URL, query, or typed source. Status and run-manifest rows are preserved. Use 0 for no per-source cap. |
searchDiscovery | string | optional | How searchQueries are handled. relatedOnly attempts Quora's public search and otherwise emits search_unavailable; bestEffortIndex queries a public web index for indexed Quora URLs. Native Quora search is never claimed. |
maxRelatedQuestions | integer | optional | Related questions to enqueue per question/topic seed. The page usually exposes 26-35; zero disables related-question discovery. |
searchResultsPerQuery | integer | optional | Maximum indexed URLs to use per search query when searchDiscovery=bestEffortIndex. |
maxAnswerPages | integer | optional | Bound QuestionPagedListPaginationQuery pages. Zero derives a bound from maxAnswersPerQuestion. |
maxTopicFeedPages | integer | optional | Bound topic-feed GraphQL pages when includeTopicFeed is enabled. |
maxSpaceFeedBundles | integer | optional | Bound Space feed bundles read from the SSR multifeed. |
includeCommentReplies | boolean | optional | Include nested repliesConnection rows. Requires includeComments. Comments are robots-sensitive and opt-in; zero-row/blocked results produce a status row. |
includeQuoraPlusPreview | boolean | optional | Emit preview-only answers flagged answer_is_quora_plus/answer_text_completeness=preview. The gated remainder is never fetched or presented as full text. |
includeProfileStats | boolean | optional | Emit follower/following/answer/question/post/view counts, joined date, credentials and expertise topics for profile rows. |
includeTopicStats | boolean | optional | Emit topic follower count and most-viewed authors for topic rows. |
includeSpaceStats | boolean | optional | Emit Space follower/member/contributor/moderator/admin counts, description, icon and cover. |
includePostStats | boolean | optional | Emit post upvotes/views/shares/comments, effective access and question-summary flag. |
includeAnswerStats | boolean | optional | Emit answer counters, dates, AI flag, effective access and Quora+ preview flags. |
includeTopicFeed | boolean | optional | Opt into the browser topic-feed GraphQL endpoint. robots-sensitive: keep off unless you accept that compliance caveat. |
includeQuestionAudit | boolean | optional | Opt into the public question /log page to fill question_created_at, question_view_count and the asker fields. robots.txt disallows /*/log for generic agents; the actor never includes this in the default prefill. |
includeProfileHistory | boolean | optional | Opt into the public profile answer/question tab GraphQL and emit bounded profile_activity rows. robots-sensitive and rate-sensitive; on a 400/403 the run keeps a profile_history_unavailable status row. |
maxProfileHistoryPages | integer | optional | Bound for each profile history list (answers and questions) when includeProfileHistory is enabled. Zero derives three pages; one page is roughly ten items. |
maxRelationshipPages | integer | optional | Bound for each public identity-list query (question followers, answer upvoters, profile followers/following, Space roles) after the relationship allowlist and privacy gates. Zero is treated as one page. |
3. Start an Actor run
Set APIFY_TOKEN in your environment. Python 3.10 or newer is sufficient; this recipe uses no third-party packages.
Execution may incur charges. Review current pricing on Apify. The request asks for a 120-second timeout and a charge limit; these settings are subject to Apify’s pricing and do not guarantee an all-inclusive spending cap.
import json
import os
import urllib.request
# Save the schema-derived example as input.json; review and edit it first.
with open("input.json", encoding="utf-8") as stream:
actor_input = json.load(stream)
token = os.environ["APIFY_TOKEN"]
request = urllib.request.Request(
"https://api.apify.com/v2/acts/uAivkcIOWnHGsAnKT/runs?timeout=120&maxTotalChargeUsd=0.50",
data=json.dumps(actor_input).encode("utf-8"),
headers={"Authorization": "Bearer " + token, "Content-Type": "application/json"},
method="POST",
)
# This starts a potentially billable run. Do not automatically retry this POST.
with urllib.request.urlopen(request, timeout=40) as response:
run = json.load(response)["data"]
print("Run ID:", run["id"])
print("Status:", run["status"])
# Inspect and download results in Apify Console once the run succeeds.
4. Inspect the actual results
Open the run in Apify Console and wait for a successful completion. Use the published output schema to locate results and the dataset schema to understand declared views. Missing or optional record fields must not be treated as guaranteed values.
Normalized Quora questions, answers, comments, profiles, Spaces, relationship edges, status rows, and a run manifest.
Questions before you run
Which inputs are required?
No top-level fields are marked required in the published schema. Conditional requirements can still apply; check the README.
What does this guide verify?
The fields, prefills, and schema links are rendered from the published build. This is a usage recipe, not a paid live extraction test. Source availability and collection results can vary.
What if the run request times out?
Check Apify Console before retrying. The first request may already have created a run. Do not automatically repeat a run-creation POST.
Generated from source build yJ2UZKFwANg4ScxlV and its README and schemas. Source changes regenerate this guide; no third-party editorial claims are inferred.