Stack Exchange 主题
将单个 Stack Exchange 问题页面解析为结构化 JSON,包含问题正文、每个回答和评论、得分、采纳状态、作者、声望以及时间戳。
API 用法
在 Crawling API 请求中添加 &scraper=stackexchange-thread。在 url 参数中对目标 URL 进行 URL 编码。
curl 'https://api.crawlbase.com/?token=YOUR_TOKEN' \
--data-urlencode 'url=https://stackoverflow.com/questions/2861071/how-to-modify-a-text-file' \
--data-urlencode 'scraper=stackexchange-thread' -Gfrom crawlbase import CrawlingAPI
api = CrawlingAPI({'token': 'YOUR_TOKEN'})
res = api.get(
'https://stackoverflow.com/questions/2861071/how-to-modify-a-text-file',
{'scraper': 'stackexchange-thread'}
)
import json
data = json.loads(res['body'])const { CrawlingAPI } = require('crawlbase');
const api = new CrawlingAPI({ token: 'YOUR_TOKEN' });
const res = await api.get(
'https://stackoverflow.com/questions/2861071/how-to-modify-a-text-file',
{ scraper: 'stackexchange-thread' }
);
const data = JSON.parse(res.body);require 'crawlbase'
api = Crawlbase::API.new(token: 'YOUR_TOKEN')
res = api.get('https://stackoverflow.com/questions/2861071/how-to-modify-a-text-file', scraper: 'stackexchange-thread')
data = JSON.parse(res.body)示例输入 URL
url 参数支持任意 Stack Exchange 问题 URL,来自网络中的任意站点。例如:
https://stackoverflow.com/questions/2861071/how-to-modify-a-text-file
https://superuser.com/questions/441895/automate-opening-html-and-printing-to-pdf
https://askubuntu.com/questions/1206658/why-is-apt-held-back
https://serverfault.com/questions/439471/which-openvpn-cipher-should-i-use
https://mathoverflow.net/questions/44265/is-the-riemann-hypothesis响应结构
JSON 响应体。当源页面省略对应值时,字段类型可能为 null。
问题本身。
Stack Exchange 问题 id。
问题标题。
问题的规范 URL。
问题正文文本。
问题得分(顶数减去踩数)。
问题的浏览数量。
应用于问题的标签。
问题创建时间(ISO 8601)。
问题的作者。
问题作者的显示名称。
问题作者的个人资料 URL。
问题作者的声望分数。
问题上的评论。
评论 id。
评论得分。
评论文本。
评论作者的显示名称。
评论作者的个人资料 URL。
评论创建时间(ISO 8601)。
answers 中返回的回答数量。问题的回答,按列表顺序排列。
回答 id。
回答得分(顶数减去踩数)。
当这是已采纳的回答时为 true。
回答正文文本。
回答的作者。
回答作者的显示名称。
回答作者的个人资料 URL。
回答作者的声望分数。
回答创建时间(ISO 8601)。
回答上的评论。
评论 id。
评论得分。
评论文本。
评论作者的显示名称。
评论作者的个人资料 URL。
评论创建时间(ISO 8601)。
示例响应
{
"question": {
"id": "2861071",
"title": "How to modify a text file?",
"url": "https://stackoverflow.com/questions/2861071/how-to-modify-a-text-file",
"body": "I'm using Python and I need to insert a line at the start of a text file without loading the whole file into memory. Is there a way to do this in place, or do I have to rewrite the file?",
"score": 312,
"viewCount": 486201,
"tags": ["python", "file", "text-files"],
"askedAt": "2010-05-18T20:11:33Z",
"author": {
"name": "Nathan Fellman",
"url": "https://stackoverflow.com/users/8460/nathan-fellman",
"reputation": 127843
},
"comments": [
{
"id": "2954120",
"score": 3,
"body": "Do you need to preserve the original file, or is rewriting acceptable?",
"author": "Greg Hewgill",
"authorUrl": "https://stackoverflow.com/users/893/greg-hewgill",
"createdAt": "2010-05-18T20:19:04Z"
}
]
},
"answerCount": 2,
"answers": [
{
"id": "2861108",
"score": 401,
"isAccepted": true,
"body": "You cannot insert into the middle of a file without rewriting it. Read the file into a list of lines, insert your new line, then write it all back:\n\n with open('file.txt') as f:\n lines = f.readlines()\n lines.insert(0, 'new first line\\n')\n with open('file.txt', 'w') as f:\n f.writelines(lines)\n",
"author": {
"name": "Roberto Bonvallet",
"url": "https://stackoverflow.com/users/193568/roberto-bonvallet",
"reputation": 30215
},
"createdAt": "2010-05-18T20:15:52Z",
"comments": [
{
"id": "2954260",
"score": 12,
"body": "For very large files, stream through a temporary file instead of holding everything in memory.",
"author": "John Machin",
"authorUrl": "https://stackoverflow.com/users/253537/john-machin",
"createdAt": "2010-05-18T21:02:11Z"
}
]
},
{
"id": "2861180",
"score": 47,
"isAccepted": false,
"body": "If the file is large, use fileinput with inplace=True to edit it line by line without loading it all at once.",
"author": {
"name": "codeape",
"url": "https://stackoverflow.com/users/18770/codeape",
"reputation": 98412
},
"createdAt": "2010-05-18T20:24:39Z",
"comments": []
}
]
}