LangChain

Use SocialScrape from LangChain and LangGraph with the langchain-socialscrape package, a toolkit of eight agent tools and a document loader for RAG.

langchain-socialscrape is the official LangChain integration. It gives an agent tools for the jobs agents do most (a profile, its posts, one post, the comments, a video transcript, a search), a tool that runs any other endpoint by id, and a document loader that turns any endpoint's result into LangChain Documents. It works with LangChain and LangGraph agents and builds on the Python SDK, so retries, timeouts and long transcription jobs are handled for you.

Install#

1pip install -U langchain-socialscrape2export SOCIALSCRAPE_API_KEY="YOUR_KEY"

Python 3.11 or newer. Create a key in the dashboard; a new account starts with 100 free credits. The source is on GitHub and the package is on PyPI.

Give an agent the tools#

1from langchain.agents import create_agent2from langchain_socialscrape import SocialScrapeToolkit34tools = SocialScrapeToolkit().get_tools()   # reads SOCIALSCRAPE_API_KEY5agent = create_agent("openai:gpt-5", tools=tools)67result = agent.invoke({"messages": [8    ("user", "What did NASA post on YouTube this week, and what are people saying in the comments?")9]})10print(result["messages"][-1].content)

Any chat model that supports tool calling works. get_tools() returns eight tools that share one client:

Tool	What it does
`socialscrape_get_profile`	A profile, channel or artist: name, bio, links, verification and follower counts.
`socialscrape_list_posts`	The latest posts or videos of a profile, channel or page, one page per call.
`socialscrape_get_post`	One post, video, tweet, pin or clip from its URL.
`socialscrape_list_comments`	The comments on a post or video, one page per call.
`socialscrape_get_transcript`	The transcript of a video from its URL. See AI transcription for what happens when a video has no captions.
`socialscrape_search`	Keyword search on one platform, one page per call.
`socialscrape_list_endpoints`	Every endpoint id with its parameters and price. It makes no request, so it is free.
`socialscrape_call_endpoint`	Runs any endpoint in the API reference by its id, with its parameters by name.

The platform is an argument, for example {"platform": "tiktok", "handle": "nasa"}. For a post, its comments or a transcript the platform is read from the link. Tools that list things return a cursor; pass it back for the next page, as described in pagination.

Use one tool directly#

1from langchain_socialscrape import SocialScrapeProfileTool, SocialScrapeTranscriptTool23SocialScrapeProfileTool().invoke({"platform": "tiktok", "handle": "nasa"})4SocialScrapeTranscriptTool().invoke({"url": "https://www.youtube.com/watch?v=dQw4w9WgXcQ"})

Every tool answers with a JSON string that holds data (the object, or a list page with items, cursor and has_more), credits_charged and credits_remaining. The objects are the same models the REST API returns.

To reach an endpoint that has no tool of its own, look it up and call it by id:

1from langchain_socialscrape import SocialScrapeCallEndpointTool, SocialScrapeEndpointsTool23SocialScrapeEndpointsTool().invoke({"search": "ad library"})4SocialScrapeCallEndpointTool().invoke({5    "endpoint": "facebook.ad_library.search",6    "params": {"query": "running shoes"},7})

Parameters are checked against the endpoint before the request is made, so a misspelt name is answered at once and costs nothing.

Errors#

A request the API refuses is free, as described in errors. By default a tool returns the error as its answer, with the error code first (not_found: ...), so an agent can read it and try something else. Pass handle_tool_error=False to have the tool raise LangChain's ToolException instead. A missing API key always raises.

Options#

Each tool, and the toolkit, takes these arguments:

Argument	What it does
`api_key`	The API key. Left out, it is read from `SOCIALSCRAPE_API_KEY`.
`max_age_days`	Accept a cached result up to this many days old. A cached result costs 0 credits; see caching.
`trim`	On by default: asks for the smaller response, without null fields and the `extra` object, which saves context.
`client`, `async_client`	A `SocialScrape` or `AsyncSocialScrape` client from the SDK that you configured yourself.
`base_url`	API base URL override.

The tools work with invoke and ainvoke.

Load documents for RAG#

SocialScrapeLoader takes an endpoint id and its parameters and returns Documents, ready for a text splitter or a vector store.

1from langchain_socialscrape import SocialScrapeLoader23comments = SocialScrapeLoader(4    "youtube.video.comments",5    {"url": "https://www.youtube.com/watch?v=dQw4w9WgXcQ"},6    max_pages=3,7).load()89transcript = SocialScrapeLoader(10    "transcript",11    {"url": "https://www.youtube.com/watch?v=dQw4w9WgXcQ"},12).load()

A list endpoint gives one Document per item and is followed across pages up to max_pages. The default is 1, because each page is one request. max_items stops early.
Any other endpoint, such as a profile, one post or a transcript, gives one Document.
The page content is the item's text: a post's title and text, a comment's text, a transcript's text, a profile's name and bio.
The item's other fields, its counts and its author go to metadata, flat, with the item's URL as source.
lazy_load() yields Documents as each page arrives.

Credits#

A tool call or a loader page is one API request and is charged like one: see credits. socialscrape_list_endpoints makes no request and is free.

Other agent frameworks#

For Claude, ChatGPT, Cursor and other MCP clients, use the MCP server instead.