Start with 100 free credits. No card needed. Get an API key

CrewAI

Use SocialScrape from CrewAI agents with the crewai-socialscrape package, eight tools for profiles, posts, comments, transcripts and search.

crewai-socialscrape is the official CrewAI integration. It gives an agent tools for the jobs agents do most (a profile, its posts, one post, the comments, a video transcript, a search) and a tool that runs any other endpoint by id. It builds on the Python SDK, so retries, timeouts and long transcription jobs are handled for you.

Install

Shell
pip install -U crewai-socialscrape
export SOCIALSCRAPE_API_KEY="YOUR_KEY"

Python 3.11 or newer. Create a key in the dashboard; a new account starts with 100 free credits. The source is on GitHub and the package is on PyPI.

Give an agent the tools

Python
from crewai import Agent, Crew, Task
from crewai_socialscrape import get_socialscrape_tools

analyst = Agent(
    role="Social media analyst",
    goal="Answer questions about creators and what their audiences say",
    backstory="You look at the data before you answer and you say where each number came from.",
    tools=get_socialscrape_tools(),   # reads SOCIALSCRAPE_API_KEY
)

task = Task(
    description=(
        "Compare NASA's TikTok and YouTube channels: followers, the latest posts, "
        "and what people say in the comments of the newest YouTube video."
    ),
    expected_output="A short comparison with the numbers and three points from the comments.",
    agent=analyst,
)

result = Crew(agents=[analyst], tasks=[task]).kickoff()
print(result.raw)

get_socialscrape_tools() returns eight tools that share one client:

ToolWhat it does
socialscrape_get_profileA profile, channel or artist: name, bio, links, verification and follower counts.
socialscrape_list_postsThe latest posts or videos of a profile, channel or page, one page per call.
socialscrape_get_postOne post, video, tweet, pin or clip from its URL.
socialscrape_list_commentsThe comments on a post or video, one page per call.
socialscrape_get_transcriptThe transcript of a video from its URL. See AI transcription for what happens when a video has no captions.
socialscrape_searchKeyword search on one platform, one page per call.
socialscrape_list_endpointsEvery endpoint id with its parameters and price. It makes no request, so it is free.
socialscrape_call_endpointRuns any endpoint in the API reference by its id, with its parameters by name.

The platform is an argument, for example platform="tiktok", handle="nasa". For a post, its comments or a transcript the platform is read from the link. Tools that list things return a cursor; pass it back for the next page, as described in pagination.

The tool classes are SocialScrapeProfileTool, SocialScrapePostsTool, SocialScrapePostTool, SocialScrapeCommentsTool, SocialScrapeTranscriptTool, SocialScrapeSearchTool, SocialScrapeEndpointsTool and SocialScrapeCallEndpointTool. Give an agent only the ones it needs:

Python
from crewai_socialscrape import SocialScrapeCommentsTool, SocialScrapeTranscriptTool

researcher = Agent(
    role="Video researcher",
    goal="Summarise what a video says and how viewers react",
    backstory="You read the transcript and the comments before you write.",
    tools=[SocialScrapeTranscriptTool(), SocialScrapeCommentsTool()],
)

Use one tool directly

Python
from crewai_socialscrape import SocialScrapeProfileTool, SocialScrapeTranscriptTool

SocialScrapeProfileTool().run(platform="tiktok", handle="nasa")
SocialScrapeTranscriptTool().run(url="https://www.youtube.com/watch?v=dQw4w9WgXcQ")

Every tool answers with a JSON string that holds data (the object, or a list page with items, cursor and has_more), credits_charged and credits_remaining. The objects are the same models the REST API returns.

To reach an endpoint that has no tool of its own, look it up and call it by id:

Python
from crewai_socialscrape import SocialScrapeCallEndpointTool, SocialScrapeEndpointsTool

SocialScrapeEndpointsTool().run(search="ad library")
SocialScrapeCallEndpointTool().run(
    endpoint="facebook.ad_library.search",
    params={"query": "running shoes"},
)

Parameters are checked against the endpoint before the request is made, so a misspelt name is answered at once and costs nothing.

Errors

A request the API refuses is free, as described in errors. A tool returns the error as its answer, with the error code first (Error: not_found: ...), so an agent can read it and try something else. A missing API key raises at the first call.

Options

Each tool, and get_socialscrape_tools(), takes these arguments:

ArgumentWhat it does
api_keyThe API key. Left out, it is read from SOCIALSCRAPE_API_KEY.
max_age_daysAccept a cached result up to this many days old. A cached result costs 0 credits; see caching.
trimOn by default: asks for the smaller response, without null fields and the extra object, which saves context.
client, async_clientA SocialScrape or AsyncSocialScrape client from the SDK that you configured yourself.
base_urlAPI base URL override.

The tools work with run and arun.

Credits

A tool call is one API request and is charged like one: see credits. socialscrape_list_endpoints makes no request and is free. Since an agent decides how many calls to make, CrewAI's max_usage_count on a tool is a simple way to cap a run.

Other agent frameworks

For LangChain and LangGraph there is a toolkit and a document loader. For Claude, ChatGPT, Cursor and other MCP clients, use the MCP server instead.