SocialScrape
Get an API key
  1. Tutorials
  2. How to scrape Instagram posts by hashtag
  3. JavaScript

How to scrape Instagram posts by hashtag with JavaScript

A JavaScript script that fetches Instagram posts by hashtag, pages through the results and saves them to a file.

Get an API key; the examples use api.socialscrape.dev.

Set up

You need Node 18 or newer. It has fetch built in, so there is nothing to install.

1export SOCIALSCRAPE_API_KEY=ss_live_...

The script

Save as scrape.mjs and run node scrape.mjs. It calls GET /v1/instagram/search/hashtag.

1import { writeFileSync } from "node:fs";23const BASE = "https://api.socialscrape.dev/v1/instagram/search/hashtag";4const KEY = process.env.SOCIALSCRAPE_API_KEY;5const PARAMS = {6  hashtag: "baking",7  enrich: false,8};9const MAX_ITEMS = 100;1011// Follow a path through nested objects; undefined if any step is missing or null.12const pick = (obj, ...path) => path.reduce((o, k) => (o == null ? undefined : o[k]), obj);13const sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms));1415// One request. Failures cost nothing; retry the ones the API marks retryable.16async function get(params) {17  for (let attempt = 0; attempt < 3; attempt++) {18    const url = new URL(BASE);19    url.search = new URLSearchParams(params).toString();20    const body = await (await fetch(url, { headers: { "x-api-key": KEY } })).json();21    if (body.ok) return body;22    const { code, message, retryable } = body.error;23    if (!retryable || attempt === 2) throw new Error(`${code}: ${message}`);24    await sleep(2 ** attempt * 1000);25  }26}2728let items = [];29let params = { ...PARAMS };30let body;31while (items.length < MAX_ITEMS) {32  body = await get(params);33  items.push(...body.data.items);34  if (!body.has_more) break;35  params = { ...params, cursor: body.cursor };36}37items = items.slice(0, MAX_ITEMS);3839writeFileSync("out.json", JSON.stringify(items, null, 2));40for (const item of items.slice(0, 5)) {41  console.log(pick(item, "text"), pick(item, "id"), pick(item, "stats", "likes"));42}43console.log(`saved ${items.length} items to out.json, credits left: ${body.credits_remaining}`);

How it works

  1. get() sends one request and retries the failures the API marks retryable. A failed request costs nothing.
  2. The loop follows cursor until has_more is false or 100 items are collected. Each page costs its own price.
  3. pick() reads nested fields and gives undefined instead of crashing when the platform returned null.
  4. Items go to out.json, and the first five are printed.

What you get back

These fields come from a recorded sample. The full field list is on the endpoint page.

SampleFields in the script
1{2  "text": "Reappearing on your feed, mine's on the reel cover😭 #baking #handmade #aesthetic #pinterest",3  "id": "De1XqvcOC4p",4  "stats": {5    "likes": 1530006  }7}

The short version

Without paging, saving or retries, the request is this.

1const url = new URL("https://api.socialscrape.dev/v1/instagram/search/hashtag");2url.search = new URLSearchParams({3  hashtag: "baking",4  enrich: "false",5}).toString();6const res = await fetch(url, {7  headers: { "x-api-key": process.env.SOCIALSCRAPE_API_KEY },8});9const body = await res.json();1011if (body.ok) {12  console.log(body.data);13} else {14  console.error(body.error.code, body.error.message);15}

With the SDK

The JavaScript client does the retries and the paging for you: see the SDKs.

1import { SocialScrape } from "socialscrape";23const client = new SocialScrape(); // reads SOCIALSCRAPE_API_KEY45for await (const item of client.paginate(client.instagram.searchHashtag, { hashtag: "baking", enrich: false }, { maxItems: 100 })) {6  console.log(item.text);7}

What it costs

1 credit. A failed request costs nothing. Add max_age_days to reuse a cached result for 0 credits. Results are kept 30 days.

Next