Need help with instagram-crawler?
Click the “chat” button below for chat support from the developer who created it, or find similar developers for support.

About the developer

huaying
761 Stars 246 Forks MIT License 141 Commits 55 Opened issues

Description

Get Instagram posts/profile/hashtag data without using Instagram API

Services available

!
?

Need anything else?

Contributors list

Instagram Crawler Build Status

Below is what you can do with this program: - Get Instagram posts/profile/hashtag data without using Instagram API.

crawler.py
- Like posts automatically.
liker.py

This crawler could fail due to updates on instagram’s website. If you encounter any problems, please contact me.

Install

  1. Make sure you have Chrome browser installed.
  2. Download chromedriver and put it into bin folder:
    ./inscrawler/bin/chromedriver
  3. Install Selenium:
    pip3 install -r requirements.txt
  4. cp inscrawler/secret.py.dist inscrawler/secret.py

User Auth

  1. Open
    inscrawler/secret.py
    file.
  2. Change the
    username
    and
    password
    variables' value to the ones corresponding to your Instagram account.
    username = 'my_ig_username'
    password = '***********'
    

Crawler

Usage

positional arguments:
  mode
    options: [posts, posts_full, profile, hashtag]

optional arguments: -n NUMBER, --number NUMBER number of returned posts -u USERNAME, --username USERNAME instagram's username -t TAG, --tag TAG instagram's tag name -o OUTPUT, --output OUTPUT output file name(json format)

--debug see how the program automates the browser

--fetch_comments fetch comments

Turning on the flag might take forever to fetch data if there are too many commnets.

--fetch_likes_plays fetch like/play number

--fetch_likers fetch all likers

Instagram might have rate limit for fetching likers. Turning on the flag might take forever to fetch data if there are too many likes.

--fetch_mentions fetch users who are mentioned in the caption/comments (startwith @)

--fetch_hashtags fetch hashtags in the caption/comments (startwith #)

--fetch_details fetch username and photo caption

only available for "hashtag" search

Example

python crawler.py posts_full -u cal_foodie -n 100 -o ./output
python crawler.py posts_full -u cal_foodie -n 10 --fetch_likers --fetch_likes_plays
python crawler.py posts_full -u cal_foodie -n 10 --fetch_comments
python crawler.py profile -u cal_foodie -o ./output
python crawler.py hashtag -t taiwan -o ./output
python crawler.py hashtag -t taiwan -o ./output --fetch_details
python crawler.py posts -u cal_foodie -n 100 -o ./output # deprecated
  1. Choose mode
    posts
    , you will get url, caption, first photo for each post; choose mode
    posts_full
    , you will get url, caption, all photos, time, comments, number of likes and views for each posts. Mode
    posts_full
    will take way longer than mode
    posts
    . [
    posts
    is deprecated. For the recent posts, there is no quick way to get the post caption]
  2. Return default 100 hashtag posts(mode: hashtag) and all user's posts(mode: posts) if not specifying the number of post
    -n
    ,
    --number
    .
  3. Print the result to the console if not specifying the output path of post
    -o
    ,
    --output
    .
  4. It takes much longer to get data if the post number is over about 1000 since Instagram has set up the rate limit for data request.
  5. Don't use this repo crawler Instagram if the user has more than 10000 posts.

The data format of

posts
: screen shot 2018-10-11 at 2 33 09 pm

The data format of

posts_full
: Screen Shot 2019-03-17 at 11 02 24 PM

Liker

Liker Preivew

Set up your username/password in

secret.py
or set them as environment variables.

Usage

positional arguments:
  tag

optional arguments: -n NUMBER, --number NUMBER (default 1000) number of posts to like

Example

python liker.py foodie

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.