Skip to main content

Data Collection on Twitter

Author by Alex Zhang

Data Collection on Twitter Through Python

Target User & Introduction

If you are wondering how we can utilize the data on Social Media in order for us to do some research, this will be a greater chance to share what I have recently done. Using Twitter is my choice, yet by doing something similar, we can analyze most of the data on social media.

Getting Start

First thing first, we need a twitter developer account!1.png


Next, Create an APP Follow the introduction on the website.2.png

It is definitely not such a difficult job to finish, and by answering several questions we should be able to get our first app ready very easily, and we should also be able to access our API. From now, we can start to code. However, before we can actually get data from the database, we have to install the library. Here is the library I have installed:

•    library(base64enc)

•    library(tm)

•    library(syuzhet)

•    library(ggmap)

•    library(rtweet)

•    library(tidyverse)

•    library(ggplot2)

•    library(plotly)

•    library(wordcloud)


Most of these libraries are used for analyzing, yet don’t forgets to “pip install tweepy”. Finally, use the following code to pull data out from twitter’s database:

## import the libraries
import tweepy

## fill in your Twitter credentials
consumer_key = ' '
consumer_secret = ' '
access_token = ' '
access_token_secret = ' '

## let Tweepy set up an instance of the REST API
auth = tweepy.AppAuthHandler(consumer_key, consumer_secret)
#auth.set_access_token(access_token, access_token_secret)
api = tweepy.API(auth, wait_on_rate_limit=True,wait_on_rate_limit_notify=True)

Done! From now, we have got all the data (within 10 days), and about what to do with the data will be your choice.

Tags in this Article