Twitter Follower Finder (Tool)

McNair Project

Twitter Follower Finder (Tool)
Project Information
Project Title
Start Date
Deadline
Primary Billing
Notes
Has project status
Copyright © 2016 edegan.com. All Rights Reserved.

Description

This crawler takes as input the twitter handle of a person we think posts similar content to us or is an account we admire. It completes the following steps to use their information to find people we should consider following: 1) Crawls the tweets of that user and notes, for each tweet, how many times a buzzword (entrepreneur((s)hip), research(ers), innovat(e)(ion)) appears 2) Composes a list of the best tweets (most buzzwords) produced by the account in it's most recent 50 tweets. 3) Crawls the people who retweeted the tweets with the most buzzwords. 4) Makes note of how many times a buzzwords was used, for each of the retweeters. 5) Outputs a csv file which gives the usernames and a score (number of buzzwords) for each of the users.

Development

Functions authenticationAndAccess_interface, jsonDataAcquisition, retweetersIdAcquisition, retweetersShortnameAcquisition and generate_pandas_table_filledWithZeroes were taken from http://mcnair.bakerinstitute.org/wiki/Twitter_Webcrawler_(Tool) aka Gunny's existing Twitter Crawler.

I had major issues with Rate Limits and eventually found a solution (or so I think) here: http://python-twitter.readthedocs.io/en/latest/rate_limits.html and read about the Rate Limit nonsense in general here: https://dev.twitter.com/rest/public/rate-limiting

I used this page http://python-twitter.readthedocs.io/en/latest/twitter.html#module-twitter.api for an examination of all the methods I can use(and probably will use after I finish this application of the crawler)

Test Plan

1) Construct a list of twitter handles that meet the following criteria:

  a. Frequently post tweets containing buzzwords 
  b. Have a lot of followers 
  c. Are retweeted frequently
  d. Post original content (don't just retweet other people)

2) Run Twitter Follower Finder on five of these handles per day that I work (10 handles a week) 3) Examine the top results (at least top 5, plus anyone who scores over a 10) 4) Determine whether or not BakerMcNair should follow them based on the following criteria (could create a more specialized crawler for this purpose but haven't done it yet. I think I should wait to do it until trying the process manually).

  a. Frequently retweet people that they follow 
  b. Have a follower to following ratio close to 1:1 but no more than 2:1 
  c. Have content that will not annoy our feed (we could always mute these people though)

5) Make an excel sheet of who we chose to follow. 6) After one week, examine:

  a. Did they follow us back? 
  b. Did they retweet or favorite any of our posts? 

7) Determine the success of the program via

  a. the percent of followed people who followed us back
  b. If the new followers engage with our content 

8) Make adjustments to algorithm and criteria for following.

Log:

October 20, 2016 Followed: CristiTranulea MissOrtiz1612 startuplondon StartUps_Angel NYUEntrepreneur factory dgilgenmann rplutecki Kisura_Official FactoryBerlin TimLampkin aplaudsophia Intrapreneur1S niclasberlin igorfonsec The_Mack_ alexanderme18