Twitter Follower Finder (Tool)

Project

Twitter Follower Finder (Tool)
Project Information
Has title	Twitter Follower Finder (Tool)
Has owner	Christy Warden
Has start date	Fall 2016
Has deadline date
Has keywords	Webcrawler, Database, Twitter, API, Python, Tool
Has project status	Complete
Has sponsor	McNair Center
Has project output	Tool
Copyright © 2019 edegan.com. All Rights Reserved.

Description

This crawler takes as input the twitter handle of a person we think posts similar content to us or is an account we admire. It completes the following steps to use their information to find people we should consider following: 1) Crawls the tweets of that user and notes, for each tweet, how many times a buzzword (entrepreneur((s)hip), research(ers), innovat(e)(ion)) appears 2) Composes a list of the best tweets (most buzzwords) produced by the account in it's most recent 50 tweets. 3) Crawls the people who retweeted the tweets with the most buzzwords. 4) Makes note of how many times a buzzwords was used, for each of the retweeters. 5) Outputs a csv file which gives the usernames and a score (number of buzzwords) for each of the users.

Development

Functions authenticationAndAccess_interface, jsonDataAcquisition, retweetersIdAcquisition, retweetersShortnameAcquisition and generate_pandas_table_filledWithZeroes were taken from http://mcnair.bakerinstitute.org/wiki/Twitter_Webcrawler_(Tool) aka Gunny's existing Twitter Crawler.

I had major issues with Rate Limits and eventually found a solution (or so I think) here: http://python-twitter.readthedocs.io/en/latest/rate_limits.html and read about the Rate Limit nonsense in general here: https://dev.twitter.com/rest/public/rate-limiting

I used this page http://python-twitter.readthedocs.io/en/latest/twitter.html#module-twitter.api for an examination of all the methods I can use(and probably will use after I finish this application of the crawler)

Test Plan

1) Construct a list of twitter handles that meet the following criteria:

  a. Frequently post tweets containing buzzwords 
  b. Have a lot of followers 
  c. Are retweeted frequently
  d. Post original content (don't just retweet other people)

2) Run Twitter Follower Finder on five of these handles per day that I work (10 handles a week) 3) Examine the top results (at least top 5, plus anyone who scores over a 10) 4) Determine whether or not BakerMcNair should follow them based on the following criteria (could create a more specialized crawler for this purpose but haven't done it yet. I think I should wait to do it until trying the process manually).

  a. Frequently retweet people that they follow 
  b. Have a follower to following ratio close to 1:1 but no more than 2:1 
  c. Have content that will not annoy our feed (we could always mute these people though)

5) Make an excel sheet of who we chose to follow. 6) After one week, examine:

  a. Did they follow us back? 
  b. Did they retweet or favorite any of our posts? 

7) Determine the success of the program via

  a. the percent of followed people who followed us back
  b. If the new followers engage with our content 

8) Make adjustments to algorithm and criteria for following.

Log:

October 20, 2016

Followed:

Via: @GoogleForEntrep CristiTranulea MissOrtiz1612 startuplondon StartUps_Angel NYUEntrepreneur factory dgilgenmann rplutecki Kisura_Official FactoryBerlin TimLampkin aplaudsophia Intrapreneur1S niclasberlin igorfonsec The_Mack_ alexanderme18

Via: @Entrepreneur GovYummy zippylab JamieLutzi realDeepPatel _Networker_ AidenDuarte vettedbyohub IzzyJDavies AltimaBusiness oGoing

Via: @businessinsider Risto_Matti DCarsonCPA_MFC DCarsonCPA_MA OctavianoTatau robertojirusta

Via: @Inc gatgman Dluxedad

October 25, 2016

Followed:

Via: @WhartonEntrep AlexEsteves ravi9007 dhairyapujara KormanMetro WhartonSocial DrLeslieGrace

Via: @YCombinator StartupDevKit The_VC_Bot jamesbregenzer careersingrowth merrybubbles RamonVillasante itsDrManhattan bkolko rttimmer marek_novy feamarketplace InventusLaw HmnMarius ApptologyCEO b_ragone nw_sme_funding KormanMetro juancarlos_an StevenPedian the__LP KME_SF letranger14 Andra_Bria SuchetChachra ShlnzuSteve IndikonTweets Slmmeez EUACADEMY808

Via: @yCenterLearning TommasoGritti YcenterLearning dhairyapujara styllshlawrence JoseGSantamarla Bozza943 aiexaaandrasays

Via: @CaseFoundation TheSiliconHill amymwofford KStCapital MHSecreto sherrling ErinMichelson ZimbaWomen redspireusnc LITEMemphis sbarnat jknox78251 M_E_Cat impart_nyc Greyeminence6 phelynskillcent

Via: @enventuremed RiceOwlSpark TMCInnovation ManageDiscovery cgjohnson09 BioNorthTX

Via: @angelashah BioHouston sandeepburug reversegremlin sarahsw55

Via: @StanfordEng Plastipolis

Via: @DukeCIP wswarren12 EshipAtDuke

Via: @Houston_Angels TMCInnovation TMCBiodesign ravibrahmbhatt stationhouston HouTechCenter

October 25, 2016

Followed:

Via: @KauffmanFDN jd_harrison InnovateEconomy _RobWilliams StartlandNews LettieriDC BrianVision Launch_KC deldelp davidaptedsr Stel_Hernandez larryjacobjr sandropiancone1 juliehn UHVictoriaSBDC networkkansas Sell2SMBiz oGoing GerardJRego eRAJ25 anaswatchlng buildinghugelis

Via: @InnovateEconomy SteveCase Patrick_J_Sims

Via: @MedTechAwards lifesciencevc reneeryan378 mindsofmalady AdvaMedConf AstierBio george_sidis sandeepburug chelsbells4 graemethickins jbhungry SamISureAm MedtechReed

Via: @Neil_Irwin LettieriDC infomodernist weverjd Anushtup555 roses_war DeepakJishnu xandi_R mattbentone Crf001 _Nick_May pawelc2 JJGomez127 rjredmer iamthebobs DrMikeThompson iarroyo48 eduardom drumice henrykopesky brettbeletz ScottCPhilips wolfeb99

Via: @BrookingsEcon TJAdethink mornyngstar AnnCleven SharonEnoch3 AkrurBarua JReed193 Sec2Bruin compliancedivis thepeternorris Elizacat4 Noriaki2013 kevins_89 iacob007

Via: @1776 BeingExample ZackHuhn ryanlouie elogerquist LaShandrow lifelong_AmyB jamesbregenzer edvaldez8888 ELIANATARDIO USAFundsOrg MadeinMenlo tplusr TEQWORKS

Via: @elogerquist careerevolved SabotageVC HarryAlford3 lifelong_AmyB tan_jianxiang chlofooo MrAShukla

October 25, 2016

Starting ratio of the day: 191:330

Followed:

Via: @pjtentrepreneur AlexEsteves PaigeHendrix bwise_bwiseusa jeremy_chauvet getadventurely JessWorldwide CaseFoundation joincakeapp SparklightIdeas PDMyriam IsmailAhlem SerenaScanz what_an_aura mcdonoughsu

Via: @CaseFoundation TheMilwaukee7 pjtentrepreneur WDSPRMaven TheSiliconHill santoshajmeera TechMoranLtd uniquehnd SEAlliance tessydeee CallahanFndtn BrandRadianz NoBullGrape LuannaZapula InclusiveUSA pamela_hanratty

Via: @SkollFoundation FastITzone houstonpeschl thisiswick

Via: @KauffmanFDN NewAgeMediaLDN TheMilwaukee7 AlexEsteves jd_harrison amyaddventure uzupay _RobWilliams vc0 AugustVenture InnovateEconomy Optimize_Biz kenanfikri SiliconPrairie TestimonialR jeremy_chauvet genemurphy oGoing LettieriDC BreedloveSteph livinginacave WDSPRMaven KauffmanFellows iWeev LandonGaryYoung arnobiomorelix ReginaAnnCampbe eRAJ25 jessie_maggard M_E_Cat rainforestbook Ktinsv TheHConnect 5amStartups Stel_Hernandez IanHathaway donnawhite keithkcvc stevebarham CarlaMays timreha VC_Atty shyduroff neilmartis17

Via: @US_EDA SobolikJan RCBI4Mfg acedc1 aileengemma EdgarCAPX CommerceGov Ceshoop RebeccaCorbin7 ASPIREhomes biQMD IMC_help neilmartis17 1kt_boo_boo_boo

Nov 8, 2016

Starting ratio of the day: 208:353

Followed:

Via: @endeavor_global Endeavoreg maroferrari mangelferrero EndeavorMIA EndeavorSA SALTBizJacqui EndeavorGr IiMaydon CathTownshend Endeavor_Lou GSbeirut AlanMain4 TricapPartners aglobalvc jmandrewsv Sal71299

Via: @KaffmanFDN lionsafrica NewAgeMediaLDN uzupay TheMilwaukee7 AlexEsteves TestimonialR kapsd kffitzgerald InnovateEconomy Innovate_Africa 5amStartups kenanfikri vc0 genemurphy amyaddventure TheHConnect livinginacave jeremy_chauvet DavidKCapelli Optimize_Biz arnobiomorelix DiverseBizBlog M_E_Cat keithkcvc danielslater811 ASBTDC oGoing iWeev KauffmanFellows Ktinsv risingtidenetwk rainforestbook brbelieves LettieriDC LandonGaryYoung WDSPRMaven shyduroff VC_Atty

Via: @sandyspeicher InnovateRI KristaMclark jcburke_sf NicolasTurcat dougreid mare_morgan mjcraw dmcbrideg tfiala3333 commac sky_astropolis maelorin jjj5819 Maxlubin08 paola_pupla

Via: @villagecapital WaiteMarilyn bftp_sep Chistorro63 lajuanda_marie MoFoImpact

Via: @HitachiFdn StartupAggies AnneClaire60923

Via: @BrookingsEcon akogidc CountterPoint DiegoElias75 mainiac1820 Takasho0001 SharonEnoch3 moore_cjm MalminderWrites DavidRedelinghu

Nov 8, 2016

Starting ratio of the day: 220:376

Followed:

Via: @endeavor_global