Difference between revisions of "Twitter Follower Finder (Tool)"
| (17 intermediate revisions by the same user not shown) | |||
| Line 1: | Line 1: | ||
| − | {{McNair | + | {{Project |
| − | | | + | |Has project output=Tool |
| − | | | + | |Has sponsor=McNair Center |
| − | + | |Has title=Twitter Follower Finder (Tool) | |
| − | | | + | |Has owner=Christy Warden |
| − | | | + | |Has start date=Fall 2016 |
| − | | | + | |Has keywords=Webcrawler, Database, Twitter, API, Python,Tool |
| − | | | + | |Has notes= |
| − | | | + | |Is dependent on= |
| + | |Depends upon it= | ||
| + | |Has project status=Complete | ||
}} | }} | ||
== '''People to Follow Crawl''' == | == '''People to Follow Crawl''' == | ||
| Line 517: | Line 519: | ||
''Via: @endeavor_global'' | ''Via: @endeavor_global'' | ||
| − | |||
Latest revision as of 12:47, 21 September 2020
| Twitter Follower Finder (Tool) | |
|---|---|
| Project Information | |
| Has title | Twitter Follower Finder (Tool) |
| Has owner | Christy Warden |
| Has start date | Fall 2016 |
| Has deadline date | |
| Has keywords | Webcrawler, Database, Twitter, API, Python, Tool |
| Has project status | Complete |
| Has sponsor | McNair Center |
| Has project output | Tool |
| Copyright © 2019 edegan.com. All Rights Reserved. | |
People to Follow Crawl
Description
This crawler takes as input the twitter handle of a person we think posts similar content to us or is an account we admire. It completes the following steps to use their information to find people we should consider following: 1) Crawls the tweets of that user and notes, for each tweet, how many times a buzzword (entrepreneur((s)hip), research(ers), innovat(e)(ion)) appears 2) Composes a list of the best tweets (most buzzwords) produced by the account in it's most recent 50 tweets. 3) Crawls the people who retweeted the tweets with the most buzzwords. 4) Makes note of how many times a buzzwords was used, for each of the retweeters. 5) Outputs a csv file which gives the usernames and a score (number of buzzwords) for each of the users.
Development
Functions authenticationAndAccess_interface, jsonDataAcquisition, retweetersIdAcquisition, retweetersShortnameAcquisition and generate_pandas_table_filledWithZeroes were taken from http://mcnair.bakerinstitute.org/wiki/Twitter_Webcrawler_(Tool) aka Gunny's existing Twitter Crawler.
I had major issues with Rate Limits and eventually found a solution (or so I think) here: http://python-twitter.readthedocs.io/en/latest/rate_limits.html and read about the Rate Limit nonsense in general here: https://dev.twitter.com/rest/public/rate-limiting
I used this page http://python-twitter.readthedocs.io/en/latest/twitter.html#module-twitter.api for an examination of all the methods I can use(and probably will use after I finish this application of the crawler)
Test Plan
1) Construct a list of twitter handles that meet the following criteria:
a. Frequently post tweets containing buzzwords b. Have a lot of followers c. Are retweeted frequently d. Post original content (don't just retweet other people)
2) Run Twitter Follower Finder on five of these handles per day that I work (10 handles a week) 3) Examine the top results (at least top 5, plus anyone who scores over a 10) 4) Determine whether or not BakerMcNair should follow them based on the following criteria (could create a more specialized crawler for this purpose but haven't done it yet. I think I should wait to do it until trying the process manually).
a. Frequently retweet people that they follow b. Have a follower to following ratio close to 1:1 but no more than 2:1 c. Have content that will not annoy our feed (we could always mute these people though)
5) Make an excel sheet of who we chose to follow. 6) After one week, examine:
a. Did they follow us back? b. Did they retweet or favorite any of our posts?
7) Determine the success of the program via
a. the percent of followed people who followed us back b. If the new followers engage with our content
8) Make adjustments to algorithm and criteria for following.
Log:
October 20, 2016
Followed:
Via: @GoogleForEntrep CristiTranulea MissOrtiz1612 startuplondon StartUps_Angel NYUEntrepreneur factory dgilgenmann rplutecki Kisura_Official FactoryBerlin TimLampkin aplaudsophia Intrapreneur1S niclasberlin igorfonsec The_Mack_ alexanderme18
Via: @Entrepreneur GovYummy zippylab JamieLutzi realDeepPatel _Networker_ AidenDuarte vettedbyohub IzzyJDavies AltimaBusiness oGoing
Via: @businessinsider Risto_Matti DCarsonCPA_MFC DCarsonCPA_MA OctavianoTatau robertojirusta
Via: @Inc gatgman Dluxedad
October 25, 2016
Followed:
Via: @WhartonEntrep AlexEsteves ravi9007 dhairyapujara KormanMetro WhartonSocial DrLeslieGrace
Via: @YCombinator StartupDevKit The_VC_Bot jamesbregenzer careersingrowth merrybubbles RamonVillasante itsDrManhattan bkolko rttimmer marek_novy feamarketplace InventusLaw HmnMarius ApptologyCEO b_ragone nw_sme_funding KormanMetro juancarlos_an StevenPedian the__LP KME_SF letranger14 Andra_Bria SuchetChachra ShlnzuSteve IndikonTweets Slmmeez EUACADEMY808
Via: @yCenterLearning TommasoGritti YcenterLearning dhairyapujara styllshlawrence JoseGSantamarla Bozza943 aiexaaandrasays
Via: @CaseFoundation TheSiliconHill amymwofford KStCapital MHSecreto sherrling ErinMichelson ZimbaWomen redspireusnc LITEMemphis sbarnat jknox78251 M_E_Cat impart_nyc Greyeminence6 phelynskillcent
Via: @enventuremed RiceOwlSpark TMCInnovation ManageDiscovery cgjohnson09 BioNorthTX
Via: @angelashah BioHouston sandeepburug reversegremlin sarahsw55
Via: @StanfordEng Plastipolis
Via: @DukeCIP wswarren12 EshipAtDuke
Via: @Houston_Angels TMCInnovation TMCBiodesign ravibrahmbhatt stationhouston HouTechCenter
October 25, 2016
Followed:
Via: @KauffmanFDN jd_harrison InnovateEconomy _RobWilliams StartlandNews LettieriDC BrianVision Launch_KC deldelp davidaptedsr Stel_Hernandez larryjacobjr sandropiancone1 juliehn UHVictoriaSBDC networkkansas Sell2SMBiz oGoing GerardJRego eRAJ25 anaswatchlng buildinghugelis
Via: @InnovateEconomy SteveCase Patrick_J_Sims
Via: @MedTechAwards lifesciencevc reneeryan378 mindsofmalady AdvaMedConf AstierBio george_sidis sandeepburug chelsbells4 graemethickins jbhungry SamISureAm MedtechReed
Via: @Neil_Irwin LettieriDC infomodernist weverjd Anushtup555 roses_war DeepakJishnu xandi_R mattbentone Crf001 _Nick_May pawelc2 JJGomez127 rjredmer iamthebobs DrMikeThompson iarroyo48 eduardom drumice henrykopesky brettbeletz ScottCPhilips wolfeb99
Via: @BrookingsEcon TJAdethink mornyngstar AnnCleven SharonEnoch3 AkrurBarua JReed193 Sec2Bruin compliancedivis thepeternorris Elizacat4 Noriaki2013 kevins_89 iacob007
Via: @1776 BeingExample ZackHuhn ryanlouie elogerquist LaShandrow lifelong_AmyB jamesbregenzer edvaldez8888 ELIANATARDIO USAFundsOrg MadeinMenlo tplusr TEQWORKS
Via: @elogerquist careerevolved SabotageVC HarryAlford3 lifelong_AmyB tan_jianxiang chlofooo MrAShukla
October 25, 2016
Starting ratio of the day: 191:330
Followed:
Via: @pjtentrepreneur AlexEsteves PaigeHendrix bwise_bwiseusa jeremy_chauvet getadventurely JessWorldwide CaseFoundation joincakeapp SparklightIdeas PDMyriam IsmailAhlem SerenaScanz what_an_aura mcdonoughsu
Via: @CaseFoundation TheMilwaukee7 pjtentrepreneur WDSPRMaven TheSiliconHill santoshajmeera TechMoranLtd uniquehnd SEAlliance tessydeee CallahanFndtn BrandRadianz NoBullGrape LuannaZapula InclusiveUSA pamela_hanratty
Via: @SkollFoundation FastITzone houstonpeschl thisiswick
Via: @KauffmanFDN NewAgeMediaLDN TheMilwaukee7 AlexEsteves jd_harrison amyaddventure uzupay _RobWilliams vc0 AugustVenture InnovateEconomy Optimize_Biz kenanfikri SiliconPrairie TestimonialR jeremy_chauvet genemurphy oGoing LettieriDC BreedloveSteph livinginacave WDSPRMaven KauffmanFellows iWeev LandonGaryYoung arnobiomorelix ReginaAnnCampbe eRAJ25 jessie_maggard M_E_Cat rainforestbook Ktinsv TheHConnect 5amStartups Stel_Hernandez IanHathaway donnawhite keithkcvc stevebarham CarlaMays timreha VC_Atty shyduroff neilmartis17
Via: @US_EDA SobolikJan RCBI4Mfg acedc1 aileengemma EdgarCAPX CommerceGov Ceshoop RebeccaCorbin7 ASPIREhomes biQMD IMC_help neilmartis17 1kt_boo_boo_boo
Nov 8, 2016
Starting ratio of the day: 208:353
Followed:
Via: @endeavor_global Endeavoreg maroferrari mangelferrero EndeavorMIA EndeavorSA SALTBizJacqui EndeavorGr IiMaydon CathTownshend Endeavor_Lou GSbeirut AlanMain4 TricapPartners aglobalvc jmandrewsv Sal71299
Via: @KaffmanFDN lionsafrica NewAgeMediaLDN uzupay TheMilwaukee7 AlexEsteves TestimonialR kapsd kffitzgerald InnovateEconomy Innovate_Africa 5amStartups kenanfikri vc0 genemurphy amyaddventure TheHConnect livinginacave jeremy_chauvet DavidKCapelli Optimize_Biz arnobiomorelix DiverseBizBlog M_E_Cat keithkcvc danielslater811 ASBTDC oGoing iWeev KauffmanFellows Ktinsv risingtidenetwk rainforestbook brbelieves LettieriDC LandonGaryYoung WDSPRMaven shyduroff VC_Atty
Via: @sandyspeicher InnovateRI KristaMclark jcburke_sf NicolasTurcat dougreid mare_morgan mjcraw dmcbrideg tfiala3333 commac sky_astropolis maelorin jjj5819 Maxlubin08 paola_pupla
Via: @villagecapital WaiteMarilyn bftp_sep Chistorro63 lajuanda_marie MoFoImpact
Via: @HitachiFdn StartupAggies AnneClaire60923
Via: @BrookingsEcon akogidc CountterPoint DiegoElias75 mainiac1820 Takasho0001 SharonEnoch3 moore_cjm MalminderWrites DavidRedelinghu
Nov 8, 2016
Starting ratio of the day: 220:376
Followed:
Via: @endeavor_global
