Existing Crawlers

Twitter Crawler

http://mcnair.bakerinstitute.org/wiki/Twitter_Webcrawler_(Tool) https://github.com/scroungemyvibe/mcnair_center_builds/blob/master/Twitter_Webcrawler_Build.py

What it does: Given the input of a twitter username, builds a chart of their most recent 200 tweets and information related to their tweets including content of tweet, number of retweeters, number of mentions, number (and what kind) of hashtags and number of favorites.

Potential (Simple) Uses to Us: - Organize the tweets by how many Retweets they get, then see what hashtags were used that could be useful to our tweets or lead our content in a specific direction - Build a count of people who RT tweets that have content similar to ours, assign scores to these users based on how often they engage with McNairish content, how many followers they have etc to produce lists of people we should follow/ try to respond to

Event Brite Crawler

https://github.com/scroungemyvibe/mcnair_center_builds/blob/master/EventBrite_Webcrawler_Build.py

What it does: Given the ID of an event organizer, creates a dataset of all the events being put on by that organizer. You can also query by name of event "startup weekend" and by category "science and technology" for example

Potential Uses to Us: Basically find content for our twitter. Although it is already pretty easy to find good content by just searching on the actual website, we could work to refine search queries using the crawler to automatically return a list of events that are potentially interesting to our twitter follower whenever the people in charge of the twitter want to post about events.