83
edits
Changes
Jump to navigation
Jump to search
no edit summary
It might be possible to go state by state. California has 385, Massachusetts has 36, New York has 141, etc. But again, this is '''limited to the incubator type'''.
==Crawler==
We decided to build a webcrawler using selenium to search for incubators using the domain for angelList companies ''https://angel.co/companies?'' with the ''locations[]='' option appended to the end as a specified state (50 states and the district of columbia).The crawler loaded the page as specified and then clicked the load more button while there were still more results to load. No state exceeded 500 results. Then the crawler collected information for all of the companies listed including state, name of company, a brief description, and the url for the company within angelList. This information was stored in a tab delimitated text file.
===Crawler By Company Type===
This crawler appended ''company_types[]=Incubator'' to the url so that the companies appearing in the search results were only those with the listed company type of incubator. It yielded 1068 results. The script (angelList_companyTypeIncubator.py) and the data it generated (AngelList_companyTypeIncubator.txt) are on the RDP in the folder AngelList.
===Crawler By Keyword===
This crawler clicked on the search bar and entered the keyword "incubator" so that companies appeared in the results contained the keyword incubator somewhere on their company page. It yield 840 results. The script (angelList_keywordIncubator.py) and the data it generated (AngelList_keywordIncubator.txt) are on the RDP in the folder AngelList.
