Old Completed Work on Hubs

This page is referenced in Hubs (Academic Paper)

Venture Capital Data General Overview

The main goal of the data set is to aggregate company, fund, and round level data to be analyzed at a combined MSA and year level. The data set is compromised of two major parts: a granular company/fund/round and an aggregated CMSA-Year. The data includes all United States Venture Capital transactions (moneytree) from the twenty-five year period of 1990 through 2015.

The Hubs data set, from SDC Platinum, has been constructed in the server:

Data files are in 128.42.44.181/bulk/Hubs
All files are in 128.42.44.182/bulk/Projects/Hubs
psql Hubs2

Sql files:

E:\McNair\Projects\Hubs\Data Script v10.txt

Note: We need to check that everything in Data Script v9 Ariel.txt has been incorporated into v10

Table Header Rows + 5 lines:

E:\McNair\Projects\Hubs\Data Table List v2.txt

Note: This was generated by Data Script v10.txt

Procedure - Granular Table

1. Start with separate raw datasets for Companies, Funds, and Rounds - Locate Raw Datasets and Determine Pedigree
2. Add Data to Each Individual dataset (e.g. add MSA code)
3. Clean and standardize names (e.g. company or fund name) for each dataset
4. Join the Datasets (here we need to exclude undisclosed companies)

Procedure - CMSA-Year Table

1. Create a consistent CMSA-Year table to be used later
2. Using the tables from the granular table, parse out the right data
3. Join the parsed out data with the CMSA-Year Table
4. Join these Tables

VC Specific Tables and Procedure

Raw data tables

1. Funds: fund name, first investment date, last investment date, fund closing date, address, known investment, average investment, number of companies invested, MSA, MSA code.
2. Rounds: round date, company name, state, round number, stage 1, stage 2, stage 3
3. Combined Rounds: company name, round date, disclosed amount, investor
4. Companies: company name, first investment, last investment, MSA, MSA code, address, state, date founded, known funding, industry
5. MSA List: MSA, MSA code, CMSA, CMSA code
6. Industry List: changes 6 industry categories to 4— ICT, Life Sciences, Semiconductors, Other

Granular Table (Fund-Round-Company)

The final table here contains all venture capital transactions by disclosed funds and portfolio companies, together with their CMSAs. To get the table, we processed the raw data sets in the following steps:

1. Clean Company data
   1. Import raw data companies
   2. Add variable 'CMSA' from data set MSA list, update variable 'industry' by joining data set industry list
   3. Remove duplicates and remove undisclosed companies
2. Clean Fund data
   1. Import raw data funds
   2. Add variable 'CMSA'
   3. Remove duplicates and remove undisclosed funds
   4. Match fund names with itself using The Matcher to get the standard fund names
3. Clean Round data
   1. Import raw data rounds and combined rounds
   2. Add variables 'number of investment', 'estimated investment' and 'year'
   3. Remove duplicates and remove undisclosed funds
4. Combine Companies and Rounds
   1. Combine cleaned companies and rounds data table on company names
   2. Add variable 'round number' and 'stage'
   3. Remove duplicates
5. Combine Funds and rounds-companies
   1. Match fund names in rounds data table with standard fund names using The Matcher to standardize fund names in rounds data table
   2. Join standard fund names to rounds-companies table
   3. Join cleaned funds table to rounds-companies table on standard fund names

Note: This was done by Ariel and then edited by Todd.

CMSA-Year Aggregated Table

The original MSA to CMSA was done by Rachel and used here. LOCATE THE FILE!!!

The final table contains number of companies and amount of investment, categorized by distance and stages, of each CMSA.

We processed data as follows:

1. Create the CMSA-Year Table
   1. Create single variable tables: Distinct CMSA, year, stage, found year of fund and found year of company.
   2. Create the cross production tables: CMSA-year, CMSA-year-fund year founded and CMSA-year-company year founded
2. Draw data from cleaned companies, funds and rounds tables
   1. Create a table with 'CMSA', 'number of companies' and 'year Founded' from cleaned companies table and join it to CMSA -year founded
   2. Create a table with 'Company CMSA', 'round year', 'disclosed amount' from rounds-companies combined table, and add stage binary variables. Join it to CMSA-year-company year founded
   3. Create a table with 'CMSA', 'fund year', 'number of investors' from cleaned funds table and join it to CMSA-year-fund year founded
3. Create near-far and stages table
   1. Add fund data to rounds-companies
   2. Create near-far and stages binary variable
   3. Count investment and deals by CMSA and year, categorized by near-far and stages
4. Combine all tables by CMSA and round-year

Supplementary Data Sets

Supplementary data sets are cleaned and joined back to CMSAyear table on CMSA and year:

1. Number of STEM graduate student, by university and year(2005 to 2014).
2. University R&D spending, by university and year(2004 to 2014).
3. Income per capital, by MSA and year(2000 to 2012)
4. Wages and salaries, by MSA and year(2000 to 2012)

All of these files were created originally by Rachel. Some were cleaned in Excel. No new data was added (some extra cols, no extra rows).

The datasets can respectively be found at:

E:\McNair\Projects\Hubs\STEM grads for upload v2.xls
  --Contains: university	zipcode	newmsacode	msa	msacode	cmsa	cmsacode	year	nostudents
  --CMSA code inside sheet seems to be ours. Check with Ariel.
E:\McNair\Projects\Hubs\NSF spending for upload.xls
  --Contains: Institution	MSA	CMSA code	Year	Spending
  --We think the CMSA Code is ours. Check with Ariel. 
E:\McNair\Projects\Hubs\Income per capita upload.xls
  --Contains: Fips	Area	Year	Income
  --Lookup to CMSA was done using VLOOKUPs in Excel. See Matcher Helper vTR.xls, and other Matcher Helper ???.xls files
E:\McNair\Projects\Hubs\Wage for upload v2.xls
  --Contains: Fips	MSA	Year	Wage
  --Lookup to CMSA was done using VLOOKUPs in Excel. See Matcher Helper vTR.xls, and other Matcher Helper ???.xls files