HiraF at 16:15, 20 June 2017

2017-06-20T16:15:28Z

HiraF: Created page with "This page is referenced in Hubs (Academic Paper) ==Venture Capital Data General Overview== The main goal of the data set is to aggregate company, fund, and round level da..."

2017-06-20T16:14:48Z

Created page with "This page is referenced in Hubs (Academic Paper) ==Venture Capital Data General Overview== The main goal of the data set is to aggregate company, fund, and round level da..."

New page

This page is referenced in [[Hubs (Academic Paper)]]

==Venture Capital Data General Overview==
The main goal of the data set is to aggregate company, fund, and round level data to be analyzed at a combined MSA and year level. The data set is compromised of two major parts: a granular company/fund/round and an aggregated CMSA-Year. The data includes all United States Venture Capital transactions (moneytree) from the twenty-five year period of 1990 through 2015.

The Hubs data set, from SDC Platinum, has been constructed in the server:
Data files are in 128.42.44.181/bulk/Hubs
All files are in 128.42.44.182/bulk/Projects/Hubs
psql Hubs2

Sql files:
:<code>E:\McNair\Projects\Hubs\Data Script v10.txt</code>
Note: We need to check that everything in '''Data Script v9 Ariel.txt''' has been incorporated into v10

Table Header Rows + 5 lines:
:<code>E:\McNair\Projects\Hubs\Data Table List v2.txt</code>
Note: This was generated by '''Data Script v10.txt'''

===Procedure - Granular Table===
#Start with separate raw datasets for Companies, Funds, and Rounds - '''Locate Raw Datasets and Determine Pedigree'''
#Add Data to Each Individual dataset (e.g. add MSA code)
#Clean and standardize names (e.g. company or fund name) for each dataset
#Join the Datasets (here we need to exclude undisclosed companies)

===Procedure - CMSA-Year Table===
#Create a consistent CMSA-Year table to be used later
#Using the tables from the granular table, parse out the right data
#Join the parsed out data with the CMSA-Year Table
#Join these Tables

==VC Specific Tables and Procedure==
===Raw data tables===
#'''Funds''': fund name, first investment date, last investment date, fund closing date, address, known investment, average investment, number of companies invested, MSA, MSA code.
#'''Rounds''': round date, company name, state, round number, stage 1, stage 2, stage 3
#'''Combined Rounds''': company name, round date, disclosed amount, investor
#'''Companies''': company name, first investment, last investment, MSA, MSA code, address, state, date founded, known funding, industry
#'''MSA List''': MSA, MSA code, CMSA, CMSA code
#'''Industry List''': changes 6 industry categories to 4— ICT, Life Sciences, Semiconductors, Other

===Granular Table (Fund-Round-Company)===
The final table here contains all venture capital transactions by disclosed funds and portfolio companies, together with their CMSAs.
To get the table, we processed the raw data sets in the following steps:
#Clean '''Company''' data
##Import raw data companies
##Add variable 'CMSA' from data set MSA list, update variable 'industry' by joining data set industry list
##Remove duplicates and remove undisclosed companies
#Clean '''Fund''' data
##Import raw data funds
##Add variable 'CMSA'
##Remove duplicates and remove undisclosed funds
##Match fund names with itself using [[The Matcher (Tool) |The Matcher]] to get the standard fund names
#Clean '''Round''' data
##Import raw data rounds and combined rounds
##Add variables 'number of investment', 'estimated investment' and 'year'
##Remove duplicates and remove undisclosed funds
#'''Combine''' '''Companies''' and '''Rounds'''
##Combine cleaned companies and rounds data table on company names
##Add variable 'round number' and 'stage'
##Remove duplicates
#'''Combine''' '''Funds''' and '''rounds-companies'''
##Match fund names in rounds data table with standard fund names using [[The Matcher (Tool) |The Matcher]] to standardize fund names in rounds data table
##Join standard fund names to rounds-companies table
##Join cleaned funds table to rounds-companies table on standard fund names

Note: This was done by Ariel and then edited by Todd.

===CMSA-Year Aggregated Table===

The original MSA to CMSA was done by Rachel and used here. '''LOCATE THE FILE!!!'''

The final table contains number of companies and amount of investment, categorized by distance and stages, of each CMSA.

We processed data as follows:
#Create the '''CMSA-Year''' Table
##Create single variable tables: Distinct CMSA, year, stage, found year of fund and found year of company.
##Create the cross production tables: CMSA-year, CMSA-year-fund year founded and CMSA-year-company year founded
#Draw data from cleaned companies, funds and rounds tables
##Create a table with 'CMSA', 'number of companies' and 'year Founded' from cleaned companies table and join it to CMSA -year founded
##Create a table with 'Company CMSA', 'round year', 'disclosed amount' from rounds-companies combined table, and add stage binary variables. Join it to CMSA-year-company year founded
##Create a table with 'CMSA', 'fund year', 'number of investors' from cleaned funds table and join it to CMSA-year-fund year founded
#Create '''near-far''' and stages table
##Add fund data to rounds-companies
##Create near-far and stages binary variable
##Count investment and deals by CMSA and year, categorized by near-far and stages
#Combine all tables by CMSA and round-year

==Supplementary Data Sets==

Supplementary data sets are cleaned and joined back to CMSAyear table on CMSA and year:

#Number of STEM graduate student, by university and year(2005 to 2014).
#University R&D spending, by university and year(2004 to 2014).
#Income per capital, by MSA and year(2000 to 2012)
#Wages and salaries, by MSA and year(2000 to 2012)

All of these files were created originally by Rachel. Some were cleaned in Excel. No new data was added (some extra cols, no extra rows).

The datasets can respectively be found at:
E:\McNair\Projects\Hubs\STEM grads for upload v2.xls
--Contains: university zipcode newmsacode msa msacode cmsa cmsacode year nostudents
--CMSA code inside sheet seems to be ours. Check with Ariel.
E:\McNair\Projects\Hubs\NSF spending for upload.xls
--Contains: Institution MSA CMSA code Year Spending
--We think the CMSA Code is ours. Check with Ariel.
E:\McNair\Projects\Hubs\Income per capita upload.xls
--Contains: Fips Area Year Income
--Lookup to CMSA was done using VLOOKUPs in Excel. See Matcher Helper vTR.xls, and other Matcher Helper ???.xls files
E:\McNair\Projects\Hubs\Wage for upload v2.xls
--Contains: Fips MSA Year Wage
--Lookup to CMSA was done using VLOOKUPs in Excel. See Matcher Helper vTR.xls, and other Matcher Helper ???.xls files

← Older revision		Revision as of 16:15, 20 June 2017
Line 110:		Line 110:
	--Contains: Fips MSA Year Wage		--Contains: Fips MSA Year Wage
	--Lookup to CMSA was done using VLOOKUPs in Excel. See Matcher Helper vTR.xls, and other Matcher Helper ???.xls files		--Lookup to CMSA was done using VLOOKUPs in Excel. See Matcher Helper vTR.xls, and other Matcher Helper ???.xls files
		+
		+	[[category:Internal]]

Old Completed Work on Hubs - Revision history

HiraF at 16:15, 20 June 2017

HiraF: Created page with "This page is referenced in Hubs (Academic Paper) ==Venture Capital Data General Overview== The main goal of the data set is to aggregate company, fund, and round level da..."