Incubator Seed Data - Revision history

Ed at 17:41, 21 September 2020

2020-09-21T17:41:43Z

Ed at 17:34, 21 September 2020

2020-09-21T17:34:18Z

Ed at 16:29, 20 September 2020

2020-09-20T16:29:13Z

Ed at 16:29, 20 September 2020

2020-09-20T16:29:11Z

Ed: /* Assembling the data */

2020-03-10T17:44:09Z

Assembling the data

Ed at 23:34, 4 October 2019

2019-10-04T23:34:31Z

Ed: /* Assembling the data */

2019-10-04T21:57:12Z

Assembling the data

Ed: /* Chosen Sources */

2019-10-03T23:52:11Z

Chosen Sources

Ed: /* Assembling the data */

2019-10-03T23:50:56Z

Assembling the data

Ed: /* Assembling the data */

2019-10-03T23:50:08Z

Assembling the data

@@ Line 1: / Line 1: @@
 {{Project
-|Has project output=
+|Has project output=Data
 |Has sponsor=Kauffman Incubator Project
 |Has sponsor=Kauffman Incubator Project

@@ Line 1: / Line 1: @@
 {{Project
 |Has sponsor=Kauffman Incubator Project
 |Has sponsor=Kauffman Incubator Project

@@ Line 1: / Line 1: @@
 {{Project
 |Has sponsor=Kauffman Incubator Project
 |Has title=Incubator Seed Data

@@ Line 1: / Line 1: @@
 {{Project
 |Has title=Incubator Seed Data
 |Has owner=Anne Freeman,

← Older revision		Revision as of 17:44, 10 March 2020
Line 346:		Line 346:
	*source --2137		*source --2137

−	~~Note that it~~ was ~~surprising that there wasn't greater overlap within~~ the ~~Crunchbase-INBIA-AngelList (CIA) data, or between~~ the ~~CIA data and US~~ Incubators. This ~~suggests that either each source is capturing different types of incubators~~, ~~or that we are unlikely~~ to ~~have near~~-~~population coverage.~~	+	The URL field was then processed using the cleanurl function to create WHOIS parsable domains. A new table called IncubatorWCount was created combining the information in Incubators with the counts of distinct domains. This was then processed by hand in Excel. The resulting clean file was re-imported as IncubatorsProcessed, and restricted to keep=1 in IncubatorsClean. The result has 1999 records with the following coverage:
		+	*statecode --1999
		+	*url --1872
		+	*description --1389
		+	*city --1854
		+	*address --909
		+	*zip --578

@@ Line 4: / Line 4: @@
 |Has project status=Active
 |Is dependent on=Crunchbase Database, INBIA, Google Crawler
 }}
 Requirement: Determine at least 4 primary data sources, or secure licenses to extract ‘seed data’ from these sources, as measured by program records.

← Older revision		Revision as of 21:57, 4 October 2019
Line 334:		Line 334:
	The CIA data is then combined with [[US Incubators]] data, which is separately available in '''USIncubators.txt''', and everything is matched using name based matching to try to remove duplicates (within states) and produce the best information. The result can then be matched back to Crunchbase. There were 2155 distinct orgnames, 37 of which had internal name matches.		The CIA data is then combined with [[US Incubators]] data, which is separately available in '''USIncubators.txt''', and everything is matched using name based matching to try to remove duplicates (within states) and produce the best information. The result can then be matched back to Crunchbase. There were 2155 distinct orgnames, 37 of which had internal name matches.
	perl Matcher.pl -mode=2 -file1="DistinctIncubatorOrgNames.txt" -file2="DistinctIncubatorOrgNames.txt"		perl Matcher.pl -mode=2 -file1="DistinctIncubatorOrgNames.txt" -file2="DistinctIncubatorOrgNames.txt"
		+
		+	The result is the table '''Incubators''' and text file '''Incubators.txt''' with 2137 records and the following coverage:
		+	*orgnamestd --2137
		+	*orgname --2137
		+	*statecode --2137
		+	*url --2031
		+	*description --1447
		+	*city --1955
		+	*address --970
		+	*zip --624
		+	*source --2137
		+
		+	Note that it was surprising that there wasn't greater overlap within the Crunchbase-INBIA-AngelList (CIA) data, or between the CIA data and US Incubators. This suggests that either each source is capturing different types of incubators, or that we are unlikely to have near-population coverage.

@@ Line 21: / Line 21: @@
 *[[AngelList Database|AngelList]]
 *[[Google Crawler]]
-*[[Yi Ma]]'s work assembling [[US Incubators]], state-by-state
+*[[Yi Ma]]'s work assembling [[US Incubators]], state-by-state, for this project
 *ClusterMapping
 *Wharton entrepreneurship club

@@ Line 332: / Line 332: @@
 *21 (very mixed type) in Wharton.txt
-The CIA data is then combined with [[US Incubator]] data and everything is matched using name based matching to try to remove duplicates (within states) and produce the best information. The result can then be matched back to Crunchbase. There were 2155 distinct orgnames, 37 of which had internal name matches.
+The CIA data is then combined with [[US Incubators]] data, which is separately available in '''USIncubators.txt''', and everything is matched using name based matching to try to remove duplicates (within states) and produce the best information. The result can then be matched back to Crunchbase. There were 2155 distinct orgnames, 37 of which had internal name matches.
   perl Matcher.pl -mode=2 -file1="DistinctIncubatorOrgNames.txt" -file2="DistinctIncubatorOrgNames.txt"

@@ Line 327: / Line 327: @@
 *zip --415
-We also have three sources that have a mix of types:
+We also have three sources that have a mix of types, which are not yet loaded into this data:
 *361 (with some non-incubators) in Gaebler.txt
-*292 (mixed type) in ClusterMapping.txt
+*292 (very mixed type) in ClusterMapping.txt
-*21 (mixed type) in Wharton.txt
+*21 (very mixed type) in Wharton.txt