Talk:Accelerator Seed List (Data)

Hi Veeral,

Intro

Welcome to the project. The documents are here: E:\Mcnair\Projects\Accelerators

SQL documents are here: E:\Mcnair\Projects\Accelerators\SQL_Data

Database Drive is here: Z:\Bulk\Accelerators

The database is called accelerator

Important docs

The SDC pull that includes all of the round data since 1999: E:\Mcnair\Projects\Accelerators\VC_Data_Repeated_Down.txt or E:\Mcnair\Projects\Accelerators\"VC Data.xlsx"

The Cohorts of accelerators (under the Updated tab on the bottom): E:\Mcnair\Projects\Accelerators\"Clean Cohort Data.xlsx"

The Crunchbase Snapshots of organizations: E:\Mcnair\Projects\Accelerators\"Crunchbase Snapshot"\organizations.csv

To-do list

1. Filter out actual accelerators from the Crunchbase organizations data

• Possibly by running accelerator_keywords.py
• Possibly by using string searching in organizations.csv
• Watch out for Venture capital companies (the organizations file has many of these and we'll probably pick up a lot in our "accelerator" filtered list

2. Match this list against the current list of accelerators

• We have our own copy of the matcher in the accelerators E drive (try mode 1 and mode 2 for different results, mode 2 might be more helpful)
• This will tell you whether it was part of the old list or not (and therefore whether we need to get data for it or not)

3. Find cohort data for all of the new accelerators (ones not previously on the list & if they're not accelerators take them off the list)

• We used regex for this
• once you find the cohort data put it into the updated cohort data list excel file
  • You just need the cohort company name and the name of the accerator

4. Match the cohort data against the round data from SDC

• Make sure to get both the accelerator name and the cohort company name in the first document
• In the second document (to match against the first) put the list of all companies funded in rounds (from SDC)
• in summary: File1 = Accelerator Cohorts and File2 = SDC data

5. Upload the match file into the psql database, then follow the code in accelerators.sql

• making new code with your new uploaded tables and documents, you should just be able to follow what we've already done to get a similar percentVC table
• The previous percent VC table you'll want it to look like is PercentVc4

^this above is all for the VC percentage rankings

For more info you can use the whoisparser which will get data on website registration (location, time, who, potentailly age if you consider the website registration date as an age) You can also do an automated google lookup (this will harvest addresses that are within google)

^These two will get you the information of where & how old

Don't worry about this stuff

Rank on VC

• Getting a VC percentage for each Accelerator

Also categorize

• Age
• Nonprofit or not
• Location

RegEx Code for repeating data down for the round data from SDC:

\n([^\t]+\t[^\t]*\t[^\t]*\t[^\t]*\t[^\t]*\t[^\t]*\t[^\t]*\t[^\t]*\t[^\t]*\t[^\t]*\t)(.*)\n\t\t\t\t\t\t\t\t\t\t

\n\1\2\n\1

=if(isnumber(search("blah",B2))=TRUE,1,0) where blah is the substring (what you're searching for), B2 is the string (what your searching in) and 1 represents that it's present and 0 means it isn't.

=sum(A1:C1) This just sums the cells from A1 to C1

Veeral's Summer Work

All New Files in E:\McNair\Projects\Accelerators and what they contain:

Cleaned Cohort Data (Excel) - The sheet named "Veeral - Updated"

Complete Master List of Accelerators

(Note: all files are found and stored under E:\McNair\Projects\Accelerators)

Transfer all of the organizations data into Access

• Done - Organizations.accdb

Use Access keyword queries with the short descriptions of each organization to accumulate a list of Potential Accelerators from Organizations data

• Companies with atleast 2 keywords from [accel, startup, mentor, seed, program, week, pitch, found, stage, incubat]
• Companies with location_country_code = USA
  • 381 Potential Accelerators (These are not exclusively Accelerators -- some VC firms and startup firms snuck into the list from initial glance. Plan is to match it with list of accelerators and then eliminate the ones that do not match that are not accelerators in that step.

Match Potential Accelerators with Cleaned Cohort Data using The Matcher (Tool).

1. List of current accelerators obtained from Cleaned Cohort Data is in Organizations.accdb under the query, "List of Accelerators". The 381 Potential Accelerators are under the "Potential Accelerators" Query.

2. Matched the Cleaned Cohort Data accelerator list with the potential accelerators obtained from the 2013 Crunchbase snapshot. There were 329 potential accelerators.

3.Manually went through the 329 potential accelerators by google searching and came up with 101 new accelerators - Can be found at ____________ (TBD)

4. Finding all of the cohorts of each new accelerator.

• Organized each cohort so the Name is in the first column and Description is in the second column.
• Saved each cohort txt file under the format "..Cohort Name..".cohort - for example, the cohorts of Velocity Accelerator would be saved under "Velocity Accelerator.cohort"

5. I am now going to add the new accelerators to our existing list and cross check our new, updated list of accelerators with all of the sources of accelerators that we've gone through so far plus the new 2017 Crunchbase data.

Updated Cleaned Cohort List

Using the 70 or so new accelerators obtained from the Crunchbase snapshot, I ran Peter's "parse_cohort_data" script located in E:\McNair\Projects\Accelerators\Code+Final_Data on the new accelerator cohort files, all in the New Crunchbase Accelerator Cohorts Folder in Data (E:\McNair\Projects\Accelerators\Data\New Crunchbase Accelerator Cohorts)

RESULTS

1. Cleaned Cohort Data (E:\McNair\Projects\Accelerators\Cleaned Cohort Data.xls)

• Updated 7-12-2017 sheet - Most up to date Accelerator cohort data
• Updated sheet - Accelerator cohort data minus the new 70~ Crunchbase 2013-snapshot-obtained accelerators

2. Updated Cleaned Cohort Data 7-12-2017 - (E:\McNair\Projects\Accelerators\Updated Cleaned Cohort Data 7-12-2017.txt)

Global Accelerator Network Parser Spec

HTML File - E:\McNair\Projects\Accelerators\GAN_data.txt

An entry:

 <div class="member_entry clear">
  ...
 </div>
 

Within an entry:

Logo:

 <header class="member">
  	<div class="logo">
  	<a href="http://gan.co/members/view/desai-accelerator"><img alt="123_large" src="./GAN_files/123_large.png"></a>
  	</div>
 

Name:

 
 <header class="member">
 	<h2 class="name">
 	<a href="http://gan.co/members/view/desai-accelerator">Desai Accelerator</a>
 	</h2>
 

Location:

  <header class="member">
 	<h3 class="location">
 	Ann Arbor, MI, USA
 	</h3>
 

For Statistics on Companies:

We want stats for -- section class = "companies", "companies_funded", "companies_funded_raised", "funding_raised", "exits", "exit_funding", "employees", "mentors", "years"

 <section class="stats clear">
 <ul class="single_stats clear">
 <a href="http://gan.co/members/standalone_filter?label=total_companies&span=20,-1" class="companies">
 	<span class="icon hide_text">GAN Compass</span>
 	<strong class="number">Under 20</strong>
  	<em class="caption">
    		Graduated Companies
   	</em>
 </a>
 

For Terms of Companies

We want terms for equity stake and S25k seed capital

 <div class="terms_holder clear">
 	<section class="terms">
 		<h4>Terms</h4>
 		<p>
 			<a href="http://gan.co/members/standalone_filter?label=terms_equity&span=4,-1">0% equity stake</a>
 			for
 			<a href="http://gan.co/members/standalone_filter?label=terms_seed&span=26,20">$25k seed capital</a>
 		</p>
 	</section>
 	</div>
 

Parser Results

The code and the resulting tab-separated text file are located here:

    E:\McNair\Projects\Accelerators\Web Scraping for Accelerators