7,612
edits
Changes
Jump to navigation
Jump to search
===Timeline of Submission===The BibTeX reference is (pending update with volume and number):
This paper had a storied submission process:*The deadline There were three reviewers for resubmission was June 15th, 2020. Before this deadline, I emailed the editors and offered them either this version of the paper, which contains no empirics, or an empirical paper without examples or definitions. I received no response.*This version of the paper was submitted as an 2nd R&R to a Special Issue of Research Policy on June 10th, 2020, with manuscript number RESPOL-D-19-01438R1. *On September 15th, I sent an email to the editors requesting information but received no response. *I wrote to the editors again on October 27th, this time using the Elvesier form, to request another update. The last status reported by Elvesier (https://ees.elsevier.com/respol/default.asp) was 'Required Reviews Complete' on October 9th, 2020.*On October 28th, I received an email saying: "Hello Ed. I hope to get back to you shortly. I have two good reviews and I’m waiting on a third. This most definitely will be another R&R. More soon"*On November 8th, I got an official email about the paper that said: "We have now received the referees' reports on your paper, copies of which I enclose below for your information. As you will see, the referees make various Reviewer 2 never returned any comments and suggestions for improvement. I have given up on the third reviewer and want to return Reviewer 1 accepted the paper to you." However, this email only contained one review. I requested clarification and noted that Reviewer 3 had asked for empirics.On November 11th I got an email that said: "Hello Ed, This is strange. The comments were in the comments to the editor. Here they are and they are not worth that much. This special issue has a specific purpose. '''You do not need to run regressions!'''" The comments are below as Reviewer 1. They indicate that the reviewer accepted the paperrevision.
===2nd R&R===The editor said:
Note :"We would be glad to reconsider a resubmitted paper, revised in the light of the referees' comments.:If you decide to revise the paper, it would be very useful if you could also include an author's response to the referees, listing what changes you have (or have not) made and where.:If you choose to revise your paper, could you please ensure that it is resubmitted on or before Feb 06, 2021. (If there Reviewer 2 never returned any commentsis too long a gap, referees may have forgotten what they said previously or be unwilling to review the revised paper, causing further delays.)"
At this point in my career, It's Reviewer 3's comments that I'm unique placed need to give the response that every academic really wants address. Jim kindly reminded me to give ascribe only normative motivations to the Reviewer (rather than positive theories). He also said to ''self-aggrandizing idiot' that somehow always ends up controlling follow the fate review''' and respond to each point with one of our hard work, namelythree sentiments:*Disagree*Good point but beyond the scope of the paper*Good point and I address it like this...
:"Dear Reviewer. After carefully considering your commentsHe also reminded me that the median review is a reject, I would like and that this a `good' review based on the offer distribution of reviews. The reviewer does say that the following responsepaper is useful and helpful: I find your suggestions for my beautiful :"The defining of key terms is a useful contribution of this paper to puerile/irrelevant/narcissistic/useless/stupid/all-, as are the identification of-potentially useful metrics of HGHT entrepreneurship. Furthermore, the-above (delete as appropriate), so I'm going to ignore them and, by extension, you. Up yoursexamples are often helpful in highlighting their various applications."
HoweverThe reviewer listed the following three major flaws::"In my view, it does seem that '''some''' of reviewer 3's comments would lead to improvements the major flaws in the paper. These study are:#Being clearer that standardized measures aren't a panacea#A better discussion not (1) considering possible downsides of gaming and incentives#Better framing each individual metric, (2) considering possible downsides in using the front end entire battery of why these measures make sense , and (likely using the points from RFP for the special issue3)testing alternatives to this measurement approach."
In my letter to Also for reference, here are the measures in the reviewer, I can explain that this paper is for a special issue and that latest version of the editors and I have agreed that this should not be and an empirical paper :*Measure 1 (Apportioned investment and exit value)*Measure 2 (MOOMI ratio)*Measure 3 (Pipeline)*Measure 4 (Raise Rate)*Measure 5 (Cost per raise)*Measure 6 (Repeat VC)*Measure 7 (ESO Expertise) The old Startup Ranking measure has been dropped. Also, there are two definitions that should not contain regressions or other testsare close to being measures:*Definition 5 (Local VC)*Definition 8 (Expert) This version also adds Measure 5 (Cost per raise) as a numbered measure. That might tamp down their vitriol somewhat Measures 1 and 2 provide ways to calculate proxies for return quartiles. Whether I deliver back a "major revision" is in Measures 3, 4 and 5 are the eye central measures of the beholder, framework. Measures 6 and there's no need 7 are alternative ways to draw attention to this demandassess the performance of pipeline components (measure 7 is possible without VC investment data).
====RP Constraints====
The files are This paper came from a presentation that I made to the Kauffman UMM Grant Cohort. I originally attempted to add empirics but this approach necessarily reduced the coverage of material: Although the framework is simple and used in practice, it is also on the frontier of research, so there aren't any published academic papers with the empirics. So I opted to break the original submission in: E:\projects\MeasuringHGHTEcosystems /bulk/vcdb4two - breaking the empirics back out and leaving this as the best attempt I could make at a narrative-based exploration of the whole framework. It is, as a consequence, a very unusual paper. But most people I showed it to were enthusiastic. It is also reference-bait. Outside the review process, some readers were both amused and worried about its snarky tone, which I'm still trying to address.
==Notice==This paper had a storied resubmission process:*The deadline for resubmission was June 15th, 2020. Before this deadline, I emailed the editors and offered them either this version of the paper, which contains no empirics, or an empirical paper without examples or definitions. I received no response.This *The first revision of the paper was broken into twosubmitted as an R&R to a Special Issue of Research Policy on June 10th, 2020, with manuscript number RESPOL-D-19-01438R1. *On September 15th, I sent an email to the editors requesting information but received no response. *I wrote to the editors again on October 27th, this time using the Elvesier form, to request another update. The last status reported by Elvesier (https://ees.elsevier.com/respol/default.asp) was 'Required Reviews Complete' on October 9th, 2020.*Measuring HGHT Entrepreneurship EcosystemsOn October 28th, I received an email saying: "Hello Ed. I hope to get back to you shortly. I have two good reviews and I’m waiting on a third. This most definitely will be another R&R. More soon"*On November 8th, I got an official email about the paper that said: "We have now contains received the definitionsreferees' reports on your paper, measurescopies of which I enclose below for your information. As you will see, the referees make various comments and suggestions for improvement. I have given up on the third reviewer and examplewant to return the paper to you. It is an informal" However, by-example theory paperthis email only contained one review. I requested clarification and noted that Reviewer 3 had asked for empirics.*[[Determinants of Future Investment On November 11th, I got an email that said: "Hello Ed, This is strange. The comments were in Uthe comments to the editor. Here they are and they are not worth that much. This special issue has a specific purpose.S '''You do not need to run regressions!'''" (Note that the comments indicate that reviewer 1 accepted the paper. Startup Cities]]: The empirical analysis )*On February 6th, 2021, I submitted the second revision of ESOs is now in this the paper!.
Measuring High-Growth High-Technology Entrepreneurship Ecosystems (view source)
Revision as of 16:46, 8 June 2021
, 16:46, 8 June 2021→Data and Analysis
|Has title=Measuring High-Growth High-Technology Entrepreneurship Ecosystems
|Has author=Ed Egan,
|Has paper status=R and RPublished
}}
==Current Final Version==
*The current final versionwas accepted to Research Policy on May 17th, 2021. which *The 50-day share link is currently : https://authors.elsevier.com/a 2nd R&R at Research /1d8SaB5ASINVf *The title was changed to "A Framework for Assessing Municipal High-Growth High-Tech Entrepreneurship Policy is:"
<pdf>File:MeasuringHGHTEntrepreneurshipEcosystemsV3Egan_(2021)_-3_A_Framework_for_Assessing_Municipal_High-Growth_High-Tech_Entrepreneurship_Policy.pdf</pdf>
@article{EGAN2021104292, title = {A framework for assessing municipal high-growth high-technology entrepreneurship policy}, journal = {Research Policy}, pages = {104292}, year = {2021}, issn = {0048-7333}, doi = {https://doi.org/10.1016/j.respol.2021.104292}, url = {https://www.sciencedirect.com/science/article/pii/S0048733321000937}, author = {Edward J. Egan}, keywords = {Entrepreneurship, Ecosystem, Measurement, High-growth high-technology, Venture capital, Ecosystem support organization, Pipeline, Raise rate, Policy cartel}, abstract = {This paper came from advances a presentation that I made to the Kauffman UMM Grant Cohortframework for making rudimentary need, impact, and cost–benefit assessments of municipal high-growth high-tech entrepreneurship policy. I originally attempted to add empirics but this approach necessarily reduced the coverage of material: Although the The framework is simple views ecosystem support organizations like accelerators, incubators, and used hubs as components in practicea city’s venture pipeline. A component’s pipeline size, raise rate, and cost per raise measure its performance. In total, it is also the framework consists of eight objective and reproducible measures based on quantities and qualities of venture capital investment and 16 definitions of related terms-of-the frontier of research-art. These measures and definitions are illustrated in 26 real-world policy examples, so there aren't any published academic papers with which assess initiatives in Houston and St. Louis over the empiricslast 20 years. So I opted to break the original submission The examples reveal an enormous variation in two welfare effects, and some policies appear welfare destroying. Many non- breaking the empirics back out profit organizations claim success (and win awards and leaving this as the best attempt I could make acclaim) using non-standard measures despite performing at a narrative-based exploration of the whole frameworkless than half benchmark levels. It isPolicy cartels, as a consequencewhich control startup policy in many U.S. cities, a very unusual paperalso engage in non-market actions to protect their rents.} } The final file series was v4-6-2 in: E:\projects\MeasuringHGHTEcosystems /bulk/vcdb4 Egan (2021) - A Framework for Assessing Municipal High-Growth High-Tech Entrepreneurship Policy. But most people I showed it pdf Production files (sent to were enthusiasticResPol):*MeasuringHGHTEntrepreneurshipEcosystemsV4-6-2. It is also referencetex*MeasuringHGHTEntrepreneurshipEcosystemsV4-6-2-baitTitlePage. Outside tex*References.bib*HoustonPipelineV4.png*HoustonVCRaiseRateWithBenchmarkV4.png*econ.bst ==Notice== The original Measuring HGHT Entrepreneurship Ecosystems paper was broken into two:*:'''A Framework for Assessing Municipal High-Growth High-Technology Entrepreneurship Policy''' now contains the review processdefinitions, measures, some readers were both amused and worried about its snarky toneexamples. It is an inductive, which I'm still trying to addresscase-study paper.*[[Determinants of Future Investment in U.S.Startup Cities]]: The empirical analysis of ESOs is now in this paper! ==2nd R&R==
Summarizing reviewer 1's comment comments (i.e., reading between the lines):
*The writing is currently pretty good: '''it is well done'''... '''the paper is polished'''... '''very nicely done'''.
*It The paper works as a whole. The reviewer didn't want anything cut: '''It is a collection of case studies and definitions''', and '''I don't have ... major comments'''. Reviewer 3's comments are more problematic. As is often the case, I wonder whether the reviewer actually read the paper:*The paper advances seven new measures, not 15 as the reviewer claims*Policy cartels are introduced in section 4.1 (out of 5), and aren't the main focus of the paper per se*I never use the entire battery of measures -- different measures are applicable in different contexts *A key point of the paper is to stop organizations from self-selecting into measures that they do well on*All bar two sentences of the 'substantive material' in the review (i.e., from "First..." to the end) don't mention anything to do with the paper!
I would need put these as points 1, 2, and 3, and added material from the rest of his review (see below) to create the list of the bullet-points that I'll address Reviewer 3's comments point-by-point, so here they are:#Consider possible downsides of each the individual metricmetrics.#Consider possible downsides in using the entire battery of measures.
#Test alternatives to this measurement approach
#Discuss when the welfare implications of putting more information in the hands of policymakers, including:##It is not necessarily true that "standardized measurement" improves outcomes, when it doessuch as policy, overall. All frameworks are not equal and perhaps howthat any conceived framework is not necessarily better than nothing.#Discuss #Measurements are rarely if ever neutral (vis-à-vis behavior). There can be systematic bias due to gaming metrics incentivized by rewards attached to the measurement (old and new). Specifically, note that this behavioral change might have nothing to do with #Consider the relationship between the underlying conceptual phenomenon and how it is operationalized:##Discuss alternative calculations of interesta measure when applicable (i. (Think "Rewarding A While Hoping For B", etce., multitask, etcfor the ranking measure).)#Justify the choices in the reduction of the #Discuss that measurement space Measurement is often reductive: What is left out? Is it important? .##Discuss the difference between the conceptual phenomenon and how it is operationalizedsystematic bias.
#Test the effects of using a framework on various outcomes.
Although no-one has asked me to, I need to cut down the length of the paper. RP asks for a maximum word count of 8,000-10,000, though does allow exceptions. Using pdftotext and wc, or doing a count in the tex file, it looks like I'm up near 17k. However, both approaches drastically inflate the total with table entries, TeX commands, etc. I expect the current true count is closer to 14k15k. I need to get it down to at most 12k.
A different way of coming at the problem is to say that I'm using 12pt font with 1" margins and one-half line spacing (as implemented using \onehalfspacing in LaTeX, which [https://tex.stackexchange.com/questions/65849/confusion-onehalfspacing-vs-spacing-vs-word-vs-the-world isn't the same as 1.5 spacing in word]). AER restricts to 40 pages using this format[https://eml.berkeley.edu//~sdellavi/wp/pagelimits12-12-11.pdf], and Management Science restricts to 32 pages[https://pubsonline.informs.org/page/mnsc/submission-guidelines]. With averages of perhaps 15 words per line and 32 lines per page, this would give just shy of 500 words per page, making 10k words, 20 pages of dense text, and perhaps 30 pages with figures, tables, quotes, headings, definitions, etc.
Because the last round was also an R&R, I presumably met all the other constraints.
===Special Issue Questions===
The special issue posed a series of questions (the full list is below) that I might rephrase to statements as follows:
#Startups (venture-backed and pre-venture backed) are the fundamental constructs, so they are what should be measured, and cities (census places geographically define ecosystems)
#Consistent measurement is key!
#The pipeline framework lays out the key relationships as it maps the mechanisms by which ecosystems get established, mature, decline, or get renewed.
#Venture capital measures add quality dimensions. Using the pipeline framework on top of this quality measure gives more finely grained evaluations of the effectiveness of policy instruments
#Venture capital has timing issues but exits are noisiers and longer term (short vs. long term measures)
#What do existing rankings for ecosystems measure?
#Beyond direct pipeline components, we should think about universities, corporations, and other participants.
#The measures should be able to assess the question: To what extent is government policy accelerating or inhibiting the progress of entrepreneurial ecosystems?
===Reviewers Comments===
The raw reviewer's comments are as follows:
====Reviewer 1's Comments====
In summary, more skepticism about the usefulness of the metrics and the framework, and more empiricism is necessary.
==FilesTimeline of Submission==
==Research Policy Special Issue==
==Data and Analysis==
The paper will use uses [[vcdb4VCDB20]] and [[US Startup City Ranking]], as well as a wealth of old McNair material. Sources include (copied to the project folder unless otherwise noted):
*[[Hubs]]: Hubs Data v2_'16.xlsx
*[[Federal Grant Data]], including NIH, NSF and other grant data, especially SBIR/STTR. Possibly also contract data.
