Changes

|Has paper status=In developmentPublished

This The original Measuring HGHT Entrepreneurship Ecosystems paper was broken in into two:*Measuring HGHT :'''A Framework for Assessing Municipal High-Growth High-Technology Entrepreneurship EcosystemsPolicy''' now contains the definitions, measures, and examples. It is an inductive, case-study paper.*[[Determinants of Future Investment in U.S. Startup Cities]]: The empirical analysis of ESOs is now in this paper! ==2nd R&R== There were three reviewers for the 2nd R&R but Reviewer 2 never returned any comments. Reviewer 1 accepted the paper. Reviewer 3 asked for a revision. The editor said: :"We would be glad to reconsider a resubmitted paper, revised in the light of the referees' comments.:If you decide to revise the paper, it would be very useful if you could also include an author's response to the referees, listing what changes you have (or have not) made and where.:If you choose to revise your paper, could you please ensure that it is resubmitted on or before Feb 06, 2021. (If there is too long a gap, referees may have forgotten what they said previously or be unwilling to review the revised paper, causing further delays.)" Summarizing reviewer 1's comments (i.e., reading between the lines):*The writing is currently pretty good: '''it is well done'''... '''the paper is polished'''... '''very nicely done'''.*The paper works as a whole. The reviewer didn't want anything cut: '''It is a collection of case studies and definitions''', and '''I don't have ... major comments'''. It's Reviewer 3's comments that I need to address. Jim kindly reminded me to ascribe only normative motivations to the Reviewer (rather than positive theories). He also said to '''follow the review''' and respond to each point with one of three sentiments:*Disagree*Good point but beyond the scope of the paper*Good point and I address it like this... He also reminded me that the median review is a reject, and that this a `good' review based on the distribution of reviews. The reviewer does say that the paper is useful and helpful::"The defining of key terms is a useful contribution of this paper, as are the identification of potentially useful metrics of HGHT entrepreneurship. Furthermore, the examples are often helpful in highlighting their various applications." The reviewer listed the following three major flaws::"In my view, the major flaws in the study are not (1) considering possible downsides of each individual metric, (2) considering possible downsides in using the entire battery of measures, and (3) testing alternatives to this measurement approach." I put these as points 1, 2, and 3, and added material from the rest of his review (see below) to create the list of the bullet-points that I'll address:#Consider possible downsides of the individual metrics.#Consider possible downsides in using the entire battery of measures.#Test alternatives to this measurement approach#Discuss the welfare implications of putting more information in the hands of policymakers, including:##It is not necessarily true that "standardized measurement" improves outcomes, such as policy, overall. All frameworks are not equal and that any conceived framework is not necessarily better than nothing.##Measurements are rarely if ever neutral (vis-à-vis behavior). There can be systematic bias due to gaming metrics incentivized by rewards attached to the measurement.#Consider the relationship between the conceptual phenomenon and how it is operationalized:##Discuss alternative calculations of a measure when applicable (i.e., for the ranking measure).##Discuss that measurement is reductive.##Discuss systematic bias.#Test the effects of using a framework on various outcomes. Also for reference, here are the measures in the latest version of the paper:*Measure 1 (Apportioned investment and exit value)*Measure 2 (MOOMI ratio)*Measure 3 (Pipeline)*Measure 4 (Raise Rate)*Measure 5 (Cost per raise)*Measure 6 (Repeat VC)*Measure 7 (ESO Expertise) The old Startup Ranking measure has been dropped. Also, there are two definitions that are close to being measures:*Definition 5 (Local VC)*Definition 8 (Expert) This version also adds Measure 5 (Cost per raise) as a numbered measure. Measures 1 and 2 provide ways to calculate proxies for return quartiles. Measures 3, 4 and 5 are the central measures of the framework. Measures 6 and 7 are alternative ways to assess the performance of pipeline components (measure 7 is possible without VC investment data). ===RP Constraints=== Although no-one has asked me to, I need to cut down the length of the paper. RP asks for a maximum word count of 8,000-10,000, though does allow exceptions. Using pdftotext and wc, or doing a count in the tex file, it looks like I'm up near 17k. However, both approaches drastically inflate the total with table entries, TeX commands, etc. I expect the current true count is closer to 15k. I need to get it down to at most 12k. A different way of coming at the problem is to say that I'm using 12pt font with 1" margins and one-half line spacing (as implemented using \onehalfspacing in LaTeX, which [https://tex.stackexchange.com/questions/65849/confusion-onehalfspacing-vs-spacing-vs-word-vs-the-world isn't the same as 1.5 spacing in word]). AER restricts to 40 pages using this format[https://eml.berkeley.edu//~sdellavi/wp/pagelimits12-12-11.pdf], and Management Science restricts to 32 pages[https://pubsonline.informs.org/page/mnsc/submission-guidelines]. With averages of perhaps 15 words per line and 32 lines per page, this would give just shy of 500 words per page, making 10k words, 20 pages of dense text, and perhaps 30 pages with figures, tables, quotes, headings, definitions, etc.  Because the last round was also an R&R, I presumably met all the other constraints. ===Special Issue Questions=== The special issue posed a series of questions (the full list is below) that I might rephrase to statements as follows:#Startups (venture-backed and pre-venture backed) are the fundamental constructs, so they are what should be measured, and cities (census places geographically define ecosystems)#Consistent measurement is key!#The pipeline framework lays out the key relationships as it maps the mechanisms by which ecosystems get established, mature, decline, or get renewed. #Venture capital measures add quality dimensions. Using the pipeline framework on top of this quality measure gives more finely grained evaluations of the effectiveness of policy instruments#Venture capital has timing issues but exits are noisiers and longer term (short vs. long term measures)#What do existing rankings for ecosystems measure?#Beyond direct pipeline components, we should think about universities, corporations, and other participants.#The measures should be able to assess the question: To what extent is government policy accelerating or inhibiting the progress of entrepreneurial ecosystems? ===Reviewers Comments=== The raw reviewer's comments are as follows: ====Reviewer 1's Comments==== First of all, my sincere apologies to you and the authors for my very late report on the manuscript. Please excuse me for the delay.  The authors study measures of high-growth, high-tech entrepreneurship activity across the U.S. and provide concrete examples of how municipalities can use these to assess various policy initiatives. The authors claim that policy interventions at the municipal level have a significant impact on pre-venture startups, and that this has been missing from the extant academic literature.  Regarding my take on the paper: it is well done and given that it has already been revised once, I don't have many more major comments, but I do have one thing I would like to bring to your attention -- although the paper is polished, it doesn't seem like an academic paper with extensive empirical analysis (in fact, it doesn't have even a single regression) or with an analytical model that enhances our understanding of theory (doesn't have a single equation either). It is a collection of case studies and definitions, albeit very nicely done. Given my area of expertise in entrepreneurial finance and my own experience publishing an empirical study in ResPol, I feel ill-equipped to offer a recommendation on whether or not this paper fits the scope of ResPol. I believe that is largely an editorial decision and you would be the best judge for it. So, I will let you decide on this bigger question and will skip my minor suggestions for the author. I hope that is alright.  ====Reviewer 3's Comments====  Reviewer #3: I'm recommending a major revision to the paper, which would likely constitute a lot of additional work, but I feel would greatly strengthen its contribution. The objective of this paper is establishing fifteen measures of HGHT entrepreneurship activity, and giving examples of their application and potential usefulness especially with regard to the behavior of what the author terms policy cartels. The defining of key terms is a useful contribution of this paper, as are the identification of potentially useful metrics of HGHT entrepreneurship. Furthermore, the examples are often helpful in highlighting their various applications. In my view, the major flaws in the study are not (1) considering possible downsides of each individual metric, (2) considering possible downsides in using the entire battery of measures, and (3) testing alternatives to this measurement approach. I briefly elaborate on these three points in the following paragraphs. In the conclusion, the author states that two antecedents to improved policy are "standardized measures, to reduce the information asymmetry between policymakers and constituents…and…a simple but grounded framework that can reduce the expertise required to develop and enact productive startup policy." First, it is not necessarily true that "standardized measurement" improves outcomes, such as policy, overall. Measurements are rarely if ever neutral (vis-à-vis behavior). Two examples are,* First, we often implicitly assume that more information in the hands of decision makers is unambiguously good. But this requires a lot of assumptions that do not hold in real life. An additional measure, say, would influence decision making but perhaps the distortion is welfare decreasing. Nassim Taleb gives the example of "value at risk" (VAR), a metric commonly used in finance, in the foreword of the book Lecturing Birds on Flying (pp. xvi).* Second, systematic bias due to gaming metrics incentivized by rewards attached to the measurement. Said differently, an agent would have the incentive to change behavior in the least costly way in order to maximize the payoff associated with the measurement---this behavioral change might have nothing to do with the underlying phenomenon of interest. By way of example, Weisbrod, Ballou, Asch in their book Mission and Money, discuss various measures used in university rankings published by US News & World Report, and how these are often finessed by schools in ways that have little to do with education (see pp. 64-65 for one such discussion). The point is, any conceived standardized framework is not necessarily better than nothing, and not all frameworks would be of equal value. I would have liked to see a more rigorous discussion of the merits of the framework purposed, which analysis would include at least the following:* Discussion of alternative calculations of a measure when applicable. For example, the paper's Measure 1 is a composite measure of three sub-measures: why was this particular normalization of sub-measures (i.e., ranking) used? Why was this particular aggregation method (i.e., summation) used? What are the upsides and downsides of this and other approaches? For example, "the flow of dollars" would have a long right tail, which is obliterated when transformed to a rank. Is this a good or bad thing and why?* Measurement is often reductive (in the sense that they constitute a mapping from a high-dimensional, potentially complex space, to a far simpler space). What is left out? Is it important? In part this translates to discussing the difference between the conceptual phenomenon and how it is operationalized?* Might a measure be systematically biased or lead to bias if implemented?* Tests of the effects of using a framework on various outcomes.In summary, more skepticism about the usefulness of the metrics and the framework, and more empiricism is necessary. ==Timeline of Submission==

This paper had a storied resubmission process:*The deadline for resubmission was June 15th, 2020. Before this deadline, I am going to try shocking emailed the editors and offered them either this version of the number paper, which contains no empirics, or an empirical paper without examples or definitions. I received no response.*The first revision of ESOs using the political party paper was submitted as an R&R to a Special Issue of Research Policy on June 10th, 2020, with manuscript number RESPOL-D-19-01438R1. *On September 15th, I sent an email to the incumbent mayoreditors requesting information but received no response. Data is available from:*I wrote to the editors again on October 27th, this time using the Elvesier form, to request another update. The last status reported by Elvesier (https://libguidesees.princetonelsevier.educom/elections#s-lg-box-10082744respol/default.asp) was 'Required Reviews Complete' on October 9th, 2020.*httpsOn October 28th, I received an email saying://dataverse"Hello Ed.harvardI hope to get back to you shortly.edu/datasetI have two good reviews and I’m waiting on a third.xhtml?persistentId=doi:10This most definitely will be another R&R.7910/DVN/SJBWC3More soon"**FIPSOn November 8th, year (non-continuous -- election years)I got an official email about the paper that said: "We have now received the referees' reports on your paper, mayor_party_final (Dcopies of which I enclose below for your information. As you will see,R, etcthe referees make various comments and suggestions for improvement.), month, FIPS_Place_ID, I have given up on the third reviewer and otherswant to return the paper to you." However, this email only contained one review.I requested clarification and noted that Reviewer 3 had asked for empirics. From 1945 to 2014!***FIPS On November 11th, I got an email that said: "Hello Ed, This is 6 or 7dg (107000 BIRMINGHAM CITY, AL strange. The comments were in the comments to 5613900 CHEYENNE CITY, WY)the editor. Here they are and they are not worth that much. This special issue has a specific purpose. FIPS_Place_ID is 4 '''You do not need to 5dg run regressions!'''" (7000 to 13900Note that the comments indicate that reviewer 1 accepted the paper.)***compare with placefp (5dg) 07000 BurminghamOn February 6th, AL and 13900 Cheyenne2021, WY, or better still GEOID (used in vcdb4) 0107000 Birmingham, AL and 5613900 Cheyenne, WYI submitted the second revision of the paper.

The paper will use uses [[vcdb4VCDB20]] and [[US Startup City Ranking]], as well as a wealth of old McNair material. Sources include (copied to the project folder unless otherwise noted):