BBMC and Baseball: Ten Years In And Going Strong
It's the end of 2019 and it occurred to me that it represents the ten year mark for BBMC. A decade!
Looking back I am especially thankful for the clients who engaged my services, some since the beginning. I am also very thankful for the recommendations of my work that they made to their colleagues. It is because of these recommendations that I have yet to advertise and still I have been busy each and every year...sometimes too busy. Just ask my wife.
I never imagined that the last 10 years would be as rewarding and as fun as it has been. So I also send thanks to my clients for making my life so interesting and rewarding. I am very thankful for the wonderful people that I have met both in person and on-line, for the places I have visited to give workshops about preparing an application and/or to help prepare grant applications. I have also been invited to places like India and Australia, places that I probably wouldn't have otherwise visited, to make presentations about tracking and analyzing progress in science using automated analytic tools. At the same time I am also blown away by the new ideas and science I have seen, and the advances that have been made by BBMC clients in their respective fields. During the past decade the rate of progress in sciences seems to have gone at warp speed. It seems that the half-life of a new idea has gone from months and weeks to days and hours.
My first impulse in writing this update to What’s Happening at BBMC was to use the opportunity to look back and review the ground that we have covered since 2009. But as Satchel Paige, a baseball hero of mine was fond of saying, "Don't look back. Something might be gaining on you." Besides, only looking back without a view to the future doesn't fit with the BBMC tag line "Working with you for excellence in research", something that I take very seriously, is future-looking, and continues to guide the way I work with my clients on a daily basis and will continue to do so as long as I continue to do consulting work.
Having mentioned one of my favorite baseball players and since I'm writing this the morning after my favorite team, the Washington Nationals, won the World Series against one of the most talented teams in major league baseball, the Houston Astros, baseball is on my mind. The Nationals did it in the most unlikely manner...by being a wild card team at the start of the post-season, then coming from behind in every "do-or-die" game that followed their last minute win against the Milwaukee Brewers, then sweeping the St. Louis Cardinals in four games to become the National League Champions, and then winning all of their World Series games on the Astros' home field. All of this is an incredible once in a lifetime happening that has put me in a baseball state of mind. So, let's talk baseball and grant writing for a bit.
One of the highlights in analyzing the Nationals-Astros match up was that the Astros decision making was based on what has become known as "analytics." The Nationals? Not so much. The Astros used what is called sabermetrics, the empirical analysis of baseball, especially the analysis of in-game activity to validate every managerial decision ranging from which players to include in the lineup and where to place them in the batting order, to tendencies of opposing pitchers and the sequence of pitches they use, to where to position players on the field for each batter, and on and on. It's not nearly as sophisticated as predictive models of disease used in epidemiology, but it relies on equations and numbers, some of which are taken to more than 5 significant decimal places. And for those of you who watched the World Series you saw the Astros manager or one of his assistants with laptop computer in hand all the time.
So all of the elements of science is there. But is it science? Hardly...the number of significant decimal places doesn't make something into science nor does reliance on a computer. Also, as one of the characters said in the movie Bull Durham, one of my favorites, "Baseball is a simple game. You throw the ball, you catch the ball, you hit the ball." So maybe the stats are, at the same time, too sophisticated or not sophisticated enough to capture what goes on both before, during and after the game is played. And I'm pretty sure that none of the sabermetrics takes into account what some people call intangibles, the single most important factor cited by many sports writers in explaining why the National's are the 2019 World Series Champion...or as my favorite coach, Gus, (yes I played baseball at a high level when I was young) was fond of saying, "When one round object (a bat) hits another round object (a ball), anything can happen."
All of this talk about intangibles in explaining wins and losses in baseball also felt very familiar to me in relation to a very different part of my life, science, and more specifically, about grant writing. Scientists who reach the "professional" or elite level of our profession have streaks during which it seems that all or many of their studies go well and their grant applications get funded. They also have other streaks when they just can't seem to catch a break and get meaningful results or an application funded. I've seen this happen to colleagues during my careers in academia, at NIH and for the last 10 years as a consultant. Is this also the result of intangibles? If it is, what are the intangibles..."lab decorum"? More importantly, what can you do to prevent a slump from happening? And how could there even be anything close to intangibles in the peer review and funding processes of organizations that provide funding for research?
Almost all funders make everything except for the names of the reviewers who actually review your application is well defined and available to everyone. In the case of NIH we know from the start the exact criteria that reviewers use in assessing the scientific merit of an application (Significance, Investigators, Innovation, Approach, Environment). We know what reviewers are asked to look for in evaluating each of those criteria. And we know what they are told to assess when evaluating the single criterion, Scientific Impact, that is used in making funding decisiona. In addition, we know the meaning of the numerical scores reviewers use in summarizing the strengths and weaknesses corresponding to each scored criterion.
In addition we have access to all kinds of analyses through NIH RePORT about success and funding rates, the amount of money each IC has available to fund both investigator-initiated projects as well as solicited projects (RFAs) along with their current funding interests as well as new areas of research (PAs). We can use RePORT to find out who has already received funding and at least a summary of the proposed research plan, its specific aims, the name and institution of affiliation of the PI, the amount funded, the institute program to which the project has been assigned for administration and the contact information of the NIH Program Official, and many other things including publications resulting from the grant. We can also find information about the review group and the scientific expertise of its members...although we won't know which of the individuals listed on the roster of the review group was assigned to assess the scientific merit of any given application by providing a written review.
What possibly could remain as an intangible, that is the unknown that can change everything involved in success or failure, in such a well delineated system? More importantly, as the applicant what can you do about it? I think the answer can be found in a favorite bit of advice that a former and very valued colleague, Dr. William Raub, gave to potential applicants. It went something like...
“No amount of grantsmanship will turn a bad idea into a fundable one…but there are many ways to hide outstanding ideas by poor grantsmanship.”
It seems to me that the intangible in the system is about what he referred to as grantsmanship, that is how an applicant develops and writes about the idea(s) in their application...in the specific aims, significance, innovation, investigators and approach sections. It's about the language and logic used, starting with the Specific Aims, that keeps the reviewer engaged and eager to see what comes next. None of these alone or in combination will turn a bad idea into a fundable one, but without them it is clear that an otherwise fundable idea can be successfully masked and, as a consequence, not be recommended for funding.
The intangibles go beyond the specific content of an application and factoids used to support the importance of the problem under investigation. They have to do with how the reader, the peer reviewer, interprets the presentation style as it pertains to the writer's ability to do outstanding and impactful science. They also involve how easy it is for the reviewers to follow the logic and to quickly gain an understanding of the importance of the problem and how it will move the field forward. And, from a consultant's and former program official's point of view, I believe that they also involve just how quickly the application writer catches on and responds appropriately to the dynamics of the review situation including that: (a) the reviewer is the gatekeeper to the money that the applicant wants/needs; (b) ideas aren't inherently "important," "unique," or "paradigm changing," they require more than adjectives...specifically content provided in the application; and (c) the content of the application is to help the reviewer do their job in order to arrive at one or more of those adjectives.
So, in my view, those are some of the intangibles in the world of writing a competitive grant application. But what about the future? I plan to continue to help applicants unravel the mysteries of how to get funding by translating the intangibles in getting funding to tangibles...things that are dependent on making sure that we don't hide otherwise excellent ideas.
And so I end this reflection on baseball and the last 10 years of consulting with a the words of Mother Theresa: “Yesterday is gone. Tomorrow has not yet come. We have only today. Let us begin.”
And once again a final thanks you to all the clients that have used my services over the past 10 years. It has been wonderful working with you for excellence in research and I thank you for passing my name on to your colleagues.
Norm Braveman
Rockville, MD
December, 2019
|