Leading in Tough Times: Can We Stay Savvy About Statistics?

Court Leader's Advantage

September 19th, 2023, Court Leader’s Advantage Podcast Episode

As court leaders, we are inundated with statistics and research. We have all been taught to have a healthy skepticism of statistics. We are given advice that includes: ask questions, go to the source of the research, and expose the assumptions behind the analysis. We have seen what happens when research is not properly evaluated. As a consequence, we are compelled to ensure that statistical analysis is objective and unbiased, valid and reliable, able to be replicated, and relevant to the work of your court.

Having a healthy skepticism is all the more important in this day and age of polarizing misinformation. Most of us have heard that old joke “73.6% of all statistics are made up on the spot.” We don’t want to be the one who is naïvely hooked by questionable statistical research. This advice is all so easy to give, but just how do we hone our critical thinking skills?

Imagine this common scenario, you are in a session with 500 attendees at a national conference. The presenter describes an exciting but controversial concept. At some point, that presenter voices that time-worn introduction “as studies have clearly shown . . .” How do you, as a critical thinker, react? How do you overcome some of the very real logistical issues this scenario presents? For example:

· In the Q&A at the end of the session, do you stand and challenge the presenter to describe the referenced statistical research? Many find this kind of challenge awkward or possibly even rude.

·Do you simply assume someone else in the audience will assess those studies and get back to you?

·Do you spend the time to locate the studies and try to understand them? What if you are curious about the topic? Do you start to think that it is a drain on your work time?

· What if you actually locate the studies and spend the time analyzing them. You end up with some legitimate questions. The sample size seems small or the consultants doing the evaluation were paid by the very program they were evaluating. Do you tell your judges; do you tell your fellow professionals in other courts? How do you do that?

· Or do you just regress into becoming generally negative about all information and default to Mark Twain’s popular adage, there are “lies, damn lies, and statistics”?

This month we’re looking at how we can remain savvy critical thinkers about statistics without ending up cynical and jaded.

Today’s Panelists

Rick Pierce, Judicial Programs Administrator for the Pennsylvania Administrative Office of the Courts

Cheryl Stone, Court Administrator for the Superior Court in Clark County, Washington

Greg Lambard, Trial Court Administrator for the Superior Court in Middlesex Vicinage, New Brunswick, New Jersey

Kent Pankey, Senior Planner, Judicial Services Department for the Supreme Court of Virginia; and

Liz Rambo – Trial Court Administrator for the Circuit Court in Lane County, Oregon.