orgtheory.net

evidence-based administration?

I recently suggested to a colleague at another institution that they turn the college into a laboratory and discover how they want to implement their new first year student program. The college wanted to revamp their existing “freshman seminar” in order to improve learning outcomes amid demands for greater accountability and enhanced assessment.

She was selected to sit on the committee examining current practice, assessing benchmarking opportunities, and figuring out how in the world to localize this best practice (i.e., rigorous and systematic skills training for incoming freshman). During the meeting, she suggested just as we had discussed: select two or three models deemed most fit for the college, implement them, and then study them closely for short- and long-term benefits and liabilities, and then, with all the data, make an informed decision instead of going “all in” on a program they hope will work well locally.

What did she get in response to her suggestion that the university practice  evidence-based administration? A concentrated pool of experts that made up the working committee burst into laughter.

Instead, they are gearing-up to implement a new model they think will work; however, when she asked, “how can you be sure?” The room, formerly filled with laughter, was silent.

Written by Nicholas

May 19, 2010 at 6:10 pm

8 Responses

Subscribe to comments with RSS.

  1. I am a teaching assistant at a very large university. I am already into my 5th TAship in 2 years.

    What is most interesting is that my department seems to be uninterested in gauging performance of their teachers or ta’s. Evaluations, in my opinion, are not useful because they are often disconnected from the substance of the course.

    Being a good little scientist and someone who actually wants to teach well, I have been doing analysis on final marks and tutorial attendance. I played around with idea of giving a pretest to all my tutorial students so that I could gauge the effect of tutorials controlled for by a general knowledge test. I decided against it because I wasn’t sure if my school would have a problem with it.

    This all reminds me of the book “Moneyball” by Michael Lewis. Baseball wasn’t interested in performance-based management, so why would academia? In your colleague’s situation, I am sure those laughing in the room had a tinge of fear in their hearts.

    Like

    Lenin3

    May 19, 2010 at 6:41 pm

  2. I was about to write a snarky comment, but there is a serious point. Some orgs have a culture of excellence and evidence gather, while others don’t. An extremely important question for orgtheory is how to introduce this culture.

    Like

    fabiorojas

    May 19, 2010 at 6:42 pm

  3. @Lenin3 — After I posted this, I showed it to my colleague and your intuition was right: my colleague told me today that she heard from another faculty member about the meeting who said [imagine someone imitating a prominent member of administration] “Some young punk wants data, hell, if we start making decisions with data, where the do I fit in!?”

    Like

    nirowlan

    May 19, 2010 at 6:48 pm

  4. @Fabio — snarky comments can be fun too. Still, you’re right: getting organizations to adopt evidence-based approaches to excellence is hard to do when your bottom-line may not demand it. Now, whether this is a more accurate depiction of higher education as compared to for-profit firms, that is an empirical question.

    Like

    nirowlan

    May 19, 2010 at 6:56 pm

  5. Underlying this is not only a question about making evidenced based decisions, but a much broader issue about fostering an environment that allows for comparisons that have been tested in a particular local environment. A number of organizations are willing to adopt some version of global best practice standards. But that presumes generalizability that rarely holds. There is something more uniquely threatening about evidence based decision making that comes from putting several models to the test in the local environment. Though this strategy seems to be rarely utilized intentionally, it is an important theme in my work with the Ghanaian state.

    For that matter, I found it interesting that a federal style solution to the US health care reform never really became a viable option. Why not come up with several options and allow states to select into them?

    Like

    erinmcdonnell

    May 20, 2010 at 3:13 am

  6. While I am sympathetic to the situation here, and agree that systematic evaluation of such things would be a great idea, it’s important to recognize that the maxim “there are no data without theory” matters here too. In medicine, the “Evidence-Based” buzzword too often stands in for not thinking theoretically, and so I would worry about the implicit biases involved in what counts as “data” for, e.g., the first-year seminar program.

    I’ve been running into issues akin to this in my work on grade policy, and while I do ultimately come down on the side of collecting and distributing data widely, it remains crucial to recognize that data never speak for themselves, the data that remain uncollected doubly so.

    Like

    andrewperrin

    May 20, 2010 at 12:27 pm

  7. ” it remains crucial to recognize that data never speak for themselves, the data that remain uncollected doubly so.”

    Amen.

    Like

    Guillermo

    May 20, 2010 at 1:07 pm

  8. ”it remains crucial to recognize that data never speak for themselves, the data that remain uncollected doubly so.”

    Amen, too.

    Like

    nirowlan

    May 21, 2010 at 12:22 pm


Comments are closed.

%d bloggers like this: