Archive for the ‘education’ Category

my deep burning hatred of letters of recommendation

with 13 comments

Econjeff mentions my long standing critique of letters of recommendation (LoRs). Here, I describe my personal experience with them and then I restate the massive empirical research showing that LoRs are worthless.

Personal experience: In graduate school, I had enormous difficulty extracting three letters from faculty. For example, during my first year, when I was unfunded, I asked an instructor, who was very well known in sociology, for a letter. He flat out refused and told me that he didn’t think I’d succeed in this profession. In the middle of graduate school, I applied for an external fellowship and was informed by the institution that my third letter was missing. Repeatedly, I was told, “I will do it.” Never happened. Even on the job market, I had to go with only two letters. A third professor (different than the first two cases) simply refused to do it. Luckily, a sympathetic professor in another program wrote my third letter so I could be employed. Then, oddly, that recalcitrant member submitted a letter after I had gotten my job.

At that point, I had assumed that I was some sort of defective graduate student. Maybe I was just making people upset so they refused to write letters. When I was on the job, I realized that lots and lots of faculty never submit letters. During job searches at Indiana, I saw lots of files with missing letters, perhaps a third were missing at least one letter. Some were missing all letters. It was clear to me that l was not alone. Lots of faculty simply failed to complete their task of evaluating students due to incompetence, malice, or cowardice.

Research: As I grew older, I slowly realized that there are researchers in psychology, education and management dedicated to studying employment practices. Surely, if we demanded all these letters and we tolerated all these poor LoR practices, then surely there must be research showing the system works.

Wrong. With a few exceptions, LoRs are poor instruments for measuring future performance. Details are here, but here’s the summary: As early as 1962, researchers realized LoRs don’t predict performance. Then, in 1993, Aamondt, Bryan and Whitcomb show that LoRs work – but only if they are written in specific ways. The more recent literature refines this – medical school letters don’t predict performance unless the writer mentions very specific things; letter writers aren’t even reliable – their evaluations are all over the place; and even in educational settings, letters seem to have a very small correlation with a *few* outcomes. Also, recent research suggests that LoRs seem to biased against women in that writers are less likely to use “standout language” for women.

The summary from one researcher in the field: “Put another way, if letters were a new psychological test they would not come close to meeting minimum professional criteria (i.e., Standards) for use in decision making (AERA, APA, & NCME, 1999).”

The bottom line is this: Letters are unreliable (they vary too much in their measurements). They draw attention to the wrong things (people judge the status of the letter writer). They rarely focus on the few items that do predict performance (like explicit comparison). They have low correlations with performance and they used codes that bias against women.

50+ chapters of grad skool advice goodness: Grad Skool Rulz ($2!!!!)/From Black Power/Party in the Street

Written by fabiorojas

October 6, 2015 at 12:01 am

new and old thoughts about teaching mathematical proofs

with one comment

A long, long time ago, I used to teach math. One of the central questions in mathematical education at the college level is how to teach mathematical proofs. Sometimes, you had pessimistic conversations. People simply had “mathematical maturity” and there wasn’t much you could do about it. There is truth to this –  some people simply can’t grasp what a proof would entail.

Beyond this simple observation, there was remarkably little thinking about how to teach proofs. Of course, there are occasional books that try to break down the process of creating and writing proofs, such as How to Prove It. Still, I felt there was something missing in the conversation about proof teaching. This blog post is my modest contribution to the topic.

My hypothesis: An important barrier to teaching math proofs is that they combine two very, very hard skills and that most math teachers only focus on one of the those skills. Specifically, proofs entail (a) symbolic manipulation and (b) recipes that get you from A to B. Math teachers and books are actually pretty good at (a). For example, almost every text will teach you about the symbols – set theory; formal logic; deltas and epsilons; etc. What is almost completely overlooked is that students find it hard to glimmer the “recipes” that make up proofs and there is no theory, or set of instructional strategies, for helping students intuitively understand recipes. In practice, you simply take courses on various topics (numerical analysis or matrix theory) and you mimic the proofs that people give you. Not great, but better than nothing.

The old Dolciani high school text books had an interesting response to this issue. In the geometry text, the proofs would always have two parts: “analysis” (outline of the idea) and “proof” (traditional proof with all details). You also see this in advanced texts and journal articles. When a long, hard proof is coming up, the author will present an outline.

Here is my modest suggestion: When teaching proofs, always outline the proof as a flow chart. In other words, take the old notion of the proof outline (or “analysis” in Dolcian’s terms), make it visual, and then put it in front of all proofs that require more than a few sentences. By repeatedly visualizing proofs as chains, teachers will be forced to extract the recipe from the text in a way that more students can understand. They will also more easily identify common themes that appear in multiple visualizations of proofs. Also, pictures are easier to remember than dense, equation filled masses of text.

50+ chapters of grad skool advice goodness: Grad Skool Rulz ($2!!!!)/From Black Power/Party in the Street

Written by fabiorojas

October 2, 2015 at 12:01 am

let the children play! it’s good for their mental health

with 3 comments

Psychology Today has an article on a new analysis of play and the mental health of young people. The gist is that (a) recently, we let kids have less unstructured play and (b) unstructured play increases the belief that one has direct control over their life, which in turn has positive effect on mental health and various measures of well being. From the article:

The standard measure of sense of control is a questionnaire developed by Julien Rotter in the late 1950s called the Internal-External Locus of Control Scale. The questionnaire consists of 23 pairs of statements. One statement in each pair represents belief in an Internal locus of control (control by the person) and the other represents belief in an External locus of control (control by circumstances outside of the person). The person taking the test must decide which statement in each pair is more true. One pair, for example, is the following:

  • (a) I have found that what is going to happen will happen.
  • (b) Trusting to fate has never turned out as well for me as making a decision to take a definite course of action.

In this case, choice (a) represents an External locus of control and (b) represents an Internal locus of control.

Many studies over the years have shown that people who score toward the Internal end of Rotter’s scale fare better in life than do those who score toward the External end.[2] They are more likely to get good jobs that they enjoy, take care of their health, and play active roles in their communities—and they are less likely to become anxious or depressed.


In a research study published a few years ago, Twenge and her colleagues analyzed the results of many previous studies that used Rotter’s Scale with young people from 1960 through 2002.[3] They found that over this period average scores shifted dramatically—for children aged 9 to 14 as well as for college students—away from the Internal toward the External end of the scale. In fact, the shift was so great that the average young person in 2002 was more External than were 80% of young people in the 1960s. The rise in Externality on Rotter’s scale over the 42-year period showed the same linear trend as did the rise in depression and anxiety.  

[Correction: The locus of control data used by Twenge and her colleagues for children age 9 to 14 came from the Nowicki-Strickland Scale, developed by Bonnie Strickland and Steve Nowicki, not from the Rotter Scale. Their scale is similar to Rotter’s, but modified for use with children.]

It is reasonable to suggest that the rise of Externality (and decline of Internality) is causally related to the rise in anxiety and depression. When people believe that they have little or no control over their fate they become anxious: “Something terrible can happen to me at any time and I will be unable to do anything about it.” When the anxiety and sense of helplessness become too great people become depressed: “There is no use trying; I’m doomed.”

Wow. Later in the article, they talk about how this shift correlates with mental health outcomes. So don’t schedule them or boss the, let them play.

50+ chapters of grad skool advice goodness: Grad Skool Rulz ($2!!!!)/From Black Power/Party in the Street

Written by fabiorojas

October 1, 2015 at 12:01 am

Posted in education, fabio, psychology

movements and inhabited institutions: the case of latino student groups

leave a comment »

A key insight from research on student activism is that the college environment has a strong influence on how that activism expresses itself. We saw that in Amy Binder and Kate Wood’s study of conservative groups. Daisy Reyes has an article in Sociology of Education that explores this issue with Latino groups and links it to institutional theory:

To comply with ideals of multiculturalism and diversity, postsecondary institutions incorporate Latino students into distinct campus cultures. These cultures influence how students interact with one another, the university community at large, and communities outside of campus, ultimately shaping how students inhabit Latino politics. Drawing on data from 20 months of ethnographic fieldwork with six student organizations and 60 in-depth interviews, I compare Latino student organizations in a liberal arts college, a research university, and a regional public university. Building on inhabited institutional theory, I identify dimensions of campus cultures that work in interaction with students to produce three divergent forms of ethnic political expression: deliberative, divisive, and contentious. Inhabited institutionalism helps explain why Latino politics takes distinct forms in specific academic contexts and suggests that strong collegiate incorporation may paradoxically serve to suppress Latino student engagement in political activism outside the campus gates.

Read the entire article here. Recommended.

50+ chapters of grad skool advice goodness: Grad Skool Rulz ($2!!!!)/From Black Power/Party in the Street

Written by fabiorojas

September 29, 2015 at 12:01 am

types of social network analysis courses

with 5 comments

We live in a golden age of network analysis. It’s booming as science and booming as business. This raises questions for the teacher – what course should you teach? A few options:

  1. Bare bones: A course designed for folks with little to no mathematical background. You would teach descriptive stats, visualization, and applications.
  2. Stats+/Models : In this course, you’d assume some basic background. Maybe micro for econ students or stats for other social science students. Then, you’d dig deep into different centrality measures, power laws, clustering/community detection, etc. A follow up course would deal with p*, ERGM, Sieana and other advanced issues.
  3. Programmers: Here, you’d lightly gloss over the math and proofs and instead focus on how to scrape the net for data, how to write simulations, and how to manipulate big data sets.
  4. Elite stats: This is for a very small number of students in math, stats, or econometrics. It would be exclusively proofs of fairly advanced issues (like the graph models underlying p*).

Currently, I teach a course for sociology seniors between 1 and 2. I get soc students, a handful from econ/psych/poli sci, and one or two informatics students. I also get one or two grad students. The elite soc programs, where students often have science backgrounds or simply a lot of mojo, are now seeing Programmers courses. Old school networks courses (a la Ed Laumann or John Padgett at Chicago) offer a version of #2. Elite stats is exceptionally rare in that if students are that advanced, they can often read the papers themselves. Add your own comments about networks education.

50+ chapters of grad skool advice goodness: Grad Skool Rulz ($2!!!!)/From Black Power/Party in the Street

Written by fabiorojas

September 25, 2015 at 12:01 am

what i did and did not say about critical thinking

Two weeks ago, we had quite a discussion about critical thinking. There was one strand of debate the baffled me. Thomas wrote the following:

“If I don’t teach critical thinking, then what do I teach?” you ask. “Turns out that there is a simple answer: sociology.” It sounds good at first. But what you’re really saying is that you won’t acknowledge a kind of critique that isn’t sociological. That’s certainly the line Goffman’s defenders are taking. If you think she’s wrong, they say, it’s because you don’t understand how ethnographers think.

That is not what I was saying. What I said is this:

Aside from a very simple general rules of thumb, such as “don’t be emotional in arguing” or “show my your evidence,” the best way to be improve your thinking is to learn from those who have spent a lifetime actually trying to figure out specific problems.

I did not say: “only trust experts.” I said: “learn from.” In other words, there are people who have encountered certain problems and they have tried and tested the ideas that might have occurred to you. In the processes of trying out those ideas, they have probably learned important things about the phenomena that you are looking at. You should probably learn those lessons. Once you have absorbed those ideas, you are free to criticize as you will. No one has a monopoly on truth, even the experts. But that doesn’t mean experts are clueless fools. Certainly, “critical thinking” must have a place for learning from other people.

50+ chapters of grad skool advice goodness: Grad Skool Rulz ($2!!!!)/From Black Power/Party in the Street

Written by fabiorojas

September 17, 2015 at 12:01 am

Posted in education, fabio

the most overlooked trend in U.S. higher education

State defunding of public higher education has received a lot of attention in recent years. And budget cuts like the $250 million one Scott Walker made this year to the University of Wisconsin mean this trend continues to get media play.

Less visible in the media, but still well known, is that as public funding has eroded, colleges have become more dependent on tuition dollars for revenue. For public institutions, this has meant both tuition increases for in-state students and, where possible, a greater percentage of out-of-state and international students. While the net price of college hasn’t increased nearly as much as the sticker price, it’s still beat the cost of inflation year after year.

Both of these narratives are completely true. Yet this story of a shift from public to private funding overlooks one critical factor: the expansion of federal student aid.

During the past two decades, as state appropriations per postsecondary student flattened then declined, federally supported financial aid made massive gains. In 2002 its volume passed that of state appropriations, and by 2010 it was twice as large.


Stunning, right? This suggests a very different story than the one about the privatization of public universities we hear so much about. Instead, it looks like there’s been a shift from state funding of higher ed to federal funding. So what’s going on here?

Well, a couple of things. First, the federal aid figures include both grants and loans. Data sources like the College Board and the Delta Cost Project include loans as part of net tuition, not as federal funding. That makes sense, if you’re interested in the financial burden of college on students and their families. And the loans don’t cost the government anything like their face value.

But counting this way downplays the fact that those loans ultimately exist because the federal government makes them possible. Colleges are doubly dependent in this scenario: on students’ choices about where to attend, but also on the feds to make them available in the first place. And if you’re coming at this from an organizational perspective, we should expect resource dependence — whether on students, on the feds, or both — to have effects.

Second, this chart collapses public, private non-profit, and for-profit institutions together. The state appropriations are only going to publics (which also enroll about three-quarters of the students). But as of 2010, more than a quarter of student aid was going to the 10% of students enrolled in for-profit institutions. Moreover, because private colleges are so much more expensive than public colleges, they also receive a disproportionate fraction of federal loans. I haven’t pulled these numbers apart by institution type. But if we just compared state appropriations and federal aid to students at public institutions, the chart would surely be less dramatic.

It would be misrepresenting reality to say that public institutions have experienced a substantial shift from state to federal dependence (at least without substantially more number crunching). And it would be similarly wrong to argue that schools haven’t become more tuition dependent (since loans do come to schools via individual students).

But you can absolutely make the case that at the field level, higher education has increased its dependence on the federal government relative to state governments. And this makes colleges susceptible to a whole wave of federal demands that simply weren’t there before. The college ratings system Obama proposed and then abandoned is one example of this. Education Secretary Arne Duncan’s drumbeating for accountability is another.

Colleges have a lot of political clout and are well-organized. They ground the ratings proposal into a shadow of its former self. And it will take a lot of doing before we see No College Student Left Behind.

Nevertheless, if organization theory tells us anything, it’s that resource dependence matters. When, five years down the road, we get a Race to the Top rewarding colleges that meet completion and job placement goals at a given tuition cost, I know where I’ll be looking: at that point in 2002 where higher ed waved goodbye to the states and hello to the feds.

[Data from the College Board’s Trends in Student Aid 2014 and Grapevine reports, various years, deflated with BLS CPI.]

Written by epopp

August 31, 2015 at 12:34 pm


Get every new post delivered to your Inbox.

Join 3,328 other followers