How truthful is ethnographic research?

In 2014, Alice Goffman’s On the Run: Fugitive Life in an American City was acclaimed in this magazine as “a truly wonderful book that identifies the casualties of the war on drugs that extend beyond the prison walls…The detail is incredible. The research is impeccable. Read it and weep.”

But not everybody was so impressed by Goffman’s harrowing account of the six years she spent carrying out immersive ethnographic research in a deprived area of Philadelphia. Some sociologists raised questions of “positionality”: whether a white, middle-class researcher had a right to interpret, and so build her academic career upon, the lives of those in an impoverished black community. Yet one of the fiercest critiques came from far outside sociology. Writing for the online book review site The New Rambler, Steven Lubet, Williams memorial professor of law at Northwestern University in Illinois, made two central points. First, Goffman seemed to have been involved in a “conspiracy to commit murder” when she agreed to drive an armed man seeking to avenge the killing of his close friend, Chuck. And, second, she described several episodes that just didn’t ring true. Could an 11-year-old boy, pulled over by the police while driving with his brother in a stolen car, really have been “placed on three years of juvenile probation on the charge of ‘accessory’ to receiving stolen goods”? And could three new fathers really have been arrested on the same day in the maternity ward of a single hospital?

In challenging these claims, Lubet made reference to his “two years in a legal services office on the West Side of Chicago, and another decade as a defense lawyer in the Cook County [Illinois] juvenile and criminal courts”. He also consulted a prosecutor, two former public defenders and a contact in the police department. None gave any credence to Goffman’s accounts.

In her reply, Goffman – now visiting assistant professor of sociology at Pomona College in California – stated that “at no time did I intend to engage in any criminal conduct in the wake of Chuck’s death”, since the drives around the neighbourhood were “about expressing anger and about grieving, not about doing actual violence”. On the factual questions, she addressed Lubet’s specific points and then referred to sociologist Howard Becker’s notion of “a hierarchy of credibility”, whereby “the accounts of people at the top tend to be taken as true, their perspectives validated and upheld, while the perspectives and experience of people at the bottom are presumed to be implausible, biased and self-serving. Professor Lubet’s critique is based on just this hierarchy. He uses the expertise and authority of people at the top – police, lawyers and law professors – to discredit the experience of people at the bottom.”

This might be taken to imply that “people at the bottom” are in essence truth tellers – a notion that sounds highly implausible and could be described as either romanticising or patronising – while accounts contained in official documents can, almost by definition, be dismissed when they clash with the claims of the marginalised. Yet if an ethnographic account includes a lot of material about police brutality or racism, for example, it must surely be important to distinguish alleged from real examples, partly because any activism needs to focus on the latter but also because political opponents will use any factual errors to discredit the more general case. (On the Run certainly attracted criticism from conservative commentators such as Heather MacDonald, who noted that Goffman “looks at [an] unending stream of lawless behavior and sees only the helpless pawns of a mindlessly draconian criminal-justice system…‘Felony convictions’ do not simply fall from the sky; they result from serious criminal activity.”)

Alice Goffman, author of On the Run: Fugitive Life in an American City

Alice Goffman, author of On the Run: Fugitive Life in an American City

But, writing to Times Higher Education, Goffman denies that she believes that marginalised people are always truth tellers: “No group of people is telling the truth all the time.” Nor does she believe “that it is the job of an ethnographer to allow people to tell their stories without interrogating those stories…Much of the power of long-term fieldwork lies in the ability to compare your own notes of events – police beatings, for example – to other people’s accounts of those same events, and then to compare those fresh notes and accounts to the stories people tell about those events years later. For the events you don’t get to observe first-hand, you triangulate.”

Many ethnographers certainly put themselves on the line for their work. A recent article in Criminal Justice Studies by Robert M. Worley, associate professor of criminal justice from Lamar University in Texas, and colleagues, titled “There were ethical dilemmas all day long!: harrowing tales of ethnographic criminology and criminal justice”, includes some striking examples. One researcher carried out fieldwork among active burglars and armed robbers. Another worked as an exotic dancer to study the subculture of stripping. A third found employment for 19 months as a correctional officer in a penitentiary and was “physically assaulted by an aggressive inmate”. A fourth lived among gangs, occasionally took part in fights with rival gangs and was “the victim of severe violence on multiple occasions”.

The last two, report the authors offhandedly, “managed to use these attacks to their advantage. Fieldworkers must be aware of the inherent risks of conducting ethnographic research in perilous places; nevertheless, if, in the unfortunate event, a fieldworker is harmed, he or she should make every effort to use this as a currency to gain acceptance within the subcultures they are studying.”

It is a core principle of ethnography that researchers must protect their sources. There have been cases of researchers jailed for refusing to hand over their field notes to the police, or burning such notes in advance so they can never be pressured into providing them. Ethnographers may also need to fictionalise accounts so that places and people cannot be identified. Yet this means that readers cannot tell whether the poignant and dramatic details of an account are accurate, hearsay or just a smokescreen introduced by the author. In the words of Carol Rambo, associate professor of sociology at the University of Memphis in Tennessee, “The reader needs to be told [but often isn’t] that something was radically fictionalised because it was traceable.”

Ethnographers certainly do fieldwork among groups they find unsympathetic. Rachel O’Neill, a feminist postdoctoral research fellow in sociology at the University of York, carried out research within “the seduction community”, which eventually led to her recent book Seduction: Men, Masculinity and Mediated Intimacy. From the start, she was “always conscious that the views and claims of my participants reflected a certain set of assumptions, many of which were at odds with my own experiences of the world, as well as my knowledge base as a researcher”, notably with regard to “just how pervasive dynamics of coercion and violence are in the context of heterosexual sex”. So although she tried to “get something approximating an ‘inside’ view”, she “never had the impulse to ‘go native’ ”.

Yet it is at least as common for ethnographers to produce empathetic and even sympathetic accounts of communities that are typically marginalised or stigmatised. Some of the ethical and methodological issues this raises are explored in Lubet’s new book, Interrogating Ethnography: Why Evidence Matters, written after his dispute with Goffman led him to read more widely in the literature. He brings to the topic, he explains, a long-term interest in “the study of proof” and “a decades-long exploration of the quality of facts and assertions in different settings”. And he uses this, together with his knowledge of the criminal justice system, a willingness to pursue paper trails and a kind of forensic common sense, to assess some of the factual claims made by ethnographers.

Source: 

Getty

Grief-stricken: Kaeinji Jackson mourns the loss of her brother Maurice Granton Jr during a vigil in Chicago on 7 June 2018. Granton was shot and killed by police

Although he finds much to praise, he also flags many concerns. His essential point is that some ethnographers are over-identified with the people they write about and make little effort to check the truth of what they are saying, and so end up in effect acting as advocates.

Why is it impossible, for example, to find independent verification for the murders of “Babycake Jackson” and “Big Cat”, described in Off the Books: The Underground Economy of the Urban Poor by self-described “rogue sociologist” Sudhir Venkatesh (whose writings have also attracted much controversy)? Why does Philippe Bourgois, in In Search of Respect: Selling Crack in El Barrio, let a drug dealer called Primo give his bitter account of trying to “go straight” in an office job – under a woman he describes as “a prejudiced boss”, “a cheap bitch” and even “a fucking ‘ho’ ” – without letting us hear her side of the story (or revealing until later Primo’s history of extreme violence against women)?

Lubet also points to anomalies in Kathryn J. Edin and H. Luke Shaefer’s $2 A Day: Living on Almost Nothing in America. Why do they report that a woman got pregnant because “antibiotics she had been prescribed had apparently neutralized her birth control”, even though the commonly prescribed antibiotics do not have such an effect? And was it plausible that a group of kids from the Mississippi Delta saw a lift for the first time during a school trip to Washington DC (“some of them didn’t believe that the box behind the doors could actually transport them from one floor to another”), given that lifts featured prominently in many popular television programmes of the time?

Venkatesh, who has now swapped his professorship at Columbia University for a position at Facebook, did not respond to requests for comments, although his book Gang Leader for a Day: A Rogue Sociologist Takes to the Streets implicitly makes the case for deep immersive ethnography in its description of his failure to administer a traditional multiple-choice questionnaire within gangland: “How do you feel about being black and poor? Very bad/Bad/Neither good nor bad/Somewhat good/Very good”.

Other authors criticised by Lubet took the opportunity, when contacted by THE, to explain why the examples he cites actually demonstrate the value of their kind of ethnography in illuminating hidden worlds.

Bourgois, professor of anthropology at the University of California, Los Angeles, argues that “it is hard for people who haven’t spent time in segregated settings to comprehend how painful it can be when one leaves one’s inner-city setting, where one has status, and suddenly discovers that one looks like a buffoon, monster or moron to the folks in the dominant world, where all the money and power is concentrated”.

Regarding his depiction of Primo’s ill-fated period as an office worker, he says that “the argument that institutionalised racism structures the workplace encounter is not contradicted by Primo’s being objectively a lousy worker or [the suspicion that he might be] misrepresenting his boss”. He denies that the book assigns any “credit” to Primo’s “routinised misogynist language”. On the contrary, by reporting it (as well as detailing Primo’s history of rape and domestic violence), “I am revealing the pervasive brutality of gender power relations, as well as the infrastructure of racism that guides workplace interactions. If I were arguing like a lawyer, I would have sanitised Primo.”

Source: 

Paul Fusco/Magnum Photos

Friends grieve for Alberta Spruill, who died of a heart attack after police threw a stun grenade into her New York apartment, which they had been told was being used by a drug dealer

Shaefer, associate professor of social work and public policy at the University of Michigan, concedes that she and her co-author got the point about antibiotics and contraception wrong – perhaps, she says, because the woman was passing on incorrect information given to her by a healthcare professional. But the elevator incident was “the best example of why we need ethnography. We heard the story from the teacher who witnessed it himself, as well as [from] the respondent – and we were able to corroborate similar stories from people who worked in the region…We feel quite certain that story as reported is accurate, and it is important to think how it could be true. If someone lives in a very poor neighbourhood, how do they interpret what they see on TV? Does it strike them as reality?”

The “central insight” of such work, according to Shaefer, is that “lots of things happen at the bottom of society that those who live higher up the income ladder wouldn’t believe”.

So where do other ethnographers stand on the issues raised by Lubet?

Scott Decker, foundation professor of criminology and criminal justice at Arizona State University, suggests that “qualitative researchers (particularly ethnographers) have a responsibility to try to validate their findings no matter which social strata they come from. Individuals are often self-serving or [turn] a blind eye to the consequences and nature of their own acts, whether they work in the boardroom, in a hospital or in street settings.”

This, he says, makes it important to seek “multiple data sources and perspectives”. In addition, in his own practice, “we often read back our conclusions and supporting evidence to our subjects for their views”.

Patricia Adler, professor emerita at the University of Colorado at Boulder, notes that while “there is a very strong feeling among ethnographers that we are giving voice to people whose voices are not heard”, it is also important to “demonstrate some independence analytically – you’re not just there to parrot back what your subjects tell you”.

However, she is sceptical about the idea of checking subjects’ accounts against official sources when doing so could have practical consequences for those subjects. For instance, she and her husband and fellow ethnographer Peter “were offered the opportunity through a friend of a friend to interview an agent from the Drug Enforcement Agency when we were doing our drug-dealing research. We could have used that connection to check if the way that the drug dealers thought the police behaved was actually true. But we declined, because if it was known that we were doing this close-up ethnography, living with our respondents and hanging out with them all the time, then all they had to do was follow us around and know who we were writing about. We thought that was too dangerous.”

Wasted: Donald Rayfield, known on the street as ‘Detroit’, smokes crack cocaine and looks at a pornographic magazine in an underground storm drain in Los Angeles, California

Martín Sánchez-Jankowski, professor of sociology (and director of the Institute for the Study of Societal Issues) at the University of California, Berkeley, describes following 37 gangs over 10 years for his first book, Islands in the Street: Gangs and American Urban Society. His work always focused on two questions: “After 150 years of trying to eradicate gangs with all kinds of material and human resources, why do these entities still exist? And why do some gangs rise and thrive, while others decline and die?” He didn’t, therefore, see his goal as “giving voice to the marginalised” and was delighted that his work was cited by those across the political spectrum.

Yet it is very evident that much of the debate about ethnography touches on the polarisation endemic in today’s politics.

Lubet stresses that he is “pretty much in complete agreement with the dominant political views of urban ethnographers and I am enthusiastic about the project of giving voice to the marginalised”. But, for him, “it nonetheless remains essential to distinguish ‘perspectives’, impressions or beliefs, on the one hand, from verified events on the other. Donald Trump’s perspective is that Mexican immigrants are dangerous criminals, but the facts prove otherwise. Both social science and progressivism will suffer if we sacrifice accurate reporting for the sake of making a point.”

Quite apart from a general requirement for scholarship to be truthful and accurate, Lubet goes on, “we are living in a time (in the US) when political forces are attempting to destabilise reality with accusations of witch-hunts, conspiracies, the so-called Deep State and even the unreliability of science. It is the job of academics to provide a counterbalance to this trend, which cannot be done if we fail to insist on the rigorous accuracy of our own work.”

Jeff Ferrell, professor of sociology at Texas Christian University, takes a very different view.

Although his “absolute hallmark in doing ethnography is to never falsify or make up anything”, he believes that “there is a politics to any attack on or critique of ethnographic work these days”. This is partly because it is “the primary method that is holding the line” against the move to base sociological research on big data, while the marginal perspective it often articulates “serves as a critique of official accounts and pervasive ideological misinterpretation of various groups”. So, “whether by intention or not, work like Lubet’s has the potential to do some serious damage – and thus makes me wonder about the motivations behind it”.

Like Goffman, Ferrell is wary of using official accounts as a “benchmark”, given that “altered police records, plea deals that encode a different crime than the one (allegedly) committed, police discretion, falsified police accounts, [and] on and on…are pervasive”. Moreover, he concludes, the questions being raised about ethnographers’ integrity are no different from those that relate to all academic research: “We can be no more sure that a lab experimenter didn’t fudge her experimental model, or that a survey researcher didn’t misreport or misrecord his survey data, than we can that an ethnographer didn’t invent an incident.” 

Steven Lubet’s Interrogating Ethnography: Why Evidence Matters was published earlier this year by Oxford University Press.