DroneAcharya
BANNED
- Joined
- Sep 5, 2012
- Messages
- 226
- Reaction score
- 0
Counting civilian casualties in CIAs drone war.
By Meg Braun Friday, November 2, 2012 - 12:15 PM
A report released in September by human rights researchers at Stanford Law School and New York University Law School sets out to demonstrate that U.S. drone policies are "damaging and counterproductive."
Media outlets from CNN to BBC hailed the report as new evidence of the U.S. government's false narrative on drones and the New York Times' Scott Shane described the study as "among the most thorough on the subject to date."
While the Stanford-NYU report certainly presents a comprehensive review of existing drone research, its new contributions to the evidentiary record are far more modest than its sweeping conclusions would suggest.
Number Crunching
The U.S. government claims that civilian casualties caused by drones are in the "single digits" during Obama's years in office, while the Stanford-NYU report seeks to establish that there is significant evidence that U.S. drone strikes have killed and injured a larger number of civilians. This is a low bar, and would merely necessitate proving that more than 10 civilians have been killed by drones since Obama assumed office, a claim that has been made by all three major databases aggregating information on drone strikes. The Stanford-NYU report goes further, claiming that between 474 and 881 civilians have been killed since 2004.
However, this does not represent new evidence. Stanford and NYU researchers made no attempt to offer new statistical analysis on the number of civilian casualties caused by drones. Rather, their report is essentially an extended endorsement of a database compiled by The Bureau of Investigative Journalism, a small team of journalists based out of City University, London. In doing so, the report rejects the findings of two other widely cited databases, The Long War Journal, which reports 138 civilian deaths and New America Foundation's Year of the Drone, which lists 152-191 civilian deaths and the deaths of 130-268 "unknowns."
If the Stanford-NYU team wants to paint a picture of the U.S. as a human rights abuser, then the Bureau, whose casualty statistics dwarf the more conservative estimates of the other databases, is best suited to that purpose, but that does not make it the most reliable source.
While a comprehensive assessment of the Bureau's data should have been conducted before giving it a resounding endorsement, just a cursory review of its account of 2012's drone strikes reveals problems:
- On May 5, 2012 the Bureau reports that a strike in North Waziristan killed 8-10 people, of whom between zero and ten are listed as civilians. Upon reviewing the accompanying sources one find that early reports said the identity of the dead could not be determined, but subsequent reports from government officials identified them as militants. Of the 15 sources cited by the Bureau, not one states that the victims were civilians.
-On June 2, 2012 the Bureau reports that 2-4 people were killed and between zero and two of them were civilians. These civilian deaths were inferred based on a report that a motorbike was accidentally hit, but the Bureau fails to take note of a later report from The Express Tribune identifying the motorbike victims as suspected militants Khalil Yargul Khail and Rehmanullah Gangi Khail.
-On July 23, 2012 the Bureau references 25 sources for a drone strike, only one of which described the victims as local residents and based on this the Bureau reports that up to 14 civilians were killed.
The Stanford-NYU report critiques the Long War Journal for not making its data available in a verifiable strike-by-strike format, over-relying on U.S. intelligence sources, and identifying all victims as militants unless they are specifically identified as civilians. These are reasonable criticisms, but when the critique turns to New America Foundation research, it becomes logically inconsistent.
While Stanford-NYU researchers admit that all three databases rely on "the same universe of publicly available press reports," they criticize only the New America Foundation for over-relying on the anonymous Pakistani government officials cited in such news reports. The Stanford-NYU report laments that these officials are cited in 88% of articles referenced in New America's 2012 data but fails to consider that these same articles are referenced by the Bureau. Furthermore, the only reason the researchers were able to generate that statistic is because New America compiles a highly transparent summary of media sources relied on for each strike with a list of which sources reported which casualty estimates, something no other database provides.
The Stanford-NYU report goes on to question the "deep reporting" capabilities of New America's sources (The New York Times, Reuters, AP, BBC, etc.) while uncritically accepting the veracity of reports from fringe outlets such as the Kuwait News Agency, the South Asian News Agency, Central Asia Online, and Punjab News, which are sometimes the sole sources for the Bureau's reports of civilian casualties.
This is not to suggest that the Bureau's research is without merit, but it should be noted that it represents an interpretation of the facts with a bias toward reporting civilian casualties. The Bureau's database does not even have a category for reporting militant deaths. It seems counter-intuitive that the organization's researchers can have such absolute faith in their civilian casualty estimates, but not have the confidence to label any of the deceased as militants.
And the Stanford-NYU team was not impartial. By the report's own admission, the research project it undertook in Pakistan to interview family members of drone strike victims was commissioned by Reprieve, a UK based advocacy organization. Reprieve, which filed two lawsuits on behalf of alleged drone victims in May of 2012, arranged, paid for and collaborated on many of the victim interviews conducted by the Stanford-NYU researchers.
No database is perfect, but a strong argument could be made that the New America Foundation's policy of attempting to identify both militants and civilians while maintaining an explicit category for "unknowns" to represent the uncertainty surrounding many of the strikes is a more balanced representation of the facts. After all, if the public is to assess the efficacy of drone strikes then we need to consider the strikes that hit their targets as well as the ones that do not.
Stanford and NYU's analysis of the three drone databases, in some respects, misses the forest for the trees. A comparison of the annual data collected by the Bureau with that compiled by the New America Foundation suggests that while their numbers may differ, their underlying findings are substantially similar. Both the Bureau and New America have observed a general decline in the percentage of civilian fatalities since the high in 2006, with a sharp decrease during the Obama administration. According to the Bureau's data, the proportion of civilians killed in drone strikes fell from 35 percent in 2008 under President Bush to 9 percent thus far in 2012. Similarly, New America reports that the rate of civilian and unknown casualties decreased from 23 percent in 2008 to 2 percent in 2012. This stands to reason when we consider that the Bureau tends to err on the side of assuming that unidentified dead or individuals of disputed identity are civilians, while New America prefers to accommodate ambiguity by listing "unknowns" and only reports a civilian or militant casualty when it is reported by two independent sources.
If one takes the average number of civilian and unknown deaths counted by the New America Foundation from 2004-2012 as a percentage of the total number of people killed in drone strikes, the civilian and unknown deaths represent 15 percent of the total. The Bureau's data suggests that civilian deaths account for 22 percent of the total killed. These numbers are in fact, not so very far apart. And while the Bureau has criticized New America for reporting that in 2012, civilian deaths are approaching zero, the Bureau's own data suggests that the percentage of civilian casualties in 2012 is at nine percent, an all-time low and significantly less than the 23 percent the Bureau reports during Obama's first year in office. This suggests that the choice is not between two very different data sets, but rather two different interpretations of similar evidence. Contrary to the Stanford-NYU report's claims, the New America Foundation is not underreporting civilian casualties; its researchers have simply chosen not to feign certainty in the face of ambiguity.
What They Didn't Say
The Stanford-NYU researchers also fail to incorporate evidence that might temper their claims. Their report makes numerous references to an AP investigative report that interviewed 80 villagers at the sites of the ten deadliest attacks in 2011 and 2012 but neglects its conclusions that "a significant majority of the dead were combatants" and the numbers of casualties gathered by AP investigators "turned out to be very close to those given by Pakistani intelligence on the day of each strike."
Numerous media accounts, including the LA Times, have highlighted that "the [Stanford-NYU] study concludes that only about 2 percent of drone casualties are top militant leaders." The study reached no such conclusion. This claim is based on research conducted by New America and published last month on CNN. While the Stanford-NYU report summarily dismissed New America's militant and civilian casualty estimates, they have been quick to make this statistic a centerpiece of their argument. In doing so, they also take the conclusion out of context, and ignore the possible tactical utility of deliberately targeting low-level militants. Indeed, destroying communication centers, training camps and vehicles undermines the operational effectiveness of al-Qaeda and the Taliban, and quotes from operatives of the Pakistan-based Haqqani Network reveal that drones have forced them into a "jungle existence" where they fear for the lives on a daily basis.
A second key contention of the Stanford-NYU report is that drone strikes are "damaging and counterproductive" to U.S. national security. However, it is shortsighted to evaluate U.S. security in isolation from Pakistani domestic order. Terrorist attacks have been a pervasive problem in Pakistan, especially since 2007. While the report references these attacks and the frequent assassinations carried out by Taliban forces, it makes no reference to recent research by an analyst at the RAND Corporation, who identifies a negative correlation between drone strikes and militant violence inside Pakistan, indicating that the rate of violence has gone down as the rate of drone strikes has gone up. Analysis by New America suggests that while only 10 percent of drone strikes target al-Qaeda operatives, approximately 50 percent hit Taliban targets and in 2009, 19 of the first 32 drone strikes targeted the organization of Baitullah Mehsud, the leader of the Pakistani Taliban who was ultimately responsible for the deaths of thousands of Pakistanis, and the alleged mastermind of the assassination of former Pakistani Prime Minister Benazir Bhutto. Mehsud was a far greater threat to Pakistani security than that of the United States.
Key Findings
These criticisms are not to suggest that the Stanford-NYU report is without merit. The anecdotal evidence on the adverse mental health impact of drones, the economic hardships created by attacks and the restriction of movement caused by FATA residents' anxiety about potential strikes are all valuable contributions to the public's understanding of drones, as is the report's succinct and cogent summary of some of the key legal issues. Both of these topics merit further research and the teams at Stanford and NYU seem well poised to continue contributing in these areas.
However it would be a mistake to unequivocally accept all of the report's conclusions or its stance on the civilian casualty debate. The U.S. government's claims that civilian casualties from drone strikes during Obama's term in office are in the single digits are manifestly untrue, but there is no need to overstate the rate of civilian deaths to make the point that drones strikes are legally suspect and morally hazardous.
Meg Braun is a Rhodes Scholar and MPhil candidate in International Relations at Oxford, where she is researching the evolution of U.S. drone policy. She was an intern at the New America Foundation during summer 2012, where she worked to revise and update its drone database.
By Meg Braun Friday, November 2, 2012 - 12:15 PM
A report released in September by human rights researchers at Stanford Law School and New York University Law School sets out to demonstrate that U.S. drone policies are "damaging and counterproductive."
Media outlets from CNN to BBC hailed the report as new evidence of the U.S. government's false narrative on drones and the New York Times' Scott Shane described the study as "among the most thorough on the subject to date."
While the Stanford-NYU report certainly presents a comprehensive review of existing drone research, its new contributions to the evidentiary record are far more modest than its sweeping conclusions would suggest.
Number Crunching
The U.S. government claims that civilian casualties caused by drones are in the "single digits" during Obama's years in office, while the Stanford-NYU report seeks to establish that there is significant evidence that U.S. drone strikes have killed and injured a larger number of civilians. This is a low bar, and would merely necessitate proving that more than 10 civilians have been killed by drones since Obama assumed office, a claim that has been made by all three major databases aggregating information on drone strikes. The Stanford-NYU report goes further, claiming that between 474 and 881 civilians have been killed since 2004.
However, this does not represent new evidence. Stanford and NYU researchers made no attempt to offer new statistical analysis on the number of civilian casualties caused by drones. Rather, their report is essentially an extended endorsement of a database compiled by The Bureau of Investigative Journalism, a small team of journalists based out of City University, London. In doing so, the report rejects the findings of two other widely cited databases, The Long War Journal, which reports 138 civilian deaths and New America Foundation's Year of the Drone, which lists 152-191 civilian deaths and the deaths of 130-268 "unknowns."
If the Stanford-NYU team wants to paint a picture of the U.S. as a human rights abuser, then the Bureau, whose casualty statistics dwarf the more conservative estimates of the other databases, is best suited to that purpose, but that does not make it the most reliable source.
While a comprehensive assessment of the Bureau's data should have been conducted before giving it a resounding endorsement, just a cursory review of its account of 2012's drone strikes reveals problems:
- On May 5, 2012 the Bureau reports that a strike in North Waziristan killed 8-10 people, of whom between zero and ten are listed as civilians. Upon reviewing the accompanying sources one find that early reports said the identity of the dead could not be determined, but subsequent reports from government officials identified them as militants. Of the 15 sources cited by the Bureau, not one states that the victims were civilians.
-On June 2, 2012 the Bureau reports that 2-4 people were killed and between zero and two of them were civilians. These civilian deaths were inferred based on a report that a motorbike was accidentally hit, but the Bureau fails to take note of a later report from The Express Tribune identifying the motorbike victims as suspected militants Khalil Yargul Khail and Rehmanullah Gangi Khail.
-On July 23, 2012 the Bureau references 25 sources for a drone strike, only one of which described the victims as local residents and based on this the Bureau reports that up to 14 civilians were killed.
The Stanford-NYU report critiques the Long War Journal for not making its data available in a verifiable strike-by-strike format, over-relying on U.S. intelligence sources, and identifying all victims as militants unless they are specifically identified as civilians. These are reasonable criticisms, but when the critique turns to New America Foundation research, it becomes logically inconsistent.
While Stanford-NYU researchers admit that all three databases rely on "the same universe of publicly available press reports," they criticize only the New America Foundation for over-relying on the anonymous Pakistani government officials cited in such news reports. The Stanford-NYU report laments that these officials are cited in 88% of articles referenced in New America's 2012 data but fails to consider that these same articles are referenced by the Bureau. Furthermore, the only reason the researchers were able to generate that statistic is because New America compiles a highly transparent summary of media sources relied on for each strike with a list of which sources reported which casualty estimates, something no other database provides.
The Stanford-NYU report goes on to question the "deep reporting" capabilities of New America's sources (The New York Times, Reuters, AP, BBC, etc.) while uncritically accepting the veracity of reports from fringe outlets such as the Kuwait News Agency, the South Asian News Agency, Central Asia Online, and Punjab News, which are sometimes the sole sources for the Bureau's reports of civilian casualties.
This is not to suggest that the Bureau's research is without merit, but it should be noted that it represents an interpretation of the facts with a bias toward reporting civilian casualties. The Bureau's database does not even have a category for reporting militant deaths. It seems counter-intuitive that the organization's researchers can have such absolute faith in their civilian casualty estimates, but not have the confidence to label any of the deceased as militants.
And the Stanford-NYU team was not impartial. By the report's own admission, the research project it undertook in Pakistan to interview family members of drone strike victims was commissioned by Reprieve, a UK based advocacy organization. Reprieve, which filed two lawsuits on behalf of alleged drone victims in May of 2012, arranged, paid for and collaborated on many of the victim interviews conducted by the Stanford-NYU researchers.
No database is perfect, but a strong argument could be made that the New America Foundation's policy of attempting to identify both militants and civilians while maintaining an explicit category for "unknowns" to represent the uncertainty surrounding many of the strikes is a more balanced representation of the facts. After all, if the public is to assess the efficacy of drone strikes then we need to consider the strikes that hit their targets as well as the ones that do not.
Stanford and NYU's analysis of the three drone databases, in some respects, misses the forest for the trees. A comparison of the annual data collected by the Bureau with that compiled by the New America Foundation suggests that while their numbers may differ, their underlying findings are substantially similar. Both the Bureau and New America have observed a general decline in the percentage of civilian fatalities since the high in 2006, with a sharp decrease during the Obama administration. According to the Bureau's data, the proportion of civilians killed in drone strikes fell from 35 percent in 2008 under President Bush to 9 percent thus far in 2012. Similarly, New America reports that the rate of civilian and unknown casualties decreased from 23 percent in 2008 to 2 percent in 2012. This stands to reason when we consider that the Bureau tends to err on the side of assuming that unidentified dead or individuals of disputed identity are civilians, while New America prefers to accommodate ambiguity by listing "unknowns" and only reports a civilian or militant casualty when it is reported by two independent sources.
If one takes the average number of civilian and unknown deaths counted by the New America Foundation from 2004-2012 as a percentage of the total number of people killed in drone strikes, the civilian and unknown deaths represent 15 percent of the total. The Bureau's data suggests that civilian deaths account for 22 percent of the total killed. These numbers are in fact, not so very far apart. And while the Bureau has criticized New America for reporting that in 2012, civilian deaths are approaching zero, the Bureau's own data suggests that the percentage of civilian casualties in 2012 is at nine percent, an all-time low and significantly less than the 23 percent the Bureau reports during Obama's first year in office. This suggests that the choice is not between two very different data sets, but rather two different interpretations of similar evidence. Contrary to the Stanford-NYU report's claims, the New America Foundation is not underreporting civilian casualties; its researchers have simply chosen not to feign certainty in the face of ambiguity.
What They Didn't Say
The Stanford-NYU researchers also fail to incorporate evidence that might temper their claims. Their report makes numerous references to an AP investigative report that interviewed 80 villagers at the sites of the ten deadliest attacks in 2011 and 2012 but neglects its conclusions that "a significant majority of the dead were combatants" and the numbers of casualties gathered by AP investigators "turned out to be very close to those given by Pakistani intelligence on the day of each strike."
Numerous media accounts, including the LA Times, have highlighted that "the [Stanford-NYU] study concludes that only about 2 percent of drone casualties are top militant leaders." The study reached no such conclusion. This claim is based on research conducted by New America and published last month on CNN. While the Stanford-NYU report summarily dismissed New America's militant and civilian casualty estimates, they have been quick to make this statistic a centerpiece of their argument. In doing so, they also take the conclusion out of context, and ignore the possible tactical utility of deliberately targeting low-level militants. Indeed, destroying communication centers, training camps and vehicles undermines the operational effectiveness of al-Qaeda and the Taliban, and quotes from operatives of the Pakistan-based Haqqani Network reveal that drones have forced them into a "jungle existence" where they fear for the lives on a daily basis.
A second key contention of the Stanford-NYU report is that drone strikes are "damaging and counterproductive" to U.S. national security. However, it is shortsighted to evaluate U.S. security in isolation from Pakistani domestic order. Terrorist attacks have been a pervasive problem in Pakistan, especially since 2007. While the report references these attacks and the frequent assassinations carried out by Taliban forces, it makes no reference to recent research by an analyst at the RAND Corporation, who identifies a negative correlation between drone strikes and militant violence inside Pakistan, indicating that the rate of violence has gone down as the rate of drone strikes has gone up. Analysis by New America suggests that while only 10 percent of drone strikes target al-Qaeda operatives, approximately 50 percent hit Taliban targets and in 2009, 19 of the first 32 drone strikes targeted the organization of Baitullah Mehsud, the leader of the Pakistani Taliban who was ultimately responsible for the deaths of thousands of Pakistanis, and the alleged mastermind of the assassination of former Pakistani Prime Minister Benazir Bhutto. Mehsud was a far greater threat to Pakistani security than that of the United States.
Key Findings
These criticisms are not to suggest that the Stanford-NYU report is without merit. The anecdotal evidence on the adverse mental health impact of drones, the economic hardships created by attacks and the restriction of movement caused by FATA residents' anxiety about potential strikes are all valuable contributions to the public's understanding of drones, as is the report's succinct and cogent summary of some of the key legal issues. Both of these topics merit further research and the teams at Stanford and NYU seem well poised to continue contributing in these areas.
However it would be a mistake to unequivocally accept all of the report's conclusions or its stance on the civilian casualty debate. The U.S. government's claims that civilian casualties from drone strikes during Obama's term in office are in the single digits are manifestly untrue, but there is no need to overstate the rate of civilian deaths to make the point that drones strikes are legally suspect and morally hazardous.
Meg Braun is a Rhodes Scholar and MPhil candidate in International Relations at Oxford, where she is researching the evolution of U.S. drone policy. She was an intern at the New America Foundation during summer 2012, where she worked to revise and update its drone database.
