The past week (really, the past 4 1/2 years), America has had a public debate over the war in Iraq. These numbers have been thrown against those numbers, raising the question "how do you know what you know?" This is a question various professors have asked in my classes again and again.
When Kenneth Pollack (now my professor) and Michael O'Hanlon wrote their op-ed in the New York Times, titled A War We Just Might Win (they did not choose the title), they were criticized for basing their claims off of a week-long tour of US military personnel and installations in Iraq, contextualized by an expertise in mostly conventional military operations. Glenn Greenwald, who I'd guess I share many political views with, is 110% convinced in his own righteousness in criticizing O'Hanlon for cherry-picking numbers (O'Hanlon's rebuttal largely rehashes his original points, showing how Greenwald and O'Hanlon are talking past each other).
When the seven soldiers from the 82nd Airborne wrote their op-ed, coming to opposite conclusions from Pollack and O'Hanlon, they were criticized because their knowledge was so immediate that they couldn't possibly see the larger picture.
Choosing objective measurable metrics, like Eli is trying to do, is a possible fix to this problem. But as he acknowledges, just because you can measure something doesn't mean it's valuable. The corollary to this is that just because something is valuable doesn't mean it's measurable. A further problem is just because a number matters today does not mean it will matter tomorrow - as Eli argues, the strategy has to drive the metrics. Strategies change over time, meaning metrics change over time. This leads to the possibility that the situation might be measurable today but not tomorrow. We are then back to square one. I disagreed with a lot of what LtC Gian Gentile said at this Heritage Foundation event, but he was right when he said that the best way to communicate the military/political situation in an insurgency is through narrative.
Recently Dan at TDAXP has frustrated me with his claims that the continent of Africa has sunk into an abyss of genocide, economic disaster and decay. His argument is that a Western recolonization of Africa is Africa's only hope.
I violently want to disagree with him - I think his data is either sketchy or irrelevant (why would Africans be too unintelligent to turn failed states into functioning states without foreign domination, but other failed societies self-corrected independently?) but I don't have the necessary data to back up my own claims (I actually don't think the data exists to prove or disprove it yet). Essentially my argument is similar to Nassim Nicholas Taleb's argument in The Black Swan - You think you know a lot and I think I know very little, but since in reality we both know very little, I'm ahead of you.
What touched off this rant was watching a video of Chris Abani's talk at TED titled "Learning the Stories of Africa." It's about the importance of narrative, and everybody should take 18 minutes of their time to go watch it.
How to measure insurgencies
Here's an essay from my friend Eli, a fellow student in Georgetown's Security Studies Program. I might not agree with everything he says in this piece, but it deserves more of an audience than us arguing in a classroom (and probably more of an audience than most of the op-eds published in the Washington Post anyway).
How to measure insurgencies
Eli Margolis
Earlier this week, America’s top two officials in Iraq testified before Congress about the war in Iraq. Ambassador Crocker described slow but sure progress; General Petraeus spoke more strongly, citing goals met and “substantial” progress.
I was surprised. After a steady public debate of stalemate and withdrawal, the pair put forward recommendations to remain. The disconnect between how America has judged Iraq and how our two most knowledgeable professionals have is great.
Why?
I believe that the answer lies in measures. Media reports and independent assessments like the Brookings Institution’s “Iraq Index” have opened the floodgates on statistics. Analyses abound. But, as a recent Salon piece demonstrates, not all have been disciplined. Indeed, the public discourse has abandoned methodology entirely.
In an unusual move, however, Gen. Petraeus took time away from his testimony to assure Congress that he hasn’t. The military, he said, uses “a methodology that has been in place for well over a year” to ensure “rigor and consistency” in its analyses. Then he called in a second opinion: “Two U.S. intelligence agencies recently reviewed our methodology and they concluded that the data we produce is the most accurate and authoritative in Iraq.”
What is this methodology? Or, more broadly, how do we measure insurgencies?
To answer that question, I began to rummage around, uncovering a number of studies outlining insightful conceptual approaches. They hardly agree. But, taken together, they highlight five important principles.
First is the firm assertion that there are no magic numbers—not troops deployed, not dollars spent, not total number of insurgent attacks. As one of West Point’s “Irregular Warfare Messages of the Month” notes bluntly, “trying to reduce success or failure to one or two criteria is risky if not irresponsible.” Instead, suggests Craig Cohen of the U.S. Institute of Peace, it is better “to devise an aggregate index of indicators.” With measures, more may not always be better, but a handful will always be too few.
Second, analysts need a framework that attaches meaning to each metric. As James Clancy and Chuck Crossett explain in one of the Army’s leading journals, different officials too often find different meaning in the same numbers because they have no common reference. To one, falling casualties may be good news. But, to another, it is a sign of decreasing patrols—a possible indicator of heightened instability. The Army’s Douglas Jones phrases it simply: “it is only through agreement of definitions and a common framework of insurgency that applying measures of effectiveness to counterinsurgency operations becomes useful.” Without a framework, a pile of statistics can be made to fit almost any position.
Third, measures must be important, not just convenient. Counting heads at a graduation parade is far easier than measuring public opinion in a war zone or tracking insurgent financing. But it is a poorer measure of effectiveness. As Frederick W. Kagan notes in the Armed Forces Journal, such tallies of casualties, attacks, and trained locals “are measures of convenience, reflecting the ease with which data can be collected and presented rather than its inherent importance.” Honest assessment begins with honest data, even if it is difficult or dangerous to collect.
Fourth, outputs are more important than inputs. Measuring inputs like total dollars spent or the number of bases constructed gauges effort, not effectiveness. As Craig Cohen notes, progress should not be “judged in large part on the basis of international resources expended or programs implemented rather than on the basis of actual results produced.” In some ways, this is related to the problem of convenience; analysts can track coalition actions much more readily than their effects. But it is the effects—not efforts—that ultimately matter most.
Fifth—and perhaps most important—is the recognition that the strategy must determine the metrics. The two must be tied. If one campaign goal is to disrupt insurgent operations, for instance, a count of local cell phones would be little more than a statistical distraction. In their approaches, researchers from USIP, the Rand Corporation, the Johns Hopkins University, the Brookings Institution, and the Army’s Command and General Staff College all follow this principle. They start high and move down the ladder—from strategy to goals, from goals to measures, and from measures to specific metrics. As in a chain of command, each metric reports to a goal, and that goal back up to the strategy. This approach both highlights needed metrics and removes unneeded metrics—the cell phone counts of some government fact sheets.
So, before the testimony, how did the measures in America’s public discourse hold up next to these principles? In a word, poorly.
Media reports were misleading. Major newspapers continue to announce casualties and troop levels daily, encouraging a “magic number” mindset. The Washington Post’s series “Weighing the Surge” cites inputs like dollars of oil revenue spent or the number of Baghdad security outposts, and convenient counts like the number of market stalls opened, Iraqis trained, or barrels of oil produced. A graphic published the day before report is the perfect example. And a recent overview in the New York Times presents an array of measures without any mention of the author’s insightful framework.
But the media follows the lead of others. While interning with a government agency working to rebuild Iraq, I saw firsthand the random—and always positive—fact sheets that once circled Washington. Today’s Congressional benchmarks are only slightly better. The legislative requirements, for instance, are measures of convenience; in such a corrupt place, laws are cheap markers of government effectiveness and social change. Further, at least five of the eighteen benchmarks measure inputs and not outputs. And nor are they tied to strategy. As last Wednesday’s GAO report notes, they were derived not from methodical assessment, but from public statements made by severely pressured politicians.
Notably, such an application to Iraq should be taken with a degree of due reflection. These principles apply to insurgencies; the war in Iraq is more than an insurgency. Among others, tribal warlords, political opportunists, criminal networks, foreign intelligence services, and terrorist organizations complicate the picture considerably. Further, these troubles with measures do not necessarily discredit the broad conclusions of public discourse. Plainly, Iraq remains stubbornly unstable and violent.
But the guidelines outlined above can help—and not just with Iraq. The United States faces an international environment simmering with active and possible insurgencies. It is a challenge that will not go away. Bringing America’s public measures nearer its professional ones may help us develop clearer consensus—ensuring that our conflicts abroad do not also become our conflicts at home.
Update: Eli has graduated from Politics and Soccer and moved on to post this piece on Small Wars Journal. I'm only slightly jealous.
How to measure insurgencies
Eli Margolis
Earlier this week, America’s top two officials in Iraq testified before Congress about the war in Iraq. Ambassador Crocker described slow but sure progress; General Petraeus spoke more strongly, citing goals met and “substantial” progress.
I was surprised. After a steady public debate of stalemate and withdrawal, the pair put forward recommendations to remain. The disconnect between how America has judged Iraq and how our two most knowledgeable professionals have is great.
Why?
I believe that the answer lies in measures. Media reports and independent assessments like the Brookings Institution’s “Iraq Index” have opened the floodgates on statistics. Analyses abound. But, as a recent Salon piece demonstrates, not all have been disciplined. Indeed, the public discourse has abandoned methodology entirely.
In an unusual move, however, Gen. Petraeus took time away from his testimony to assure Congress that he hasn’t. The military, he said, uses “a methodology that has been in place for well over a year” to ensure “rigor and consistency” in its analyses. Then he called in a second opinion: “Two U.S. intelligence agencies recently reviewed our methodology and they concluded that the data we produce is the most accurate and authoritative in Iraq.”
What is this methodology? Or, more broadly, how do we measure insurgencies?
To answer that question, I began to rummage around, uncovering a number of studies outlining insightful conceptual approaches. They hardly agree. But, taken together, they highlight five important principles.
First is the firm assertion that there are no magic numbers—not troops deployed, not dollars spent, not total number of insurgent attacks. As one of West Point’s “Irregular Warfare Messages of the Month” notes bluntly, “trying to reduce success or failure to one or two criteria is risky if not irresponsible.” Instead, suggests Craig Cohen of the U.S. Institute of Peace, it is better “to devise an aggregate index of indicators.” With measures, more may not always be better, but a handful will always be too few.
Second, analysts need a framework that attaches meaning to each metric. As James Clancy and Chuck Crossett explain in one of the Army’s leading journals, different officials too often find different meaning in the same numbers because they have no common reference. To one, falling casualties may be good news. But, to another, it is a sign of decreasing patrols—a possible indicator of heightened instability. The Army’s Douglas Jones phrases it simply: “it is only through agreement of definitions and a common framework of insurgency that applying measures of effectiveness to counterinsurgency operations becomes useful.” Without a framework, a pile of statistics can be made to fit almost any position.
Third, measures must be important, not just convenient. Counting heads at a graduation parade is far easier than measuring public opinion in a war zone or tracking insurgent financing. But it is a poorer measure of effectiveness. As Frederick W. Kagan notes in the Armed Forces Journal, such tallies of casualties, attacks, and trained locals “are measures of convenience, reflecting the ease with which data can be collected and presented rather than its inherent importance.” Honest assessment begins with honest data, even if it is difficult or dangerous to collect.
Fourth, outputs are more important than inputs. Measuring inputs like total dollars spent or the number of bases constructed gauges effort, not effectiveness. As Craig Cohen notes, progress should not be “judged in large part on the basis of international resources expended or programs implemented rather than on the basis of actual results produced.” In some ways, this is related to the problem of convenience; analysts can track coalition actions much more readily than their effects. But it is the effects—not efforts—that ultimately matter most.
Fifth—and perhaps most important—is the recognition that the strategy must determine the metrics. The two must be tied. If one campaign goal is to disrupt insurgent operations, for instance, a count of local cell phones would be little more than a statistical distraction. In their approaches, researchers from USIP, the Rand Corporation, the Johns Hopkins University, the Brookings Institution, and the Army’s Command and General Staff College all follow this principle. They start high and move down the ladder—from strategy to goals, from goals to measures, and from measures to specific metrics. As in a chain of command, each metric reports to a goal, and that goal back up to the strategy. This approach both highlights needed metrics and removes unneeded metrics—the cell phone counts of some government fact sheets.
So, before the testimony, how did the measures in America’s public discourse hold up next to these principles? In a word, poorly.
Media reports were misleading. Major newspapers continue to announce casualties and troop levels daily, encouraging a “magic number” mindset. The Washington Post’s series “Weighing the Surge” cites inputs like dollars of oil revenue spent or the number of Baghdad security outposts, and convenient counts like the number of market stalls opened, Iraqis trained, or barrels of oil produced. A graphic published the day before report is the perfect example. And a recent overview in the New York Times presents an array of measures without any mention of the author’s insightful framework.
But the media follows the lead of others. While interning with a government agency working to rebuild Iraq, I saw firsthand the random—and always positive—fact sheets that once circled Washington. Today’s Congressional benchmarks are only slightly better. The legislative requirements, for instance, are measures of convenience; in such a corrupt place, laws are cheap markers of government effectiveness and social change. Further, at least five of the eighteen benchmarks measure inputs and not outputs. And nor are they tied to strategy. As last Wednesday’s GAO report notes, they were derived not from methodical assessment, but from public statements made by severely pressured politicians.
Notably, such an application to Iraq should be taken with a degree of due reflection. These principles apply to insurgencies; the war in Iraq is more than an insurgency. Among others, tribal warlords, political opportunists, criminal networks, foreign intelligence services, and terrorist organizations complicate the picture considerably. Further, these troubles with measures do not necessarily discredit the broad conclusions of public discourse. Plainly, Iraq remains stubbornly unstable and violent.
But the guidelines outlined above can help—and not just with Iraq. The United States faces an international environment simmering with active and possible insurgencies. It is a challenge that will not go away. Bringing America’s public measures nearer its professional ones may help us develop clearer consensus—ensuring that our conflicts abroad do not also become our conflicts at home.
Update: Eli has graduated from Politics and Soccer and moved on to post this piece on Small Wars Journal. I'm only slightly jealous.
USA vs Brazil
Yesterday, in addition to the Revs vs. DC game, the United States (minus its players on New England) took on a full-strength Brazil. By all accounts they played well and lost 4-2, the final two Brazil goals being a free kick and a penalty kick. I didn't get to watch it, as I was at RFK stadium, but here are the highlights.
Revolution lose to DC United
I went to see the Revolution play at DC United's RFK stadium today. Highlights are below. First, some notes:
1. DC had a goal called back for offsides that was a near thing;
2. They then scored a clearly offsides goal that was allowed to stand, giving me the impression of a makeup call for a call that was probably correct anyway.
3. Taylor Twellman scored a Golazo!
4. Their winning goal started as Ben Olsen (who is one of the dirtiest players in the league but for some reason is DC United's posterchild, after Moreno) hacked at Khano Smith's ankles from behind. Smith fell, no call from the ref, and Olsen took the ball down the wing and crossed it for a goal.
5. We sat on the upper deck, across the stadium from the Barra Brava and Screaming Eagles (DC supporters clubs), and they were in fine voice - DC has the best atmosphere stadium-wide of any MLS team I've seen. The Revs' stadium is cavernous, drowning all the sound the Riders make.
The level of play has risen every year in MLS, but the level of officiating has yet to keep up.
1. DC had a goal called back for offsides that was a near thing;
2. They then scored a clearly offsides goal that was allowed to stand, giving me the impression of a makeup call for a call that was probably correct anyway.
3. Taylor Twellman scored a Golazo!
4. Their winning goal started as Ben Olsen (who is one of the dirtiest players in the league but for some reason is DC United's posterchild, after Moreno) hacked at Khano Smith's ankles from behind. Smith fell, no call from the ref, and Olsen took the ball down the wing and crossed it for a goal.
5. We sat on the upper deck, across the stadium from the Barra Brava and Screaming Eagles (DC supporters clubs), and they were in fine voice - DC has the best atmosphere stadium-wide of any MLS team I've seen. The Revs' stadium is cavernous, drowning all the sound the Riders make.
The level of play has risen every year in MLS, but the level of officiating has yet to keep up.
Come to my conference!
For my job, I'm helping put on what is shaping to be a pretty interesting conference. September 25 and 26, The Battle of Ideas: Messages, Mediums and Methods will look at how conflicting ideologies influence the conduct of the war on the ground. A bunch of interesting presentations, including one by another guy in my program (US army vet) about his experiences doing information operations in Ramadi.
Confirmed presenters include Marc Lynch, author of Voices of the New Arab Public, John Robb, author of Brave New War, TX Hammes, author of The Sling and the Stone, Todd Helmus and Christopher Paul, coauthors of Enlisting Madison Avenue, and Kyle Teamey, coauthor of the US Counterinsurgency Field Manual.
The seminar is free and all are welcome. Click on one of the links below for more info. When the official link on the NDU website is live I'll post a link to it as well.
Around the web:
Transformation National Security (I post on here from work).
MountainRunner
Small Wars Council
John Robb
Confirmed presenters include Marc Lynch, author of Voices of the New Arab Public, John Robb, author of Brave New War, TX Hammes, author of The Sling and the Stone, Todd Helmus and Christopher Paul, coauthors of Enlisting Madison Avenue, and Kyle Teamey, coauthor of the US Counterinsurgency Field Manual.
The seminar is free and all are welcome. Click on one of the links below for more info. When the official link on the NDU website is live I'll post a link to it as well.
Around the web:
Transformation National Security (I post on here from work).
MountainRunner
Small Wars Council
John Robb
Interview with Jeff Larentowicz
Here's an interview with Revolution midfielder Jeff Larentowicz:
I had bought his jersey once he got a couple starts under his belt, so last season when the Revs came to DC, I found his email through the Brown alumni system and got his autograph. Then when Pat and I were at Brown's reunion this past summer, we saw him eating lunch outside with (I assume) his parents, and we chatted him up for a few minutes. He was like "Oh, you're the guy that adds all those nicknames on my Wikipedia!" He's a very cool guy - you'd never guess he was a professional athlete who can sell his own posters. Hopefully he'll score a bunch against DC on Sunday...
Q: Give us a little preview of what [goal celebrations] could be coming up as the games get bigger and the goals more important?I watched Jeff Larentowicz (aka Big Red, Ginger Jeff, or the Ginja Ninja) play his college soccer at Brown University, and so it's been pretty exciting watching him get into MLS, then get his first minutes, then get his first start, and finally be recognized as one of the better defensive midfielders in the league.
A: It’s a little more difficult to come up with good celebrations when you’re the redhead defensive midfielder who used to wear goggles... I would just look for a lot of fist pumps and yelling.
I had bought his jersey once he got a couple starts under his belt, so last season when the Revs came to DC, I found his email through the Brown alumni system and got his autograph. Then when Pat and I were at Brown's reunion this past summer, we saw him eating lunch outside with (I assume) his parents, and we chatted him up for a few minutes. He was like "Oh, you're the guy that adds all those nicknames on my Wikipedia!" He's a very cool guy - you'd never guess he was a professional athlete who can sell his own posters. Hopefully he'll score a bunch against DC on Sunday...
UN catches on to mobile phones
Via Chris Albon at Human Security Review, we see that the UN is using text messages to inform Iraqi refugees in Syria about food aid:
The United Nations has sent about 10,000 text messages on mobile phones to help inform Iraqi refugees in Syria that an international food distribution program for them begins tomorrow...I wonder how soon it will be before some enterprising terrorist hacks the UN's database of phone numbers, sends out a false pickup time and place, and detonates a car bomb there.
...This is the first large-scale text message campaign to provide information about a humanitarian assistance program, said World Food Program spokesman Khaled Mansouri, speaking in a telephone interview from Cairo. "This is a technology that will be used more and more in the future."
Text messages to mobile phones are one of the most effective ways to communicate with communities of refugees who may not have stable addresses, the UN said.
Irony or lunacy?
Inspired by Glenn Greenwald's examination of the nutcases that want to bomb Iran, Digby digs up an article from 2003 by Michael Ledeen at the National Review. An excerpt:
Of course this would have to be an extremely wide-spread Sokal-conspiracy, involving Douglas MacKinnon as well, who has his own crazy ramblings.
So the French and the Germans struck a deal with radical Islam and with radical Arabs: You go after the United States, and we'll do everything we can to protect you, and we will do everything we can to weaken the Americans.Michael Ledeen is actually taken seriously on by many Republicans and members of the current administration. His grand conspiracy theory is especially ironic, considering today's news:
I continue to hold out hope that people like Ledeen are actually engaged in a long-term punking of the right-wing media (National Review, FrontPageMag, WorldNetDaily, etc.). Perhaps they were inspired by Alan Sokal and are engaged in a long-term project to destroy right-wing credibility on matters of national security.Three suspects allegedly trained in Pakistan by an Al Qaeda-linked group have been arrested for plotting massive car-bomb attacks on US troops and other Americans near US military bases and German airports, authorities said yesterday...
The trio in Germany allegedly planned simultaneous strikes on three soft targets that might have included nightclubs, bars, restaurants, or airports frequented by American soldiers and tourists, according to German and US law enforcement officials. Because the confiscated materials could have produced the equivalent of about 1,000 pounds of TNT, the casualty toll could have far exceeded the transport bombings that killed 52 people in London in 2005 and 191 people in Madrid in 2004, officials said.
Of course this would have to be an extremely wide-spread Sokal-conspiracy, involving Douglas MacKinnon as well, who has his own crazy ramblings.
The surge, updated
Looks like my earlier thoughts on the Surge were correct. From the IHT:
For now, though, Bush told the author, Robert Draper, in a later session, "I'm playing for October-November." That is when he hopes the Iraq troop increase will finally show enough results to help him achieve the central goal of his remaining time in office: "To get us in a position where the presidential candidates will be comfortable about sustaining a presence," and, he said later, "stay longer."
Loose nukes...
A B-52 flew some cruise missiles from North Dakota to Louisiana to be dismantled last Thursday, August 30th. All well and good, but somebody forgot to remove their nuclear warheads. Whoops! Stories from the AP, Guardian and the original one at an NBC affiliate.
Nobody was in danger and there was no possibility of the weapons ever detonating, but it's a more than a minor goof. The Air Force squadron commander was immediately removed from command and lost his nuclear certification. He will probably never be promoted again, which means his career is over and he'll leave the military.
The first thing I thought of when I read this was this line from Paul Yingling's essay from last May, A Failure in Generalship, in which he states:
Major General Taguba was essentially forced out for doing his job well, investigating the Abu Ghraib incidents. Shinsheki as we all know was forced out for doing his job in giving his honest opinion about the forces required for invading Iraq. On the other hand, there was no accountability to the generals who thought the insurgency would be defeated by just killing more people. Meanwhile, the people who were and are successful in Iraq, like Col. McMaster, are repeatedly passed over for promotion.
It is of course important to handle nuclear weapons carefully and to keep track of them at all times. But it is more important to win wars...
Nobody was in danger and there was no possibility of the weapons ever detonating, but it's a more than a minor goof. The Air Force squadron commander was immediately removed from command and lost his nuclear certification. He will probably never be promoted again, which means his career is over and he'll leave the military.
The first thing I thought of when I read this was this line from Paul Yingling's essay from last May, A Failure in Generalship, in which he states:
As matters stand now, a private who loses a rifle suffers far greater consequences than a general who loses a war.An Air Force squadron commander was correctly fired for misplacing five nuclear weapons for three and a half hours. No-one to my knowledge has been fired for mismanaging the wars in Iraq or Afghanistan. Different sorts of accountability exist at different levels.
Major General Taguba was essentially forced out for doing his job well, investigating the Abu Ghraib incidents. Shinsheki as we all know was forced out for doing his job in giving his honest opinion about the forces required for invading Iraq. On the other hand, there was no accountability to the generals who thought the insurgency would be defeated by just killing more people. Meanwhile, the people who were and are successful in Iraq, like Col. McMaster, are repeatedly passed over for promotion.
It is of course important to handle nuclear weapons carefully and to keep track of them at all times. But it is more important to win wars...
Subscribe to:
Posts (Atom)