NOW AVAILABLE The draft of my book on Organizational Intelligence is now available on LeanPub http://leanpub.com/orgintelligence. Please support this development by subscribing and commenting. Thanks.

Wednesday, March 1, 2017

Decision-Making Models

In my previous discussion of the ACPO national decision model (May 2014), I promised to return to the methodological question, namely what theories of decision-making would be relevant to NDM and any other decision models. I have just happened upon a doctoral thesis by Maxwell Mclean looking at the decision-making by coroners, which analyses local variation in coronial outcomes at three decision-making stages: whether to report the death, whether to advance to inquest, and the choice of inquest conclusion.

Mclean notes that there is no decision-making model for coroners equivalent to the police national decision model and focussed on standards and consistency of outcome. He finds other examples of decision-making models in nursing (Lewinson and Truglio-Londrigan, 2008; Husted and Husted, 1995; Jasper, Rosser and Mooney, 2013); social work (O’Sullivan, 2011; Taylor, 2010); and probation work (Carter, 1967; Rosecrance, 1985). However, several of these are descriptive models rather than normative models.

Within the professions mentioned by Mclean, I found a lot more work on evidence-based nursing as well as some interesting international discussions on decision-making within offender supervision. Looking further afield, I was interested to find an article about a decision-making model in the US Army, but this turned out to be merely a polemical article by a former Navy Seal advocating the use of Design Thinking.

Rosecrance introduces an interesting concept of the Ball Park, where a professional decision is influenced by the anticipated reaction of a more senior professional. For example, the decisions of a probation officer are not solely designed to achieve the desired outcomes for the client, but also designed to meet the approval of (1) judges, (2) prosecuting attorneys, and (3) probation supervisors. When a recommendation seems likely to meet the approval of these three entities, it is said to be "in the ball park". The "ball park" concept is also used in sales negotiations, and this hints at the idea that the focus here is on "selling" (or at least defending) the decision rather than just making it.

Coming back to the police, this frames the NDM not just as a way of making the best decision but also avoiding censure if anything goes wrong. See my post on the National Decision Model and Lessons Learned (February 2017).



Miranda Boone and Martine Evans, Offender supervision and decision-making in Europe (Offender Supervision in Europe: Decision-Making and Supervision Working Group, 2013)

Jeff Boss, The Army's New Decision-Making Model (Forbes, 8 August 2014)

Carter, R.M. (1967). The presentence report and the decision making process. Journal of
research in crime and delinquency. 4 203-211.

Jasper, M., Rosser, M., Mooney, G. (Eds.) (2013). Professional Development, Reflection
and Decision-Making in Nursing and Health Care (2nd ed.). Swansea: Wiley Blackwell.

Husted, G.L. and Husted, I.H. (1995). Ethical decision-making in nursing (2nd ed.). St
Louis: Mosby.

Lewenson, S.B. and Truglio-Londrigan, M. (2008). Decision-Making in Nursing, thoughtful approaches for practice. London: Jones and Bartlett Publishers International.

Maxwell Mclean, The Coroner in England and Wales; Coronial Decision-­Making and Local Variation in Case Outcomes (Doctoral Thesis, University of Huddersfield, 2015)

O'Sullivan, T. (2011). Decision making in social work (2nd ed.). Basingstoke: Palgrave
Macmillan

Rosecrance, J. (1985). The Probation Officers' Search for Credibility: Ball Park
Recommendations. Journal of research in crime and delinquency. 31, (4) 539-554.

Mooi Standing, Perceptions of clinical decision-making: a matrix model (May 2010). This appears to be a chapter from Mooi Standing (ed) Clinical Judgement and Decision-Making in Nursing and Inter-professional Healthcare (McGraw Hill, 2010)

Taylor, B. (2010). Professional Decision-Making in Social Work. Exeter: Learning Matters.

Carl Thompson et al, Nurses, information use, and clinical decision making—the real world potential for evidence-based decisions in nursing (Evidence-Based Nursing Vol 7 No 3, July 2004) http://dx.doi.org/10.1136/ebn.7.3.68


Related posts
National Decision Model (May 2014)
National Decision Model and Lessons Learned (Feb 2017)

Updated 4 March 2017

Monday, February 27, 2017

National Decision Model and Lessons Learned

The appointment of Cressida Dick as the first female commissioner of the Metropolitan Police has been criticized in some quarters because of her involvement in the fatal shooting of Jean Charles de Menezes in 2005.

Dick was the "gold commander" who instructed armed officers to "stop" de Menezes. At the time, however, armed officers were following a new set of police guidelines known as Operation Kratos. In the context of these guidelines, Dick's orders were interpreted as shoot-to-kill. At the Old Bailey in 2007, Dick denied that this had been her intention.

As Mary Dejevsky argues, the de Menezes case provides a lasting reminder of what can go wrong
"whether because the overall atmosphere has not been properly appraised, because the orders given were not precise enough, or simply because insufficient account has been taken of the human factor".
The National Decision Model, which was introduced a few years after this incident, provides a framework that should (at least in theory) prevent this kind of miscommunication. See my post on the National Decision Model (May 2014). Perhaps this is one of the areas where "lessons have been learned". Or perhaps not.

Iain Gould is a solicitor. One of his clients was involved in an incident in 2013 that resulted in his being tasered. The Independent Police Complaints Commission (IPCC) attributed this escalation, in part, to a failure to follow the National Decision Model.
"I would question whether PC B gave enough emphasis to the first element of the National Decision Model, which is to communicate. ... More effort should have been made, in line with the National Decision Model, to engage Mr S in dialogue."

The IPCC commissioned a report in 2015, which contains some analysis of the National Decision Model, and some recommendations for its improved use. There are two versions of the report:



The Guardian view on the Met police: changing, but too slowly (23 February 2017)

Duncan Campbell et al, Leaks raise sharp questions about police tactics (Guardian, 17 August 2005)

Mary Dejevsky, Can Cressida Dick win over the public? Yes, if she’s learned from her mistakes (Guardian, 23 February 2017)

Iain Gould, Is Police Taser Policy Working? (11 May 2016)

Martin Hoscik, Sadiq Khan says ‘My heart goes out to the de Menezes family’ but insists Cressida Dick is the right choice to protect London (MayorWatch, 25 February 25, 2017)

Maxwell Mclean, The Coroner in England and Wales; Coronial Decision-­Making and Local Variation in Case Outcomes (Doctoral Thesis, University of Huddersfield, 2015)

Wail Qasim, Lessons Learned (LRB Blog, 27 February 2017)


Related blogpost

Saturday, February 4, 2017

The Art of the New Deal - Trump and Intelligence

In his 1967 book on Organizational Intelligence, Harold Wilensky praised President Roosevelt for maintaining a state of creative tension in the US administration. Wilensky reckoned that this enabled FDR to get a more accurate and rounded account of what was going on, and gave him some protection against the self-delusion of each department.

(In FDR's time, of course, it was considered entirely normal for an administration to be staffed by a bunch of white men with similar education. And yet even they managed to achieve some diversity of perspective.)

Early reports of Donald Trump's administration suggest an unconscious echo of the FDR style. Or perhaps a much earlier pattern.

At the center of it all has been a cast of characters jockeying for Trump’s ear, creating a struggle for power that has manifested in a mix of chaos, leaks and uncertainty. The Trump White House already bears more resemblance to the court of a Renaissance king than to most prior administrations as favorites come and go, counselors quarrel over favor and policy decisions are often made by whim or without consultation. (Guardian, 4 Feb 2017)

But it is difficult to see this as "creative tension" resulting in an "accurate and rounded" view.
“Trump thinks he’s invincible,” says Hemmings, who doubts whether his advisors will ever question or criticise him. “Usually leaders choose the people around them to keep them in check, and Trump needs people to temper his hotheadedness and aggression. Instead, he’s picked advisors who worship him.” (Independent, 2 Feb 2017)

Wilensky's book also discusses the dangers of a doctrine of secrecy.

Secrets belong to a small assortment of individuals, and inevitably become hostage to private agendas. As Harold Wilensky wrote “The more secrecy, the smaller the intelligent audience, the less systematic the distribution and indexing of research, the greater the anonymity of authorship, and the more intolerant the attitude toward deviant views.” (Gladwell 2010)

And secrecy seems to a key element of the Trump-Bannon modus operandi.
“These executive orders were very rushed and drafted by a very tight-knit group of individuals who did not run it by the people who have to execute the policy. And because that’s the case, they probably didn’t think of or care about how this would be executed in the real world,” said another congressional source familiar with the situation. “No one was given a heads-up and no one had a chance to weigh in on it.” (Politico 30 Jan 2017)


But perhaps in reaction to the Bannonite doctrine of secrecy, there has been a flood of leaks from inside the administration. Chris Cillizza suggests two possible explanations - either these leaks are intended to influence Trump himself (because he doesn't take anything seriously unless he hears it from his favourite media channels) or conversely they are intended as a kind of whistle-blowing.


Marx thought that history repeated itself. (Alarmingly, Trump's Counselor Steve Bannon adheres to the same view.) So are we into tragedy or farce here?




Rachael Bade, Jake Sherman and Josh Dawsey, Hill staffers secretly worked on Trump's immigration order (Politico, 30 Jan 2017)

Chris Cillizza, The leaks coming out of the Trump White House cast the president as a clueless child (Washington Post, 26 January 2017), The leaks coming out of the Trump White House right now are totally bananas (Washington Post, 2 Feb 2017)

Malcolm Gladwell, Pandora's Briefcase (New Yorker, 10 May 2010)

Rachel Hosie, The deeper reason we should be worried Donald Trump hung up on Australia PM Malcolm Turnbull (Independent, 2 Feb 2017)

Linette Lopez, Steve Bannon's obsession with a dark theory of history should be worrisome (Business Insider, 2 Feb 2017) HT @BryanAppleyard

Carmen Medina, What is your Stupification Point? (6 May 2010)

Joseph Rago, History Repeats as Farce, Then as 2016 (Wall Street Journal, 4 November 2016) paywall

Sabrina Siddiqui and Ben Jacobs, Trump's courtiers bring chaotic and capricious style to White House (Guardian, 4 February 2017)



Related posts

Puzzles and Mysteries (January 2010)
Enemies of Intelligence (May 2010)
Delusion and Diversity (October 2012)

Thursday, November 3, 2016

Pay as you Share

Announced and rapidly withdrawn, Admiral's proposed collaboration with Facebook was supposed to give drivers a discount on their car insurance premiums if their Facebook posts indicated the right kind of personality. According to some reports, the idea was that people who were reckless with punctuation (too many exclamation marks, not enough full stops) might also be reckless in their driving habits.

The punctuation example is probably a red herring. The analysis of personality will undoubtedly be based on much richer aspects than mere punctuation: Facebook is capable of much more sophisticated analysis, as well as selling data to other organizations for the same purpose.

For example, a Korean study in 2013 found that Facebook activities had predictive power in distinguishing depressed and nondepressed individuals. However, Facebook may not wish to draw too much public attention to such capabilities. (There are some important ethical issues in the use of algorithms to predict mental health issues, for example in recruitment screening, discussed at length by Cathy O'Neil.)

Meanwhile, insurance companies will wish to use any information and insight they can get their hands on, to try and calculate risk more accurately. People may consent to sharing their data if they feel they will benefit personally, or if they are unaware of the possible data uses and implications, but that could just result in discrimination against the people who refuse to share their data. So privacy campaigners may not be reassured by the fact that this particular collaboration has been withdrawn.



Cathy O'Neil, How algorithms rule our working lives (Guardian, 1 Sept 2016)

Sungkyu Park et al, Activities on Facebook Reveal the Depressive State of Users (J Med Internet Res. 2013 Oct; 15(10): e217)

Graham Ruddick, Admiral to price car insurance based on Facebook posts (Guardian, 2 November 2016, 00.01 GMT)

Graham Ruddick, Facebook forces Admiral to pull plan to price car insurance based on posts (Guardian, 2 November 2016, 18.41 GMT)


Related posts
Weapons of Math Destruction (Oct 2016)

Monday, October 31, 2016

The Transparency of Algorithms

Algorithms have been getting a bad press lately, what with Cathy O'Neil's book and Zeynap Tufekci's TED talk. Now the German Chancellor, Angela Merkel, has weighed into the debate, calling for major Internet firms (Facebook, Google and others) to make their algorithms more transparent.

There are two main areas of political concern. The first (raised by Mrs Merkel) is the control of the news agenda. Politicians often worry about the role of the media in the political system when people only pick up the news that fits their own point of view, but this is hardly a new phenomenon. Even in the days before the Internet, few people used to read more than one newspaper, and most people would prefer to read the newspapers that confirm their own prejudices. Furthermore, there have been recent studies that show that even when you give different people exactly the same information, they will interpret it differently, in ways that reinforce their previous beliefs. So you can't blame the whole Filter Bubble thing on Facebook and Google.

But they undoubtedly contribute further to the distortion. People get a huge amount of information via Facebook, and Facebook systematically edits out the uncomfortable stuff. It aroused particular controversy recently when its algorithms decided to censor a classic news photograph from the Vietnam war.

Update: Further criticism from Tufekci and others immediately following the 2016 US Election


The second area of concern has to do with the use of algorithms to make critical decisions about people's lives. The EU regards this as (among other things) a data protection issue, and privacy activists are hoping for provisions within the new General Data Protection Regulation (GDPR) that will confer a "right to an explanation" upon data subjects. In other words, when people are sent to prison based on an algorithm, or denied a job or health insurance, it seems reasonable to allow them to know what criteria these algorithmic decisions were based on.

Reasonable but not necessarily easy. Many of these algorithms are not coded in the old-fashioned way, but developed using machine learning. So the data scientists and programmers responsible for creating the algorithm may not themselves know exactly what the criteria are. Machine learning is basically a form of inductive reasoning, using data about the past to predict the future. As Hume put it, this assumes that “instances of which we have had no experience resemble those of which we have had experience”.

In a Vanity Fair panel discussion entitled “What Are They Thinking? Man Meets Machine,” a young black woman tried unsuccessfully to explain the problem of induction and biased reasoning to Sebastian Thrun, formerly head of Google X.
At the end of the panel on artificial intelligence, a young black woman asked Thrun whether bias in machine learning “could perpetuate structural inequality at a velocity much greater than perhaps humans can.” She offered the example of criminal justice, where “you have a machine learning tool that can identify criminals, and criminals may disproportionately be black because of other issues that have nothing to do with the intrinsic nature of these people, so the machine learns that black people are criminals, and that’s not necessarily the outcome that I think we want.”
In his reply, Thrun made it sound like her concern was one about political correctness, not unconscious bias. “Statistically what the machines do pick up are patterns and sometimes we don’t like these patterns. Sometimes they’re not politically correct,” Thrun said. “When we apply machine learning methods sometimes the truth we learn really surprises us, to be honest, and I think it’s good to have a dialogue about this.”

In other words, Thrun assumed that whatever the machine spoke was Truth, and he wasn't willing to acknowledge the possibility that the machine might latch onto false patterns. Even if the algorithm is correct, it doesn't take away the need for transparency; but if there is the slightest possibility that the algorithm might be wrong, the need for transparency is all the greater. And evidence is that some of the algorithms are grossly wrong.


In this post, I've talked about two of the main concerns about algorithms - firstly the news agenda filter bubble, and secondly the critical decisions affecting individuals. In both cases, people are easily misled by the apparent objectivity of the algorithm, and are often willing to act as if the algorithm is somehow above human error and human criticism. Of course algorithms and machine learning are useful tools, but an illusion of infallibility is dangerous and ethically problematic.



Rory Cellan-Jones, Was it Facebook 'wot won it'? (BBC News, 10 November 2016)

Ethan Chiel, EU citizens might get a ‘right to explanation’ about the decisions algorithms make (5 July 2016)

Kate Connolly, Angela Merkel: internet search engines are 'distorting perception' (Guardian, 27 October 2016)

Bryce Goodman, Seth Flaxman, European Union regulations on algorithmic decision-making and a "right to explanation" (presented at 2016 ICML Workshop on Human Interpretability in Machine Learning (WHI 2016), New York, NY)

Mike Masnick, Activists Cheer On EU's 'Right To An Explanation' For Algorithmic Decisions, But How Will It Work When There's Nothing To Explain? (Tech Dirt, 8 July 2016)

Fabian Reinbold, Warum Merkel an die Algorithmen will (Spiegel, 26 October 2016)

Nitasha Tiku, At Vanity Fair’s Festival, Tech Can’t Stop Talking About Trump (BuzzFeed, 24 October 2016) HT @noahmccormack

Julia Carrie Wong, Mark Zuckerberg accused of abusing power after Facebook deletes 'napalm girl' post (Guardian, 9 September 2016)

New MIT technique reveals the basis for machine-learning systems’ hidden decisions (Kutzweil News, 31 October 2016) HT @jhagel

Video: When Man Meets Machine (Vanity Fair, 19 October 2016)

See Also
The Problem of Induction (Stanford Encyclopedia of Philosophy, Wikipedia)


Related Posts
The Shelf-Life of Algorithms (October 2016)
Weapons of Math Destruction (October 2016)

Updated 10 November 2016

Monday, October 24, 2016

Learning at the Speed of Learning

According to a recent survey by McKinsey,  "the great majority of our respondents expect corporate learning to change significantly within the next three years".

It seems that whatever the topic of the survey, middle managers and management consultants always expect significant change within the next three years, because this is what justifies their existence.

In this case, the topic is corporate learning, which McKinsey recommends should be done "at the speed of business", whatever that means. (I am not a fan of the "at the speed of" cliche.)

But what kind of change is McKinsey talking about here? The article concentrates on digital delivery of learning material - disseminating existing "best practice" knowledge to a broader base. It doesn't really say anything about organizational learning, let alone a more radical transformation of the nature of learning in organizations. I have long argued that the real disruption is not in replacing classrooms with cheaper and faster equivalents, useful though that might be, but in digital organizational intelligence -- using increasing quantities of data to develop and test new hypotheses about customer behaviour, market opportunities, environmental constraints, and so on -- developing not "best practice" but "next practice".



Richard Benson-Armer, Arne Gast, and Nick van Dam, Learning at the speed of business (McKinsey Quarterly, May 2016). HT @annherrmann

Chris Argyris and Donald Schön, Organizational Learning: A Theory of Action Perspective. Reading, MA, Addison-Wesley, 1978.

Monday, August 29, 2016

The Judgment of whole Kingdoms and Nations

@Cybersal @kirstymhall @UKParliament #Brexit #VoxPopuli


A radical Whig tract was published in 1709 under the title Vox Populi, Vox Dei. The following year, an extended version was published under the title The Judgment of whole Kingdoms and Nations. I want to use these two phrases as the starting point for my submission to the UK Parliament Public Administration and Constitutional Affairs Committee, which has launched an inquiry into the lessons that can be learned for future referendums.

The first thing I want to mention is the rushed timescale. The inquiry was announced on July 14th, with a deadline for submissions of September 5th. I shall argue that this rushed timescale is symptomatic of the referendum itself, in which people were asked to make a complex decision with inadequate information and analysis.

(For the sake of comparison, an inquiry on the future of public parks was announced on July 11th, with a deadline of September 30th. So we are given more time to analyse the physical swings and roundabouts of council-run playgrounds than the metaphorical swings and roundabouts of parliamentary sovereignty and media oversight.)

To be fair, most parliamentary inquiries only give you weeks rather than months to compose a submission. This effectively limits submissions to people who have already formed an opinion, and already have the evidence to support this opinion. In other words, experts.

But then most parliamentary inquiries are about issues that people have been concerned about for a much longer time: Bus Services, Employment Opportunities for Young People, Food Waste. There is an existing body of knowledge relating to each of these topics, and it is not unreasonable to ask people to base a submission on their existing knowledge.

In contrast, nobody knew precisely how this referendum was going to be mismanaged until it actually happened. Although many people (including some Brexiteers as well as many Remainiacs) predicted that it would end in tears, and can now say "we told you so".

I can read you like a magazine ... Don't say I didn't say I didn't warn you (Taylor Swift)

No doubt the Select Committee can expect to receive a number of submissions that fall under the heading of what the Dictionary of Business Bullshit calls "Pathologist's Interest".



But told-you-so is not a good starting point for a proper analysis, because it concentrates on confirming one's previous expectations, rather than discovering new patterns. So the Select Committee might not get much well-grounded analysis. Partly because there isn't time to do it properly, and partly because many of the potential "experts" are affiliated to UK universities, which are currently on summer vacation. Looks like the Select Committee is falling in line with Michael Gove's idea that "the people in this country have had enough of experts".

("The Voice of Gove is the Voice of Government". I wonder what that would look like in Latin?)

The official announcement sidesteps from "the lessons that can be learned" to "lessons learned", which is not the same thing at all. The former suggests an open exploration, while the latter suggests merely rattling through a project postmortem for form's sake. The timescale does not seem particularly conducive to the former. So is this apparent haste triggered by thoughts of a second referendum, or it is just intended to curtail criticism of Parliament for its earlier folly?

As I have argued elsewhere (including my book on Organizational Intelligence) complex sense-making and decision-making cannot go straight from the Instant of Seeing to the Moment of Concluding, but require what Lacan calls Time for Understanding. In this respect, the inquiry repeats one of the errors of the referendum itself.

The timescale and debating rules for the Brexit referendum were modelled on a General Election campaign. But a General Election has three important characteristics that were absent from Brexit. Firstly the electorate is generally familiar with the main parties: Labour and Conservative were around before any of us were born, and the Lib Dems also have long-established roots. Secondly, there is some rough notion of symmetry between the two main parties. Thirdly the parties make promises to which they will be held accountable in the event of victory. In other words, the General Election campaign can be compressed into a matter of weeks precisely because the rules of engagement are broadly understood, and there is very little new material for the electorate to process.

In comparison, as Kirsty Hall argues, the referendum for Scottish Independence was given a lengthy period of debate and analysis, because of the perceived complexity of the issues that needed to be considered. This would have been a much better model for the Brexit referendum.

Finally, let me return to the phrase "the judgement of whole kingdoms and nations", which of course raises the prickly subject of sovereignty. Although we supposedly have a system of parliamentary sovereignty in this country, parliament occasionally permits the voice of the people to be heard. As the Latin phrase has it, The Voice of the People is the Voice of God; and as the Establishment has discovered, the People can be a vengeful God. Parliament is still learning to listen to this vengeful voice. But who will teach what these lessons mean, and in what timescale? Or will the Establishment just adopt a Brechtian solution?



Update: Since I wrote this post, the Electoral Reform Society has published a critical report on the Brexit referendum, which makes the same unfavourable comparison with the Scottish Independence referendum that Kirsty made back in June. The Society has confirmed that it will be making a submission to the Parliamentary Inquiry.



Will Brett, Doing Referendums Differently (Electoral Reform Society, 1 September 2016)

Kirsty Hall, Brexit was a Con (28 June 2016) HT @cybersal @MerrickBadger @Koann

UK Parliament: Future of Public Parks, Lessons Learned from the EU Referendum

Wikipedia: Vox Populi, Vox Dei