Amplification and Attenuation

@STS_News (Lee Vinsel) poses a couple of important questions on Twitter

• Have any of y'all seen any specific proposals for how to regulate Big Tech companies around mis/dis/harmful information?
• Are there any so-far best philosophical/ethical treatments for why platforms *should* shutdown groups like Qanon?

I think it makes sense to start with the WHY question before tackling the HOW. In my post on The Ethics of Diversion - Tobacco Example (September 2019), I listed four ethical principles that might be relevant to that case, and discussed the need to balance these appropriately.

• Minimum interference principle
• Utilitarianism
• Cautionary principle
• Conflict of interest
  

If the primary justification for shutting down certain groups is to reduce the circulation of ideas that society has agreed to classify as undesirable, for whatever reason, then the minimum interference principle urges us to consider alternative (less extreme) measures that might achieve almost as much suppression with less interference. For example, instead of banning these groups from a given platform, it might be almost as effective to allow these groups to post whatever they like, provided that these posts are not picked up and recommended to others by the platform algorithm. Indeed, it might be safer to keep these groups on a well-controlled platform, rather than push them to migrate onto a less well-controlled platform.

In other words, in order to justify total shutdown, you have to demonstrate that this will produce significantly better outcomes than lighter regulation would.

However, in the case of the social media platforms, this is complicated by conflict of interest. The platform algorithms are designed to promote the commercial interests of the platform, and this means maximizing the engagement (or even addiction) of the so-called users. So society may not feel it can trust the platforms to manage the recommendation algorithms in a way that achieves the outcomes that society wants (and avoids the outcomes that society wants to prevent). And the algorithms themselves are generally not amenable to independent inspection, being incredibly complex and wrapped in commercial secrecy.

Meanwhile, not everyone is comfortable with the idea that ethical decisions should depend solely on the consequences of this or that action. If there is something clearly objectionable in the content that a given group wishes to post, or if we can demonstrate some evil intention behind it, surely we should be allowed to object to the content itself, not merely suppress its consequences? It may also be difficult to determine these consequences with any degree of accuracy, either in advance or retrospectively. So in addition to the consequentialist arguments we have already looked at, moral philosophers have identified a different class of argument, which they call deontological.

This includes arguments based on the intrinsic value of truth, and the idea that platforms should suppress material simply because it contains certain kinds of falsehood. However, I think this idea is insufficient as a basis for regulation, as I argue in my post on Ethical Communication in a Digital Age. 

There's obviously a lot more I could say about the WHY, but let me say a few words about the HOW. One way of understanding regulation and control comes from cybernetics. As we see from the work of Stafford Beer, for example, the behaviour of complex systems can be controlled by a combination of amplification and attenuation. In the world of social media, the critical question is which messages are selected for amplification.

In her article on Amplified Propaganda, Renée DiResta suggests that traditional top-down propaganda (which Chomsky called the Manufacture of Consent) has been replaced by the bottom-up creation and selective amplification of narratives that shape reality. She calls this Ampliganda.

In many cases, these narratives appear to emerge spontaneously from the crowd - this is sometimes called going viral. However, there are increasingly sophisticated ways of engineering and orchestrating these narratives, not just with bots but by convincing real people to participate in their propagation.

Meanwhile, if some things are being amplified, other things are being suppressed or ignored. In most cases, this is not the result of anyone actively deciding to suppress them, more a consequence of the fact that there is a finite quantity of attention. If a book or film or other cultural product receives poor reviews, or perhaps nobody even bothers to review it at all, then it is unlikely to become a best-seller. Similarly, the majority of the content on the internet will be read by at most a handful of people. It doesn't make sense to accuse the platform of exercising censorship, or to complain of being cancelled, simply because something fails to achieve a mass audience.

Furthermore, there are some structural reasons why certain types of content fails to attract an audience. In politics, there are limits to the kinds of idea that people are willing to take seriously - this is known as the Overton window. (However, this can change over time, for reasons that I can't go into here.)

But while the majority of people are comfortably inside the Overton window, there may be a significant minority who are attracted to groups positioned well outside the Overton window. So instead of asking how we should regulate the use of Big Tech by these groups (or some might say the cynical exploitation of these groups by Big Tech), maybe there are some larger questions about the regulation and/or liberation of discourse inside and outside the Overton window.

---

Renée DiResta, It’s Not Misinformation. It’s Amplified Propaganda (Atlantic, 9 October 2021)

Stanford Encyclopedia of Philosophy: Consequentialism, Deontological Ethics

Wikipedia: Overton Window

Related posts: Ethical Communication in a Digital Age (November 2018), Culture War - What Is It Good For? (July 2021)

The Aim of Human Society

To what extent is the aim of human society to maintain its equilibrium, as the POSIWID principle would suggest? There is a line of French thinkers who resist the universalism to which some schools of cybernetics aspire, and see the construction of social norms as political rather than teleological or quasi-biological. Xavier Guchet traces the position of Canguilhem and Simondon back to Bergson’s The Two Sources of Morality and Religion

where it is argued that closed morality and religion are without a doubt morality and religion of conservation. Their function is to preserve the stability of the existing social order. On the contrary, open moralities and religions have the function of returning human societies to the élan of creation, of unmaking the existing social order, inventing another order and becoming something else. Guchet 2012

Whereas for Foucault and others, social norms are constructed to protect society from pathological variations that might threaten it, Simondon focused on invention and the creation of new norms.

(I am more familiar with Popper's notion of Open Society than with Bergson's. There appears to be some difference between the two notions, but I haven't done enough reading at this point to be able to explain the difference.) 

One way of talking about these questions is in terms of programming. Simon Mills quotes from a book by James Beniger, distinguishing between control (purposive influence toward a predetermined goal) and programming (setting of the goal to be achieved). Mills carries out a close reading of one technology advocate (Sandy Pentland), showing that the success stories of big data are largely based on relatively closed or autopoietic systems, delivering some degree of technocratic efficiency and resilience, but failing to answer the more fundamental question - what is the purpose of society as a whole. Where do the goals come from?

Another way of talking about these questions is in terms of organizational learning. Chris Argyris introduced the distinction between single-loop learning and double-loop learning, which very roughly corresponds to Beniger's distinction between control and programming. While single-loop learning uses simple feedback to improve the performance of a system relative to a fixed goal, double-loop learning allows for the modification of goals in the light of experience. Advanced technologies such as machine learning are not limited to single-loop learning, and may be able to do some limited double-loop learning, in suitably controlled environments. (To go beyond this, we may need some notion of triple-loop learning. But see article by Tosey Visser and Saunders problematizing such labels.)

Sometimes, technologies and sociotechnical innovations are spoken of as ethically and politically neutral instruments, which can simply be used to maintain established socioeconomic and cultural goals. So that falls naturally into the "closed" model of society identified by Bergson. But if technologies and innovations (sometimes the same ones) are described as disruptive, this seems to imply a more "open" model of society.

Obviously there are ethical issues both ways - whom does the disruption serve, but also whom does the preservation of the status quo serve?

---

Henri Bergson, The Two Sources of Morality and Religion (1932)

Geoffrey Bowker, How to be universal: some cybernetic strategies, 1943-1970 (Social Studies of Science 23, 1993) pp 107-127 

Philip Boxer, Triple-Loop Learning (Asymmetric Leadership, 8 January 2007)

Xavier Guchet, Technology, Sociology, Humanism: Simondon and the Problem of the Human Sciences (SubStance #129, Vol. 41, no. 3, 2012)

Simon Mills, Simondon and Big Data (Platform Journal of Media and Communication, Vol 6, 2015) 59-72. 

Alex "Sandy" Pentland, Reinventing Society in the Wake of Big Data (30 August 2012). Professor Pentland is also mentioned in John Thornhill, Trustworthy data will transform the world (FT, 5 March 2018, paywall)

Karl Popper, The Open Society and its Enemies (Routledge 1945)

Paul Tosey, Max Visser and Mark NK Saunders, The origins and conceptualizations of triple-loop learning: A critical review (Management Learning 2012 43: 291 originally published online 2 December 2011)

Stanford Encyclopedia of Philosophy: A cybernetic view of human nature, Henri Bergson, Karl Popper

Related posts: Three wishes (May 2009), A cybernetics view of data-driven (August 2020)

Why This Stupid Behaviour?

@NateSilver538 argues that the simple explanation for the US president's outbursts (that he has poor impulse control and/or is bigoted) is (sometimes, usually) right.

The alternative explanation generally references some positive outcome for Trump. Silver mentions a few popular theories.

• shoring up his base
• questioning bias and fairness
• driving a wedge between the Trumpian and the Republican establishment
• distracting the media from other, more serious issues 

Silver's argument is based on the fact that some of his outbursts don't appear to produce the desired result. But there are some further considerations to bear in mind.

Firstly, when Trump appears to do something stupid or selfish, this prompts a barrage of criticism from various quarters. Some voices are consistently anti-Trump, while others (including conservative media channels, members of the Republican establishment and his own administration) will firmly distance themselves from his more outrageous pronouncements. This helps to reassure Trump's base that he remains an anti-establishment champion and is not getting swamped by Washington.

Thus the desired effects may follow from the response to Trump rather than directly from Trump's own words and deeds. This suggests a delay before the effect is visible in Silver's data. This leads to my second point: there are always some effects cannot be reliably detected in real-time, and there may be some effects will never be detectable to Silver. That doesn't mean that the effects aren't real. Just because Silver can't detect a strategy doesn't mean there isn't a strategy. Maybe Trump isn't one step ahead of the media, maybe he's three steps ahead.

Thirdly, what matters is not the effect of a single outburst, or even a series of outbursts on a single topic, but the cumulative effect. Some say that being inconsistent, volatile, unpredictable is part of his shtick.

Trump’s two most recent tweets:
1) Everybody’s against me, I’m a victim
2) Everybody loves me, I’m doing a great job.

— David Frum (@davidfrum) September 27, 2017

So what is the explanation for this inconsistency? Psychologists regard arbitrary and unpredictable inconsistency as a sign of emotional abuse, while mathematician Cathy O'Neil observes the similarity between Trump's behaviour and a machine learning algorithm.

Even if Silver is right about the intent and motivation of Trump's behaviour, that doesn't fully explain it. Just dismissing Trump as stupid or bigoted is not a sufficient explanation, because there are many stupid and bigoted people who do not behave quite like Trump. What is special in Trump's case is that there are some feedback loops that strongly reinforce these particular behaviour patterns, because they have produced the desired outcomes in the recent past.

Trump's worldview (Weltanschauung) causes him to pick up certain signals and ignore others. In terms of second-order cybernetics, we can view Trump as an autopoietic system (Maturana, Varela), within which the outcome-based theory and the impulse-base theory are not mutually incompatible after all, but are connected via closed feedback loops.

---

Cathy O'Neil, Donald Trump's Path-Independent Theory of Mind (Bloomberg, 21 May 2017)

Nate Silver, The Media Needs To Stop Rationalizing President Trump’s Behavior (FiveThirtyEight, 30 September 2017)

Wikipedia: Autopoiesis, Psychological Abuse

Related post: Dark Data and the US Election (November 2020) 

All Chewed Over By Machines

#AWOBMOLG Have been watching the first part of the latest Adam Curtis documentary "All Watched Over By Machines of Loving Grace". @socialtechno reckons "It's like a man writing a love letter to someone he doesn't love."

The documentary is named after a rather soppy 1950 poem by Richard Brautigan, and opens with Ayn Rand. Curtis would like us to believe that everyone in Silicon Valley was inspired and influenced by Ayn Rand (based on the fact that a few people named their children and companies after herself or her works) and he uses the life and works of Ayn Rand to frame a powerful but logically flawed dialectic about technological capitalism.

Thesis,

"Ever since the 1970s, computer utopians in California believed that if human beings were linked by webs of computers, then together they could create their own kind of order. It was a cybernetic dream, which said that the feedback of information between all the individuals connected as nodes in the network would work to create a self-stabilizing system. The world would be stable, yet everyone would be heroic Randian beings completely free to follow their desires."

This conflates a diverse range of beliefs and theories that were circulating all around the world, certainly not just in California. Those developing these ideas were far more likely to have been inspired and influenced by Karl Marx than by Ayn Rand. (Marx wrote of a world in which "the free self-development of each would be the condition of the free self-development of all"). Or perhaps Hannah Arendt, Norman O Brown, Paul Goodman, Ivan Illich, Herbert Marcuse, Jean-Paul Sartre or any number of other postwar European and American intellectuals. Meanwhile, cybernetics was emerging from the work of mathematicians and systems thinkers - including Ross Ashby, Stafford Beer, Jay Forrester, Donella Meadows and Norbert Wiener - who posed a radical critique to the managerial philosophy of Taylorism and Fordism.

Curtis refers to this collection of ideas as "The Californian Ideology". According to Wikipedia, this term was coined by Richard Barbrook and Andy Cameron, who wrote an essay with this title in 1995 as a critique of West Coast cyber-libertarianism. The essay doesn't mention Ayn Rand, but it does mention some of the writers who appear in Curtis's film, including Stewart Brand, Kevin Kelly, Peter Schwartz and Alvin Toffler. By stringing together short clips from his interviews with these and other worthies, Curtis creates the impression that their opinions can all be lumped together into a common belief system, which he can then attack in the next part of the film.

Antithesis

Meanwhile, one of the most influential followers of Ayn Rand was Alan Greenspan, for many years the Chairman of the US Federal Reserve. Along with other members of the financial elite, Greenspan persuaded President Clinton to abdicate control of the financial sector, and leave financial stability and prosperity to the markets. In some ways this new market libertarianism was a continuation of the economic monetarism that prevailed in the 1980s under Reagan and Thatcher. During the 1990s, however, people started to tell themselves a story in which the new technology somehow replaced the old economy (boom and bust) with a new economy based on unshakable stability, perpetual productivity growth and a transfer of power from governments to the people. This story (which we now know to have been wishful thinking) echoed some elements of the Californian ideology, but it was overlaid with all sorts of other political agendas.

Greenspan himself observed that there was a puzzling mismatch between corporate profits and genuine productivity growth. If the global economy had been a viable system in the cybernetic sense, designed or emerging according to the principles of Stafford Beer and others, this kind of information would be properly shared and interpreted, and would have had significant regulatory force.

Greenspan's original interpretation of this mismatch was that there was a worm in the apple of new market liberalism: he was soon persuaded to abandon this interpretation in favour of a much more optimistic one. As Curtis shows, using the IMF's intervention into Indonesia as an example, the financial elites leveraged the market system to protect their own interests, even when this ran counter to any notion of stability or general prosperity or natural justice. Ayn Rand would probably have called such behaviour "rational". Curtis gives us a lot of background about Rand's unbalanced sex life, in order to illustrate the mutually destructive nature of Rand's selfish notion of "rationality". He perhaps intended this section of the film as an allegory for the systemic side-effects of the Myth of the Machine.

Greenspan is portrayed in the film as an anti-hero, whose indecision and folly led an unsuspecting world (along with a compliant or distracted President Clinton) into disaster. Curtis also uses the Monica Lewinsky story, a tiny amount of familiar stock footage stretched by slow motion effects, to reinforce the train-crash element of the Clinton presidency. (Although it's difficult to see how Curtis can blame Californian hippies for all that. Might just as easily blame the Cavendish Laboratory.) But this is a fairly conventional version of recent history, which doesn't seem to tell us very much about our changing relationship with technology.

Synthesis

Curtis portrays Clinton as an important pivotal figure in this story, brought down by a series of tragic conflicts. Being as arrogant and selfish as Ayn Rand in his sexual behaviour - but being forced into concealment and deception by his political position. Adoring the old-style democratic politics (says Curtis), but being lured by the evil Greenspan-Iago into the new anti-politics. (Clinton as the glamorous Othello, obviously, who remains handsome even when picking his teeth.)

There is a heroic way of narrating history that concentrates on leaders and their personal strengths and weaknesses. Curtis's previous film, The Power of Nightmares, told a fascinating tale of the leaders of the American Right and the leaders of al Qaeda, and revealed intriguing links and parallels between Bush and Bin Laden.

There is also an "objective" way of narrating history, that downplays leadership and concentrates on the deeper system forces that shape events. George Orwell noted the tendency of Marxists to overuse words like "objectively", but surely nobody can doubt that political activity can sometimes have unanticipated or even counterproductive effects. Systems thinkers including Stafford Beer and Maturana remind us that complex sociotechnical systems may sometimes have a life of their own, preserving their essential characteristics regardless of the espoused intentions and best efforts of the people who are supposed to be in charge. (POSIWID was Beer's name for this effect.) Economists appeal to the "invisible hand", which supposedly creates beneficial outcomes without conscious planning or top-down governance. (By the way, the extreme form of economic liberalism espoused by Greenspan was popular in the 19th century, so it's a bit misleading to credit Ayn Rand with inventing this idea.) And narratives about the power of The Machine tend to belong here.

I've got better things to do with my time than work out how Ayn Rand's version of "objectivism" attempts to reconcile the notion of personal heroism (while steering away from Nietzsche's version) with the notion of system forces (while steering away from Marx's version). But if Adam Curtis believes he can produce a historical account that personalizes how we are watched over by machines, without dealing with the problems introduced by an array of German intellectuals from Marx to Nietzsche, he may be as much a captive of the Ayn Rand camp as the people featured in his film. (See what I did there?)

Notes

Richard Brautigan, "All Watched Over By Machines Of Loving Grace" (via American Poems). Brautigan was briefly a poet-in-residence at CalTech, so he must have influenced loads of people in Silicon Valley mustn't he? Jarvis Cocker calls him "A Hemingway for Hippies" (Guardian 17 September 2014). See also Alexis Madrigal, Weekend Poem (Atlantic, September 2011).

Richard Barbrook and Andy Cameron, The Californian Ideology (1996)
Terry Eagleton, In Praise of Marx (Chronicle Review, April 2011)
Kevin Kelly, Out of Control (1994)
Montserrat Tovar, Pandora’s Vox: On Community in Cyberspace (1994)

Previews and reviews of "All Watched Over by Machines of Loving Grace"

• Adam Curtis (especially comments by Leeravitz and leftnotliberal)
• Chris Applegate (24 May 2011)
• Brainfisch (26 May 2011)
• Andrew Orlowski (The Register 23 May 2011)
• Ashwin Parameswaran (24 May 2011)
• Paul Rennie (24 May 2011)
• Alasdair Thompson (Bright Green 26 May 2011) (via OrgZone 25 May 2011)
• Sam Wollaston (Guardian 23 May 2011)
• Saturday Review (BBC Radio 4, 21 May 2011) 
• Wikipedia

More on Ayn Rand

• Howard Gold, Did Greenspan channel or betray Ayn Rand (MarketWatch June 2010)
• Noah Kristula-Green, Tea Party Embraces Ayn Rand (FrumForum, July 2010) via Jonathan Chait (New Republic)
• Evan McMurray, The Stunning Weirdness of Ayn Rand (Salon, 6 October 2014)
  
• Amity Shlaes, Rand’s Atlas Is Shrugging With a Growing Load (Bloomberg July 2009)
  
• Vladimir Shlapentokh, The Marxist and Bolshevik Roots of Ayn Rand’s Philosophy (Aug 2010)
• Jenny Turner, As Astonishing as Elvis (LRB Vol 27 No 23, 1 Dec 2005) with reply by Chris Sciabarra (6 Dec 2005)
• Slavoj Zizek, The actuality of Ayn Rand pdf, (Journal of Ayn Rand Studies Vol 3 No 2, Spring 2002)

More on Alan Greenspan

• Bernard Stiegler, Automatic Society 1: The Future of Work - Introduction (trans Daniel Ross, via La Deleuziana 2015)
  

More on Adam Curtis

• Brian Holmes, NEOLIB GOES NEOCON: Adam Curtis, or Cultural Critique in the 21st Century (June 2007)
• Adam Curtis: Cognitive Deterrorization (Politics of Knowledge, Feb 2008)
• Adam Curtis Alarm Clock Films (Diagonal Thoughts, March 2008)

Related posts

• The Ecosystem Myth (May 2011)
  
• Witnessing Machines Built in Secret (November 2017)
  

 

Updated 17 September 2014. Links added 19 August 2020.

Tribal Identity

The point John makes in his post on Atavistic Honour is essentially a question of viability versus identity. This is a key question in cybernetics, which was explored in depth by Maturana and Varela.

The primary purpose (POSIWID) of closed systems is to maintain their identity, and to resist all challenges to this identity. However, in complex dynamic environemnts, viability often requires responding creatively to change. Identity is therefore often in conflict with viability. Both the Labour Party in the 1980s and the Conservative Party today are clinging to some identity, which makes them non-viable in electoral terms. Tribes certainly have this quality as well (and that's why tribes struggle to survive in complex dynamic environments). However, the preservation of identity (at the expense of viability) and the neutralization of innovation can be found in lots of other systems as well.

---

Humberto Maturana and Francisco Varela, Autopoiesis and cognition: the realization of the living (Dordrecht: Reidel 1972) 

Humberto Maturana and Francisco Varela, The Tree of Knowledge: The Biological Roots of Human Understanding (Shambhala 1992)

Deming SIG Meeting

VSM event at IQA, Hyde Park Corner, London. 14 July, 2005.

Speakers include Patrick Hoverstadt, Trevor Hilder and Tony Gill. Trevor will be using POSIWID to describe a project using ICT to improve the business processes of a company which manufactures enzymes for use in life-science laboratories worldwide. His focus will be on recounting the history of this 'cybernetic intervention' and what it can tell us about the purposes being pursued by the management of the company, as compared to what the management say about the subject. Trevor is a strong advocate of the work of Stafford Beer, and his paper Viability versus Tribalism was published in the special edition of Kybernetes, dedicated to Stafford's work (Volume 33 Number 3/4 2004).

For booking details, see IQA website. Technorati Tags: Deming POSIWID Stafford Beer VSM