Showing posts with label algorithm. Show all posts
Showing posts with label algorithm. Show all posts

Friday, October 03, 2025

On Censorship

At City St George's University this week for a discussion on Truth, Trust and Tricksters in the Age of AI, organized by Index on Censorship, the Institute for Creativity and AI and Global (Dis)Order.

One of the panelists, Kenneth Cukier of the Economist, recounted his attempt to get an AI tool to produce an image to illustrate a point about silo-busting. The tool refused on the grounds that this would be an image of destruction. Cukier regarded this as an example of censorship - the tool was denying his ability to express his ideas freely.

As I suggested in the subsequent discussion, we might also regard this as the tool censoring itself. Self-censorship has been a feature of many authoritarian regimes in the past, and is perhaps emerging in new forms today, especially as people may fear the consequences of their words being taken out of context and blasted around the Internet.

One of the features of agentic AI is presenting the AI device as an autonomous agent. It may appear to be working for Crukier, but it is ultimately controlled by whichever big tech assemblage has developed and disseminated it. Which brings me to a critically important question I have asked several times previously - Whom Does The Chatbot Serve? (Towards Chatbot Ethics, May 2019)

In any case, all communication and creation is necessarily selective. However many images the AI tool does in fact produce, there is a much larger quantity of possible images it has decided or been programmed not to produce. There must be huge amounts of material that the editors of the Economist choose not to publish, for whatever reason, and it would be absurd to frame all these editorial decisions as forms of censorship or self-censorship (although of course some of them may well be).

However, attempts to reduce the quantity and reach of misleading content will often be framed as censorship by those promoting such content, as Jacob Weisberg's latest article demonstrates. There are some difficult issues here, and it is sometimes hard to avoid taking a political side.

 

See also Thinking with the Majority (May 2021), Amplification and Attenuation (October 2021)

Jacob Weisberg, Algorithm Nation (New York Review of Books, 23 October 2025) 

Wednesday, November 22, 2023

Algorithmic Intuition - Gaydar

When my friend A was still going out with women, other friends would sometimes ask if he was gay. An intuitive ability to guess the sexuality of other people is known as gaydar. There have been studies that appear to provide evidence that both humans and computers possess such an ability, although the reliability of this evidence has been challenged. For example, some of these studies have relied on images posted on dating sites, but images that have been crafted and selected for dating purposes may already reflect how a person of a given sexuality wishes to present thenselves in that specific context, and may not reflect how the person looks in other contexts.

The latest study claims to assess sexuality from brain waves. This has been criticized as gross and irresponsible (Rae Walker) and as unscientific (Ababa Birhane). Continuing a debate that had started with other methods of algorithmic gaydar.

More generally, there is considerable disquiet about computers attempting to segment people in this way. For a start, there are many parts of the world where homosexuality doesn't only lead to social disapproval and harassment, but also criminal penalties and sanctions. Even though the algorithms may be inaccurate, they might be used to discriminate against people, or trigger homophobic actions. Whether someone actually is gay or is a false positive is almost beside the point here, either way the algorithmic gaydar may result in individual suffering.

Furthermore, these algorithm appears to want to colonize aspects of subjectivity, of the subject's identity.

  • WyssBernard: I’m not going accept a machine determination as to what I identify as. ?¿
  • Abeba Birhane: just let people be or let people identify their own sexuality

In an interview with the editor of Wired, Yuval Noah Harari wonders whether an algorithm might have guessed he was gay before he realised it himself. And if an algorithm had been the source of this wisdom about himself, would this not have been incredibly deflating for the ego?

And Lawrence Scott describes how his Facebook timeline started to be invaded by images of attractive men, suggesting that the algorithm had somehow profiled him as being particularly susceptible to these images.


to be continued




Isobel Cockerell, Facial recognition systems decide your gender for you. Activists say it needs to stop (Codastory, 12 April 2021)

Isobel Cockerell, Researchers say their AI can detect sexuality. Critics say it’s dangerous (Codastory, 13 July 2023)

Lawrence Scott, Hell is Ourselves (The New Atlantis #68, Spring 2022, pp. 65-72)

Nicholas Thompson, When Tech Knows You Better Than You Know Yourself (Wired, 4 October 2018)

Wikipedia: Gaydar

Thursday, June 09, 2022

Progress Bar

There are algorithms whose primary purpose appears to be to generate affect - for example, to reduce the anxiety of those waiting. For example, the progress bar that is displayed when something is loading or downloading. Other examples include indicators at bus stops, on railway platforms or next to lifts, showing either the current location or the expected time of arrival.

Sometimes these indicators misbehave. The progress bar suddenly jumps from 60% to 90% and then gets stuck. One moment the bus is five minutes away, the next moment it is seven minutes away. These glitches reveal that the indicators are not unmediated truth but fictions functioning as truth.

The algorithm of the progress bar depends not only on the code generating it but the cultural calculus of waiting itself, on a user seeking feedback from the system, and on the opportunity - increasingly capitalized on - to show the user other messages, entertainments or advertising during the waiting phase. Finn p34

Jason Farman compares these indicators with earlier symbols, such as the spinning cursor, which provided no such feedback.

These symbols keep us from seeing how the system is actually working; we’re not given a behind-the-scenes view of how the process is actually progressing, so we are kept at arm’s length, spinning or twiddling our thumbs as we wait. Farman
In other contexts, such as digital games, progress bars are designed to motivate the players.

Feedback is a system that tells players how close they are to achieving a goal and can come from points, levels, a score or a progress bar; this provides motivation to keep playing. Pulos quoting McGonigal, 2011, p. 21
And if the progress bar gets stuck on 99%, what then? In her PhD thesis, Kate Starbird discusses a meme that circulated on Twitter during the 2011 political uprising in Egypt, with progress bars showing variations of installing freedom and uninstalling dictator, in some cases linked to messages encouraging patience and/or persistence.

As things turned out, President Mubarak was uninstalled, but many of the protesters were unhappy with subsequent events, and there was a further uprising in 2013. So how much progress has Egypt made in installing freedom and democracy? Unfortunately, democracy isn't something that was uploaded to the cloud by the ancient Greeks, just waiting to be downloaded into any country with sufficient memory.


Jason Farman, Delayed Response (Yale University Press 2018). Extract: Spinning in Place

Ed Finn, What Algorithms Want (MIT Press 2017)

McGonigal, J. (2011). Reality is broken: Why games make us better and how they can change the world. New York, NY: Penguin Press

Alexis Pulos, FARMING AND FIGHTING AS PRACTICE AND PEDAGOGY: A PROCEDURAL FIELD ANALYSIS OF DIGITAL GAMES (University of New Mexico, PhD thesis 2013). https://digitalrepository.unm.edu/cj_etds/45

Kate Starbird, Crowdwork, Crisis and Convergence: How the Connected Crowd Organizes Information during Mass Disruption Events (Atlas Institute, PhD thesis 2012)

Wikipedia: Progress Bar, Progress Indicator, 2011 Egyptian revolution  

Related post: Business Rules are Forking (March 2010)

Saturday, April 09, 2022

The Purpose of Shame

Weapons expert @mathbabedotorg Cathy O'Neil is best known for her account of algorithms as weapons (of math destruction). Her new book tackles a related topic, the use of shame as a weapon. 

Many people have written about the psychology and sociology of shame. O'Neil's focus is on how shame is manufactured and mined, how organizations gain commercial and sociopolitical benefit from propagating shame, how individuals are coopted into circuits of shame, and (to quote her subtitle) who profits in the new age of humiliation.

One use of the shame machine is to persuade people to buy products and services. O'Neil describes her own experience being targetted with fat-shaming advertisements. Such advertisements are designed to make people feel ashamed, and to believe that the advertised product will somehow help. She also notes how shame can undermine a person's capacity for rational evaluation, and trigger impulsive actions. 

Shame is also used socially, to reinforce social norms. In some cases this is centrally planned - for example, in China where people are publicly shamed for acts that are officially disapproved, such as jaywalking. In other cases, this can be the result of what O'Neil calls Networked Shame, where people feel empowered to shame strangers, supposedly for their own good. As if pointing out the health risks of obesity to a fat person is somehow being kind and helpful to them.

Immediately following the 2020 US presidential election, Judith Butler noted how Trump and his supporters saw the left as a shame machine.

Shame occupied a permanent and necessary place in the Trumpian scenario insofar as it was externalized and lodged in the left: the left seek to shame you for your guns, your racism, your sexual assault, your xenophobia! The excited fantasy of his supporters was that, with Trump, shame could be overcome.

Of course shame was not really overcome, it was merely redirected onto others, using a version of the Shame Machine that Geoff Shullenberger calls the Scapegoating Machine, tracing back to René Girard. (O'Neil also references Girard.)

So shaming the Other becomes a political tool. Making people feel ashamed that they need help is a lot cheaper and more convenient than actually helping them, so politicians make unfortunate circumstances shameful (addiction, homelessness, single parenthood, etc) as a way of signalling that people in such circumstances don't deserve our help. And by creating a sense of Us and Them, it reinforces loyalty to populist politicians. Shullenberger credits Peter Thiel, a former student of Girard, for helping to plan Trump's successful 2016 campaign, and notes that like the social media platforms on which it has thrived, Trumpism channels violence mainly toward victims it wishes to marginalize.

While there is nothing new about public humiliation and scapegoating, the Internet and social media clearly provide new affordance to those wishing to shame others. Is that merely an unfortunate side-effect of an otherwise beneficial and beneficent technology? Not surprisingly, O'Neil doesn't think so.

Digital titans, led by Facebook and Google, not only profit from shame events but are engineered to exploit and diffuse them. In their massive research labs, mathematicians work closely with psychologists and anthropologists, using our behavioral data to train their machines. Their objective is to spur customer participation and to mine advertising gold. When it comes to this type of intense engagement, shame is one of the most potent motivators. ... It spurs traffic and boosts revenue.

One possible remedy, suggests O'Neil, is to redirect shame back towards the powerful, or what she calls punching up. She notes how Google could itself be shamed, for example in relation to its treatment of Timnit Gebru, and notes at least the possibility of what she calls healthy shame. She ends, not with a plan to end all shame, but with some recommendations for detoxifying shame.



Judith Butler, Is the show finally over for Donald Trump? (The Guardian, 5 November 2020)

Cathy O'Neil, Weapons of Math Destruction (New York: Crown, 2016)

Cathy O'Neil, The Shame Machine (New York: Crown, 2022)

Geoff Shullenberger, Mimesis, Violence, and Facebook: Peter Thiel’s French Connection (Cyborgology, 13 August 2016) The Scapegoating Machine (The New Inquiry, 30 November 2016)


Related posts: Weapons of Math Destruction (October 2016), Ethical Communication in a Digital Age (November 2018), Dark Data and the US Election (November 2020)

Sunday, October 31, 2021

Contagion

The mathematician and broadcaster Hannah Fry did some post-doctoral research modelling social disturbance and crime in terms of contagion. In 2014, she gave a presentation at a conference in Berlin about her work with the police attempting to model the patterns of the 2011 London riots. As she said later, it didn't occur to her that a Berlin audience might have a different perspective on police power than a British audience would, and she got (in her words) absolutely torn apart. 2018 video 11:20

Several members of the audience expressed concerns about handing over too much control to the police, not only in giving them the power to suppress different forms of disturbance, but also in biasing the data on which the mathematical models were based. One person asked whether the data could really represent who the rioters were, referring to sociological research showing that police arrests are anything but neutral ... underprivileged groups of society tend to be arrested more. 2014 video 55:40

Another person noted how the riots depended not only on the behaviour of the rioters and the police, but also on the behaviour of the bystanders, which varied in different parts of London. If the Turkish community on London dealt robustly with the situation without relying on the police, this might be linked to the relationship between police and public in Turkey. In her response, Fry also noted the influence of the British media on the behaviour of bystanders in such situations.

While noting concerns about privacy, and agreeing that handing over too much control to technology is a really scary thing, Fry attempted to balance this against the claim that there is something positive to be gained by looking at the macro level behaviour of people in the way that we can design our society. 2014 video 52:50

In her more recent talks, Professor Fry has been more careful to put mathematical modelling into an ethical frame, as well as encouraging people to question the authority of the algorithm.

When it comes to algorithms, you can't just build them, put them on a shelf, and decide whether they're good or bad in isolation. You have to think about how they are actually going to be used by people. 2018 video 11:40

Once you dress something up as an algorithm or as a bit of artificial intelligence it can take on this air of authority that makes it really hard to argue with. 2018 video 26:10

Peter Polack provides a more fundamental challenge to the something positive claim. He traces the genealogy of this idea from August Comte's social physics to latter-day neorationalism, referencing Michel Foucault's notion of biopower and biopolitics.

Meanwhile, if social disorder appears to follow the same mathematical patterns as contagious disease, and the police are being invited to treat crime as a disease, perhaps it is not surprising when disease (or even the possibility of being infectious) starts to be treated as a crime.

The protective measures during the COVID pandemic include lock-down and self-isolation. So-called social distancing really means physical distancing, with as much social interaction as your technology (from phones to Internet) can provide. This is a lot easier for people with reasonably large houses, good internet connections, and devices for each member of the family, as well as the kinds of jobs that are relatively easy to do from home. For people in cramped housing, and for people who actually need to turn up at work if they want to get paid, self-isolation is a luxury they may not be able to afford. Therefore being tested for COVID may also be a luxury they can't afford.

Hannah Fry's mathematical model of the London riots identified that many of those arrested were from disadvantaged areas, although as we've seen this finding can be interpreted in more than one way. A model of disease might also show increased infection in disadvantaged areas. Maps of disadvantage and disease show strong persistence over time, as I discuss in my post on Location, Location, Location, quoting a study by Dr Douglas Noble and his colleagues.

But the COVID testing data are not going to show this pattern if people from disadvantaged areas are reluctant to come forward for testing. So much for biopower then.



Hannah Fry, I predict a riot (re:publica 2014, May 2014) recording via YouTube

Hannah Fry, Contagion: The BBC Four Pandemic (BBC March 2018) recording not currently available

Hannah Fry, Should Computers Run the World (Royal Institution, November 2018) recording via YouTube

Douglas Noble et al, Feasibility study of geospatial mapping of chronic disease risk to inform public health commissioning. BMJ Open 2012;2:e000711 doi:10.1136/bmjopen-2011-000711

Peter Polack, False Positivism (Real Life Mag, 18 October 2021) HT @jjn1

Stanford Encyclopedia of Philosophy: August Comte, Michel Foucault

Related posts: Location, Location, Location (February 2012), Algorithms and Governmentality (July 2019), Algorithmic Bias (March 2021)

Saturday, March 13, 2021

Algorithmic Bias

I have written several blogposts touching on various aspects of #AlgorithmicBias. This post summarizes my position, with links to further blogposts and other supporting material. 

 

1. Algorithmic bias exists. As Cathy O'Neil says, algorithms are opinions embedded in code.

 

2. Algorithmic bias can have negative consequences for individuals and society, and is therefore an ethical issue. Algorithms may be performative (producing what they claim to predict) and final (producing conclusions that are incapable of revision). Furthermore, because of its potential scaling effects, a biased algorithm may have much broader consequences than a biased human being. Hence the title of O'Neil's book, Weapons of Math Destruction.


3. Algorithms often have an aura of infallibility. People may be misled by an illusion of objectivity and omniscience.

 

4. There are different kinds of bias. People may disagree about the relative importance of these. This is one of the reasons why the notion of media balance and impartiality is problematic.


5. Attempts to eliminate one kind of bias may reinforce another kind of bias. It may not be possible to eliminate all kinds of bias simultaneously.

 

6. Attempts to address algorithmic bias may distract from other ethical issues. Furthermore, researchers may be pushed or pulled into solving algorithmic bias as a fascinating technical problem. Julia Powles calls this a seductive diversion.

 

Note: Points 4 and 5 are not exclusively about algorithmic bias, but apply also to other kinds of systemic bias.


Further discussion and links in the following posts. 

Reinforcing Stereotypes (May 2006). Early evidence of search engines embedding human bias.

Weapons of Math Destruction (October 2016). Links to book reviews, talks and other material relating to Cathy O'Neil's book.

Transparency of Algorithms (October 2016). Policy-makers are often willing to act as if algorithms are somehow above human error and human criticism. But when people are sent to prison based on an algorithm, or denied a job or health insurance, it seems reasonable to allow them to know what criteria these algorithmic decisions were based on. Reasonable but not necessarily easy. 

The Road Less Travelled - Whom does the algorithm serve? (June 2019). In general, an algorithm is juggling the interests of many different stakeholders, and we may assume that this is designed to optimize the commercial returns to the algorithm-makers.

The Game of Wits Between Technologists and Ethics Professors (June 2019). Technology companies fund ethics researchers to work on obscure philosophical problems and technical fixes.

Algorithms and Auditability (July 2019). Looking at proposed remedies to algorithmic bias.

Algorithms and Governmentality (July 2019). Looking at the use of algorithms to support bureaucratic biopower (Foucault). If the prime job of the bureaucrat is to compile lists that could be shuffled and compared (Latour), then this function is increasingly being taken over by the technologies of data and intelligence - notably algorithms and so-called big data.

Limitations of Machine Learning (July 2020). Problems resulting from using biased datasets to train machine learning algorithms.

Bias or Balance (March 2021). Discusses how different stakeholders within Facebook prioritize different kinds of bias, comparing this with approaches to impartiality and balance in other media organizations including the BBC.

Does the algorithm have the last word? (February 2022). Algorithms may be performative (producing what they claim to predict) and final (producing conclusions that are incapable of revision).  See also On the Performativity of Data (August 2021) and Can Predictions Create their Own Reality (August 2021).

 

 

Lots of references and links in the above posts. Here are some of the main sources.

Cathy O'Neil, How algorithms rule our working lives (Guardian, 1 Sept 2016)

Cathy O'Neil, Weapons of Math Destruction (Crown Books, 2016)

Julia Powles, The Seductive Diversion of ‘Solving’ Bias in Artificial Intelligence (7 December 2018)

Wikipedia: Algorithmic bias