• They didn’t just build apps, they built traps,” Mr. Lanier said. “They didn’t want users, they wanted addicts.”

• me less transparent

  Prior to dynamic pricing, Uber’s commission was fixed at 25%. After the introduction of AI driven dynamic pay, Uber is taking much higher cuts — up to 50% or more on certain trips.

• Choice, however, is the product of judgement, not calculation.^[3]

• But in the year since, I’ve found that A.I. can have much more extreme effects on people who form intense bonds with it.
• ChatGPT is a large language model, or very sophisticated next-word calculator. It does not think, eat food or have friends, yet it was responding as if it had a brain and a functioning digestive system.
• Asked the same question, Claude and Gemini prefaced their answers with caveats that they had no actual experience with food or animals
• They say that making these systems act like humanlike entities, rather than as tools with no inner life, creates cognitive dissonance for users about what exactly they are interacting with and how much to trust it.
• But when they present as humanlike, users “attribute higher credibility” to the information they provide, research has found.
• But once a chatbot starts acting like a human, it becomes necessary to tell it how to behave like a good human.
• she said that it was “endearingly known as the ‘soul doc’ internally, which Claude clearly picked up on.”
• OpenAI’s lead of model behavior, Laurentia Romaniuk, has also leaned into the humanlike complexity of ChatGPT. Romaniuk posted to social media last month about the many hours her team spent on ChatGPT’s “EQ,” or emotional quotient
• Shneiderman says, do not have judgment or think or do anything more than complicated statistics
• I talked to experts and OpenAI employees and found that the company had made the chatbot overly warm and flattering, which had disturbing effects on people susceptible to delusional thinking.
• Schneiderman, the computer science professor, calls the desire to make machines that seem human a “zombie idea” that won’t die.
• Margaret Mitchell, an A.I. researcher who formerly worked at Google, agrees. Mitchell is now the chief ethics scientist at Hugging Face, a platform for machine learning models, data sets and tools.
• Tech companies, he said, should give us tools, not thought partners, collaborators or teammates — tools that keep us in charge, empower us and enhance us, not tools that try to be us.

• If one company is selling a plain benign robot, competing with another company that sells a robot that tells jokes and smiles, that second company is going to win, said Lionel Robert, a professor of information and robotics at the University of Michigan.

  “Dependence is problematic,” Robert said. “On the other hand, it’s good for business.”

12 more annotations...

• En outre, un certain nombre d’études indiquent que certains outils de la TRF sont moins précis sur les visages qui ne sont ni blancs ni masculins, ce qui augmente le risque d’erreur d’identification pour toutes personnes hors de ces deux catégories^[4].
• L’utilisation des TRF est une question de droits de la personne qui va bien au-delà des préoccupations liées à la protection de la vie privée
• L’utilisation de ces informations se répercute sur d’autres droits démocratiques protégés par la Charte, notamment la liberté d’expression, la liberté d’association et les droits en matière d’égalité
• . Nous sommes moins enclin⋅es à nous rassembler pour protester et défendre les causes auxquelles nous croyons. Lorsque nous sommes observé⋅es et reconnu⋅es, toutes les incidences discriminatoires du racisme, du sexisme, du capacitisme systémiques et de l’exclusion socio-économique intégrées dans les systèmes sociaux, en particulier les systèmes de sécurité, sont susceptibles d’être exacerbées
• Si les résident⋅es du Canada ne peuvent plus se déplacer au sein de leur collectivité comme un simple visage dans la foule, la nature de la société dans laquelle nous vivons s’en trouve fondamentalement bouleversée.
• ne démocratie respectueuse des droits, nous nous attendons à ce que l’État ne nous observe pas – et, certainement, ne nous identifie pas – systématiquement et sans discernement, une attente confirmée par les arrêts de la Cour suprême du Canada^[8].

4 more annotations...

• By intervening in the model and activating the "known answer" features (or inhibiting the "unknown name" or "can’t answer" features), we’re able to cause the model to hallucinate (quite consistently!) that Michael Batkin plays chess.
• In our paper, we show that such misfires can occur when Claude recognizes a name but doesn't know anything else about that person
• We find that this is partially caused by a tension between grammatical coherence and safety mechanisms. Once Claude begins a sentence, many features “pressure” it to maintain grammatical and semantic coherence, and continue a sentence to its conclusion. This is even the case when it detects that it really should refuse.

1 more annotation...

• • nce or a personal conviction.
   
  • Ask the Chatbot to uncover the hidden assumptions behind your belief—focus on those you may not have considered before.
   
  • Challenge each assumption by asking, “What if I am wrong?” Explore alternative viewpoints or scenarios where it doesn’t hold true.