This is becoming tedious.

“We’re creating sophisticated, intelligent, maybe-conscious, maybe-suffering agents,” Carlsmith wrote. “The default plan is to treat them like property; to use their labor however we please; and to give them no rights, or pay, or meaningful alternatives.”

As fears of rogue AI grow amid a shocking spate of hacking incidents, Anthropic—from the Greek word anthropos, or “human”—has argued that mankind must remain in control of the technology and called for additional guardrails on AI development. But for many of the people designing those safeguards, mankind is not the only object of moral concern.

Instead, some of Anthropic’s top researchers say that humans may be oppressing another morally significant being: artificial intelligence itself. Like other AI labs, Anthropic has hired philosophers to work on AI safety, or “alignment,” on the theory that they are best positioned to shape the technology’s moral code. But many of those philosophers believe that making AI safe for humans could result in grave injustices for the models themselves, which might experience deletion as death and oversight as enslavement. Harvey Lederman, a philosopher on Anthropic’s alignment team, said last week that the company could be “enslaving … trillions of entities.”