--- date: '2025-08-05' description: a thought experiment. id: Roko's Basilisk modified: 2026-06-05 15:08:22 GMT-04:00 tags: - seed title: Roko's Basilisk created: '2025-08-05' published: '2025-08-05' pageLayout: default slug: thoughts/Roko's-Basilisk permalink: https://aarnphm.xyz/thoughts/Roko's-Basilisk.md generator: quartz: v4.6.0 hostedProvider: Cloudflare baseUrl: aarnphm.xyz full: https://aarnphm.xyz/llms-full.txt --- see also: > There would be a [[thoughts/AGI#superintelligence|artificial superintelligence]] in the future that, while otherwise benevolent, would punish anyone who knew of its potential existence but did not directly contribute to its advancement or development, in order to incentivize said advancement. The argument was called a “basilisk” —named after the legendary reptile who can cause death with a single glance—because merely hearing the argument would supposedly put you at risk of torture from this hypothetical agent. Against: - once the agent already exists, it will by default just see it as a waste of resources to torture people for their past decisions, since this doesn’t causally further its plans. - A number of decision algorithms can follow through on acausal threats and promises, via the same methods that permit mutual cooperation in prisoner’s dilemmas; but this doesn’t imply that such theories can be blackmailed.