--- date: '2024-01-11' description: in information theory id: Entropy modified: 2026-06-05 15:08:29 GMT-04:00 tags: - seed title: Entropy created: '2024-01-11' published: '2024-01-11' pageLayout: default slug: thoughts/Entropy permalink: https://aarnphm.xyz/thoughts/Entropy.md generator: quartz: v4.6.0 hostedProvider: Cloudflare baseUrl: aarnphm.xyz full: https://aarnphm.xyz/llms-full.txt ---

In particular, "good, aligned, conversational AI" is just one of many possible different rollouts. Finetuning / alignment tries to "collapse" and control the entropy to that region of the simulator. Jailbreak prompts try to knock the state into other logprob ravines.

idea: quantifies average level of uncertainty information associated with a variable’s potential states or possible outcome. > \[!definition\] Definition 1. > > Given a discrete random variable $X$, which takes a value in a set $\mathcal{X}$ distributed according to $p : \mathcal{X} \to [0,1]$, the entropy $H(X)$ is defined as > > $$ > H(X) \coloneqq - \sum_{x \in \mathcal{X}} p(x) \log p(x) > $$ Base 2 gives unit of “bits” (or “shannons”), while natural base gives “natural units” (or “nat”), and base 10 gives unit of “dits” (or “bans”, or “hartleys”) ## The Waluigi Effect [_source_](https://www.lesswrong.com/posts/D7PumeYTDPfBTp3i7/the-waluigi-effect-mega-post) talks more about the semiotic phenomena which arise within LLMs, and how hallucination come to be. ## joint _measure of uncertainty associated with a set of variables_ namely, the joint _Shannon entropy_, > \[!definition\] Definition 2. joint _Shannon entropy_ > > in bits, of two discrete random variable $X$ and $Y$ with images $\mathcal{X}$ and $\mathcal{Y}$ is defined as: > > $$ > H(X,Y) = - \sum_{x \in \mathcal{X}} \sum_{y \in \mathcal{Y}} P(x,y) \log_2 [P(x,y)] > $$ > > where $P(x,y)$ is the joint probability of both $X$ and $Y$ occurring together. For more than two random variables, this expands onto: $$ H(X_{1},\ldots,X_{n}) = - \sum_{x_{1} \in \mathcal{X}_{1}} \cdots \sum_{x_{n} \in \mathcal{X}_{n}} P(x_{1},\ldots,x_{n}) \log_2 [P(x_{1},\ldots,x_{n})] $$