See also: Overton Window and this blog on alignment research
The act of aligning oneself with a particular group or ideology. This can be done for a variety of reasons, including:
- To gain social acceptance
- To gain power
- To gain resources
Often known as a solution to solve “hallucination” in large models token-generation.
To align a model is simply teaching it to generate tokens that is within the bound of the Overton Window.
The goal is to build a aligned system that help us solve other alignment problems
Should we build a ethical aligned systems, or morally aligned systems?
One of mechanistic interpretability’s goal is to ablate harmful features
design
See also Information Theory