--- abstract: large language models will probably be the most important piece of technology in the 21st century date: '2024-02-07' description: a mathematical framework for intelligence id: LLMs layout: technical modified: 2026-10-22 11:36:15 GMT-04:00 seealso: - '[[thoughts/mechanistic interpretability]]' - '[[thoughts/Alignment|Alignment]]' slides: true tags: - ml - llm - philosophy - technical - evergreen title: LLMs created: '2024-02-07' published: '2024-02-07' pageLayout: technical slug: thoughts/LLMs permalink: https://aarnphm.xyz/thoughts/LLMs.md generator: quartz: v4.6.0 hostedProvider: Cloudflare baseUrl: aarnphm.xyz full: https://aarnphm.xyz/llms-full.txt --- Large language models builds on top of [[thoughts/Autoregressive models|autoregressive]] [[thoughts/Transformers|transformers]] to predict the [[thoughts/Autoregressive models#next token prediction|next token]]. This is often because these models are trained to “hillclimb” this specific objective, and with the recent advancement in [[thoughts/Reinforcement learning|RL]] we’ve basically achieved _pseudo-intellectualism_ There are also a whole genre of language model built on top of diffusion blocks, where it learns to reconstruct masked tokens, then generates text through repeated {{sidenotes[unmasking]: diffusion is an alternate approach to AR, but if anything it lacks the coherency formed via [[thoughts/mechanistic interpretability#superposition hypothesis|superposition]] to those of AR models (_i suspect_)}} \[@nie2025largelanguagediffusionmodels\]. [[thoughts/Speculative decoding|Speculative decoding]] techniques such as DFlash uses a lightweight block-diffusion draft model to produce a block of draft tokens in one forward pass for an autoregressive target model to verify \[@chen2026dflashblockdiffusionflash\]. > \[!note\]- GPTs and friends > > Most variants of LLMs are decoder-only \[@radford2019language\]. There are bi-directional encoder-decoder models, though their application are more domain-specific, whereas decoder-only models generalized a lot better, per recent history. Have “capabilities” to understand [[thoughts/NLP|natural language]]. Exhibits [[thoughts/emergent behaviour]] of [[thoughts/intelligence|intelligence]]. ^emergent One way or another is a form of [[thoughts/Behaviourism|behaviourism]], through [[thoughts/Machine learning|reinforcement learning]]. It is being “told” what is good or bad, and thus act accordingly towards the users. However, this induces [[thoughts/confirmation bias]] where one aligns and contains his/her prejudices towards the problem. ### Scalability Incredibly hard to scale, mainly due to their [[thoughts/large models|large]] memory footprint and tokens memory allocation. ### [[thoughts/optimization|Optimization]] - [[thoughts/quantization|Quantization]]: reduce computational and memory costs of running inference with representing the weight and activations with low-precision data type - [[thoughts/Continuous batching]]: update the batch after each decoding step so finished requests leave and waiting requests can join. [Orca](https://www.usenix.org/conference/osdi22/presentation/yu) introduced iteration-level scheduling. [[thoughts/paged attention|PagedAttention]] manages KV-cache memory in blocks; it can support this scheduling by reducing wasted memory. - Different [[thoughts/Attention|Attention]] variants, for better kernels and hardware optimisation (Think of [[thoughts/flash attention|Flash Attention 3]], [[thoughts/radix attention|Radix Attention]], [[thoughts/tree attention|TreeAttention]], etc.) - [[thoughts/Transformers#Byte-Latent Transformer]]: group bytes into variable-length patches using next-byte prediction entropy. The large transformer processes patches, while a smaller decoder generates bytes autoregressively. [^blt] [^blt]: A small byte-level model estimates how uncertain the next byte is. High uncertainty starts a new patch; predictable byte sequences can share longer patches, reducing the number of steps run by the large transformer. Entropy determines patch boundaries. Byte generation still uses the decoder’s next-byte predictions. See [2412.09871](https://arxiv.org/html/2412.09871v1) \[@pagnoni2024bytelatenttransformerpatches\] . ### on how we are being [[thoughts/education#teaching|taught]]. > \[!question\] How would we assess thinking? > > Similar to calculator, it _simplifies_ and increase accessibility to the masses, but in doing so _lost_ the value in the _action of doing_ math. > > We do math to internalise the concept, and practice to thinking coherently. Similarly, we [[thoughts/writing|write]] to help crystalised our ideas, and in the process improve through the act of putting it down. The process of rephrasing and arranging sentences poses a challenges for the writer, and in doing so, teach you how to think coherently. Writing essays is an exercise for students to articulate their thoughts, rather than testing the understanding of the materials. ### on [[thoughts/ethics|ethics]] There are ethical concerns with the act of “hallucinating” content, therefore alignment research is crucial to ensure that the model is not producing harmful content. For [[thoughts/university/twenty-four-twenty-five/engineer-4a03/finals|medicare]], ethical implications requires us to develop better [[thoughts/mechanistic interpretability|interpretable models]] ### as philosophical tool. To create a better [[thoughts/representations|representations]] of the world for both humans and machines to understand, we can truly have assistive tools to enhance our understanding of the world surround us ### AI generated content Don’t shit where you eat, **[[thoughts/Garbage in Garbage out|Garbage in, garbage out]]**. The quality of the content is highly dependent on the quality of the data it was trained on, or model are incredibly sensitive to data variances and biases. Bland doublespeak See also: [All the better to see you with](https://www.kernelmag.io/2/all-the-better-to-see-you)

Here's a real problem though. Most people find writing hard and will get AIs to do it for them whenever they can get away with it. Which means bland doublespeak will become the default style of writing. Ugh.

### machine-assisted [[thoughts/writing]] source: [creative fiction with GPT-3](https://gwern.net/gpt-3) Idea: use [[thoughts/mechanistic interpretability#sparse autoencoders]] to guide ideas generations The idea from [writing for LLMs](https://gwern.net/llm-writing) means once your writing/thoughts were embedded within the models’ [[thoughts/latent space]] ### Good-enough

"How did we get AI art before self-driving cars?" IMHO this is the single best heuristic for predicting the speed at which certain AI advances will happen. https://t.co/yAo6pwEsxD

This only occurs if you only need a “good-enough” item where value outweighs the process. However, one should always consider to put in the work, rather than being “ok” with good enough. In the process of working through a problem, one will learn about bottleneck and problems to be solved, which in turn gain invaluable experience otherwise would not achieved if one fully relies on the interaction with the models alone. ### as [[thoughts/Search|search]] These models are incredibly useful for summarization and information gathering. With the [[thoughts/taxonomy]] of [[thoughts/RAG]] or any other CoT tooling, you can pretty much augment and produce and improve search-efficiency bu quite a lot. notable mentions: - [perplexity.ai](https://perplexity.ai/): [[thoughts/RAG|RAG]]-first search engine - [explorer.globe.engineer](https://explorer.globe.engineer/): tree-based [[thoughts/information retrieval]] - [Exa labs](https://twitter.com/ExaAiLabs) ### Programming Overall should be a net positive, but it’s a double-edged sword. #### as end-users [Source](https://www.geoffreylitt.com/2023/03/25/llm-end-user-programming.html) > I think it’s likely that soon all computer users will have the ability to develop small software tools from scratch, and to describe modifications they’d like made to software they’re already using #### as developers Tool that lower of barrier of entry is always a good thing, but it often will lead to probably even higher discrepancies in quality of software Increased in productivity, but also increased in technical debt, as these generated code are mostly “bad” code, and often we have to nudge and do a lot of **[[thoughts/prompt engineering|prompt engineering]]**. ### Truthfulness Preference data to train against dense (model) Judges → evaluator models Dense models ← (reasoning) UX → how to get in front of users? Data provenance and governance ?

You don’t need frontier lab resources for frontier lab automated LLM evaluation.

To prove this, we’re open-sourcing j1-nano and j1-micro: two absurdly tiny (600M & 1.7B parameters) but mighty reward models competitive with orders-of-magnitude larger peers.

j1-nano and j1-micro were trained in <1 day on a single A100 GPU.

_outperform opus and gpt-4o_