--- aliases: - inspo cssclasses: - nolist date: '2024-10-24' description: My run-down version of are[dot]na id: are.na metadata: ebnf: |- channel = "##" , ws , heading , newline , channel_body ; channel_body = [ meta_section , newline ] , block , { block } ; block = list_entry , newline , meta_section , { note_line } ; list_entry = "-" , ws , link , [ ws "--" ws title ] , [ ws "[**]" ] ; meta_section = ws , "-" , ws , "[meta]:" , newline , meta_pair , { meta_pair } ; meta_pair = ws , ws , "-" , ws , key , ":" , ws , value ; key = "date" | "tags" | "pinned" | "later" | "unlocked" | "socials" | "view" | identifier ; value = date | tag_list | boolean | text ; tag_list = "[" , tag , { "," , ws , tag } , "]" ; tag = identifier ; date = digit , digit , "/" , digit , digit , "/" , digit , digit , digit , digit ; boolean = "true" | "false" ; note_line = ws , "-" , ws , text ; link = uri ; title = text ; heading = text ; identifier = letter , { letter | digit | "-" } ; text = { character - newline } ; uri = ? valid http uri ? ; letter = "a".."z" ; digit = "0".."9" ; character = ? any printable ascii except newline ? ; modified: 2026-09-24 00:14:31 GMT-04:00 permalinks: - /website - /tweets - /resources socials: are.na: https://www.are.na/aaron-pham/channels curius: /curius home: / print: https://print.are.na/ tags: - evergreen - llm - design - love - friend - r/pedagogy - alignment - philosophy - P-Complete - interpretability - math/linalg title: '#papers' created: '2024-10-24' published: '2024-10-24' pageLayout: default slug: arena/papers permalink: https://aarnphm.xyz/arena/papers.md generator: quartz: v4.6.0 hostedProvider: Cloudflare baseUrl: aarnphm.xyz full: https://aarnphm.xyz/llms-full.txt --- # #papers - — Leveraging exploration in off-policy algorithms via normalizing flows - \[meta]: - date: 09/21/2026 - tags: \["reinforcement learning","normalizing flows"] - later: true - [2601.21351![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2601.21351) ([Song et al., 2026](#bib-song2026analyticalprovisioningattentionffndisaggregated)) — Analytical Provisioning for Attention-FFN Disaggregated LLM Serving under Stochastic Workloads - \[meta]: - date: 09/15/2026 - tags: \["papers"] - later: true - — Alpha-MoE: A Megakernel for Faster Tensor Parallel Inference - \[meta]: - date: 09/15/2026 - tags: \["papers","inference"] - later: true - — Composer 2 Technical Report - \[meta]: - date: 09/15/2026 - tags: \["papers"] - later: true - — Comprehensive assessment of activity, specificity, and safety of hypercompact TnpB systems for gene editing - \[meta]: - date: 09/15/2026 - tags: \["papers","gene editing","biology","ai safety"] - later: true - — Edge detectors in human vision - \[meta]: - date: 09/15/2026 - tags: \["papers","vision","neuroscience"] - later: true - — FlashSampling: Fast and Memory-Efficient Exact Sampling - \[meta]: - date: 09/15/2026 - tags: \["papers"] - later: true - — High-level semi-synthetic production of the potent antimalarial artemisinin - Nature - \[meta]: - date: 09/15/2026 - tags: \["papers","biotechnology"] - later: true - — Learned Helplessness at Fifty: Insights from Neuroscience - \[meta]: - date: 09/15/2026 - tags: \["papers","psychology","neuroscience"] - later: true - — Metacognition, comprehension monitoring, and the adult reader - \[meta]: - date: 09/15/2026 - tags: \["papers"] - later: true - — The Abstraction Fallacy: Why AI Can Simulate But Not Instantiate Consciousness - \[meta]: - date: 09/15/2026 - tags: \["papers","consciousness"] - later: true - — Prior Knowledge and Its Relationship to Comprehension - \[meta]: - date: 09/15/2026 - tags: \["papers"] - later: true - — Robust Technology Regulation - \[meta]: - date: 09/15/2026 - tags: \["papers","regulation"] - later: true - — A Spectral Theory of Computation - \[meta]: - date: 09/15/2026 - tags: \["papers"] - later: true - — SPEED-Bench: A Unified and Diverse Benchmark for Speculative Decoding - \[meta]: - date: 09/15/2026 - tags: \["papers","inference"] - later: true - — Fast Adaptation via Meta Reinforcement Learning - \[meta]: - date: 09/15/2026 - tags: \["papers","meta-learning","reinforcement learning"] - later: true - — Understanding Trust - \[meta]: - date: 09/15/2026 - tags: \["papers","trust"] - later: true - — Optimal Timing for Superintelligence: Mundane Considerations for Existing People - \[meta]: - date: 09/15/2026 - tags: \["papers","superintelligence"] - later: true - — Economic Scenarios for Transformative AI - \[meta]: - date: 09/12/2026 - tags: \["ai","economics"] - later: true - — DeepSeek V4.1 Flash Technical Report - \[meta]: - date: 09/12/2026 - tags: \["deepseek","technical report"] - later: true - — The Cerebras Wafer-Scale Architecture for Deep Learning - \[meta]: - date: 09/07/2026 - tags: \["hardware","deep learning"] - later: true - — Latent Introspection - Cartoon V2 - \[meta]: - date: 09/07/2026 - tags: \["mechanistic interpretability","poster"] - later: true - — S6 - \[meta]: - date: 09/07/2026 - tags: \["mathematics","pdf"] - later: true - — Transformer Transformer: A Unified Model for Motion-Conditioned Robot Co-design - \[meta]: - date: 09/07/2026 - tags: \["robotics","transformers"] - later: true - — Inducing Task Models from Computer-Use Traces - \[meta]: - date: 08/26/2026 - tags: \["paper","computer use"] - later: true - [2507.02754![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2507.02754) ([Roy et al., 2025](#bib-roy2025fastsimplex2simplicialattention)) — Fast and Simplex: 2-Simplicial Attention in Triton - \[meta]: - date: 08/25/2026 - tags: \["paper","attention"] - later: true - [2608.08888![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2608.08888) ([Wang et al., 2026](#bib-wang2026fullbandwidthtransformer)) — Full-bandwidth transformer - \[meta]: - date: 08/25/2026 - tags: \["paper","transformers"] - later: true - [2605.31514![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2605.31514) ([de Wynter, 2026](#bib-dewynter2026llmshumanlikeattributesdoes)) — If LLMs Have Human-Like Attributes, Then So Does Age of Empires II - \[meta]: - date: 08/25/2026 - tags: \["paper"] - later: true - [2308.07633![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/pdf/2308.07633) ([Zhu et al., 2024](#bib-zhu2024surveymodelcompressionlarge)) — A Survey on Model Compression for Large Language Models - \[meta]: - date: 08/25/2026 - tags: \["paper","language models"] - later: true - [2307.14936![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2307.14936) ([Shen et al., 2023](#bib-shen2023pangucoder2boostinglargelanguage)) — PanGu-Coder2: Boosting Large Language Models for Code with Ranking Feedback - \[meta]: - date: 08/25/2026 - tags: \["paper","language models"] - later: true - [1706.03762![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/pdf/1706.03762.pdf) ([Vaswani et al., 2023](#bib-vaswani2023attentionneed)) — Attention Is All You Need - \[meta]: - date: 08/25/2026 - tags: \["paper","attention"] - later: true - [2310.01859![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2310.01859) ([Lambert et al., 2023](#bib-lambert2023variationalgaussianapproximationkushner)) — Variational Gaussian approximation of the Kushner optimal filter - \[meta]: - date: 08/25/2026 - tags: \["paper"] - later: true - — When Physics Became Undisciplined: An Essay on Econophysics - \[meta]: - date: 08/25/2026 - tags: \["paper"] - later: true - [2006.16668![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/pdf/2006.16668) ([Lepikhin et al., 2020](#bib-lepikhin2020gshardscalinggiantmodels)) — GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding - \[meta]: - date: 08/25/2026 - tags: \["paper"] - later: true - [2401.10020![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2401.10020) ([Yuan et al., 2025](#bib-yuan2025selfrewardinglanguagemodels)) — Self-Rewarding Language Models - \[meta]: - date: 08/25/2026 - tags: \["paper","language models"] - later: true - — Physics-informed Machine Learning - \[meta]: - date: 08/25/2026 - tags: \["paper"] - later: true - [1511.06434![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/1511.06434) ([Radford et al., 2016](#bib-radford2016unsupervisedrepresentationlearningdeep)) — Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks - \[meta]: - date: 08/25/2026 - tags: \["paper"] - later: true - [2304.09960![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/pdf/2304.09960) ([Jiang, 2023](#bib-jiang2023latentspacetheoryemergent)) — A Latent Space Theory for Emergent Abilities in Large Language Models - \[meta]: - date: 08/25/2026 - tags: \["paper","language models"] - later: true - [2309.08600![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/pdf/2309.08600.pdf) ([Cunningham et al., 2023](#bib-cunningham2023sparseautoencodershighlyinterpretable)) — Sparse Autoencoders Find Highly Interpretable Features in Language Models - \[meta]: - date: 08/25/2026 - tags: \["paper","language models"] - later: true - — Conformal Prediction - \[meta]: - date: 08/25/2026 - tags: \["paper"] - later: true - — The Last Question - \[meta]: - date: 08/25/2026 - tags: \["paper"] - later: true - — Notation as a Tool of Thought - \[meta]: - date: 08/25/2026 - tags: \["paper"] - later: true - — Causal Inference: A Statistical Learning Approach - \[meta]: - date: 08/25/2026 - tags: \["paper"] - later: true - [2409.14586![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/pdf/2409.14586) ([Y. Zhang et al., 2024](#bib-zhang2024backtrackingimprovesgenerationsafety)) — Backtracking Improves Generation Safety - \[meta]: - date: 08/25/2026 - tags: \["paper","safety"] - later: true - [2407.01449![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2407.01449) ([Faysse et al., 2025](#bib-faysse2025colpaliefficientdocumentretrieval)) — ColPali: Efficient Document Retrieval with Vision Language Models - \[meta]: - date: 08/25/2026 - tags: \["paper","language models"] - later: true - — Concise Machine Learning - \[meta]: - date: 08/25/2026 - tags: \["paper"] - later: true - [2410.10630![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2410.10630) ([T. Wu et al., 2024](#bib-wu2024thinkingllmsgeneralinstruction)) — Thinking LLMs: General Instruction Following with Thought Generation - \[meta]: - date: 08/25/2026 - tags: \["paper"] - later: true - [2410.01131![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2410.01131) ([Loshchilov et al., 2025](#bib-loshchilov2025ngptnormalizedtransformerrepresentation)) — nGPT: Normalized Transformer with Representation Learning on the Hypersphere - \[meta]: - date: 08/25/2026 - tags: \["paper","transformers"] - later: true - [2406.15786![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2406.15786) ([He et al., 2024](#bib-he2024matterstransformersattentionneeded)) — What Matters in Transformers? Not All Attention is Needed - \[meta]: - date: 08/25/2026 - tags: \["paper","attention"] - later: true - [1706.05806![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/pdf/1706.05806) ([Raghu et al., 2017](#bib-raghu2017svccasingularvectorcanonical)) — SVCCA: Singular Vector Canonical Correlation Analysis for Deep Learning Dynamics and Interpretability - \[meta]: - date: 08/25/2026 - tags: \["paper","interpretability"] - later: true - [2409.11340![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2409.11340) ([Xiao et al., 2024](#bib-xiao2024omnigenunifiedimagegeneration)) — OmniGen: Unified Image Generation - \[meta]: - date: 08/25/2026 - tags: \["paper"] - later: true - — Data Mining - \[meta]: - date: 08/25/2026 - tags: \["paper"] - later: true - [1712.00409![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/1712.00409) ([Hestness et al., 2017](#bib-hestness2017deeplearningscalingpredictable)) — Deep Learning Scaling is Predictable, Empirically - \[meta]: - date: 08/25/2026 - tags: \["paper"] - later: true - — ScienceDirect - \[meta]: - date: 08/25/2026 - tags: \["paper"] - later: true - [2407.09722![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/html/2407.09722v2) ([Qin et al., 2025](#bib-qin2025optimizedmultitokenjointdecoding)) — Optimized Multi-Token Joint Decoding with Auxiliary Model for LLM Inference - \[meta]: - date: 08/25/2026 - tags: \["paper","inference"] - later: true - [2306.01116![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2306.01116) ([Penedo et al., 2023](#bib-penedo2023refinedwebdatasetfalconllm)) — The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Only - \[meta]: - date: 08/25/2026 - tags: \["paper"] - later: true - — Socrates in Plato’s Symposium: A Lover of Wisdom Who Lacks Wisdom on Love - \[meta]: - date: 08/25/2026 - tags: \["paper"] - later: true - [2312.00752![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/pdf/2312.00752) ([Gu & Dao, 2024](#bib-gu2024mambalineartimesequencemodeling)) — Mamba: Linear-Time Sequence Modeling with Selective State Spaces - \[meta]: - date: 08/25/2026 - tags: \["paper"] - later: true - [2002.09402![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/pdf/2002.09402) ([Fan et al., 2021](#bib-fan2021addressinglimitationstransformersfeedback)) — Addressing Some Limitations of Transformers with Feedback Memory - \[meta]: - date: 08/25/2026 - tags: \["paper","transformers"] - later: true - — Bridging the Human-AI Knowledge Gap Through Concept Discovery and Transfer in AlphaZero - \[meta]: - date: 08/25/2026 - tags: \["paper"] - later: true - [2507.02754![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/pdf/2507.02754) ([Roy et al., 2025](#bib-roy2025fastsimplex2simplicialattention)) — Fast and Simplex: 2-Simplicial Attention in Triton - \[meta]: - date: 08/25/2026 - tags: \["paper","attention"] - later: true - [2404.17625![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/pdf/2404.17625) ([Scardapane, 2025](#bib-scardapane2025alicesadventuresdifferentiablewonderland)) — Alice’s Adventures in a Differentiable Wonderland - Volume I, A Tour of the Land - \[meta]: - date: 08/25/2026 - tags: \["paper"] - later: true - [2506.21575![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/pdf/2506.21575) ([Stoisser et al., 2025](#bib-stoisser2025structllmunifyingtabulargraph)) — STRuCT-LLM: Unifying Tabular and Graph Reasoning with Reinforcement Learning for Semantic Parsing - \[meta]: - date: 08/25/2026 - tags: \["paper","reinforcement learning"] - later: true - [2504.17033![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/html/2504.17033v2) ([Duan et al., 2025](#bib-duan2025breakingsortingbarrierdirected)) — Breaking the Sorting Barrier for Directed Single-Source Shortest Paths - \[meta]: - date: 08/25/2026 - tags: \["paper"] - later: true - [2205.14135![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/pdf/2205.14135) ([Dao et al., 2022](#bib-dao2022flashattentionfastmemoryefficientexact)) — FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness - \[meta]: - date: 08/25/2026 - tags: \["paper","attention"] - later: true - [1411.0945![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/pdf/1411.0945) ([Bartlett et al., 2014](#bib-bartlett2014extended3dimensionalbordismtheory)) — Extended 3-dimensional bordism as the theory of modular objects - \[meta]: - date: 08/25/2026 - tags: \["paper"] - later: true - [2412.19437![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/html/2412.19437v1) ([DeepSeek-AI et al., 2025](#bib-deepseekai2025deepseekv3technicalreport)) — DeepSeek-V3 Technical Report - \[meta]: - date: 08/25/2026 - tags: \["paper"] - later: true - [2405.16444![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/pdf/2405.16444) ([Yao et al., 2025](#bib-yao2025cacheblendfastlargelanguage)) — CacheBlend: Fast Large Language Model Serving for RAG with Cached Knowledge Fusion - \[meta]: - date: 08/25/2026 - tags: \["paper","language models"] - later: true - [2204.08396![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/pdf/2204.08396) ([Dai et al., 2022](#bib-dai2022stablemoestableroutingstrategy)) — StableMoE: Stable Routing Strategy for Mixture of Experts - \[meta]: - date: 08/25/2026 - tags: \["paper"] - later: true - [2305.19370![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/pdf/2305.19370) ([Liu & Abbeel, 2023](#bib-liu2023blockwiseparalleltransformerlarge)) — Blockwise Parallel Transformer for Large Context Models - \[meta]: - date: 08/25/2026 - tags: \["paper","transformers"] - later: true - [2506.05508![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/pdf/2506.05508) ([Mitra et al., 2025](#bib-mitra2025buzzpragmaticinferencedisaggregation)) — Beyond the Buzz: A Pragmatic Take on Inference Disaggregation - \[meta]: - date: 08/25/2026 - tags: \["paper","inference"] - later: true - [2512.16705![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/pdf/2512.16705) ([Müller et al., 2026](#bib-müller2026olafbringinganimatedcharacter)) — Olaf: Bringing an Animated Character to Life in the Physical World - \[meta]: - date: 08/25/2026 - tags: \["paper"] - later: true - [2511.01815![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2511.01815) ([Staniszewski & Łańcucki, 2026](#bib-staniszewski2026kvcachetransformcoding)) — KV Cache Transform Coding for Compact Storage in LLM Inference - \[meta]: - date: 08/25/2026 - tags: \["paper","inference"] - later: true - [https://www.arxiv.org/pdf/2602.00398![arXiv](/static/favicons/arxiv.avif)](https://www.arxiv.org/pdf/2602.00398) — MemoryLLM: Plug-n-Play Interpretable Feed-Forward Memory for Transformers - \[meta]: - date: 08/25/2026 - tags: \["paper","transformers"] - later: true - [2602.06964![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2602.06964) ([Luo et al., 2026](#bib-luo2026learninggenerativemetamodelllm)) — Learning a Generative Meta-Model of LLM Activations - \[meta]: - date: 08/25/2026 - tags: \["paper"] - later: true - [2602.12176![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/pdf/2602.12176) ([Guevara et al., 2026](#bib-guevara2026singleminusgluontreeamplitudes)) — Single-minus gluon tree amplitudes are nonzero - \[meta]: - date: 08/25/2026 - tags: \["paper"] - later: true - [2602.21548![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/pdf/2602.21548) ([Y. Wu et al., 2026](#bib-arxiv-260221548)) — DualPath: Breaking the Storage Bandwidth Bottleneck in Agentic LLM Inference - \[meta]: - date: 08/25/2026 - tags: \["paper","agents"] - later: true - — The Residual Stream Isn’t Linear - \[meta]: - date: 08/25/2026 - tags: \["paper"] - later: true - [2507.07101![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/pdf/2507.07101) ([Marek et al., 2025](#bib-marek2025smallbatchsizetraining)) — Small Batch Size Training for Language Models: When Vanilla SGD Works, and Why Gradient Accumulation Is Wasteful - \[meta]: - date: 08/25/2026 - tags: \["paper","language models"] - later: true - [2508.15734![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/pdf/2508.15734) ([Elsworth et al., 2025](#bib-elsworth2025measuringenvironmentalimpactdelivering)) — Measuring the environmental impact of delivering AI at Google Scale - \[meta]: - date: 08/25/2026 - tags: \["paper"] - later: true - [2601.11580![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/pdf/2601.11580) ([X. Liu et al., 2026](#bib-liu2026speculativedecodingperformanceillusion)) — Speculative Decoding: Performance or Illusion? - \[meta]: - date: 08/25/2026 - tags: \["paper"] - later: true - [2603.21852![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/html/2603.21852v2) ([Odrzywołek, 2026](#bib-odrzywołek2026elementaryfunctionssinglebinary)) — All elementary functions from a single operator - \[meta]: - date: 08/25/2026 - tags: \["paper"] - later: true - [2604.11962![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2604.11962) ([Walker et al., 2026](#bib-walker2026linearcentroidshypothesisfeatures)) — The Linear Centroids Hypothesis: Features as Directions Learned by Local Experts - \[meta]: - date: 08/25/2026 - tags: \["paper"] - later: true - [2506.05340![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2506.05340) ([Chandrasegaran et al., 2025](#bib-chandrasegaran2025exploringdiffusiontransformerdesigns)) — Exploring Diffusion Transformer Designs via Grafting - \[meta]: - date: 08/25/2026 - tags: \["paper","transformers"] - later: true - [2605.03327![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/pdf/2605.03327) ([Jin et al., 2026](#bib-jin2026dgpodistributionguidedpolicy)) — DGPO: Distribution Guided Policy Optimization for Fine Grained Credit Assignment - \[meta]: - date: 08/25/2026 - tags: \["paper"] - later: true - [2602.06036![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/pdf/2602.06036) ([Chen et al., 2026](#bib-chen2026dflashblockdiffusionflash)) — DFlash: Block Diffusion for Flash Speculative Decoding - \[meta]: - date: 08/25/2026 - tags: \["paper"] - later: true - [2605.21568![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/pdf/2605.21568) ([Kendall, 2026](#bib-kendall2026equilibriumpropagationhamiltonianinference)) — Equilibrium Propagation and Hamiltonian Inference in the Diffusive Fitzhugh-Nagumo Model - \[meta]: - date: 08/25/2026 - tags: \["paper","inference"] - later: true - — Verified Machine Learning Infrastructure: Formal Methods for Trustworthy Artificial Intelligence Deployment - \[meta]: - date: 08/25/2026 - tags: \["paper"] - later: true - [2606.12360![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2606.12360) ([Bergen et al., 2026](#bib-bergen2026anatomyposttrainingusinginterpretability)) — Anatomy of Post-Training: Using Interpretability to Characterize Data and Shape the Learning Signal - \[meta]: - date: 08/25/2026 - tags: \["paper","interpretability"] - later: true - [2606.16112![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2606.16112) ([Figliolia & Millidge, 2026](#bib-figliolia2026scalingadaptivedepthnormagnostic)) — Scaling Adaptive Depth with Norm-Agnostic Residual Networks - \[meta]: - date: 08/25/2026 - tags: \["paper"] - later: true - [2602.14486![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/pdf/2602.14486) ([Gröger et al., 2026](#bib-gröger2026revisitingplatonicrepresentationhypothesis)) — Revisiting the Platonic Representation Hypothesis: An Aristotelian View - \[meta]: - date: 08/25/2026 - tags: \["paper"] - later: true - [1701.07570![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/1701.07570) ([L. Zhang et al., 2018](#bib-zhang2018dynamicregretstronglyadaptive)) — Dynamic Regret of Strongly Adaptive Methods - \[meta]: - date: 08/25/2026 - tags: \["paper"] - later: true - [2607.03502![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/pdf/2607.03502) ([Brauer et al., 2026](#bib-brauer2026readingdotsdecodinghidden)) — Reading Between the Dots: Decoding Hidden Computation across Filler Tokens - \[meta]: - date: 08/25/2026 - tags: \["paper"] - later: true - [2301.12987![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/pdf/2301.12987v4) ([Bennett, 2024](#bib-bennett2024optimalchoicehypothesisweakest)) — The Optimal Choice of Hypothesis Is the Weakest, Not the Shortest - \[meta]: - date: 08/25/2026 - tags: \["paper"] - later: true - [2608.11612![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2608.11612) ([Amsel et al., 2026](#bib-amsel2026dion3fullstackorthogonalupdates)) — Dion3: Full-Stack Orthogonal Updates - \[meta]: - date: 08/25/2026 - tags: \["paper"] - later: true - — Know Thyself: Evaluating Self-prediction Capabilities in LM Agents - \[meta]: - date: 08/20/2026 - tags: \["agents","self-prediction"] - later: true - — Ironies of Automation - \[meta]: - date: 08/20/2026 - tags: \["automation","human factors"] - later: true - — An Extensible, Data-Oriented Architecture for High-Performance, Many-World Simulation - \[meta]: - date: 08/20/2026 - tags: \["simulation","reinforcement learning"] - later: true - — More Than Two Thirds of the Zeros of the Riemann Zeta Function Lie on the Critical Line - \[meta]: - date: 08/20/2026 - tags: \["riemann zeta","mathematics"] - later: true - — 67% of the zeroes are on the line - \[meta]: - date: 08/20/2026 - tags: \["riemann zeta","mathematics"] - later: true - [ParallelKittens: Systematic and Practical Simplification of Multi-GPU AI Kernels![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2511.13940) ([Sul et al., 2025](#bib-sul2025parallelkittenssystematicpracticalsimplification)) — ParallelKittens: Systematic and Practical Simplification of Multi-GPU AI Kernels - \[meta]: - date: 08/06/2026 - tags: \["gpu kernels","parallel programming"] - later: true - [Intelligence per Watt: Measuring Intelligence Efficiency of Local AI![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2511.07885) ([Saad-Falcon et al., 2026](#bib-saadfalcon2026intelligencewattmeasuringintelligence)) — Intelligence per Watt: Measuring Intelligence Efficiency of Local AI - \[meta]: - date: 08/06/2026 - tags: \["ai efficiency","benchmarking"] - later: true - — Claude Cycles - \[meta]: - date: 07/31/2026 - tags: \["claude","language models"] - later: true - [1910.07467![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/pdf/1910.07467) ([B. Zhang & Sennrich, 2019](#bib-zhang2019rootmeansquarelayer)) — Root Mean Square Layer Normalization - \[meta]: - date: 07/31/2026 - tags: \["normalization","transformers"] - later: true - — Value estimation with finite data - \[meta]: - date: 07/31/2026 - tags: \["value estimation","reinforcement learning"] - later: true - [2605.12888![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/pdf/2605.12888) ([Guo et al., 2026](#bib-guo2026seedbankcoopstoop)) — Seed Bank, Co-op, Stoop Swap: Metaphors for Governing Language Model Data for Creative Writing - \[meta]: - date: 07/31/2026 - tags: \["language models","creative writing"] - later: true - [2408.11049![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2408.11049) ([Sadhukhan et al., 2025](#bib-sadhukhan2025magicdecbreakinglatencythroughputtradeoff)) — MagicDec: Breaking the Latency-Throughput Tradeoff for Long Context Generation with Speculative Decoding - \[meta]: - date: 07/31/2026 - tags: \["speculative decoding","long context"] - later: true - — Pangram 4 Technical Report - \[meta]: - date: 07/31/2026 - tags: \["language models","technical report"] - later: true - [2607.05147![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/pdf/2607.05147) ([Cheng et al., 2026](#bib-cheng2026dsparkconfidencescheduledspeculativedecoding)) — DSpark: Confidence-Scheduled Speculative Decoding with Semi-Autoregressive Generation - \[meta]: - date: 07/31/2026 - tags: \["speculative decoding","semi-autoregressive"] - later: true - [2512.07805![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/pdf/2512.07805) ([Y. Zhang et al., 2026](#bib-zhang2026grouprepresentationalpositionencoding)) — Group Representational Position Encoding - \[meta]: - date: 07/31/2026 - tags: \["positional encodings","attention"] - later: true - [2602.20021![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2602.20021) ([Shapira et al., 2026](#bib-shapira2026agentschaos)) — Agents of Chaos - \[meta]: - date: 07/31/2026 - tags: \["agents","language models"] - later: true - — Claude Opus 5 System Card - \[meta]: - date: 07/28/2026 - tags: \["system card","safety"] - later: true - [2606.09589![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2606.09589) ([Riccio, 2026](#bib-riccio2026iscrollingisaw)) - \[meta]: - date: 06/30/2026 - tags: \["biology"] - [deepseek-ai/DeepSpec](https://github.com/deepseek-ai/DeepSpec/blob/main/DSpark_paper.pdf) — DSpark - \[meta]: - date: 06/29/2026 - tags: \["speculative decoding"] - later: true - [2503.24278![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2503.24278) ([Zhou et al., 2025](#bib-zhou2025autoevalautonomousevaluationgeneralist)) — AutoEval: Autonomous Evaluation of Generalist Robot Manipulation Policies in the Real World - \[meta]: - date: 06/29/2026 - tags: \["robotics","evaluation"] - later: true - — BiternionNets head orientation thread - \[meta]: - date: 05/26/2026 - tags: \["computer vision","orientation"] - later: true - — On Calibration of Modern Neural Networks - \[meta]: - date: 06/28/2026 - tags: \["ml"] - later: true - — The persona selection model - \[meta]: - date: 06/28/2026 - tags: \["alignment","agi"] - — Towards Compositional Steepest Descent - \[meta]: - date: 06/28/2026 - tags: \["optimizer","muon"] - — Neuron Populations Exhibit Divergent Selectivity with Scale - \[meta]: - date: 06/28/2026 - tags: \["theory","models","llm"] - — Future leakage in block-quantized attention - \[meta]: - date: 06/28/2026 - tags: \["sparse","attention"] - [2602.11399![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2602.11399) ([Zheng et al., 2026](#bib-zheng2026reallylearnrepresentationoptimize)) - \[meta]: - date: 06/28/2026 - tags: \["rl","scaling"] - later: true - — Scuba: Diving into Data at Facebook - \[meta]: - date: 06/28/2026 - tags: \["data","systems"] - — The Moore’s Law of Synthetic Gene Circuits - \[meta]: - date: 06/18/2026 - tags: \["synthetic biology","gene circuits"] - later: true - [2606.07878![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2606.07878v1) ([O’Neill et al., 2026](#bib-oneill2026stillamortizedkvcache)) — Still: Amortized KV Cache Compaction in a Single Forward Pass - \[meta]: - date: 06/15/2026 - tags: \["kv cache","inference"] - later: true - [2211.00593![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2211.00593) ([K. Wang et al., 2022](#bib-wang2022interpretabilitywildcircuitindirect)) — Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small - \[meta]: - date: 06/15/2026 - tags: \["interpretability","circuits"] - later: true - [2502.05795![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2502.05795) ([Sun et al., 2026](#bib-sun2026cursedepthlargelanguage)) — The Curse of Depth in Large Language Models - \[meta]: - date: 06/15/2026 - tags: \["llm","training"] - later: true - [2512.18194![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2512.18194) ([Yoon et al., 2025](#bib-yoon2025tractdisaggregatedllmserving)) - \[meta]: - date: 06/14/2026 - tags: \["kv","disaggregation"] - later: true - [2602.11729![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2602.11729) ([Jiralerspong & Bricken, 2026](#bib-jiralerspong2026crossarchitecturemodeldiffingcrosscoders)) - \[meta]: - date: 06/14/2026 - tags: \["interpretability","crosscoders"] - later: true - [2410.13780![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2410.13780) ([Ordentlich & Polyanskiy, 2025](#bib-ordentlich2025optimalquantizationmatrixmultiplication)) - \[meta]: - date: 06/14/2026 - tags: \["quantization","matrix multiplication"] - later: true - [2604.06425![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2604.06425) ([Zhuge et al., 2026](#bib-zhuge2026neuralcomputers)) - \[meta]: - date: 06/14/2026 - tags: \["architecture","neural computers"] - later: true - [2603.21852![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2603.21852) ([Odrzywołek, 2026](#bib-odrzywołek2026elementaryfunctionssinglebinary)) - \[meta]: - date: 06/14/2026 - tags: \["symbolic regression","functions"] - later: true - [2602.06036![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2602.06036) ([Chen et al., 2026](#bib-chen2026dflashblockdiffusionflash)) - \[meta]: - date: 06/14/2026 - tags: \["speculative decoding","diffusion"] - later: true - [2602.02204![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2602.02204) ([Yin et al., 2026](#bib-yin2026vllmomnifullydisaggregatedserving)) - \[meta]: - date: 06/14/2026 - tags: \["multimodal","serving"] - later: true - [2603.12201![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2603.12201) ([Bai et al., 2026](#bib-bai2026indexcacheacceleratingsparseattention)) - \[meta]: - date: 06/14/2026 - tags: \["sparse attention","inference"] - later: true - [2604.01472![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2604.01472) ([Du & Su, 2026](#bib-du2026newtonmuonoptimizer)) - \[meta]: - date: 06/14/2026 - tags: \["optimizer","muon"] - later: true - [2603.20105![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2603.20105) ([Roy et al., 2026](#bib-roy2026mathbfycombinatorllmssolvinglongcontext)) - \[meta]: - date: 06/14/2026 - tags: \["long context","recursion"] - later: true - [2102.03467![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2102.03467) ([Cazenave, 2021](#bib-cazenave2021improvingmodelsearchcomputer)) - \[meta]: - date: 06/14/2026 - tags: \["rl","game playing"] - later: true - [1611.01144![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/1611.01144) ([Jang et al., 2017](#bib-jang2017categoricalreparameterizationgumbelsoftmax)) - \[meta]: - date: 06/14/2026 - tags: \["gumbel softmax","reparameterization"] - later: true - [2411.19379![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2411.19379) ([Pan et al., 2025](#bib-pan2025marconiprefixcachingera)) - \[meta]: - date: 06/14/2026 - tags: \["prefix-caching","hybrid models"] - later: true - [2603.15031![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2603.15031) ([Team et al., 2026](#bib-kimiteam2026attentionresiduals)) - \[meta]: - date: 06/14/2026 - tags: \["architecture","residuals"] - later: true - [2509.25424![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2509.25424) ([Hamid et al., 2026](#bib-hamid2026polychromicobjectivesreinforcementlearning)) - \[meta]: - date: 06/14/2026 - tags: \["rl","objectives"] - later: true - [2103.03230![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2103.03230) ([Zbontar et al., 2021](#bib-zbontar2021barlowtwinsselfsupervisedlearning)) - \[meta]: - date: 06/14/2026 - tags: \["self-supervised learning","representation"] - later: true - [2405.04434![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2405.04434) ([DeepSeek-AI et al., 2024](#bib-deepseekai2024deepseekv2strongeconomicalefficient)) - \[meta]: - date: 06/14/2026 - tags: \["moe","models"] - later: true - [2201.02177![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2201.02177) ([Power et al., 2022](#bib-power2022grokkinggeneralizationoverfittingsmall)) - \[meta]: - date: 06/14/2026 - tags: \["grokking","generalization"] - later: true - [2605.28814![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2605.28814) ([Xu et al., 2026](#bib-xu2026selfimprovinglanguagemodelsbidirectional)) - \[meta]: - date: 06/14/2026 - tags: \["self improvement","search"] - later: true - [2605.29157![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2605.29157) ([Zuo et al., 2026](#bib-zuo2026parallaxparameterizedlocallinear)) - \[meta]: - date: 06/14/2026 - tags: \["linear attention","architecture"] - later: true - [2603.03251![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2603.03251) ([Kumar et al., 2026](#bib-kumar2026speculativespeculativedecoding)) - \[meta]: - date: 06/14/2026 - tags: \["speculative decoding","inference"] - later: true - [2410.05364![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2410.05364) ([G. Zhou et al., 2025](#bib-zhou2025diffusionmodelpredictivecontrol)) - \[meta]: - date: 06/14/2026 - tags: \["diffusion","control"] - later: true - [2603.19312![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2603.19312) ([Maes et al., 2026](#bib-maes2026leworldmodelstableendtoendjointembedding)) - \[meta]: - date: 06/14/2026 - tags: \["world models","representation"] - later: true - [2503.02113![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2503.02113) ([Wilson, 2025](#bib-wilson2025deeplearningmysteriousdifferent)) - \[meta]: - date: 06/14/2026 - tags: \["generalization","deep learning"] - later: true - [2509.14786![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2509.14786) ([Kim et al., 2025](#bib-kim2025pretraininginfinitecompute)) - \[meta]: - date: 06/14/2026 - tags: \["pretraining","scaling"] - later: true - [2605.21739![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2605.21739) ([Lubrano et al., 2026](#bib-lubrano2026attunebenchconversationbasedbenchmarkllm)) - \[meta]: - date: 06/14/2026 - tags: \["emotional intelligence","benchmark"] - later: true - [2605.31574![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2605.31574) ([Hohnen-Ford et al., 2026](#bib-hohnenford2026generativeaihelppeople)) - \[meta]: - date: 06/14/2026 - tags: \["moral-disagreement","human ai"] - later: true - [1805.02867![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/1805.02867) ([Milakov & Gimelshein, 2018](#bib-milakov2018onlinenormalizercalculationsoftmax)) - \[meta]: - date: 06/14/2026 - tags: \["softmax","optimization"] - later: true - [2606.03237![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2606.03237) ([Trivedi et al., 2026](#bib-trivedi2026solipsisticsuperintelligenceunlikelycooperative)) - \[meta]: - date: 06/14/2026 - tags: \["cooperation","ai safety"] - later: true - [2604.05192![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2604.05192) ([Schmidt et al., 2026](#bib-schmidt2026fastersuperwordtokenization)) - \[meta]: - date: 06/14/2026 - tags: \["tokenization","bpe"] - later: true - [2502.04878![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2502.04878) ([Leask et al., 2025](#bib-leask2025sparseautoencoderscanonicalunits)) - \[meta]: - date: 06/13/2026 - tags: \["interpretability","sae"] - later: true - [2508.09494![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2508.09494) ([Lin et al., 2025](#bib-lin2025learningfactsscaleactive)) - \[meta]: - date: 06/13/2026 - tags: \["memory","knowledge"] - later: true - [2509.06863![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2509.06863) ([Agrawalla et al., 2025](#bib-agrawalla2025floqtrainingcriticsflowmatching)) - \[meta]: - date: 06/13/2026 - tags: \["rl","value learning"] - later: true - [2310.07240![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2310.07240) ([Y. Liu et al., 2024](#bib-liu2024cachegenkvcachecompression)) - \[meta]: - date: 06/13/2026 - tags: \["kv","compression"] - later: true - [2405.16444![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2405.16444) ([Yao et al., 2025](#bib-yao2025cacheblendfastlargelanguage)) - \[meta]: - date: 06/13/2026 - tags: \["rag","kv"] - later: true - [2510.04871![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2510.04871) ([Jolicoeur-Martineau, 2025](#bib-jolicoeurmartineau2025morerecursivereasoningtiny)) - \[meta]: - date: 06/13/2026 - tags: \["reasoning","recursion"] - later: true - [1611.09940![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/1611.09940) ([Bello et al., 2017](#bib-bello2017neuralcombinatorialoptimizationreinforcement)) - \[meta]: - date: 06/13/2026 - tags: \["rl","optimization"] - later: true - [2601.16175![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2601.16175) ([Yuksekgonul et al., 2026](#bib-yuksekgonul2026learningdiscovertesttime)) - \[meta]: - date: 06/13/2026 - tags: \["test time","discovery"] - later: true - [2601.15727![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2601.15727) ([Yu et al., 2026](#bib-yu2026automatedkernelgenerationera)) - \[meta]: - date: 06/13/2026 - tags: \["kernels","codegen"] - later: true - [2504.11816![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2504.11816) ([Kim, Kim, et al., 2025](#bib-kim2025costefficientllmservingcloud)) - \[meta]: - date: 06/13/2026 - tags: \["kv","offloading"] - later: true - [2512.19428![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2512.19428) ([Chong, 2025](#bib-chong2025attentionneed)) - \[meta]: - date: 06/13/2026 - tags: \["architecture","attention"] - later: true - [2601.16979![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2601.16979) ([Kalra et al., 2026](#bib-kalra2026scalablemeasurelosslandscape)) - \[meta]: - date: 06/13/2026 - tags: \["training","curvature"] - later: true - [2302.00805![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2302.00805) ([Hubinger et al., 2023](#bib-hubinger2023conditioningpredictivemodelsrisks)) - \[meta]: - date: 06/13/2026 - tags: \["safety","prediction"] - later: true - [2601.22401![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2601.22401) ([Feng et al., 2026](#bib-feng2026semiautonomousmathematicsdiscoverygemini)) - \[meta]: - date: 06/13/2026 - tags: \["math","agents"] - later: true - [2601.07933![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2601.07933) ([Lam & Litt, 2026](#bib-lam2026pcurvaturenonabeliancohomology)) - \[meta]: - date: 06/13/2026 - tags: \["math","cohomology"] - later: true - [2601.18778![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2601.18778) ([Sundaram et al., 2026](#bib-sundaram2026teachingmodelsteachthemselves)) - \[meta]: - date: 06/13/2026 - tags: \["reasoning","learnability"] - later: true - [2505.22756![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2505.22756) ([T. Qin et al., 2025](#bib-qin2025decomposingelementsproblemsolving)) - \[meta]: - date: 06/13/2026 - tags: \["rl","reasoning"] - later: true - [2412.08905![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2412.08905) ([Abdin et al., 2024](#bib-abdin2024phi4technicalreport)) - \[meta]: - date: 06/13/2026 - tags: \["models","data"] - later: true - [2602.12429![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2602.12429) ([Janson et al., 2026](#bib-janson2026stabilizingnativelowrankllm)) - \[meta]: - date: 06/13/2026 - tags: \["pretraining","low rank"] - later: true - [2602.07193![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2602.07193) ([Poonsiriwong et al., 2026](#bib-poonsiriwong2026deathchatbotinvestigatingdesigning)) - \[meta]: - date: 06/13/2026 - tags: \["human ai","safety"] - later: true - [2602.11287![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2602.11287) ([Y. Luo et al., 2026](#bib-luo2026hifloat4formatlanguagemodel)) - \[meta]: - date: 06/13/2026 - tags: \["quantization","inference"] - later: true - [2410.13787![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2410.13787) ([Binder et al., 2024](#bib-binder2024lookinginwardlanguagemodels)) - \[meta]: - date: 06/13/2026 - tags: \["introspection","llms"] - later: true - [2603.07685![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2603.07685) ([Yan et al., 2026](#bib-yan2026scalabletrainingmixtureofexpertsmodels)) - \[meta]: - date: 06/13/2026 - tags: \["moe","training"] - later: true - [2603.02298![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2603.02298) ([Cecka, 2026](#bib-cecka2026cutelayoutrepresentationalgebra)) - \[meta]: - date: 06/13/2026 - tags: \["cute","tensor"] - later: true - [2602.01469![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2602.01469) ([Hui et al., 2026](#bib-hui2026peagleparalleldraftingeaglescalable)) - \[meta]: - date: 06/13/2026 - tags: \["speculative decoding","eagle"] - later: true - [2212.11279![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2212.11279) ([Schmidhuber, 2025](#bib-schmidhuber2025annotatedhistorymodernai)) - \[meta]: - date: 06/13/2026 - tags: \["history","ai"] - later: true - [2003.07892![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2003.07892) ([Desai & Durrett, 2020](#bib-desai2020calibrationpretrainedtransformers)) - \[meta]: - date: 06/13/2026 - tags: \["calibration","transformers"] - later: true - [2510.27484![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2510.27484) ([Macar et al., 2026](#bib-macar2026thoughtbranchesinterpretingllm)) - \[meta]: - date: 06/13/2026 - tags: \["cot","interpretability"] - later: true - [2410.04691![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2410.04691) ([Q. Yin et al., 2024](#bib-yin2024deeperinsightsupdatespower)) - \[meta]: - date: 06/13/2026 - tags: \["icl","fine tuning"] - later: true - [2001.08361![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2001.08361) ([Kaplan et al., 2020](#bib-kaplan2020scalinglawsneurallanguage)) - \[meta]: - date: 06/13/2026 - tags: \["scaling","language models"] - later: true - [2408.02442![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2408.02442) ([Tam et al., 2024](#bib-tam2024letspeakfreelystudy)) - \[meta]: - date: 06/13/2026 - tags: \["structured generation","evaluation"] - later: true - [2601.00397![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2601.00397) ([Agrawal et al., 2026](#bib-agrawal2026revatitransparentgpufreetimewarp)) - \[meta]: - date: 06/13/2026 - tags: \["simulation","serving"] - later: true - [2502.06785![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2502.06785) ([Heddes et al., 2025](#bib-heddes2025deepcrossattentionsuperchargingtransformerresidual)) - \[meta]: - date: 06/13/2026 - tags: \["architecture","residuals"] - later: true - [2507.10524![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2507.10524) ([Bae et al., 2025](#bib-bae2025mixtureofrecursionslearningdynamicrecursive)) - \[meta]: - date: 06/13/2026 - tags: \["adaptive compute","recursion"] - later: true - [2603.15339![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2603.15339) ([Koenig & Negrello, 2026](#bib-koenig2026neurosciencetransformers)) - \[meta]: - date: 06/13/2026 - tags: \["neuroscience","transformers"] - later: true - [2502.12131![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2502.12131) ([Fernando & Guitchounts, 2025](#bib-fernando2025transformerdynamicsneuroscientificapproach)) - \[meta]: - date: 06/13/2026 - tags: \["interpretability","dynamics"] - later: true - [2407.21092![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2407.21092) ([Yang, 2024](#bib-yang2024entropythermodynamicsgeometrizationlanguage)) - \[meta]: - date: 06/13/2026 - tags: \["thermodynamics","language models"] - later: true - [2409.02387![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2409.02387) ([Niu et al., 2025](#bib-niu2025largelanguagemodelscognitive)) - \[meta]: - date: 06/13/2026 - tags: \["cognitive science","llms"] - later: true - [2506.04374![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2506.04374) ([Carson & Reisizadeh, 2025](#bib-carson2025statisticalphysicslanguagemodel)) - \[meta]: - date: 06/13/2026 - tags: \["statistical physics","reasoning"] - later: true - [2503.01840![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2503.01840) ([Li et al., 2025](#bib-li2025eagle3scalinginferenceacceleration)) - \[meta]: - date: 06/13/2026 - tags: \["speculative decoding","eagle"] - later: true - [2505.07203![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2505.07203) ([K. Du et al., 2025](#bib-du2025prefillonlyinferenceengineprefillonly)) - \[meta]: - date: 06/13/2026 - tags: \["prefill","inference"] - later: true - [2603.13358![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2603.13358) ([Z. Li et al., 2026](#bib-li2026prefillsequalppddisaggregation)) - \[meta]: - date: 06/13/2026 - tags: \["disaggregation","serving"] - later: true - [2405.05254![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2405.05254) ([Y. Sun et al., 2024](#bib-sun2024cacheoncedecoderdecoderarchitectures)) - \[meta]: - date: 06/13/2026 - tags: \["kv","architecture"] - later: true - [2409.19606![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2409.19606) ([D. Zhu et al., 2025](#bib-zhu2025hyperconnections)) - \[meta]: - date: 06/12/2026 - tags: \["architecture","residuals"] - later: true - [2408.08147![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2408.08147) ([Y. Jin et al., 2024](#bib-jin2024pdserveservingdisaggregatedlarge)) - \[meta]: - date: 06/12/2026 - tags: \["inference","disaggregation"] - later: true - [2401.11181![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2401.11181) ([Hu et al., 2024](#bib-hu2024inferenceinterferencedisaggregatellm)) - \[meta]: - date: 06/12/2026 - tags: \["inference","disaggregation"] - later: true - [2510.01070![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2510.01070) ([Cywiński et al., 2025](#bib-cywiński2025elicitingsecretknowledgelanguage)) - \[meta]: - date: 06/12/2026 - tags: \["alignment","interpretability"] - later: true - [2508.19559![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2508.19559) ([R. Li et al., 2025](#bib-li2025tamingchaoscoordinatedautoscaling)) - \[meta]: - date: 06/12/2026 - tags: \["inference","autoscaling"] - later: true - [2401.09670![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2401.09670) ([of DistServe, 2024](#bib-distserve2024osdi)) - \[meta]: - date: 06/12/2026 - tags: \["inference","disaggregation"] - later: true - [2303.01037![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2303.01037) ([Y. Zhang et al., 2023](#bib-zhang2023googleusmscalingautomatic)) - \[meta]: - date: 06/12/2026 - tags: \["speech","models"] - later: true - [2301.13310![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2301.13310) ([Baykal et al., 2023](#bib-baykal2023alternatingupdatesefficienttransformers)) - \[meta]: - date: 06/12/2026 - tags: \["architecture","efficiency"] - later: true - [2411.07501![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2411.07501) ([Menghani et al., 2025](#bib-menghani2025laurellearnedaugmentedresidual)) - \[meta]: - date: 06/12/2026 - tags: \["architecture","residuals"] - later: true - [2506.06644![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2506.06644) ([You et al., 2025](#bib-you2025sparktransformerreactivatingsparsity)) - \[meta]: - date: 06/12/2026 - tags: \["sparsity","architecture"] - later: true - [2512.10942![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2512.10942) ([D. Chen et al., 2026](#bib-chen2026vljepajointembeddingpredictive)) - \[meta]: - date: 06/12/2026 - tags: \["vlm","representation"] - later: true - [1804.08838![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/1804.08838) ([C. Li et al., 2018](#bib-li2018measuringintrinsicdimensionobjective)) - \[meta]: - date: 06/12/2026 - tags: \["optimization","geometry"] - later: true - [2601.01206![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2601.01206) ([Elyasi et al., 2026](#bib-elyasi2026mentalgamepredictingpersonalityjobfitness)) - \[meta]: - date: 06/12/2026 - tags: \["software","psychology"] - later: true - [2412.19437![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2412.19437) ([DeepSeek-AI et al., 2025](#bib-deepseekai2025deepseekv3technicalreport)) - \[meta]: - date: 06/12/2026 - tags: \["models","architecture"] - later: true - [2406.06484![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2406.06484) ([S. Yang et al., 2025](#bib-yang2025parallelizinglineartransformersdelta)) - \[meta]: - date: 06/12/2026 - tags: \["linear attention","parallelism"] - later: true - [2312.06635![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2312.06635) ([S. Yang et al., 2024](#bib-yang2024gatedlinearattentiontransformers)) - \[meta]: - date: 06/12/2026 - tags: \["linear attention","training"] - later: true - [2601.03220![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2601.03220) ([Finzi et al., 2026](#bib-finzi2026entropyepiplexityrethinkinginformation)) - \[meta]: - date: 06/12/2026 - tags: \["information theory","intelligence"] - later: true - [2601.04603![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2601.04603) ([Cunningham et al., 2026](#bib-cunningham2026constitutionalclassifiersefficientproductiongrade)) - \[meta]: - date: 06/12/2026 - tags: \["safety","jailbreaks"] - later: true - [2601.00417![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2601.00417) ([Y. Zhang, Liu, et al., 2026](#bib-zhang2026deepdeltalearning)) - \[meta]: - date: 06/12/2026 - tags: \["architecture","attention"] - later: true - [2502.06643![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2502.06643) ([Go & Mahajan, 2025](#bib-go2025moetuneroptimizedmixtureexpert)) - \[meta]: - date: 06/12/2026 - tags: \["moe","inference"] - later: true - [2512.24601![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2512.24601) ([A. L. Zhang, Kraska, et al., 2026](#bib-zhang2026recursivelanguagemodels)) - \[meta]: - date: 06/12/2026 - tags: \["architecture","recursion"] - later: true - [2406.09264![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2406.09264) ([H. Shen et al., 2025](#bib-shen2025positionbidirectionalhumanaialignment)) - \[meta]: - date: 06/12/2026 - tags: \["alignment","human ai"] - later: true - [1905.01517![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/1905.01517) ([D. Sun et al., 2020](#bib-sun2020realdifferencesotcrdt)) - \[meta]: - date: 06/12/2026 - tags: \["crdt","collaboration"] - later: true - [math/0501118![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/math/0501118) ([Euler, 2008](#bib-euler2008observationscertaintheoremfermat)) - \[meta]: - date: 06/12/2026 - tags: \["math","number theory"] - later: true - [2503.08679![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2503.08679) ([Arcuschin et al., 2026](#bib-arcuschin2026chainofthoughtreasoningwildfaithful)) - \[meta]: - date: 06/12/2026 - tags: \["cot","reasoning"] - later: true - [2507.07101![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2507.07101) ([Marek et al., 2025](#bib-marek2025smallbatchsizetraining)) - \[meta]: - date: 06/12/2026 - tags: \["training","optimization"] - later: true - [2510.09665![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2510.09665) ([Y. Liu et al., 2025](#bib-liu2025lmcacheefficientkvcache)) - \[meta]: - date: 06/12/2026 - tags: \["kv","inference"] - later: true - [2601.05401![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2601.05401) ([Guo, Ledo, et al., 2026](#bib-guo2026protosamplingenablingfreeformconvergence)) - \[meta]: - date: 06/12/2026 - tags: \["generative ai","interface"] - later: true - [2511.09030![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2511.09030) ([Meyerson et al., 2025](#bib-meyerson2025solvingmillionstepllmtask)) - \[meta]: - date: 06/12/2026 - tags: \["agents","reliability"] - later: true - [2511.11907![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2511.11907) ([H. Zhang et al., 2025](#bib-zhang2025kvswapdiskawarekvcache)) - \[meta]: - date: 06/12/2026 - tags: \["kv","offloading"] - later: true - [1904.09751![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/1904.09751) ([Holtzman et al., 2020](#bib-holtzman2020curiouscaseneuraltext)) - \[meta]: - date: 06/12/2026 - tags: \["decoding","language models"] - later: true - [2512.12167![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2512.12167) ([Gelberg et al., 2025](#bib-gelberg2025extendingcontextpretrainedllms)) - \[meta]: - date: 06/12/2026 - tags: \["long-context","positional embeddings"] - later: true - [2506.06266![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2506.06266) ([Eyuboglu et al., 2025](#bib-eyuboglu2025cartridgeslightweightgeneralpurposelong)) - \[meta]: - date: 06/12/2026 - tags: \["long context","representation"] - later: true - [2510.12402![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2510.12402) ([L. Chen et al., 2026](#bib-chen2026cautiousweightdecay)) - \[meta]: - date: 06/12/2026 - tags: \["optimizer","training"] - later: true - [2601.07372![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2601.07372) ([Cheng, Zeng, et al., 2026](#bib-cheng2026conditionalmemoryscalablelookup)) - \[meta]: - date: 06/12/2026 - tags: \["memory","sparsity"] - later: true - [2505.01618![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2505.01618) ([Dey et al., 2026](#bib-dey2026dontlazycompletepenables)) - \[meta]: - date: 06/04/2026 - tags: \["architecture","muon"] - [2512.12087![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2512.12087) ([J. Yuan et al., 2026](#bib-yuan2026blasstdynamicblockedattention)) - \[meta]: - date: 06/04/2026 - tags: \["kv","systems"] - [2404.14469![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2404.14469) ([Y. Li et al., 2024](#bib-li2024snapkvllmknowslooking)) - \[meta]: - date: 06/02/2026 - tags: \["kv","compression"] - [2604.03515![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2604.03515) ([Rombaut, 2026](#bib-rombaut2026insidescaffoldsourcecodetaxonomy)) - \[meta]: - date: 05/25/2026 - tags: \["design","llms"] - [2410.06205![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2410.06205) ([Barbero, Vitvitskyi, et al., 2025](#bib-barbero2025roundroundgomakes)) - \[meta]: - date: 05/23/2026 - tags: \["algorithm","interpretability"] - [2605.12290![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2605.12290) ([Herring et al., 2026](#bib-herring2026targetedneuronmodulationcontrastive)) - \[meta]: - date: 05/19/2026 - tags: \["interpretability"] - later: true - [2604.18556![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2604.18556) ([Dadgarnia et al., 2026](#bib-dadgarnia2026gsqhighlyaccuratelowprecisionscalar)) - \[meta]: - date: 05/19/2026 - tags: \["quantization"] - [2605.02105![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2605.02105) ([Watts et al., 2026](#bib-watts2026sharpnessawarepretrainingmitigatescatastrophic)) - \[meta]: - date: 05/12/2026 - tags: \["optimizer"] - — Adaptive Parallel Reasoning: The Next Paradigm in Efficient Inference Scaling - \[meta]: - date: 05/12/2026 - tags: \["inference","ttt"] - — DeepSeek V4 Technical Reports - \[meta]: - date: 04/28/2026 - tags: \["infrastructure","models"] - — Artful Mathematics: The Heritage of M.C. Escher - \[meta]: - date: 03/24/2026 - tags: \["complex analysis","art"] - — Streaming Transformer for Hardware Efficient Voice Trigger Detection and False Trigger Mitigation - \[meta]: - date: 03/12/2026 - tags: \["continual learning","models"] - later: true - — CodeAct: Your LLM Agent Acts Better when Generating Code - \[meta]: - date: 03/12/2026 - tags: \["agentic","codegen","rl"] - later: true - — A Review of Parallel Implementations for the Smith–Waterman Algorithm - \[meta]: - date: 03/12/2026 - tags: \["path","optimization","graph theory"] - later: true - — The widespread adoption of large language model-assisted writing across society - \[meta]: - date: 03/09/2026 - tags: \["writing"] - later: true - [2408.07666![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2408.07666) ([E. Yang et al., 2024](#bib-arxiv-240807666)) - \[meta]: - date: 02/23/2026 - tags: \["distillation"] - later: true - [2601.14175![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2601.14175) ([Raju & Netrapalli, 2026](#bib-arxiv-260114175)) - \[meta]: - date: 01/26/2026 - tags: \["errors","interpretability"] - [1906.07983![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/1906.07983) ([Dombrowski et al., 2019](#bib-arxiv-190607983)) - \[meta]: - date: 01/19/2026 - tags: \["interpretability","geometry","manifold"] - later: true - [2512.23966![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2512.23966) ([C. Zhang et al., 2025](#bib-arxiv-251223966)) - \[meta]: - date: 01/15/2026 - tags: \["scaling","long context"] - — Objective Epistemic Consequentialism - \[meta]: - date: 01/15/2026 - tags: \["continential","metaethics"] - later: true - [2512.23675![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2512.23675) ([Tandon et al., 2025](#bib-arxiv-251223675)) - \[meta]: - date: 01/14/2026 - tags: \["scaling","test time compute"] - [2206.13353![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2206.13353) ([Carlsmith, 2024](#bib-carlsmith2024powerseekingaiexistentialrisk)) - \[meta]: - date: 01/08/2026 - tags: \["alignment"] - later: true - [2101.03961![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2101.03961) ([Fedus et al., 2022](#bib-fedus2022switchtransformersscalingtrillion)) - \[meta]: - date: 01/08/2026 - tags: \["moe"] - pinned: true - - [2512.15712![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2512.15712) ([Huang et al., 2025](#bib-huang2025predictiveconceptdecoderstraining)) - \[meta]: - date: 12/21/2025 - tags: \["interpretability"] - later: true - [2512.15674![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2512.15674) ([Karvonen et al., 2025](#bib-karvonen2025activationoraclestrainingevaluating)) - \[meta]: - date: 12/21/2025 - tags: \["interpretability"] - later: true - [1609.09106![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/1609.09106) ([Ha et al., 2016](#bib-ha2016hypernetworks)) - \[meta]: - date: 12/20/2025 - tags: \["network","highway network"] - [2501.19393![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2501.19393) ([Muennighoff et al., 2025](#bib-muennighoff2025s1simpletesttimescaling)) - \[meta]: - date: 12/17/2025 - tags: \["scaling","mamba","hybrid attention"] - [2307.08691![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2307.08691) ([Dao, 2023](#bib-dao2023flashattention2fasterattentionbetter)) - \[meta]: - date: 12/17/2025 - tags: \["fa2","optimization","kernel"] - [2507.19427![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2507.19427) ([StepFun et al., 2025](#bib-stepfun2025step3largeaffordablemodelsystem)) - \[meta]: - date: 12/17/2025 - tags: \["models","disaggregation","architecture"] - [2112.05682![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2112.05682) ([Rabe & Staats, 2022](#bib-rabe2022selfattentiondoesneedon2)) - \[meta]: - date: 12/17/2025 - tags: \["optimization","memory","attention"] - [1701.06538![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/1701.06538) ([Shazeer et al., 2017](#bib-shazeer2017outrageouslylargeneuralnetworks)) - \[meta]: - date: 12/17/2025 - tags: \["moe"] - see also: [MoE](/thoughts/MoE) - [1904.10509![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/1904.10509) ([Child et al., 2019](#bib-child2019generatinglongsequencessparse)) - \[meta]: - date: 12/17/2025 - tags: \["long context","inference"] - [1911.02150![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/1911.02150) ([Shazeer, 2019](#bib-shazeer2019fasttransformerdecodingwritehead)) - \[meta]: - date: 12/17/2025 - tags: \["speculative decoding"] - — Sectional and Ricci Curvature, Variations of Arc-Length and Diameter of Riemannian Manifold - \[meta]: - date: 12/17/2025 - tags: \["manifold","topology"] - later: true - — Looking Out from the Isolator: David’s Perception of the World - \[meta]: - date: 12/16/2025 - tags: \["mind","dualism","psychology"] - — Nemotron 3 - \[meta]: - date: 12/16/2025 - tags: \["architecture","byte level"] - — The Complexity of Domino Tiling Problems - \[meta]: - date: 12/16/2025 - tags: \["cs","optimization"] - — Packing Trominoes is NP-Complete, [P-Complete](/tags/P-Complete) and ASP-Complete - \[meta]: - date: 12/15/2025 - tags: \["cs","bin packing"] - later: true - [2505.17120![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2505.17120) ([Plunkett et al., 2025](#bib-plunkett2025selfinterpretabilityllmscomplexinternal)) - \[meta]: - date: 12/14/2025 - tags: \["interpretability","recursion"] - later: true - [2505.15813![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2505.15813) ([M. Yu et al., 2025](#bib-yu2025metalearningincontexttransformermodel)) - \[meta]: - date: 12/14/2025 - tags: \["in-context learning","meta learning"] - later: true - [2206.00759![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2206.00759) ([Wäldchen et al., 2024](#bib-wäldchen2024interpretabilityguaranteesmerlinarthurclassifiers)) - \[meta]: - date: 12/14/2025 - tags: \["interpretability","theory","classifier"] - later: true - [2507.05526![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2507.05526) ([Dhir et al., 2025](#bib-dhir2025estimatinginterventionaldistributionsuncertain)) - \[meta]: - date: 12/14/2025 - tags: \["interpretability","causal reasoning"] - later: true - [2505.12075![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2505.12075) ([Davidson et al., 2025](#bib-davidson2025differentpromptingmethodsyield)) - \[meta]: - date: 12/14/2025 - tags: \["in context learning","behaviour"] - later: true - [2411.01783![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2411.01783) ([A. Yang et al., 2025](#bib-yang2025contextparallelismscalablemilliontoken)) - \[meta]: - date: 12/14/2025 - tags: \["inference","context parallelism"] - later: true - [1803.04585![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/1803.04585) ([Manheim & Garrabrant, 2019](#bib-manheim2019categorizingvariantsgoodhartslaw)) - \[meta]: - date: 12/14/2025 - tags: \["law","cs","economics"] - later: true - Goodhart’s Law: failure modes for overoptimizastion of systems on the basis of metrics, to determine whether or not further optimization is either ineffective or _harmful_ - Variants of Goodhart’s Law: - Regressional Goodhart: select for true goal and the difference between the proxy and the goal M=G+\operatonamenormal(μ,σ2) - Extremal Goodhart: worlds in which proxy takes an extreme value _may be very different_ from the ordinary worlds in which the relationship between the proxy and the goal was observed. - Model insufficiency: M=G(si​)+G′(si​) - Change in Regime: G={M+x,M+y,​where M≤awhere M>a​ - Causal Goodhart: causal path between proxy and goal is indirect, intervening can change the relationship between the measure and the proxy. ![](./thoughts/images/causal-goodhart.webp) - Shared Cause Intervention - Intermediary Intervention - Metric Manipulation - [2310.04363![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2310.04363) ([E. J. Hu et al., 2024](#bib-hu2024amortizingintractableinferencelarge)) - \[meta]: - date: 12/14/2025 - tags: \["inference","scaling"] - [2406.11717![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2406.11717) ([Arditi et al., 2024](#bib-arditi2024refusallanguagemodelsmediated)) - \[meta]: - date: 12/14/2025 - tags: \["rejection sampling","interpretability"] - later: true - [2507.08218![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2507.08218) ([A. Wang et al., 2025](#bib-wang2025simplemechanisticexplanationsoutofcontext)) - \[meta]: - date: 12/14/2025 - tags: \["interpretability","OOD"] - later: true - [2303.05514![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2303.05514) ([Rudolph, 2023](#bib-rudolph2023terryvsairound)) - \[meta]: - date: 12/14/2025 - tags: \["physics","scenarios"] - later: true - [1810.09113![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/1810.09113) ([Nielsen & Nock, 2018](#bib-nielsen2018bregmanchorddivergence)) - \[meta]: - date: 12/14/2025 - tags: \["theory","distance"] - later: true - [2212.07677![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2212.07677) ([von Oswald et al., 2023](#bib-vonoswald2023transformerslearnincontextgradient)) - \[meta]: - date: 12/14/2025 - tags: \["training","double descent"] - pinned: true - [2506.10892![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2506.10892) ([Sahoo et al., 2025](#bib-sahoo2025diffusionduality)) - \[meta]: - date: 12/14/2025 - tags: \["diffusion","theory"] - later: true - [2412.06769![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2412.06769) ([Hao et al., 2025](#bib-hao2025traininglargelanguagemodels)) - \[meta]: - date: 12/14/2025 - tags: \["continual learning"] - later: true - [1704.01444![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/1704.01444) ([Radford et al., 2017](#bib-radford2017learninggeneratereviewsdiscovering)) - \[meta]: - date: 12/14/2025 - tags: \["representation"] - later: true - [2510.05056![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2510.05056) ([Ross et al., 2025](#bib-ross2025modelingstudentlearning38)) - \[meta]: - date: 12/14/2025 - tags: \["representation","interpretability"] - later: true - [2507.12638![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2507.12638) ([Ward et al., 2025](#bib-ward2025reasoningfinetuningrepurposeslatentrepresentations)) - \[meta]: - date: 12/14/2025 - tags: \["distilation","fine tuning"] - [2508.21258![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2508.21258) ([Jafari et al., 2025](#bib-jafari2025relpfaithfulefficientcircuit)) - \[meta]: - date: 12/14/2025 - tags: \["interpretability"] - [2511.04694![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2511.04694) ([Z. Zheng et al., 2025](#bib-zheng2025reasoninginstructionladdercontrollable)) - \[meta]: - date: 12/14/2025 - tags: \["cot","reasoning","anchors"] - [2512.02556![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2512.02556) ([DeepSeek-AI, 2025](#bib-deepseekai2025deepseekv32)) - \[meta]: - date: 12/14/2025 - tags: \["models"] - [2502.11089![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2502.11089) ([J. Yuan et al., 2025](#bib-yuan2025nativesparseattentionhardwarealigned)) - \[meta]: - date: 12/14/2025 - tags: \["inference"] - see also [notes](/lectures/430/notes#native-sparse-attention) - [2501.00070![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2501.00070) ([C. F. Park et al., 2025](#bib-park2025iclrincontextlearningrepresentations)) - \[meta]: - date: 12/14/2025 - tags: \["interpretability"] - later: true - [2310.16410![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2310.16410) ([Schut et al., 2023](#bib-schut2023bridginghumanaiknowledgegap)) - \[meta]: - date: 12/14/2025 - tags: \["distillation"] - [2512.09742![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2512.09742) ([Betley et al., 2025](#bib-betley2025weirdgeneralizationinductivebackdoors)) - \[meta]: - date: 12/13/2025 - tags: \["alignment","contrastive"] - Effects of contrastive steering in actions, based on [SAE](/thoughts/sparse-autoencoder) features - What does this say about grokking in general? - see also: - [2511.21689![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2511.21689) ([Su et al., 2025](#bib-su2025toolorchestraelevatingintelligenceefficient)) - \[meta]: - date: 12/13/2025 - tags: \["tools","benchmark"] - later: true - — Forecasting the progression of human civilization on the Kardashev Scale through 2060 with a machine learning approach - \[meta]: - date: 12/11/2025 - tags: \["progress"] - [2407.08516![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2407.08516) ([Xiong et al., 2024](#bib-xiong2024convergingparadigmssynergysymbolic)) - \[meta]: - date: 12/08/2025 - tags: \["emergent","paradigm"] - [2410.22264![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2410.22264) ([Block et al., 2025](#bib-block2025provablemetalearninglowrankadaptations)) - \[meta]: - date: 12/01/2025 - tags: \["lora","meta learning"] - [2511.20102![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2511.20102) ([Z. Shen et al., 2025](#bib-shen2025ssasparsesparseattention)) - \[meta]: - date: 12/01/2025 - tags: \["attention"] - later: true - [2109.13916![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2109.13916) ([Hendrycks et al., 2022](#bib-hendrycks2022unsolvedproblemsmlsafety)) - \[meta]: - date: 12/01/2025 - tags: \["interpretability"] - [DeepSeek-V3.2.pdf](/thoughts/pdfs/DeepSeek-V3.2.pdf) — DeepSeek V3.2 Technical Report - \[meta]: - date: 12/01/2025 - tags: \["models","attention","dsa"] - [2406.02069![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2406.02069) ([Cai et al., 2025](#bib-cai2025pyramidkvdynamickvcache)) - \[meta]: - date: 11/27/2025 - tags: \["kv","inference"] - [2511.21631![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2511.21631) ([S. Bai et al., 2025](#bib-bai2025qwen3vltechnicalreport)) - \[meta]: - date: 11/27/2025 - tags: \["vlm","rl","moe"] - Hmm, scaling interleaved MRoPE - [2412.06769![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2412.06769) ([Hao et al., 2025](#bib-hao2025traininglargelanguagemodels)) - \[meta]: - date: 11/26/2025 - tags: \["cot","llm"] - — Raft consensus algorithm \[\*\*] - \[meta]: - date: 11/23/2025 - socials: {"extended":"https\://raft.github.io/raft.pdf"} - tags: \["crdt","distributed system"] - pinned: true - highlighted: true - [2301.04709![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2301.04709) ([Geiger et al., 2025](#bib-geiger2025causalabstraction)) - \[meta]: - date: 11/15/2025 - tags: \["interpretability"] - [2511.05408![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2511.05408) ([Fierro & Roger, 2025](#bib-fierro2025steeringlanguagemodelsweight)) - \[meta]: - date: 11/14/2025 - tags: \["interpretability"] - [2507.07120![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2507.07120) ([Bhatia et al., 2025](#bib-bhatia2025helixparallelismrethinkingsharding)) - \[meta]: - date: 11/11/2025 - tags: \["parallelism"] - pinned: true - [2509.23202![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2509.23202) ([Egiazarian et al., 2025](#bib-egiazarian2025bridginggappromiseperformance)) - \[meta]: - date: 11/09/2025 - tags: \["inference"] - [2509.25149![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2509.25149) ([NVIDIA et al., 2025](#bib-nvidia2025pretraininglargelanguagemodels)) - \[meta]: - date: 11/09/2025 - tags: \["training","nvfp4"] - [2309.00667![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2309.00667) ([Berglund et al., 2023](#bib-berglund2023takencontextmeasuringsituational)) - \[meta]: - date: 11/09/2025 - tags: \["alignment"] - [interpretability](/tags/interpretability) on LLM’s situational awareness - [2510.20075![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2510.20075) ([Norelli & Bronstein, 2025](#bib-norelli2025llmshidetexttext)) - \[meta]: - date: 11/09/2025 - tags: \["interpretability"] - [2412.01784![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2412.01784) ([Tice et al., 2024](#bib-tice2024noiseinjectionrevealshidden)) - \[meta]: - date: 11/05/2025 - tags: \["interpretability"] - [2404.03715![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2404.03715) ([Rosset et al., 2024](#bib-rosset2024directnashoptimizationteaching)) - \[meta]: - date: 10/28/2025 - tags: \["rl","nash optimization"] - [2510.13786![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2510.13786) ([Khatri et al., 2025](#bib-khatri2025artscalingreinforcementlearning)) - \[meta]: - date: 10/27/2025 - tags: \["rl"] - later: true - [2504.13181![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2504.13181) ([Bolya et al., 2025](#bib-bolya2025perceptionencoderbestvisual)) - \[meta]: - date: 10/27/2025 - tags: \["perception","embeddings"] - [2510.18234![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2510.18234) ([Wei et al., 2025](#bib-wei2025deepseekocrcontextsopticalcompression)) - \[meta]: - date: 10/27/2025 - tags: \["ocr"] - [2405.15319![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2405.15319) ([W. Du et al., 2024](#bib-du2024stackingtransformerscloserlook)) - \[meta]: - date: 10/27/2025 - tags: \["pretraining"] - [2310.07177![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2310.07177) ([X. Liu et al., 2024](#bib-liu2024onlinespeculativedecoding)) - \[meta]: - date: 10/24/2025 - tags: \["inference"] - [2501.16975![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2501.16975) ([H. Huang et al., 2025](#bib-huang2025overtokenizedtransformervocabularygenerally)) - \[meta]: - date: 10/16/2025 - tags: \["pretraining","bytes","representations"] - [2008.00044![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2008.00044) ([L. Li & Nikolov, 2020](#bib-li2020computationalcomplexitylineardiscrepancy)) - \[meta]: - date: 10/16/2025 - tags: \["tcs","linear discrepancy"] - [2412.06464![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2412.06464) ([S. Yang, Kautz, et al., 2025](#bib-yang2025gateddeltanetworksimproving)) - \[meta]: - date: 10/16/2025 - tags: \["hybrid attention","gated delta"] - [2506.20790![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2506.20790) ([Bushnaq et al., 2025](#bib-bushnaq2025stochasticparameterdecomposition)) - \[meta]: - date: 10/09/2025 - tags: \["interpretability"] - [2505.12049![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2505.12049) ([Shakerinava et al., 2025](#bib-shakerinava2025scalarrewardsaxiomaticframework)) - \[meta]: - date: 10/08/2025 - tags: \["rl"] - [2410.01104![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2410.01104) ([Veličković et al., 2025](#bib-veličković2025softmaxforsharpsize)) - \[meta]: - date: 10/06/2025 - tags: \["optimizer","softmax"] - — Mind, brains, programs - \[meta]: - date: 10/06/2025 - tags: \["brain","intentionality"] - [1912.02279![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/1912.02279) ([B. Chen et al., 2019](#bib-chen2019191202279)) - \[meta]: - date: 10/06/2025 - tags: \["cnn","vision"] - [2501.15225![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2501.15225) ([Lee et al., 2025](#bib-lee2025sealscalingemphasizeattention)) - \[meta]: - date: 10/06/2025 - tags: \["inference","long context"] - [2401.08383![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2401.08383) ([Yao et al., 2024](#bib-yao2024exploitinginterlayerexpertaffinity)) - \[meta]: - date: 10/06/2025 - tags: \["inference","moe"] - Inter-Layer Expert Affinity - [2305.15054![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2305.15054) ([Stolfo et al., 2023](#bib-stolfo2023mechanisticinterpretationarithmeticreasoning)) - \[meta]: - date: 10/06/2025 - tags: \["interpretability"] - [2509.21519![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2509.21519v3) ([Tian, 2025](#bib-tian2025provablescalinglawsfeature)) - \[meta]: - date: 10/06/2025 - tags: \["scaling law","interpretability"] - [1607.06450![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/1607.06450) ([Ba et al., 2016](#bib-ba2016layernormalization)) - \[meta]: - date: 10/05/2025 - tags: \["models"] - pinned: true - [2507.21509![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2507.21509) ([R. Chen et al., 2025](#bib-chen2025personavectorsmonitoringcontrolling)) - \[meta]: - date: 10/05/2025 - tags: \["interpretability","engineering"] - persona vectors, i.e linear probes to intercept models’ behaviour. - [2412.14093![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2412.14093) ([Greenblatt et al., 2024](#bib-greenblatt2024alignmentfakinglargelanguage)) - \[meta]: - date: 10/05/2025 - tags: \["alignment"] - - [2506.18032![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2506.18032) ([Sheshadri et al., 2025](#bib-sheshadri2025languagemodelsfakealignment)) - \[meta]: - date: 10/03/2025 - tags: \["alignment"] - Rater Sycophancy - Instrumental Goal Guarding - Low coherence - Terminal goal guarding - F.4: Another hypothesis that could indicate instrumental goal guarding is model behavior that tries to minimize future harm that other AIs might cause. These models would comply more if they believe their outputs will only be used to train a different model to be more harmful, with a training method such as Direct Preference Optimization (DPO). [2401.08671![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2401.08671) ([Holmes et al., 2024](#bib-holmes2024deepspeedfastgenhighthroughputtextgeneration)) - [2506.06941![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2506.06941) ([Shojaee et al., 2025](#bib-shojaee2025illusionthinkingunderstandingstrengths)) - \[meta]: - date: 10/03/2025 - tags: \["rl","reasoning"] - The argument being certain reasoning capabilities collapses after certain complexities. - [Critique of this paper](/thoughts/papers/illusion_of_illusion_of_thinking.pdf) illustrates the shortcomings of careful experiments calibrations, leading to a pre-empted conclusions with regards to models’ reasoning capabilities. - [2505.24832![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2505.24832) ([Morris et al., 2025](#bib-morris2025languagemodelsmemorize)) - \[meta]: - date: 10/03/2025 - tags: \["emergent properties"] - in terms of how much model memorises under compressions for a specific types of generalizations. - OOD study. - [2405.07987![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2405.07987) ([Huh et al., 2024](#bib-huh2024platonicrepresentationhypothesis)) - \[meta]: - date: 10/03/2025 - tags: \["hypothesis"] - pinned: true - All AI models converges to a global minima, among multiple domains - cf [Plato](/thoughts/Plato)‘s concepts of an _ideal reality_ - [2501.16007![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2501.16007) ([Ong et al., 2025](#bib-ong2025toploclocalitysensitivehashing)) - \[meta]: - date: 10/03/2025 - tags: \["distributed","verifier","cryptographic"] - [2504.02922![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2504.02922) ([Minder et al., 2025](#bib-minder2025overcomingsparsityartifactscrosscoders)) - \[meta]: - date: 10/03/2025 - tags: \["interpretability"] - [LW![LessWrong](/static/favicons/lesswrong.avif)](https://www.lesswrong.com/posts/xmpauEXEerzYcJKNm/what-we-learned-trying-to-diff-base-and-chat-models-and-why) - see also: [sparse crosscoders](/thoughts/sparse-crosscoders) - [2505.05315![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2505.05315) ([Y. Xu et al., 2025](#bib-xu2025scalablechainthoughtselastic)) - \[meta]: - date: 10/03/2025 - tags: \["rl","reasoning"] - [SalesforceAIResearch/Elastic-Reasoning](https://github.com/SalesforceAIResearch/Elastic-Reasoning) - [2412.14093![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2412.14093) ([Greenblatt et al., 2024](#bib-greenblatt2024alignmentfakinglargelanguage)) - \[meta]: - date: 10/03/2025 - tags: \["alignment"] - [2403.09629![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2403.09629) ([Zelikman et al., 2024](#bib-zelikman2024quietstarlanguagemodelsteach)) - \[meta]: - date: 10/03/2025 - tags: \["rl","reasoning"] - [1910.02054![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/1910.02054) ([Rajbhandari et al., 2020](#bib-rajbhandari2020zeromemoryoptimizationstraining)) - \[meta]: - date: 10/03/2025 - tags: \["optimizer"] - [2506.19143![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2506.19143) ([Bogdan et al., 2025](#bib-bogdan2025thoughtanchorsllmreasoning)) - \[meta]: - date: 08/05/2025 - tags: \["interpretability"] - which thought trace matters? - [2503.18292![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2503.18292) ([C. Zhang, Du, et al., 2025](#bib-zhang2025jengaeffectivememorymanagement)) - \[meta]: - date: 10/03/2025 - tags: \["system","inference"] - [2310.05209![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2310.05209) ([X. Liu, Yan, et al., 2024](#bib-liu2024scalinglawsropebasedextrapolation)) - \[meta]: - date: 10/03/2025 - tags: \["inference","law"] - [2310.10537![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2310.10537) ([Rouhani et al., 2023](#bib-rouhani2023microscalingdataformatsdeep)) - \[meta]: - date: 10/03/2025 - tags: \["data format","quantization"] - mxfp4 is pretty bad, better to use nvfp4 instead. - [2104.09864![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2104.09864) ([J. Su et al., 2023](#bib-su2023roformerenhancedtransformerrotary)) - \[meta]: - date: 10/03/2025 - tags: \["inference"] - [RoPE](/thoughts/RoPE) - more popular methods: [2309.00071![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2309.00071) ([Peng et al., 2023](#bib-peng2023yarnefficientcontextwindow)) - [2508.06471![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2508.06471) ([5 Team et al., 2025](#bib-5team2025glm45agenticreasoningcoding)) - \[meta]: - date: 10/03/2025 - tags: \["model","llm"] - [1112.0510![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/1112.0510) ([S. Janson, 2011](#bib-janson2011simplygeneratedtreesconditioned)) - \[meta]: - date: 08/12/2025 - tags: \["tree","cs"] - [2504.02263![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2504.02263) ([R. Zhu et al., 2025](#bib-zhu2025megascaleinferservingmixtureofexpertsscale)) - \[meta]: - date: 10/03/2025 - tags: \["inference","moe","disaggregation"] - [2412.01784![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2412.01784) ([Tice et al., 2024](#bib-tice2024noiseinjectionrevealshidden)) - \[meta]: - date: 10/04/2025 - tags: \["interpretability"] - [2510.00184![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2510.00184) ([X. Bai et al., 2025](#bib-bai2025canttransformerslearnmultiplication)) - \[meta]: - date: 10/06/2025 - tags: \["arithmetics","llm"] - [2509.26537![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2509.26537) ([Adam et al., 2025](#bib-adam2025losskernelgeometricprobe)) - \[meta]: - date: 10/07/2025 - tags: \["kernel","interpretability"] - [2112.01898![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2112.01898) ([Charton, 2022](#bib-charton2022linearalgebratransformers)) - \[meta]: - date: 10/09/2025 - tags: \["math","llm"] - see also [linalg](/tags/math/linalg) - [2211.00170![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2211.00170) ([Charton, 2022b](#bib-charton2022mathtransformerdoing)) - \[meta]: - date: 10/09/2025 - tags: \["math","interpretability"] - [2012.14913![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2012.14913) ([Geva et al., 2021](#bib-geva2021transformerfeedforwardlayerskeyvalue)) - \[meta]: - date: 10/09/2025 - tags: \["inference","interpretability"] - [2505.21487![arXiv](/static/favicons/arxiv.avif)](https://arxiv.org/abs/2505.21487) ([Zadouri et al., 2025](#bib-zadouri2025hardwareefficientattentionfastdecoding)) - \[meta]: - date: 10/11/2025 - tags: \["system","inference"]