---
date: '2024-02-07'
description: structured state space sequence models
id: state-space models
modified: 2026-06-05 15:08:06 GMT-04:00
tags:
  - ml
title: state-space models
created: '2024-02-07'
published: '2024-02-07'
pageLayout: default
slug: thoughts/state-space-models
permalink: https://aarnphm.xyz/thoughts/state-space-models.md
generator:
  quartz: v4.6.0
  hostedProvider: Cloudflare
  baseUrl: aarnphm.xyz
full: https://aarnphm.xyz/llms-full.txt
---
[Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://arxiv.org/abs/2312.00752) \[@gu2024mambalineartimesequencemodeling\] 

<https://github.com/state-spaces/mamba>, <https://github.com/state-spaces/s4>

Idea: to use state space representations in [[thoughts/university/twenty-three-twenty-four/sfwr-3dx4|control theory]] to for sequences modeling.

## Mamba

[Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://arxiv.org/abs/2312.00752) \[@gu2024mambalineartimesequencemodeling\] [Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality](https://arxiv.org/abs/2405.21060) \[@dao2024transformersssmsgeneralizedmodels\] [Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality](https://arxiv.org/abs/2405.21060)

State-space duality (SSD): SSM + attentions layers (SMA, or structured masked [[thoughts/Attention|attention]])