---
date: '2024-12-14'
description: techniques preventing overfitting in over-parameterized models through penalty terms, early stopping, noise, and dropout.
id: regularization
modified: 2026-06-05 15:08:28 GMT-04:00
tags:
  - ml
title: regularization
created: '2024-12-14'
published: '2024-12-14'
pageLayout: default
slug: thoughts/regularization
permalink: https://aarnphm.xyz/thoughts/regularization.md
generator:
  quartz: v4.6.0
  hostedProvider: Cloudflare
  baseUrl: aarnphm.xyz
full: https://aarnphm.xyz/llms-full.txt
---
usually prone to overfitting given they are often over-parameterized

1. We can usually add regularization terms to the objective functions
2. Early stopping
3. Adding noise
4. structural regularization, via adding dropout

## dropout

a case of _structural regularization_

a technique of randomly drop each node with probability $p$