zanewalker
f3453cb0fc
- Refactor Redis backend connection handling and pool management - Update algorithm implementations with improved type annotations - Enhance config loader validation with stricter Pydantic schemas - Improve decorator and middleware error handling - Expand example scripts with better docstrings and usage patterns - Add new 00_basic_usage.py example for quick start - Reorganize examples directory structure - Fix type annotation inconsistencies across core modules - Update dependencies in pyproject.toml
313 lines
8.2 KiB
ReStructuredText
313 lines
8.2 KiB
ReStructuredText
Storage Backends
|
|
================
|
|
|
|
FastAPI Traffic needs somewhere to store rate limit state — how many requests each
|
|
client has made, when their window resets, and so on. That's what backends are for.
|
|
|
|
You have three options, each suited to different deployment scenarios.
|
|
|
|
Choosing a Backend
|
|
------------------
|
|
|
|
Here's the quick guide:
|
|
|
|
.. list-table::
|
|
:header-rows: 1
|
|
:widths: 20 30 50
|
|
|
|
* - Backend
|
|
- Use When
|
|
- Limitations
|
|
* - **Memory**
|
|
- Development, single-process apps
|
|
- Lost on restart, doesn't share across processes
|
|
* - **SQLite**
|
|
- Single-node production
|
|
- Doesn't share across machines
|
|
* - **Redis**
|
|
- Distributed systems, multiple nodes
|
|
- Requires Redis infrastructure
|
|
|
|
Memory Backend
|
|
--------------
|
|
|
|
The default backend. It stores everything in memory using a dictionary with LRU
|
|
eviction and automatic TTL cleanup.
|
|
|
|
.. code-block:: python
|
|
|
|
from fastapi_traffic import MemoryBackend, RateLimiter
|
|
from fastapi_traffic.core.limiter import set_limiter
|
|
|
|
# This is what happens by default, but you can configure it:
|
|
backend = MemoryBackend(
|
|
max_size=10000, # Maximum number of keys to store
|
|
cleanup_interval=60, # How often to clean expired entries (seconds)
|
|
)
|
|
limiter = RateLimiter(backend)
|
|
set_limiter(limiter)
|
|
|
|
**When to use it:**
|
|
|
|
- Local development
|
|
- Single-process applications
|
|
- Testing and CI/CD pipelines
|
|
- When you don't need persistence
|
|
|
|
**Limitations:**
|
|
|
|
- State is lost when the process restarts
|
|
- Doesn't work with multiple workers (each worker has its own memory)
|
|
- Not suitable for ``gunicorn`` with multiple workers or Kubernetes pods
|
|
|
|
**Memory management:**
|
|
|
|
The backend automatically evicts old entries when it hits ``max_size``. It uses
|
|
LRU (Least Recently Used) eviction, so inactive clients get cleaned up first.
|
|
|
|
SQLite Backend
|
|
--------------
|
|
|
|
For single-node production deployments where you need persistence. Rate limits
|
|
survive restarts and work across multiple processes on the same machine.
|
|
|
|
.. code-block:: python
|
|
|
|
from fastapi_traffic import SQLiteBackend, RateLimiter
|
|
from fastapi_traffic.core.limiter import set_limiter
|
|
|
|
backend = SQLiteBackend(
|
|
"rate_limits.db", # Database file path
|
|
cleanup_interval=300, # Clean expired entries every 5 minutes
|
|
)
|
|
limiter = RateLimiter(backend)
|
|
set_limiter(limiter)
|
|
|
|
@app.on_event("startup")
|
|
async def startup():
|
|
await limiter.initialize()
|
|
|
|
@app.on_event("shutdown")
|
|
async def shutdown():
|
|
await limiter.close()
|
|
|
|
**When to use it:**
|
|
|
|
- Single-server deployments
|
|
- When you need rate limits to survive restarts
|
|
- Multiple workers on the same machine (gunicorn, uvicorn with workers)
|
|
- When Redis is overkill for your use case
|
|
|
|
**Performance notes:**
|
|
|
|
- Uses WAL (Write-Ahead Logging) mode for better concurrent performance
|
|
- Connection pooling is handled automatically
|
|
- Writes are batched where possible
|
|
|
|
**File location:**
|
|
|
|
Put the database file somewhere persistent. For Docker deployments, mount a volume:
|
|
|
|
.. code-block:: yaml
|
|
|
|
# docker-compose.yml
|
|
services:
|
|
api:
|
|
volumes:
|
|
- ./data:/app/data
|
|
environment:
|
|
- RATE_LIMIT_DB=/app/data/rate_limits.db
|
|
|
|
Redis Backend
|
|
-------------
|
|
|
|
The go-to choice for distributed systems. All your application instances share
|
|
the same rate limit state.
|
|
|
|
.. code-block:: python
|
|
|
|
from fastapi_traffic import RateLimiter
|
|
from fastapi_traffic.backends.redis import RedisBackend
|
|
from fastapi_traffic.core.limiter import set_limiter
|
|
|
|
@app.on_event("startup")
|
|
async def startup():
|
|
backend = await RedisBackend.from_url(
|
|
"redis://localhost:6379/0",
|
|
key_prefix="myapp:ratelimit", # Optional prefix for all keys
|
|
)
|
|
limiter = RateLimiter(backend)
|
|
set_limiter(limiter)
|
|
await limiter.initialize()
|
|
|
|
@app.on_event("shutdown")
|
|
async def shutdown():
|
|
await limiter.close()
|
|
|
|
**When to use it:**
|
|
|
|
- Multiple application instances (Kubernetes, load-balanced servers)
|
|
- When you need rate limits shared across your entire infrastructure
|
|
- High-availability requirements
|
|
|
|
**Connection options:**
|
|
|
|
.. code-block:: python
|
|
|
|
# Simple connection
|
|
backend = await RedisBackend.from_url("redis://localhost:6379/0")
|
|
|
|
# With authentication
|
|
backend = await RedisBackend.from_url("redis://:password@localhost:6379/0")
|
|
|
|
# Redis Sentinel for HA
|
|
backend = await RedisBackend.from_url(
|
|
"redis://sentinel1:26379/0",
|
|
sentinel_master="mymaster",
|
|
)
|
|
|
|
# Redis Cluster
|
|
backend = await RedisBackend.from_url("redis://node1:6379,node2:6379,node3:6379/0")
|
|
|
|
**Atomic operations:**
|
|
|
|
The Redis backend uses Lua scripts to ensure atomic operations. This means rate
|
|
limit checks are accurate even under high concurrency — no race conditions.
|
|
|
|
**Key expiration:**
|
|
|
|
Keys automatically expire based on the rate limit window. You don't need to worry
|
|
about Redis filling up with stale data.
|
|
|
|
Switching Backends
|
|
------------------
|
|
|
|
You can switch backends without changing your rate limiting code. Just configure
|
|
a different backend at startup:
|
|
|
|
.. code-block:: python
|
|
|
|
import os
|
|
from fastapi_traffic import RateLimiter, MemoryBackend, SQLiteBackend
|
|
from fastapi_traffic.core.limiter import set_limiter
|
|
|
|
def get_backend():
|
|
"""Choose backend based on environment."""
|
|
env = os.getenv("ENVIRONMENT", "development")
|
|
|
|
if env == "production":
|
|
redis_url = os.getenv("REDIS_URL")
|
|
if redis_url:
|
|
from fastapi_traffic.backends.redis import RedisBackend
|
|
return RedisBackend.from_url(redis_url)
|
|
return SQLiteBackend("/app/data/rate_limits.db")
|
|
|
|
return MemoryBackend()
|
|
|
|
@app.on_event("startup")
|
|
async def startup():
|
|
backend = await get_backend()
|
|
limiter = RateLimiter(backend)
|
|
set_limiter(limiter)
|
|
await limiter.initialize()
|
|
|
|
Custom Backends
|
|
---------------
|
|
|
|
Need something different? Maybe you want to use PostgreSQL, DynamoDB, or some
|
|
other storage system. You can implement your own backend:
|
|
|
|
.. code-block:: python
|
|
|
|
from fastapi_traffic.backends.base import Backend
|
|
from typing import Any
|
|
|
|
class MyCustomBackend(Backend):
|
|
async def get(self, key: str) -> dict[str, Any] | None:
|
|
"""Retrieve state for a key."""
|
|
# Your implementation here
|
|
pass
|
|
|
|
async def set(self, key: str, value: dict[str, Any], *, ttl: float) -> None:
|
|
"""Store state with TTL."""
|
|
pass
|
|
|
|
async def delete(self, key: str) -> None:
|
|
"""Delete a key."""
|
|
pass
|
|
|
|
async def exists(self, key: str) -> bool:
|
|
"""Check if key exists."""
|
|
pass
|
|
|
|
async def increment(self, key: str, amount: int = 1) -> int:
|
|
"""Atomically increment a counter."""
|
|
pass
|
|
|
|
async def clear(self) -> None:
|
|
"""Clear all data."""
|
|
pass
|
|
|
|
async def close(self) -> None:
|
|
"""Clean up resources."""
|
|
pass
|
|
|
|
The key methods are ``get``, ``set``, and ``delete``. The state is stored as a
|
|
dictionary, and the backend is responsible for serialization.
|
|
|
|
Backend Comparison
|
|
------------------
|
|
|
|
.. list-table::
|
|
:header-rows: 1
|
|
|
|
* - Feature
|
|
- Memory
|
|
- SQLite
|
|
- Redis
|
|
* - Persistence
|
|
- ❌
|
|
- ✅
|
|
- ✅
|
|
* - Multi-process
|
|
- ❌
|
|
- ✅
|
|
- ✅
|
|
* - Multi-node
|
|
- ❌
|
|
- ❌
|
|
- ✅
|
|
* - Setup complexity
|
|
- None
|
|
- Low
|
|
- Medium
|
|
* - Latency
|
|
- ~0.01ms
|
|
- ~0.1ms
|
|
- ~1ms
|
|
* - Dependencies
|
|
- None
|
|
- None
|
|
- redis package
|
|
|
|
Best Practices
|
|
--------------
|
|
|
|
1. **Start with Memory, upgrade when needed.** Don't over-engineer. Memory is
|
|
fine for development and many production scenarios.
|
|
|
|
2. **Use Redis for distributed systems.** If you have multiple application
|
|
instances, Redis is the only option that works correctly.
|
|
|
|
3. **Handle backend errors gracefully.** Set ``skip_on_error=True`` if you'd
|
|
rather allow requests through than fail when the backend is down:
|
|
|
|
.. code-block:: python
|
|
|
|
@rate_limit(100, 60, skip_on_error=True)
|
|
async def endpoint(request: Request):
|
|
return {"status": "ok"}
|
|
|
|
4. **Monitor your backend.** Keep an eye on memory usage (Memory backend),
|
|
disk space (SQLite), or Redis memory and connections.
|