Hot partition detection

Detect when a single partition key gets too much traffic.

DynamoDB partitions have limits: ~1000 WCU/s for writes and ~3000 RCU/s for reads. When you hit these limits, DynamoDB throttles your requests. The hot partition detector warns you before that happens.

Key features

Track writes and reads per partition key
Sliding window counter (configurable duration)
Per-table threshold overrides
Per-model threshold overrides via ModelConfig
Logs warnings when thresholds are exceeded

Getting started

Create a HotPartitionDetector and pass it to your client:

hot_partition_basic.py

"""Basic hot partition detection setup."""

from pydynox import DynamoDBClient
from pydynox.diagnostics import HotPartitionDetector

# Create detector with thresholds
detector = HotPartitionDetector(
    writes_threshold=500,  # Warn after 500 writes per partition
    reads_threshold=1500,  # Warn after 1500 reads per partition
    window_seconds=60,  # Count operations in 60-second window
)

# Pass detector to client
client = DynamoDBClient(
    region="us-east-1",
    diagnostics=detector,
)

# Now all operations are tracked
# If a partition gets too hot, you'll see a warning in logs:
# WARNING: Hot partition detected - table="events" pk="EVENTS" had 500 writes in 60s

When a partition exceeds the threshold, you'll see a warning in your logs:

WARNING:pydynox.diagnostics:Hot partition detected - table="events" pk="EVENTS" had 500 writes in 60s

Parameters

Parameter	Type	Description
`writes_threshold`	int	Max writes per window before warning
`reads_threshold`	int	Max reads per window before warning
`window_seconds`	int	Sliding window duration in seconds

All parameters are required. There are no defaults.

Per-table overrides

Some tables handle more traffic than others. Use set_table_thresholds() to set different limits:

hot_partition_table_override.py

"""Override thresholds for specific tables."""

from pydynox import DynamoDBClient
from pydynox.diagnostics import HotPartitionDetector

detector = HotPartitionDetector(
    writes_threshold=500,
    reads_threshold=1500,
    window_seconds=60,
)

# High-traffic table needs higher threshold
detector.set_table_thresholds(
    "events",
    writes_threshold=2000,
    reads_threshold=5000,
)

# Cache table has lots of reads
detector.set_table_thresholds(
    "config_cache",
    reads_threshold=10000,
)

client = DynamoDBClient(
    region="us-east-1",
    diagnostics=detector,
)

Per-model overrides

You can also set thresholds in ModelConfig. This is useful when different models share the same client:

hot_partition_model_override.py

"""Override thresholds per model using ModelConfig."""

from pydynox import DynamoDBClient, Model, ModelConfig, set_default_client
from pydynox.attributes import StringAttribute
from pydynox.diagnostics import HotPartitionDetector

detector = HotPartitionDetector(
    writes_threshold=500,
    reads_threshold=1500,
    window_seconds=60,
)

client = DynamoDBClient(
    region="us-east-1",
    diagnostics=detector,
)
set_default_client(client)


# Normal model uses client's default thresholds
class User(Model):
    model_config = ModelConfig(table="users")
    pk = StringAttribute(hash_key=True)
    name = StringAttribute()


# High-traffic model overrides thresholds
class Event(Model):
    model_config = ModelConfig(
        table="events",
        hot_partition_writes=2000,  # Override client's 500
        hot_partition_reads=5000,  # Override client's 1500
    )
    pk = StringAttribute(hash_key=True)
    event_type = StringAttribute()

Model thresholds take precedence over client thresholds.

How it works

The detector uses a sliding window counter:

Each write/read operation is recorded with a timestamp
Old entries (outside the window) are cleaned up
When the count exceeds the threshold, a warning is logged

The tracking happens in Rust for speed. The Python wrapper adds logging and threshold overrides.

Choosing thresholds

DynamoDB partition limits are:

~1000 WCU/s per partition (writes)
~3000 RCU/s per partition (reads)

But these are theoretical maximums. In practice, set thresholds lower to catch problems early:

Use case	Writes	Reads	Window
Conservative	500	1500	60s
Moderate	800	2500	60s
High traffic	2000	5000	60s

Tip

Start with conservative thresholds. If you get too many false positives, increase them.

Methods

Method	Description
`record_write(table, pk)`	Record a write operation
`record_read(table, pk)`	Record a read operation
`get_write_count(table, pk)`	Get current write count for a partition
`get_read_count(table, pk)`	Get current read count for a partition
`set_table_thresholds(table, writes_threshold, reads_threshold)`	Override thresholds for a table
`clear()`	Reset all tracked data

What to do when you see warnings

Hot partition warnings mean you need to rethink your access pattern:

Spread the load - Use a more distributed partition key
Add randomness - Append a random suffix to partition keys
Use write sharding - Split writes across multiple partitions
Cache reads - Reduce read pressure with caching
Review your data model - Maybe the partition key design needs work