Posts tagged sharding

Avoiding Over-Sharding in OpenSearch with Filtered Aliases per Environment

Blog Post Publish Date: 2026/11/11

This blog post covers one of the strategies I use to avoid over-sharding in OpenSearch. Instead of per-environment indexes, I write all logs into one index series and expose each environment as a filtered alias. The rule behind it is simple: the environment is an attribute of each document, so it belongs in a field, not in the index name. The cluster keeps fewer, fuller shards and one mapping, and OpenSearch Dashboards users still get a per-environment view that behaves like a dedicated index.

Read more ...


Scaling Down and Resizing OpenSearch Data Nodes Without Losing Data

Blog Post Publish Date: 2026/11/10

This blog post covers the patterns and API calls I use to scale down and resize OpenSearch data nodes without losing data. The rule behind all of them is simple: the data on a node you remove has to end up somewhere else first. If you take nodes away faster than the cluster can relocate their shards, you lose data.

Read more ...


OpenSearch Shard Allocation: Balanced by Count, Rebalanced by Watermarks

Blog Post Publish Date: 2026/11/04

This blog post explains how OpenSearch distributes shards across data nodes. It covers how the default allocator works, why it can leave disk usage uneven in clusters with many small indices, and how disk watermarks correct that. It also covers other allocation mechanisms and a limitation I run into often: watermarks are one cluster-wide setting, even in clusters with different storage tiers.

Read more ...


OpenSearch Concepts: Index and Shards

Blog Post Publish Date: 2025/01/05

This blog post explains two of the most basic concepts in OpenSearch administration: indices and shards. It covers what primary and replica shards are, how their counts affect performance and fault tolerance, and the API calls I use most to inspect indices and shards while troubleshooting.

Read more ...