-
Notifications
You must be signed in to change notification settings - Fork 322
RS: new Flex docs #2871
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
RS: new Flex docs #2871
Changes from all commits
2a95478
f029f92
c23b6f1
d244b0d
b66c654
0a9fc1e
6524936
eeee481
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,93 @@ | ||
| --- | ||
| Title: Flex databases | ||
| alwaysopen: false | ||
| categories: | ||
| - docs | ||
| - operate | ||
| - rs | ||
| - rc | ||
| description: Extend Redis databases with flash storage for large-scale, cost-efficient deployments. | ||
| hideListLinks: true | ||
| linktitle: Flex databases | ||
| weight: 38 | ||
| --- | ||
|
|
||
| Flex extends your database capacity by combining RAM and flash (SSD) storage. This tiered architecture keeps frequently accessed (hot) data in RAM for sub-millisecond latency while storing less active (warm) data on flash to reduce costs and increase capacity. | ||
|
|
||
| Flex databases work with your existing Redis applications and the Redis API without modification. | ||
|
|
||
| ## How Flex works | ||
|
|
||
| ### Automatic data tiering | ||
|
|
||
| Flex moves data between RAM and flash based on access patterns: | ||
|
|
||
| - Frequently accessed data stays in high-speed RAM. | ||
| - Less active data moves to cost-efficient flash storage. | ||
| - Data accessed from flash promotes back to RAM automatically. | ||
|
|
||
| Redis uses an [LRU (least recently used)]({{< relref "/develop/reference/eviction#apx-lru" >}}) eviction policy to manage data placement. When memory pressure increases, Flex identifies cold objects, transfers them to flash, and frees RAM for new or frequently accessed keys. | ||
|
|
||
| This process requires no application changes. Your existing Redis commands work across both storage tiers. | ||
|
|
||
| ### Storage engine | ||
|
|
||
| Flex uses [Speedb](https://docs.speedb.io), a high-performance key-value storage engine optimized for flash drives: | ||
|
|
||
| - Redis handles all data operations in memory. | ||
| - Speedb manages the flash storage layer. | ||
|
|
||
| This design delivers predictable latency and throughput as datasets grow beyond RAM limits. | ||
|
|
||
| ## Compatibility | ||
|
|
||
| Flex is compatible with the Redis API and supports all Redis data types, including JSON and probabilistic data structures (Bloom filters, Count-Min Sketch, Top-K). | ||
|
|
||
| The following features are not yet supported with Flex: | ||
|
|
||
| - Redis Search | ||
| - Time series | ||
| - Active-Active | ||
|
|
||
| ## When to use Flex | ||
|
|
||
| Use Flex when you need to: | ||
|
|
||
| - Run Redis at terabyte scale while maintaining high throughput and sub-10 ms latency | ||
| - Power real-time feature stores for machine learning applications | ||
| - Operate large distributed caches with elastic scaling and consistent performance under heavy load | ||
| - Reduce infrastructure costs by combining high-speed RAM with cost-efficient flash storage | ||
|
|
||
| {{<note>}} | ||
| Flex does not replace long-term data persistence. For workloads that require durability and recovery across restarts or failures, use Redis persistence features like [AOF (Append-Only File)]({{< relref "/operate/oss_and_stack/management/persistence#append-only-file" >}}), [RDB snapshots]({{< relref "/operate/oss_and_stack/management/persistence#snapshotting" >}}), or both. For more information, see [Database persistence]({{< relref "/operate/rs/databases/configure/database-persistence" >}}). | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I suggest adding a dedicated "What Flex isn't" section with the wording similar to the doc: https://docs.google.com/document/d/1PuFejCooJpy_gs0MoM5Pf9MLS46x6gcNUmzYfPjIHik/edit?disco=AAABz_t08H0&tab=t.0 |
||
| {{</note>}} | ||
|
|
||
| ## Flex and Auto Tiering | ||
|
|
||
| Flex replaces [Auto Tiering]({{< relref "/operate/rs/7.22/databases/auto-tiering" >}}) (formerly known as Redis on Flash). Redis Software selects the implementation based on your Redis version: | ||
|
|
||
| | Redis database version | Flex | Auto Tiering | | ||
| |------------------------|------|--------------| | ||
| | 8.0 and later | <span title="Supported">✅</span> | <span title="Not supported">❌</span> | | ||
| | 7.4 | <span title="Supported">✅</span> | <span title="Supported">✅</span>| | ||
| | 7.2 and earlier | <span title="Not supported">❌</span> | <span title="Supported">✅</span> | | ||
|
|
||
| For Redis Software version 7.22.2-22 or earlier, see [Auto Tiering]({{< relref "/operate/rs/7.22/databases/auto-tiering" >}}). | ||
|
|
||
| ### Differences between Flex and Auto Tiering | ||
|
|
||
| - Key and value offloading | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Add a short explanation on why it matters: |
||
| - Auto Tiering offloads only values to flash while keys remain in RAM. | ||
| - Flex offloads both keys and values, which increases dataset density per node and reduces RAM consumption. | ||
| - RAM population strategy | ||
| - Auto Tiering fills all available RAM before offloading data to flash. This maximizes hot-data performance but can cause non-linear performance changes at high utilization. | ||
| - Flex uses utilization-aware RAM population. When database utilization is below 50%, Flex uses up to 50% of configured RAM for hot data. Above 50% utilization, Flex uses both RAM and flash proportionally, following the configured RAM-to-flash ratio. This provides a stable performance curve, consistent RAM hit-rate, and predictable throughput and latency. | ||
| - Storage engine | ||
| - Auto Tiering uses either RocksDB or Speedb as the storage engine. | ||
| - Flex uses Speedb only. | ||
|
|
||
| ## Next steps | ||
|
|
||
| - [Plan your deployment]({{< relref "/operate/rs/flex/plan" >}}): Review hardware requirements, sizing guidelines, and limitations. | ||
| - [Get started]({{< relref "/operate/rs/flex/get-started" >}}): Configure Flex on your cluster. | ||
| - [Scale your deployment]({{< relref "/operate/rs/flex/scale" >}}): Learn scaling strategies and best practices. | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,132 @@ | ||
| --- | ||
| Title: Get started with Flex databases for Redis Software | ||
| alwaysopen: false | ||
| categories: | ||
| - docs | ||
| - operate | ||
| - rs | ||
| description: Configure Flex databases on your Redis Software cluster. | ||
| linkTitle: Get started | ||
| weight: 20 | ||
| --- | ||
| This page guides you through a quick setup of [Flex]({{< relref "/operate/rs/flex" >}}) with a single node for testing and demo purposes. | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. consider labeling it as a non-production quick start and refer to the planning page for production grade deployments.
Collaborator
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This sentence and the following sentence "For production environments" already indicate that this quick start is for testing and not intended for production, but I went ahead and linked to the planning doc too. |
||
|
|
||
| For production environments, see [Plan a Flex deployment for Redis Software]({{<relref "/operate/rs/flex/plan">}}) and more detailed installation instructions in the [install and setup]({{< relref "/operate/rs/installing-upgrading" >}}) section. | ||
|
|
||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Consider adding before you begin clause with version requirements.
Collaborator
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I added a version requirements section to the quick start. |
||
| To set up a Redis Software cluster using Flex with a single node: | ||
|
|
||
| 1. Review the [version requirements](#version-requirements). | ||
|
|
||
| 1. [Install Redis Software](#install-redis-software) or run it in a [Docker container](#dockerbased-installation). | ||
|
|
||
| 1. [Prepare and format flash memory](#prepare-and-format-flash-memory). | ||
|
|
||
| 1. [Set up a cluster](#setup-cluster-flash) with flash storage. | ||
|
|
||
| 1. [Create a new database](#create-a-database) with flash enabled. | ||
|
|
||
| 1. [Connect to your new database](#connect-to-your-database). | ||
|
|
||
| ## Version requirements | ||
|
|
||
| To create Flex databases, you need: | ||
|
|
||
| - Redis Software cluster version 8.0.2-17 or later | ||
|
|
||
| - Redis database version 8.2 or later | ||
|
|
||
| ## Install Redis Software | ||
|
|
||
| ### Bare metal, VM, Cloud instance | ||
|
|
||
| To install on bare metal, a virtual machine, or an instance: | ||
|
|
||
| 1. Download the Redis Software binaries from the [download center](https://redis.io/downloads/#Redis_Software). | ||
|
|
||
| 1. Upload the binaries to a Linux-based operating system. | ||
|
|
||
| 1. Extract the image: | ||
|
|
||
| ```sh | ||
| tar -vxf <downloaded tar file name> | ||
| ``` | ||
|
|
||
| 1. After the `tar` command completes, you can find a new `install.sh` script in the current directory: | ||
|
|
||
| ```sh | ||
| sudo ./install.sh -y | ||
| ``` | ||
|
|
||
| ### Docker-based installation {#dockerbased-installation} | ||
|
|
||
| For testing purposes, you can run a Redis Software Docker container: | ||
|
|
||
| ```sh | ||
| docker run -d --cap-add sys_resource --name rp -p 8443:8443 -p 12000:12000 redislabs/redis:latest | ||
| ``` | ||
|
|
||
| ## Prepare and format flash memory | ||
|
|
||
| After you [install Redis Software](#install-redis-software), use the `prepare_flash` script to prepare and format flash memory: | ||
|
|
||
| ```sh | ||
| sudo /opt/redislabs/sbin/prepare_flash.sh | ||
| ``` | ||
|
|
||
| This script finds unformatted disks and mounts them as RAID partitions in `/var/opt/redislabs/flash`. | ||
|
|
||
| To verify the disk configuration, run: | ||
|
|
||
| ```sh | ||
| sudo lsblk | ||
| ``` | ||
|
|
||
| ## Set up a cluster with flash storage {#setup-cluster-flash} | ||
|
|
||
| 1. Direct your browser to `https://localhost:8443` on the host machine to | ||
| see the Redis Software Cluster Manager UI. | ||
|
|
||
| {{<note>}} | ||
| Depending on your browser, you may see a certificate error. | ||
| Choose "continue to the website" to go to the setup screen. | ||
| {{</note>}} | ||
|
|
||
| 1. Click **Create new cluster**. | ||
|
|
||
| 1. Set up account credentials for a cluster administrator, then click **Next** to proceed to cluster setup. | ||
|
|
||
| 1. Enter your cluster license key if you have one. Otherwise, the cluster uses the trial version. | ||
|
|
||
| 1. Provide a cluster FQDN such as `mycluster.local`, then click **Next**. | ||
|
|
||
| 1. In the **Storage configuration** section, turn on the **Enable flash storage** toggle. | ||
|
|
||
| 1. Click **Create cluster**. | ||
|
|
||
| 1. Click **OK** to confirm that you are aware of the replacement of the HTTPS TLS certificate on the node, and proceed through the browser warning. | ||
|
|
||
| ## Create a database | ||
|
|
||
| On the **Databases** screen: | ||
|
|
||
| 1. Select **Quick database**. | ||
|
|
||
| 1. Verify **Flash** is selected for **Runs on**. | ||
|
|
||
| {{<image filename="images/rs/screenshots/databases/quick-db-flash-7-8-2.png" alt="Create a quick database with Runs on Flash selected." >}} | ||
|
|
||
| 1. Enter `12000` for the endpoint **Port** number. | ||
|
|
||
| 1. Optionally, select **Full options** to see available alerts. | ||
|
|
||
| 1. Click **Create**. | ||
|
|
||
| You now have a Flex database. | ||
|
|
||
| ## Connect to your database | ||
|
|
||
| After you create the database, you can connect to it and store data. See [Test client connection]({{<relref "/operate/rs/databases/connect/test-client-connectivity">}}) for connection options and examples. | ||
|
|
||
| ## Next steps | ||
|
|
||
| To see the true performance and scale of Flex, you must tune your I/O path and set the flash path to the mounted path of SSD or NVMe flash memory. | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,102 @@ | ||
| --- | ||
| title: Plan a Flex deployment for Redis Software | ||
| alwaysopen: false | ||
| categories: | ||
| - docs | ||
| - operate | ||
| - rs | ||
| description: Hardware requirements, sizing guidelines, best practices, and limitations for Flex databases on Redis Software. | ||
| hideListLinks: true | ||
| linkTitle: Plan deployment | ||
| weight: 10 | ||
| --- | ||
|
|
||
| Review the following hardware requirements, sizing guidelines, best practices, and known limitations before you deploy a Redis Software cluster for Flex databases. | ||
|
|
||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Consider adding a short checkbox: `Before you deploy Flex, verify:
|
||
| ## Version requirements | ||
|
|
||
| To create Flex databases, you need: | ||
|
|
||
| - Redis Software cluster version 8.0.2-17 or later | ||
|
|
||
| - Redis database version 8.2 or later | ||
|
|
||
| ## Hardware requirements | ||
|
|
||
| When planning a Flex deployment, consider the following flash drive requirements: | ||
|
|
||
| - Flash storage must be locally attached. Network-attached storage (NAS), storage area networks (SAN), or solutions such as AWS Elastic Block Storage (EBS) are not supported. | ||
|
|
||
| - Flash storage must be dedicated to Flex database data and keys. It should not be used for durability, binaries, or persistence. | ||
|
|
||
| - For the best performance, the SSDs should be NVMe based. NVMe Gen 5 or Gen 4 is recommended. Gen 3 is also supported, but not recommended for best performance. | ||
|
|
||
| - Flash storage must be greater than the total provisioned database size to account for write buffers, space amplification, and more. | ||
|
|
||
| See the general Redis Software [hardware requirements]({{<relref "/operate/rs/installing-upgrading/install/plan-deployment/hardware-requirements">}}) for additional requirements. | ||
|
|
||
| ## Sizing guidelines | ||
|
|
||
| Size your Redis Flex databases based on data volume, shard count, RAM-to-flash ratio, and CPU allocation to balance capacity, throughput, and latency. | ||
|
|
||
| ### Shard size | ||
|
|
||
| Flex databases consist of shards—independent data partitions that handle a portion of the dataset and request load. Each shard runs as a self-contained Redis process with dedicated memory, CPU, and flash resources. | ||
|
|
||
| Use these standard building blocks to plan capacity: | ||
|
|
||
| - 50 GB per shard | ||
|
|
||
| - Allocate 1 vCPU per shard | ||
|
|
||
| For example, a 1 TB dataset requires 20 shards, with 50 GB per shard. | ||
|
|
||
| ### CPU allocation | ||
|
|
||
| Throughput capacity scales with both CPU cores. At minimum, allocate 1 vCPU per 50 GB shard. | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. "... with both CPU cores and the RAM-to-flash ration". |
||
|
|
||
| ### RAM-to-flash ratio | ||
|
|
||
| The RAM-to-flash ratio directly affects throughput and latency: | ||
|
|
||
| - More RAM increases throughput and lowers latency. | ||
|
|
||
| - Less RAM lowers throughput and increases latency, but reduces cost. | ||
|
|
||
| Actual performance can vary based on your data model, commands, and network latency. | ||
|
|
||
| ## Feature and data type compatibility | ||
|
|
||
| | Data type/feature | Flex support | | ||
| |------------------|--------------| | ||
| | [Active-Active databases]({{<relref "/operate/rs/databases/active-active">}}) | <span title="Not supported">❌</span>Not supported | | ||
| | [JSON]({{<relref "/develop/data-types/json">}}) | <span title="Supported">✅</span> Supported | | ||
| | [Probabilistic data structures]({{<relref "/develop/data-types/#probabilistic-data-types">}}) | <span title="Supported">✅</span> Supported | | ||
| | [Redis Search]({{<relref "/develop/ai/search-and-query">}}) | <span title="Not supported">❌</span>Not supported | | ||
| | Standard [Redis data types]({{<relref "/develop/data-types">}}) | <span title="Supported">✅</span> Supported | | ||
| | [Time series]({{<relref "/develop/data-types/timeseries">}}) | <span title="Not supported">❌</span>Not supported | | ||
|
|
||
| ## Best practices | ||
|
|
||
| - Store small keys and values when possible. Avoid objects larger than 10 KB, which can reduce performance because the entire value moves between RAM and flash. | ||
|
|
||
| - Start with a balanced ratio, such as 20% RAM to 80% flash, and monitor latency and throughput metrics. | ||
|
|
||
| - Increase RAM percentage if 99th-percentile latency exceeds your target or cache hit rates drop. | ||
|
|
||
| - Decrease RAM percentage if memory utilization is high but latency remains stable to lower cost and improve efficiency. | ||
|
|
||
| - Re-evaluate the ratio of RAM to flash periodically as dataset size and access patterns evolve. | ||
|
|
||
| ## Known limitations | ||
|
|
||
| - Flex databases cannot store keys or values larger than 4GB in flash storage. Larger keys or values will be stored in RAM only, and warnings will appear in the Redis logs: | ||
|
|
||
| ```sh | ||
| # WARNING: key too big for disk driver, size: 4703717276, key: subactinfo:htable | ||
| ``` | ||
|
|
||
| ## Next steps | ||
|
|
||
| - [Get started]({{< relref "/operate/rs/flex/get-started" >}}): Configure Flex on your cluster. | ||
| - [Scale your deployment]({{< relref "/operate/rs/flex/scale" >}}): Learn scaling strategies. | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"Flex frees RAM for hot data, higher RAM hit‑rate, bigger datasets per node”