Introducing Foreign Keys and more: Exploring New Features in TiDB v8.5.0

TiDB v8.5.0 introduces powerful enhancements to scalability, performance, and security, along with the key features like accelerated table creation, MVCC in-memory engine and client-side encryption. These updates strengthen TiDB’s ability to handle complex workloads, ensuring reliability and efficiency for diverse database needs.

Scalability

Schema Cache Memory Limit (GA):

In certain SaaS scenarios, where the number of tables can reach hundreds of thousands or even millions, schema metadata may consume a substantial amount of memory. Enabling this feature allows TiDB to utilize the Least Recently Used (LRU) algorithm to cache and evict schema metadata, effectively optimizing memory usage.

Starting from v8.4.0, this feature is enabled by default with a default value of 512 MiB. You can adjust it as needed using the variable tidb_schema_cache_size.

Active PD Follower (GA):

In TiDB clusters with a large number of Regions, the PD leader may face high CPU usage due to the overhead of handling heartbeats and scheduling tasks, potentially leading to PD service unavailability. To ensure high availability, TiDB v7.6.0 introduced Active PD Follower as an experimental feature. 

In v8.5.0, this feature becomes generally available (GA). You can enable it by setting the system variable pd_enable_follower_handle_region to ON. Once enabled, TiDB distributes Region information requests evenly across all PD servers, allowing PD followers to handle these requests and alleviating CPU pressure on the PD leader.

Performance

Accelerated Table Creation (GA):

TiDB v7.6.0 introduces accelerated table creation as an experimental feature by the system variable tidb_ddl_version and renamed to tidb_enable_fast_create_table in v8.0.0. This feature becomes generally available in v8.5.0 and is enabled by default. It significantly reduces data migration and cluster initialization time by enabling rapid creation of millions of tables.

MVCC In-Memory Engine (GA):

When records are frequently updated, or TiDB needs to retain historical versions for extended periods (e.g., 24 hours), the accumulation of MVCC versions can negatively impact scan performance. The TiKV MVCC in-memory engine addresses this by caching the latest MVCC versions in memory and employing a fast GC mechanism to remove historical versions from memory.

Starting with v8.5.0, TiKV introduces the MVCC in-memory engine. If excessive accumulation of MVCC versions in the TiKV cluster degrades scan performance, you can enable this engine to improve scan efficiency by configuring the in-memory-engine.enable parameter in TiKV settings.

Reliability

Support limiting the maximum rate and concurrency of requests processed by PD:

When a sudden influx of requests is sent to PD, it can lead to high workloads and potentially affect PD performance. Starting from v8.5.0, you can use pd-ctl to limit the maximum rate and concurrency of requests processed by PD, improving its stability.

SQL

Foreign Key (GA):

Starting from v8.5.0, foreign key constraints become generally available, enabling relationships between tables with support for cascading updates and deletions. This simplifies data management and ensures consistency, especially for applications with complex relationships.

ADMIN ALTER DDL JOBS statement to support online modification of DDL jobs :

Starting from v8.3.0, the variables tidb_ddl_reorg_batch_size and tidb_ddl_reorg_worker_cnt can be set at the session level. Consequently, setting these variables globally no longer impacts all running DDL jobs. 

With TiDB v8.5.0, the new ADMIN ALTER DDL JOBS statement allows you to adjust the variable values of specific DDL jobs online. This provides greater flexibility in balancing resource consumption and performance. The changes apply only to individual jobs, ensuring a more controllable impact. 

For example:

• Adjusts the tidb_ddl_reorg_worker_cnt of the specified DDL job online : 

        ADMIN ALTER DDL JOBS job_id THREAD = 8;

• Adjusts the tidb_ddl_reorg_batch_size of the specified job online : 

        ADMIN ALTER DDL JOBS job_id BATCH_SIZE = 256;

• Adjusts the write traffic of index data to each TiKV node online : 

        ADMIN ALTER DDL JOBS job_id MAX_WRITE_SPEED = '200MiB';

Click here to know more about TiDB's DDL jobs.

Security

BR now supports client-side encryption for both full backup and log backup data (GA):

Client-side encryption for full backup data, introduced as an experimental feature in TiDB v5.3.0, allows backup data to be encrypted on the client side using a custom fixed key.

Similarly, client-side encryption for log backup data, introduced as an experimental feature in TiDB v8.4.0, enables log backup data to be encrypted on the client side through the following methods:

• Using a custom fixed key
• Using a master key stored on a local disk
• Using a master key managed by a Key Management Service (KMS)

Starting with v8.5.0, both encryption features are generally available (GA), providing enhanced security for client-side data encryption.

TiKV encryption at rest supports Google Cloud KMS (GA):

TiKV's encryption at rest now fully supports Google Cloud KMS for master key management, providing enhanced security and seamless integration. Configure the feature by setting the security.encryption.master-key section in the TiKV configuration.

Conclusion

With significant new features and numerous bug fixes, TiDB v8.5.0 delivers improved performance, enhanced security, and streamlined operations. This release further solidifies TiDB as a reliable choice for modern database challenges. To know more about TiDB v8.5.0 refer the TiDB document.