Mafiree logo
  • About
  • Services
  • Blogs
  • Careers
  • Products
    • orbit logo Orbit
    • streamer logo Xstreami
  • Contact
Schedule a Call
Menu
  • About
  • Services
  • Blogs
  • Careers
  • Products
    • orbit logo Orbit
    • streamer logo Xstreami
  • Contact
  • Schedule a Call
Database
Database Managed Database Services
MySQL MySQL
MySQL Consulting
MySQL Migration Services
MySQL Optimization & Query Tuning
MySQL Database Administration
MySQL Backup & Recovery
MySQL Security & Maintenance
MySQL Cloud Services (AWS RDS, Aurora, Google Cloud SQL, Azure)
MySQL for Ecommerce
MySQL High Availability & Replication
MongoDB MongoDB
MongoDB Consulting
MongoDB Migration Services
MongoDB Optimization & Query Tuning
MongoDB Database Administration
MongoDB Backup & Recovery
MongoDB Security & Maintenance
MongoDB Cloud (Atlas)
MongoDB Solutions by Industry
MongoDB High Availability & Replication
PostgreSQL PostgreSQL
PostgreSQL Consulting
PostgreSQL Migration & Upgrades
Performance Tuning & Query Optimization
PostgreSQL Administration & Managed Services
High Availability, Clustering & Replication
PostgreSQL Backup, Recovery & Disaster Planning
PostgreSQL Security, Compliance & Auditing
PostgreSQL for Analytics & Data Warehousing
PostgreSQL on Cloud & Containers
PostgreSQL Extensions & Open-Source Integrations
PostgreSQL for Every Industry
SQL Server MSSQL
MSSQL Consulting
MSSQL Migration Services
MSSQL Optimization & Query Tuning Services
MSSQL Database Administration Services
MSSQL Backup & Recovery Services
MSSQL High Availability & Replication Services
MSSQL Security & Compliance Services
MSSQL Performance Monitoring & Health Checks
MSSQL Solutions by Industry
Aerospike Aerospike
Aerospike Consulting
Aerospike Migration Services
Aerospike Performance Optimization & Tuning
Aerospike Backup & Recovery
Aerospike High Availability
Aerospike Cloud & Hybrid Deployments
Aerospike for Real-Time Applications (AdTech, FinTech, Retail, IoT)
Clickhouse Clickhouse
ClickHouse Consulting
ClickHouse Migration Services
ClickHouse Optimization & Query Tuning
ClickHouse Database Administration
ClickHouse Backup & Recovery
ClickHouse Security & Maintenance
ClickHouse Cloud Services (ClickHouse Cloud, AWS, GCP, Azure)
ClickHouse Solutions by Industry
ClickHouse High Availability & Replication
TiDB TiDB
TiDB Consulting
TiDB Administration & Maintenance
TiDB Security and Privacy Maintenance
TiDB Performance & Query Optimization
TiDB Migration Services
TiDB Backup & Disaster Recovery
TiDB High Availability Solutions
TiDB Solutions by Industry
TiDB Cloud Services
ScyllaDB ScyllaDB
ScyllaDB Consulting
ScyllaDB Administration & Maintenance
ScyllaDB Security and Privacy Maintenance
ScyllaDB Performance & Query Optimization
ScyllaDB Migration Services
ScyllaDB Backup & Disaster Recovery
ScyllaDB High Availability Solutions
ScyllaDB Solutions by Industry
ScyllaDB Cloud Services
DevOps
DevOps DevOps Services
Version Control Version Control
Kubernetes Kubernetes
Infrastructure Infrastructure Management
Web Servers Web Servers
Networking
Networking Networking Services
Basic Basic
Advanced Advanced
MySQL MySQL
MongoDB MongoDB
PostgreSQL PostgreSQL
MSSQL MSSQL
Aerospike Aerospike
Clickhouse Clickhouse
TiDB TiDB
ScyllaDB ScyllaDB
Version Control Version Control
Kubernetes Kubernetes
Infrastructure Infrastructure Management
Web Servers Web Servers
Basic Basic
Advanced Advanced
MySQL Consulting
MySQL Migration Services
MySQL Optimization & Query Tuning
MySQL Database Administration
MySQL Backup & Recovery
MySQL Security & Maintenance
MySQL Cloud Services (AWS RDS, Aurora, Google Cloud SQL, Azure)
MySQL for Ecommerce
MySQL High Availability & Replication
MongoDB Consulting
MongoDB Migration Services
MongoDB Optimization & Query Tuning
MongoDB Database Administration
MongoDB Backup & Recovery
MongoDB Security & Maintenance
MongoDB Cloud (Atlas)
MongoDB Solutions by Industry
MongoDB High Availability & Replication
PostgreSQL Consulting
PostgreSQL Migration & Upgrades
Performance Tuning & Query Optimization
PostgreSQL Administration & Managed Services
High Availability, Clustering & Replication
PostgreSQL Backup, Recovery & Disaster Planning
PostgreSQL Security, Compliance & Auditing
PostgreSQL for Analytics & Data Warehousing
PostgreSQL on Cloud & Containers
PostgreSQL Extensions & Open-Source Integrations
PostgreSQL for Every Industry
MSSQL Consulting
MSSQL Migration Services
MSSQL Optimization & Query Tuning Services
MSSQL Database Administration Services
MSSQL Backup & Recovery Services
MSSQL High Availability & Replication Services
MSSQL Security & Compliance Services
MSSQL Performance Monitoring & Health Checks
MSSQL Solutions by Industry
Aerospike Consulting
Aerospike Migration Services
Aerospike Performance Optimization & Tuning
Aerospike Backup & Recovery
Aerospike High Availability
Aerospike Cloud & Hybrid Deployments
Aerospike for Real-Time Applications (AdTech, FinTech, Retail, IoT)
ClickHouse Consulting
ClickHouse Migration Services
ClickHouse Optimization & Query Tuning
ClickHouse Database Administration
ClickHouse Backup & Recovery
ClickHouse Security & Maintenance
ClickHouse Cloud Services (ClickHouse Cloud, AWS, GCP, Azure)
ClickHouse Solutions by Industry
ClickHouse High Availability & Replication
TiDB Consulting
TiDB Administration & Maintenance
TiDB Security and Privacy Maintenance
TiDB Performance & Query Optimization
TiDB Migration Services
TiDB Backup & Disaster Recovery
TiDB High Availability Solutions
TiDB Solutions by Industry
TiDB Cloud Services
ScyllaDB Consulting
ScyllaDB Administration & Maintenance
ScyllaDB Security and Privacy Maintenance
ScyllaDB Performance & Query Optimization
ScyllaDB Migration Services
ScyllaDB Backup & Disaster Recovery
ScyllaDB High Availability Solutions
ScyllaDB Solutions by Industry
ScyllaDB Cloud Services
  1. Home
  2. > Blogs
  3. > MySQL
  4. > Avoid deadlocks, certification failures on Galera Cluster using ProxySQL

Avoid deadlocks, certification failures on Galera Cluster using ProxySQL

Dealing with deadlocks, certification failures on Galera cluster by running single node writer using ProxySQL

sukan July 06, 2019

Subscribe for email updates

Summarize with AI: ChatGPT Google AI Perplexity Claude Grok

One of the major limitations of galera cluster is deadlocks, it is quite common across multi-writer setups. This is mainly because of cluster wise optimistic locking. If your transaction per second is high and if you are updating same rows concurrently then deadlocks will be a major setback for your application.

Yes, one can retry the failed transaction and hope that it succeed next time, obviously this is a hard way and it can increase the response time. If application is not designed to retry the failed transaction then setting the wsrep_retry_autocommit on galera nodes would automatically retry the failed transaction.

If your table is bound to have more hotspots, then the frequency of the deadlock would be very high.

Dealing with deadlocks

Our setup is a classic 3 Node MariaDB Galera cluster all three nodes serving reads and writes equally from the ProxySQL Load Balancer.

 


 


One of our tables is having a hotspot that it will update the table row very frequently, thus our application will always bounce a deadlock error and during our peak transaction hour, almost most of the transactions are retried.

Galera has few options to monitor and troubleshoot the deadlocks.

wsrep_local_bf_aborts - Total number of local transactions that were aborted by slave transactions while in execution.

wsrep_local_cert_failures - Total number of local transactions that failed certification test.


As seen above, during our peak transaction hours, deadlocks and certification failures are quite high in our systems.
With wsrep_debug=ON and wsrep_log_conflicts=ON configuration one can enable additional logging and get the queries that are failed / retried. In our case, we know our hotspot queries and these are straight forward queries so we decided to test by sending all the writes to one node and reads to the remaining nodes.
Fortunately we have ProxySQL 2.0 which has the native support to galera servers.

The Solution

We have put the ProxySQL to send write traffic to node 1 and read traffic to other nodes. This can be done by setting max_writers to 1 in mysql_galera_hostgroup_table and thus by setting up regex on mysql_query_rules we have achieved this without taking any downtime.

 


 


Now writes are sent to node 1 and reads are sent to the nodes 2 and 3. It is designed in such a way that in case of failure of node 1 other node 2 or 3 can be brought up as a writer thus eliminating the risk of having a downtime because of failed single writer.

Post these changes, we have seen a drastic dip in the number of deadlocks occurred and there are no certification failure happening in the cluster


This is a real improvement and now our applications are not facing any deadlock or certification failure.

Verdict

If your database server can single handedly serve all the writes then running galera cluster with single node write would solve all your deadlock issues.

Author Bio

sukan

Sukan is Database Team Lead at Mafiree with over a decade of experience in database systems, architecture, and performance optimization. He specializes in MySQL, MongoDB, TiDB, and ClickHouse, developing architectural improvements that make data platforms faster, more efficient, and cost-effective. Sukan writes about practical database engineering topics, real-world performance tuning, data replication, and high-scale system design, drawing from extensive hands-on experience solving complex technical challenges.

Leave a Comment

Related Blogs

MySQL Incident Response: When Does Slow Response Signal a Support Model Gap?

Slow MySQL incident resolution is often a symptom of a weak support model—not just a technical problem. Learn how to identify gaps in escalation, monitoring, automation, DBA expertise, and post-incident reviews before they lead to recurring downtime.

  71 views
MySQL Deadlock Analysis: Diagnosing and Resolving Lock Contention in High-Concurrency Workloads

A MySQL deadlock occurs when two or more transactions hold locks that each other needs, creating a circular dependency that prevents them from proceeding. InnoDB detects the deadlock and rolls back one transaction; applications typically receive MySQL error 1213 and should safely retry the transaction. To diagnose a deadlock, start with SHOW ENGINE INNODB STATUS; for recurring incidents, use innodb_print_all_deadlocks and Performance Schema data_locks and data_lock_waits to investigate lock relationships. Common causes include inconsistent transaction ordering, long-running transactions, and inefficient indexing or locking patterns. The most effective fixes are to keep transactions short, access shared resources in a consistent order, optimize the indexes used by locking statements, and implement safe retry handling.

  441 views
MySQL Consulting vs In-House DBA: Which Is Right for Your Team?

Choose an in-house DBA when you need continuous database ownership, deep knowledge of internal systems, and day-to-day operational control. Choose MySQL consulting when you need specialized expertise, performance tuning, migrations, complex troubleshooting, or additional database capacity without hiring a full-time specialist. A hybrid model can be the best fit when your team handles routine database operations internally but needs external expertise for complex projects, peak workloads, or specialized MySQL problems. The right choice depends on workload consistency, technical complexity, required coverage, internal expertise, scalability, and total cost—not simply the hourly or salary cost. Mafiree's managed database services provide ongoing MySQL support alongside specialized database expertise for organizations that need more flexibility than a purely in-house model.

  292 views
MySQL Consultant: When to Call in a Performance Expert

Most MySQL slowdowns are fixable in-house. A smaller set of problems keep coming back, keep growing, or sit outside what routine tuning can solve — and that's the line worth knowing before an incident forces the decision for you.

  364 views
MySQL Performance Issues: 7 Signs You Need Professional Tuning

MySQL performance issues are rarely sudden — they build over time through slow queries, InnoDB buffer pool misses, replication lag, lock contention, thread pile-ups, tablespace bloat, and unstable query plans. This post identifies the seven most reliable signs that your MySQL environment needs professional DBA attention, with diagnostic queries and remediation guidance for each.

  13245 views
Column-Level Security: Enterprise Data Protection Without the Infrastructure Overhead

Column-level security is a native database feature that restricts access to specific table columns by user role. Mafiree implemented this for a client as a cost-effective replacement for a planned CDC replication architecture that existed solely to strip sensitive columns. The result: zero additional infrastructure, single source of truth, full GDPR/HIPAA compliance posture, and validated in production with no performance impact.

  1256 views
MySQL Schema Migration Without Downtime: A Real Fintech Case Study

Schema changes on large MySQL tables can bring production systems to a halt if not handled correctly. This case study walks through how Mafiree helped a fintech client execute a zero-downtime MySQL schema migration on a 500M+ row production database — covering the real challenges faced, the three-phase tool strategy using gh-ost, pt-online-schema-change, and MySQL 8.0 INSTANT DDL, production configuration settings with performance benchmarks, and best practices for safely evolving your MySQL schema without impacting users

  4382 views
MySQL Architecture Explained: Performance Tuning & Troubleshooting Guide

MySQL features a unique tiered architecture that separates query processing from data storage through its pluggable storage engine model. This guide explores the core components—from connection handling and the SQL optimizer to the physical storage of data on disk. By understanding how engines like InnoDB provide ACID compliance and row-level locking, you can significantly improve your database's scalability. We also break down the query execution workflow and provide actionable tips for performance tuning, such as optimizing the buffer pool. Whether you're managing a replica set or a standalone instance, mastering MySQL’s internal structure is essential for building high-performance applications.

  2328 views
MariaDB vs MySQL: What's Different in 2026 and Which One Should You Use

Discover how MariaDB 11.x is redefining open-source databases with cutting-edge features like system-versioned tables, native AI-ready vector support, UUIDv7 for scalable inserts, and enterprise-grade security; all in the Community Edition, without the paywall.

  1759 views
MySQL’s Connection Control Plugin for Brute-Force Protection: A DBA Security Guide

“Fortify Your MySQL Security: Slow Down Attackers with Connection Control Plugin” Learn how the MySQL Connection Control Plugin helps defend against brute-force login attempts by introducing intelligent, progressive delays—without locking out legitimate users.

  13228 views
MySQL 8.4 Migration: Key Changes for IT Leaders

Explore the 8 critical changes in MySQL 8.4 you need to know before migrating your production environment. From authentication updates to InnoDB defaults, this release brings significant operational implications for developers and DBAs.

  348 views
Load Balancing in MySQL: Read and Write Traffic Optimization Using MySQL Router

Optimize MySQL traffic with MySQL Router — smart load balancing made easyRouter Architecture image widget

  2000 views
Check Constraints in MySQL

Master Database Integrity with Check Constraints: Improve Data Quality and System Performance

  13221 views
Guarding Your Account: 5 Essential Tips to Evade Common Password Mistakes.

How to Secure your mysql database with the mysql validate password component

  3584 views
Access Control & Account Management In MySQL 8

Discover how MySQL 8 enhances security through features like password expiry, password validation, account lock/unlock, privilege management, and dual passwords. Safeguard your valuable data and streamline access control in your MySQL environment.

  3910 views
Significance of using Invisible Primary key (GIPK) with MySQL 8.0

How Generated Invisible Primary Keys (GIPK) Can Boost Your Database Efficiency

  3746 views
Optimize Your Database: Exploring MySQL 8.0 JSON Functions

This blog highlights the some of the important JSON function in MySQL 8.0 with examples. Level up your SQL skills with MySQL 8.0's JSON functions.

  5183 views
Best practices on enabling Audit log on MySQL, MariaDB and ProxySQL

Importance of audit log in MySQL, MariaDB and ProxySQL. Best practices to enable audit log

  5228 views
High Availability for MySQL using Orchestrator and ProxySQL

Auto slave promotion without any DBAs intervention using Orchestrator and ProxySQL. If you are looking for a HA solution without going for synchronous replication or Aws RDS then Orchestrator with ProxySQL is a great choice.

  3535 views
Understanding the Spider Storage Engine

MariaDB Spider is a storage engine that supports federation and sharding across multiple MySQL or MariaDB servers. It helps distribute data and workloads, making it useful for scaling large datasets, balancing traffic, and improving data retrieval through parallel processing.

  1783 views
MySQL Character Set: UTF8MB4 vs UTF8 and How to Choose the Right Encoding

Choosing the right MySQL character set is critical for ensuring accurate data storage, multilingual support, and proper text comparison. While the legacy utf8 (utf8mb3) character set supports only up to 3-byte characters, it fails to handle modern Unicode requirements such as emojis and certain international symbols. In contrast, utf8mb4 provides full 4-byte Unicode support, making it the recommended standard for modern applications. This guide explains the differences between UTF8MB4 and UTF8, their impact on storage and performance, and provides step-by-step instructions to configure and migrate your MySQL databases safely.

  5880 views
Real time data streaming from MariaDB to Columnstore

How real time data from transactional database(mariadb) to analytic database(columnstore) is achieved using MariaDB's maxscale binlog streaming

  1515 views
Fixing the MariaDB Galera Bug: WSREP Lock Waits

A MariaDB Galera cluster can experience severe stalls when concurrent updates to the same primary key trigger an unexpected deadlock issue. This blog explains how we reproduced the issue in MariaDB 10.2, why the affected node became unresponsive, and the recovery and upgrade options used to resolve it.

  4606 views

Subscribe for email updates

Get in touch with us

Highlights

More than 6000 Servers Monitored

Happy Clients

Certified DBAs

24 x 7 x 365 Support

PCI

Database Services

MySQL MongoDB PostgreSQL SQL Server Aerospike Clickhouse TiDB ScyllaDB

Quick Links

Careers Blog Contact Privacy Policy Disclaimer Policy

Contacts

Linkedin Mafiree Facebook Mafiree Twitter Mafiree

Nagercoil Office

Miru IT Park, Vallankumaranvillai,

Nagercoil, Tamilnadu - 629 002.

Bangalore Office

Unit 303, Vanguard Rise,

5th Main, Konena Agrahara,

Old Airport Road, Bangalore - 560 017.

Call: +91 6383016411

Email: sales@mafiree.com


Copyright © - All Rights Reserved - Mafiree