Coder Legion - Recent posts tagged mysql-heatwave

MySQL HeatWave: Read Replicas for Scaling Read-Heavy Workloads

Fri, 28 Nov 2025 08:11:25 +0000

Read replicas are a powerful feature of MySQL HeatWave that enable you to scale read-heavy workloads by distributing queries across multiple database instances. As read-only copies of your source DB system, read replicas handle SELECT queries, analytics, and reporting workloads while the primary instance handles write operations. This architecture improves application performance, enhances availability, and supports disaster recovery strategies.

This guide covers read replica concepts, creation, connectivity, use cases, maintenance, and limitations.

What Are Read Replicas?

Read replicas are read-only copies of a MySQL HeatWave DB system that are automatically updated using asynchronous replication from the source DB system.

Key Characteristics:

Read-only: Cannot execute write operations (INSERT, UPDATE, DELETE)
Asynchronous replication: Data replicated from source with minimal lag
Full data copy: Contains complete copy of source DB system data
Independent endpoints: Each read replica has its own connection endpoint
Automatic lifecycle: Started, stopped, and deleted with source DB system

Benefits of Read Replicas

1. Redirecting Excess Traffic

Distribute read traffic across multiple instances to prevent primary instance overload.

Scenario: E-commerce application during peak shopping season

Primary handles transactions (orders, payments)
Read replicas serve product catalogs, inventory checks, search queries
Reduces primary instance load by 60-80%

2. Business Reporting and Analytics

Offload resource-intensive reporting queries to read replicas.

Scenario: Daily business intelligence reports

Complex JOIN queries with aggregations
Historical data analysis
Real-time dashboards
No impact on transactional performance

3. Geographic Distribution

Place read replicas closer to users in different regions.

Scenario: Global application

Primary in US-East
Read replica in EU-West for European users
Read replica in AP-South for Asian users
Reduced latency for read operations

4. Disaster Recovery

Read replicas serve as standby databases for failover scenarios.

Scenario: Primary DB system failure

Promote read replica to standalone DB system
Minimal data loss (replication lag only)
Faster recovery than backup restoration
Business continuity maintained

5. Development and Testing

Create isolated environments without impacting production.

Scenario: Testing new features

Read replica mirrors production data
Safe environment for experiments
No risk to production workload
Fresh data for realistic testing

Creating Read Replicas

Prerequisites

Before creating read replicas, ensure the following requirements are met:

1. OCI User Accounts with Required Privileges

Your OCI user must have permissions to:

Create and manage read replicas
Access the source DB system
Manage network resources

Required IAM Policy:

Allow group DatabaseAdmins to manage mysql-family in compartment ProductionDatabases

2. Operational DB System

An active MySQL HeatWave DB system must exist and be in ACTIVE state. You can create read replicas on standalone or high availability DB systems.

3. Compatible Shape

Read replicas are not supported on shapes that have less than 8 ECPUs or less than 4 OCPUs.

Supported Shapes:

ECPU: MySQL.8 (8 ECPUs), MySQL.16, MySQL.32, MySQL.64, MySQL.128, MySQL.256
OCPU (deprecated): MySQL.VM.Standard.E3.1.8GB (4 OCPUs) and higher

Note: OCPU shapes are deprecated for new users after September 5, 2024, and will be unavailable after March 13, 2026.

4. Network Configuration

Read replicas inherit the VCN and subnet from the source DB system by default, but you can specify a different subnet in the same VCN or different VCN.

Maximum Read Replicas

You can create a maximum of 18 read replicas per DB system. Each read replica has its own endpoint for connections.

Creating Read Replicas via OCI Console

Navigate to DB System
- Open OCI Console
- Go to Databases → HeatWave MySQL → DB Systems
- Select your source DB system
Access Read Replicas Section
- Under Resources, click Read replicas
- Click Create read replica
Configure Read Replica
- Display name: Descriptive name (e.g., "prod-analytics-replica")
- Description: Purpose description
- Shape: Inherit from source or override with different shape
- Configuration: Inherit from source or select custom configuration
- MySQL version: Inherit or override (must be same or higher)
- Networking: Select VCN and subnet (can differ from source)
- Delete protection: Enable to prevent accidental deletion
Review and Create
- Click Create read replica
- Read replica enters CREATING state
- Typically available in 15-30 minutes

Creating Read Replicas via OCI CLI

Generate configuration template:

oci mysql replica create --generate-full-command-json-input > replica-config.json

Edit configuration:

{
  "dbSystemId": "ocid1.mysqldbsystem.oc1...",
  "displayName": "analytics-read-replica",
  "description": "Read replica for BI workloads",
  "shapeName": "MySQL.16",
  "isDeleteProtected": true,
  "waitForState": ["SUCCEEDED"]
}

Create replica:

oci mysql replica create --from-json file://replica-config.json

Read Replica Properties

Inherited vs. Overridden Properties

By default, a read replica inherits shape, configuration, and version from the DB system. You can override these properties.

When Inherited:

Changing property on DB system also changes it on read replica
Replica follows source configuration automatically

When Overridden:

Changing property on DB system does NOT affect read replica
Replica maintains independent configuration

Overridable Properties:

Shape (e.g., use larger shape for heavy analytics)
Configuration (e.g., tune for read-heavy workload)
MySQL version (must be same or higher than source)

Automatic Read Replica Load Balancer

When you create the first read replica of a DB system, a read replica load balancer is created automatically, which distributes read traffic among the read replicas.

Load Balancer Features:

Distributes queries across all read replicas
Health checks ensure traffic goes to ACTIVE replicas
Automatic failover if replica becomes unavailable
No manual configuration required

Connecting to Read Replicas

Client applications cannot directly access read replicas from the internet. Connection methods mirror those for DB systems.

Connection Methods

1. Load Balancer (Recommended)

Use the automatically created read replica load balancer for balanced traffic distribution.

Load Balancer Endpoint:

<replica-load-balancer-ip>:3306

Benefits:

Automatic distribution across replicas
Health checks
Simplified application configuration
High availability

Important: When connecting via load balancer, user accounts require username only (no hostname restriction like 'user'@'%').

2. Bastion Session

Provides restricted and time-limited secure access.

Use Case: Temporary administrative access, troubleshooting

Setup:

Create bastion session in OCI Console
Session provides temporary SSH access
Connect to read replica from bastion
Session expires after defined period (30 minutes to 3 hours)

Example:

# Create bastion session
oci bastion session create-managed-ssh ...

# Connect through bastion
ssh -i <private_key> -o ProxyCommand="ssh -W %h:%p ..." replica_ip
mysql -h <REPLICA_IP> -u admin -p

3. Compute Instance

For long-term, secure access.

Setup:

Create compute instance in same VCN
Configure security lists to allow MySQL ports
Install MySQL client on compute instance
Connect to read replica endpoint

Example:

mysql -h <REPLICA_IP> -P 3306 -u admin -p

4. VPN Connection

For site-to-site connectivity from on-premises.

Options:

Site-to-Site VPN: IPSec VPN over internet
FastConnect: Dedicated private connection
OpenVPN: Client VPN access

5. Direct Application Connection

Configure application to use read replica endpoints.

Application Configuration:

# Primary for writes
primary_connection = mysql.connector.connect(
    host='primary_db_ip',
    user='app_user',
    password='password',
    database='production'
)

# Read replica for reads
replica_connection = mysql.connector.connect(
    host='replica_ip',
    user='app_user',
    password='password',
    database='production'
)

# Write operation
primary_connection.cursor().execute("INSERT INTO orders ...")

# Read operation
replica_connection.cursor().execute("SELECT * FROM products")

Monitoring Read Replicas

Read Replica Lag

Read replica lag measures how far behind the replica is from the source, measured by the read replica lag metric.

Factors Affecting Lag:

Workload on each read replica
CPU and memory resources
Network latency
Volume of writes on primary

Monitoring Lag:

-- On read replica
SHOW REPLICA STATUS\G

Check Seconds_Behind_Source value.

Creating Alarms

Set up OCI alarms to monitor read replica health:

Navigate to Monitoring service
Create alarm for ReplicaLag metric
Set threshold (e.g., > 60 seconds)
Configure notification topic
Add email or webhook destination

Alarm Query:

ReplicaLag[1m]{resourceDisplayName = "analytics-replica"}.mean() > 60

Automatic Recovery

MySQL HeatWave Service performs automatic recovery on any read replica that has failed.

Recovery Process:

Failure detected
Read replica set to UPDATING state
Cannot accept client connections during recovery
After success, set to ACTIVE state
Becomes operational again

Failed Beyond Recovery:
If replica cannot replicate from DB system anymore, it's set to NEEDS_ATTENTION state. Manual intervention required.

Read Replica Maintenance and Upgrades

Standalone DB System Upgrades

Read replicas are upgraded in parallel with source DB system. Both the source and all read replicas go offline for an update and upgrade at the same time.

Process:

Maintenance window starts
Source and all replicas enter UPDATING state
Upgrades applied simultaneously
All return to ACTIVE state together

Downtime: Brief period (typically 5-15 minutes depending on upgrade type)

High Availability DB System Upgrades

For HA DB systems, read replicas are upgraded one by one using rolling maintenance process. Only one read replica goes offline at a time.

Rolling Upgrade Process:

Replica 1 enters UPDATING state
Replica 1 upgraded, returns to ACTIVE
Replica 2 enters UPDATING state
Replica 2 upgraded, returns to ACTIVE
Continue for all replicas

Benefits:

Minimal impact on read capacity
At least N-1 replicas always available
Reduced performance impact

Duration: Longer total time but better availability

Read Replica Limitations

1. Shape Requirements

Read replicas are not supported on shapes that have less than 8 ECPUs or less than 4 OCPUs.

Unsupported:

MySQL.2 (2 ECPUs)
MySQL.4 (4 ECPUs)
MySQL.VM.Standard.E3.1.8GB (1 OCPU)
MySQL.VM.Standard.E3.1.16GB (2 OCPUs)

2. Load Balancer Account Name Restrictions

When connecting via load balancer, user accounts require username only ('user'@'%'), not hostname-specific restrictions.

Won't Work:

CREATE USER 'app_user'@'10.0.1.%' ...

Must Use:

CREATE USER 'app_user'@'%' ...

3. Outbound Replication Source

Read replicas cannot be configured as a source for outbound replication. Only the primary DB system can be a replication source.

4. IPv4 Subnet Support Only

Read replicas are only supported with IPv4 subnets. IPv6 subnets are not supported.

5. Bandwidth Limits

Read replicas support up to 8 Gbps bandwidth for client connections.

Implications:

High-throughput workloads may hit bandwidth limits
Monitor network metrics
Consider multiple replicas for very high throughput needs

6. Lifecycle Dependency

Read replicas are started when DB system is started, stopped when DB system is stopped, and deleted when DB system is deleted.

Cannot:

Start replica independently if source is stopped
Keep replica running when source is stopped
Preserve replica after deleting source

7. HeatWave Cluster Not Supported

Read replicas cannot have HeatWave clusters attached. If the source DB system has a HeatWave cluster, read replicas will not replicate HeatWave-specific data.

8. Configuration Changes

Prior to March 2024, configuration changes required deleting and recreating read replicas. As of March 26, 2024, you can now change shape and configuration of DB systems with read replicas without deleting replicas.

Best Practices

1. Use Multiple Read Replicas

For high availability and load distribution:

Minimum 2 replicas for redundancy
More replicas for higher read throughput
Maximum 18 replicas per DB system

2. Monitor Replication Lag

Set up alerts and regularly check:

SHOW REPLICA STATUS\G

If lag exceeds 60 seconds consistently:

Increase replica shape
Reduce read load
Optimize slow queries

3. Size Replicas Appropriately

Match replica shape to workload:

Light analytics: Same shape as source
Heavy reporting: Larger shape than source
Mixed workload: Multiple replicas with different shapes

4. Implement Read/Write Splitting in Applications

Direct writes to primary, reads to replicas:

def ex# Conclusionecute_query(query):
    if query.startswith('SELECT'):
        return replica_connection.execute(query)
    else:
        return primary_connection.execute(query)

5. Use Load Balancer for Simplicity

Automatic distribution and failover:

Configure application with load balancer endpoint only
No application changes when adding/removing replicas
Built-in health checks

6. Plan for Replication Lag

Design applications to tolerate eventual consistency:

Don't read immediately after write from replica
Use primary for read-after-write scenarios
Communicate lag tolerance to users

7. Test Failover Scenarios

Regularly test promoting replica to standalone:

Verify promotion procedure
Measure recovery time
Update application connection strings
Practice makes perfect for real disasters

8. Enable Delete Protection

For production replicas:

--is-delete-protected true

Prevents accidental deletion.

Read replicas are essential for scaling MySQL HeatWave beyond single-instance limitations. By distributing read traffic across multiple instances, you improve application performance, support business analytics, enable geographic distribution, and enhance disaster recovery capabilities.

Key takeaways:

Asynchronous replication: Data replicated with minimal lag
18 replicas maximum: Scale read capacity significantly
Shape requirement: Minimum 8 ECPUs or 4 OCPUs
Automatic load balancer: Simplifies traffic distribution
Multiple connection methods: Bastion, compute, VPN, load balancer
Rolling upgrades for HA: Minimal downtime during maintenance
Monitor lag: Critical for maintaining data freshness
Disaster recovery: Promote replica to standalone if needed

Whether supporting global applications, offloading analytics, or implementing disaster recovery, MySQL HeatWave read replicas provide the scalability and flexibility needed for modern database architectures.

MySQL HeatWave: Migrating Your Data to HeatWave Service

Wed, 26 Nov 2025 10:08:43 +0000

Migrating data to MySQL HeatWave is a critical step in leveraging the cloud's power for your database workloads. MySQL Shell dump utilities provide the recommended method for exporting data from source MySQL databases and importing them into HeatWave DB systems. These utilities offer HeatWave-specific compatibility checks, parallel processing, and automatic modifications to ensure smooth migrations.

This guide covers dump utilities, compatibility checks, and best practices for successful data migration to MySQL HeatWave.

Understanding Data Migration

Data migration to MySQL HeatWave involves moving data from a source MySQL database (on-premises, cloud, or other infrastructure) to a target MySQL HeatWave DB system in Oracle Cloud Infrastructure.

Migration Process:

Export data from source using MySQL Shell dump utilities
Store dump files in OCI Object Storage or locally
Run compatibility checks for HeatWave
Apply necessary modifications
Import data into HeatWave DB system using util.loadDump()

MySQL Shell Dump Utilities

MySQL Shell provides three dump utilities: util.dumpInstance(), util.dumpSchemas(), and util.dumpTables(). Each serves different migration scenarios.

1. util.dumpInstance()

Exports all compatible schemas to Object Storage bucket or local files, including users, events, routines, and triggers by default

What It Exports:

All user schemas (excludes system schemas)
User accounts and privileges
Events, routines, and triggers
Complete database structure

What It Excludes:

System schemas: mysql, sys, performance_schema, information_schema, ndbinfo

Example Usage:

util.dumpInstance("/path/to/dump", {
  ocimds: true,
  threads: 8,
  compatibility: ["force_innodb", "strip_definers", "strip_tablespaces"]
})

To Object Storage:

util.dumpInstance("", {
  osBucketName: "myMigrationBucket",
  osNamespace: "my-namespace",
  ocimds: true,
  threads: 16,
  compatibility: ["force_innodb", "strip_definers", "strip_restricted_grants", 
                  "strip_tablespaces", "create_invisible_pks"]
})

2. util.dumpSchemas()

Exports selected schemas with all tables, triggers, routines, and events of those schemas

What It Exports:

All tables in specified schemas
Triggers associated with tables
Routines (stored procedures and functions)
Events
Views

Example Usage:

util.dumpSchemas(["sales", "inventory", "customers"], "/path/to/dump", {
  ocimds: true,
  threads: 4,
  compatibility: ["force_innodb", "strip_definers"]
})

Multiple Schemas to Object Storage:

util.dumpSchemas(
  ["tpch", "employees"], 
  "", 
  {
    osBucketName: "mysqlData",
    osNamespace: "abcdefghijkl",
    ocimds: true,
    threads: 16,
    compatibility: ["force_innodb", "strip_restricted_grants", 
                    "strip_definers", "strip_tablespaces", "create_invisible_pks"]
  }
)

3. util.dumpTables()

Exports selected tables or views from a schema, including triggers of exported tables

What It Exports:

Specified tables or views
Triggers associated with those tables
Schema structure needed to recreate tables

Example Usage:

util.dumpTables("sales", ["orders", "order_items", "customers"], "/path/to/dump", {
  ocimds: true,
  threads: 2,
  compatibility: ["force_innodb"]
})

Dump Utility Options

Dump utilities support parallel dumping with multiple threads, file compression, OCI Object Storage streaming, and MySQL HeatWave compatibility checks and modifications

Essential Options

ocimds

Enables HeatWave Service compatibility checks. When set to true, the dump cannot proceed if incompatibilities exist.

ocimds: true

Critical: Dumps loaded into HeatWave must be created with ocimds: true.

threads

Number of parallel threads for dump operation. More threads = faster dumps but higher source load.

threads: 16  // Use 4-16 for most workloads

compatibility

Array of modifications to apply during export to ensure HeatWave compatibility.

compatibility: ["force_innodb", "strip_definers", "strip_tablespaces", 
                "create_invisible_pks"]

bytesPerChunk

Defines chunk size for large datasets. Default is 64MB.

bytesPerChunk: "128M"  // For large datasets

compression

Compression algorithm for dump files.

compression: "zstd"  // Options: zstd (default), gzip, none

maxRate

Maximum data read throughput during dump (in bytes per second).

maxRate: "50M"  // Limit to 50 MB/s to reduce source impact

Advanced Options

dryRun

Displays what would be dumped and results of compatibility checks without proceeding

mode: "dryrun"

Always run dryRun first to identify issues before actual export.

consistent

Ensures data consistency by using FLUSH TABLES WITH READ LOCK.

consistent: true  // Default

excludeSchemas / includeSchemas

Filter which schemas to dump.

excludeSchemas: ["test", "staging"]
// Or
includeSchemas: ["production", "analytics"]

excludeTables / includeTables

Filter tables within schemas (for dumpSchemas and dumpInstance).

excludeTables: ["sales.temp_*", "logs.old_*"]

defaultCharacterSet

Specify character set for dump files.

defaultCharacterSet: "utf8mb4"  // Recommended

Performing a Dry Run

Always perform a dry run before actual export to identify compatibility issues

Dry Run Command

util.dumpInstance("", {
  mode: "dryrun",
  ocimds: true
})

What Dry Run Does

Connects to source database
Analyzes schemas, tables, and objects
Runs HeatWave compatibility checks
Lists all objects to be dumped
Reports compatibility issues
Does not export any data

Sample Dry Run Output

Checking for compatibility with MySQL HeatWave Service

Compatibility issues with MySQL HeatWave Service 8.0.32
1) Schema 'sales'
  Table 'legacy_orders' uses MyISAM storage engine (issue: engine)
  Table 'customers' has no primary key (issue: no_pk)
  View 'active_orders' has DEFINER='admin'@'localhost' (issue: definer)

2) Schema 'inventory'
  Table 'items' uses MyISAM storage engine (issue: engine)
  
Compatibility issues found. Use 'compatibility' option to apply modifications.
Suggested options: ["force_innodb", "strip_definers", "create_invisible_pks"]

HeatWave Compatibility Checks

Setting ocimds option to true enables compatibility checks that list all issues when instance is dumped

Common Compatibility Issues

1. Non-InnoDB Storage Engines

Issue: HeatWave only supports InnoDB
Solution: Use force_innodb compatibility option

2. Missing Primary Keys

Issue: High Availability requires primary keys on all tables
Solution: Use create_invisible_pks or ignore_missing_pks

3. DEFINER Clauses

Issue: Requires special privileges to create with custom definer
Solution: Use strip_definers

4. Tablespace Clauses

Issue: HeatWave has tablespace restrictions
Solution: Use strip_tablespaces

5. Restricted Grants

Issue: Some privileges not supported in HeatWave
Solution: Use strip_restricted_grants

6. External Authentication

Issue: Some authentication plugins not supported
Solution: Use skip_invalid_accounts

HeatWave Compatibility Modifications

Compatibility options automatically modify exported data to resolve compatibility issues

Available Compatibility Options

force_innodb

Modifies ENGINE clause of CREATE TABLE statements to specify InnoDB for tables using incompatible storage engines

What It Does:

Changes ENGINE=MyISAM to ENGINE=InnoDB
Changes other engines (MEMORY, CSV, etc.) to InnoDB
Leaves InnoDB tables unchanged

Example:

-- Before
CREATE TABLE orders (...) ENGINE=MyISAM;

-- After
CREATE TABLE orders (...) ENGINE=InnoDB;

strip_definers

Removes DEFINER=account clause from views, routines, events, and triggers, changing SQL SECURITY to INVOKER

What It Does:

Removes DEFINER clause
Changes SQL SECURITY from DEFINER to INVOKER
Objects created with default definer (user loading schema)

Example:

-- Before
CREATE DEFINER=`admin`@`localhost` VIEW active_orders AS ...

-- After
CREATE VIEW active_orders AS ...

strip_tablespaces

Removes TABLESPACE clause from CREATE TABLE statements so tables are created in default tablespaces

What It Does:

Removes TABLESPACE option from CREATE TABLE
Comments out DATA DIRECTORY, INDEX DIRECTORY, ENCRYPTION options

Example:

-- Before
CREATE TABLE orders (...) TABLESPACE=ts1;

-- After
CREATE TABLE orders (...);

create_invisible_pks

Adds invisible primary keys to tables without primary keys, required for High Availability DB systems

What It Does:

Adds invisible AUTO_INCREMENT column named my_row_id
Sets as primary key
No impact on applications (column is invisible)
Requires MySQL 8.0.24+ on target

Example:

-- Before
CREATE TABLE logs (
  timestamp DATETIME,
  message TEXT
);

-- After
CREATE TABLE logs (
  my_row_id BIGINT UNSIGNED AUTO_INCREMENT INVISIBLE PRIMARY KEY,
  timestamp DATETIME,
  message TEXT
);

Alternative: ignore_missing_pks - ignores missing PKs but cannot load to HA systems.

strip_restricted_grants

Removes privileges that are restricted in MySQL HeatWave Service

Restricted Privileges:

SUPER
FILE
RELOAD
BINLOG_ADMIN
Others specific to managed service

skip_invalid_accounts

Skips user accounts with external authentication plugins not supported in HeatWave.

Also removes:

Accounts without passwords (except roles)
Accounts using unsupported plugins

Applying Compatibility Options

util.dumpInstance("", {
  osBucketName: "migrationBucket",
  ocimds: true,
  compatibility: [
    "force_innodb",
    "strip_definers",
    "strip_restricted_grants",
    "strip_tablespaces",
    "create_invisible_pks"
  ]
})

Step-by-Step Migration Process

Step 1: Prepare Source Database

Ensure MySQL Shell is installed and source is accessible:

mysqlsh --version  # Should be 8.0.21 or later

Step 2: Run Dry Run

Identify compatibility issues:

mysqlsh admin@source_server:3306 -- util dump-instance "" \
  --mode=dryrun \
  --ocimds=true

Step 3: Analyze Dry Run Output

Review compatibility issues and note suggested options:

Storage engine issues → force_innodb
Missing PKs → create_invisible_pks
Definer clauses → strip_definers
Tablespace issues → strip_tablespaces

Step 4: Export Data with Modifications

To Local Files:

mysqlsh admin@source_server:3306 -- util dump-instance /backup/migration \
  --ocimds=true \
  --threads=8 \
  --bytesPerChunk=128M \
  --compatibility='["force_innodb","strip_definers","strip_tablespaces","create_invisible_pks"]'

To Object Storage:

mysqlsh admin@source_server:3306 -- util dump-instance "" \
  --osBucketName=migrationBucket \
  --osNamespace=my-namespace \
  --ocimds=true \
  --threads=16 \
  --compression=zstd \
  --compatibility='["force_innodb","strip_definers","strip_restricted_grants","strip_tablespaces","create_invisible_pks"]'

Step 5: Import Data into HeatWave

From Object Storage (during DB system creation):

Select "Import data from Object Storage"
Provide bucket details
HeatWave automatically imports during provisioning

To Existing DB System:

mysqlsh admin@heatwave_endpoint:3306 -- util load-dump /backup/migration \
  --threads=16 \
  --progressFile=/tmp/load-progress.json

From Object Storage to Existing System:

mysqlsh admin@heatwave_endpoint:3306 -- util load-dump migrationBucket \
  --osBucketName=migrationBucket \
  --osNamespace=my-namespace \
  --threads=16

Best Practices

1. Always Use ocimds: true

Ensures HeatWave compatibility checks run during export. Dumps without this cannot be loaded into HeatWave.

2. Run Dry Run First

Identify compatibility issues before actual export to avoid failed migrations

3. Use Appropriate Thread Count

Small databases (< 10GB): 2-4 threads
Medium databases (10-100GB): 8-12 threads
Large databases (> 100GB): 16-32 threads

Balance: More threads = faster but higher source load.

4. Chunk Large Tables

For very large tables, use bytesPerChunk:

bytesPerChunk: "256M"  // For tables > 10GB

5. Use Compression

Reduces storage and transfer costs:

compression: "zstd"  // Best compression ratio and speed

6. Monitor Dump Progress

Dumps display progress including:

Total rows to dump
Rows dumped so far
Percentage complete
Throughput (rows/sec, MB/sec)

7. Test with Subset First

For large migrations, test with single schema:

util.dumpSchemas(["test_schema"], ...)

Validate before full migration.

8. Plan for Downtime

Read-only mode during export for consistency
Or use consistent: true for point-in-time snapshot
Plan maintenance window for production migrations

9. Validate After Import

-- Check row counts
SELECT COUNT(*) FROM imported_table;

-- Verify structure
SHOW CREATE TABLE imported_table;

-- Test queries
SELECT * FROM imported_table LIMIT 10;

10. Keep Dump Files

Retain dump files until migration validated:

Local backups or Object Storage
Useful for troubleshooting
Allows re-import if needed

Troubleshooting Common Issues

Issue 1: Dump Fails with Compatibility Errors

Symptoms: Export aborts with list of incompatible objects

Solution:

Run dry run to see all issues
Apply suggested compatibility options
For manual issues, fix source schema
Re-run export

Issue 2: Out of Memory During Dump

Symptoms: Process killed, "out of memory" errors

Solution:

Reduce thread count
Increase system memory
Use bytesPerChunk to process smaller chunks

Issue 3: Load Fails with "local_infile" Error

Symptoms: Import fails, local_infile error

Solution:

-- On HeatWave DB system
SET GLOBAL local_infile = ON;

(Default is ON in HeatWave, but verify)

Issue 4: Tables Missing After Import

Symptoms: Some tables not imported

Solution:

Check excluded tables in dump command
Verify tables exported in dump metadata
Check for errors in load-progress.json
Ensure compatibility options applied correctly

Migrating data to MySQL HeatWave using MySQL Shell dump utilities provides a reliable, efficient path to the cloud. By understanding the three dump utilities, running compatibility checks, and applying appropriate modifications, you can ensure smooth migrations with minimal issues.

Key takeaways:

Use util.dumpInstance() for complete database migrations
Use util.dumpSchemas() for selective schema migrations
Use util.dumpTables() for specific table exports
Always run dry run first to identify compatibility issues
Apply compatibility options to auto-fix common issues
Test with subset before full migration
Monitor and validate throughout the process

Whether migrating from on-premises, other clouds, or between HeatWave regions, MySQL Shell dump utilities provide the tools needed for successful data migration to MySQL HeatWave.

MySQL HeatWave: Replicating to MySQL Server (Outbound Replication)

Mon, 24 Nov 2025 09:16:13 +0000

Outbound replication enables you to copy data from a MySQL HeatWave DB system to external MySQL instances, supporting data distribution, disaster recovery, and hybrid cloud architectures. Unlike inbound replication where HeatWave acts as the replica, outbound replication positions HeatWave as the source, allowing you to replicate data to on-premises servers, cloud compute instances, or other DB systems.

This guide covers outbound replication scenarios, configuration, prerequisites, and best practices.

What is Outbound Replication?

Outbound replication uses a replication channel to copy transactions from a DB system to another location asynchronously. A channel connects the source DB system (HeatWave) to a replica database (external MySQL instance or another DB system), copying data changes from source to replica.

Key Characteristics:

Source: MySQL HeatWave DB system
Replica: External MySQL instance or another DB system
Asynchronous: Replica doesn't need constant connection
Channel Configuration: Always configured on the replica
Scale-Out: Enables read scaling by distributing load

Important: You can configure only a DB system as the source; you cannot configure a read replica as the source.

Supported Outbound Replication Scenarios

MySQL HeatWave supports three primary outbound replication scenarios. In all scenarios, the replica must be the same version as, or a higher version of MySQL than, the DB system source.

1. DB System to On-Premises MySQL Instance

Replicate from a DB system to an on-premises MySQL server.

Use Cases:

Data repatriation from cloud
Hybrid cloud architectures
Local reporting and analytics
Disaster recovery to on-premises
Compliance requirements for local data

2. DB System to MySQL on Cloud Compute

Replicate to a MySQL instance installed on OCI compute instance or other cloud providers (AWS, Azure, GCP).

Use Cases:

Custom MySQL configurations not available in managed service
Testing new MySQL versions
Migration staging environments
Cross-cloud data distribution

3. DB System to DB System

Replicate from one DB system to another DB system, both on OCI.

Use Cases:

Cross-region replication
Data distribution across geographies
Reporting replicas
Development/test environment refresh

Note: Prior to MySQL 8.3.0-u2, you cannot perform outbound replication from a DB system with MySQL HeatWave Lakehouse enabled. As of MySQL 8.3.0-u2+, outbound replication is supported with Lakehouse, but Lakehouse tables can only replicate to replica DB systems, not external replicas.

Outbound vs. Inbound Replication

Aspect	Outbound Replication	Inbound Replication
Source	HeatWave DB System	External MySQL or DB System
Replica	External MySQL or DB System	HeatWave DB System
Channel Config	On replica	In HeatWave Service
Use Case	Data out of cloud	Data into cloud
Management	Manual on replica	Managed in OCI Console

Prerequisites for Outbound Replication

Source DB System Requirements

1. Active DB System

An active MySQL HeatWave DB system must be available as the replication source.

2. Binary Logging Enabled

Binary logging is enabled by default on all DB systems. Verify:

-- On source DB system
SHOW VARIABLES LIKE 'log_bin';

3. Row-Based Replication

Row-based binary logging must be enabled (default):

SHOW VARIABLES LIKE 'binlog_format';
-- Should return: ROW

4. GTID Mode Enabled (Recommended)

GTID mode is disabled by default on DB systems but strongly recommended for outbound replication.

Why GTID Matters:

Simplifies replication setup with auto-positioning
Easier failover and recovery
Better tracking of transactions
Reduces configuration complexity

Note: GTID mode is a compatibility option you must consider when creating your DB system.

5. Replication User Account

Create a dedicated replication user on the source DB system with required privileges.

Create replication user:

-- Connect to source DB system
CREATE USER 'repl_user'@'%' 
  IDENTIFIED BY 'SecurePassword123!' 
  REQUIRE SSL;

GRANT REPLICATION SLAVE ON *.* TO 'repl_user'@'%';
FLUSH PRIVILEGES;

Replica MySQL Server Requirements

1. MySQL Version Compatibility

The replica must be the same version as, or a higher version than, the DB system source. For example, you can replicate from MySQL 8.0 to MySQL 8.4, but not from MySQL 8.4 to MySQL 8.0.

2. Compatible Configuration Options

The replica must have matching configuration for certain system variables:

lowercase_table_names: The source and replica must run with the same lower_case_table_names system variable value (0, 1, or 2).

Enable Partial Revoke (if used on source): If your source DB system uses partial revokes, the replica must also have this enabled:

-- On replica
SET GLOBAL partial_revokes = ON;

GTID Mode (recommended): If source uses GTID, replica should have GTID enabled:

-- On replica
SET GLOBAL enforce_gtid_consistency = ON;
SET GLOBAL gtid_mode = OFF_PERMISSIVE;
SET GLOBAL gtid_mode = ON_PERMISSIVE;
SET GLOBAL gtid_mode = ON;

Timezone Information Loaded: If your source DB system uses named timezones, load timezone information on the replica:

# On replica server
mysql_tzinfo_to_sql /usr/share/zoneinfo | mysql -u root -p mysql

3. Bi-Directional Communication Enabled

The replica must be able to connect to the source DB system:

Network path exists (VPN, FastConnect, direct connectivity)
Firewall rules allow MySQL port (3306)
Security lists/NSGs permit traffic

4. Row-Based Binary Logging

The replica should also use row-based replication:

SET GLOBAL binlog_format = 'ROW';

Network Connectivity Options

Connecting External MySQL to HeatWave Source

For external MySQL servers replicating FROM HeatWave, establish connectivity:

Option 1: Network Load Balancer

Set up a Network Load Balancer to provide external access to the DB system:

Steps:

Create Network Security Group allowing access from replica IP
Configure LB Listener (add NSG)
Configure LB Backend
Specify Health Check Policy
Configure backend to access MySQL source

Security: Enforce SSL/TLS for all connections.

Option 2: Site-to-Site VPN

For on-premises replicas:

IPSec VPN tunnel over internet
Encrypted connectivity
Up to 250 Mbps per tunnel

Option 3: FastConnect

For production on-premises replicas:

Dedicated private connection
Up to 10 Gbps bandwidth
Predictable performance
Lower latency

Loading Initial Data from DB System to MySQL Server

Before starting replication, export existing data from the source to the replica.

Step 1: Export Data from Source DB System

Use MySQL Shell Instance Dump utility to export data:

mysqlsh admin@<DB_SYSTEM_IP>:3306 -- util dump-instance /path/to/dump \
  --excludeSchemas=mysql,sys,performance_schema,information_schema \
  --ocimds=true

What This Exports:

All user schemas and data
User accounts and privileges
GTID executed position
Excludes system schemas

Why Exclude System Schemas:

System schemas managed differently
Prevents replication conflicts
User accounts can be recreated separately
Best practice for replication setup

Important: If using GTID-based replication, capture the value of the source server's system variable gtid_executed from the dump metadata.

Step 2: Import Data into Replica MySQL Server

Import the dump into your replica MySQL instance:

mysqlsh admin@<REPLICA_IP>:3306 -- util load-dump /path/to/dump \
  --includeSchemas=<schema_list> \
  --progressFile=/path/to/progress.json

Import User Accounts:

mysqlsh admin@<REPLICA_IP>:3306 -- util load-dump /path/to/dump \
  --includeUsers=true \
  --excludeUsers=admin,root

Step 3: Update gtid_purged Settings

If using GTID-based replication, set gtid_purged on the replica:

-- On replica
RESET MASTER;
SET GLOBAL gtid_purged='<gtid_executed_value>';

Example:

SET GLOBAL gtid_purged='3E11FA47-71CA-11E1-9E33-C80AA9429562:1-5';

Configuring Outbound Replication Channel

The replication channel is always configured on the replica (not in HeatWave Service for outbound replication).

Configuration Parameters

Source Connection Settings

source_host: IP address or hostname of the source DB system

SOURCE_HOST='10.0.1.100'

source_port: MySQL port (default 3306)

SOURCE_PORT=3306

source_user: Replication user account

SOURCE_USER='repl_user'

source_password: Replication user password

SOURCE_PASSWORD='SecurePassword123!'

source_ssl: Enable SSL/TLS (recommended)

SOURCE_SSL=1

Replication Positioning

GTID Auto-Positioning (recommended):

SOURCE_AUTO_POSITION=1

Manual Binary Log Positioning (if GTID not used):

SOURCE_LOG_FILE='mysql-bin.000003'
SOURCE_LOG_POS=154

Setting Up the Replication Channel

Connect to the replica MySQL server and run:

-- Using GTID auto-positioning (recommended)
CHANGE REPLICATION SOURCE TO
  SOURCE_HOST='<DB_SYSTEM_IP>',
  SOURCE_PORT=3306,
  SOURCE_USER='repl_user',
  SOURCE_PASSWORD='SecurePassword123!',
  SOURCE_SSL=1,
  SOURCE_AUTO_POSITION=1;

Without GTID (manual positioning):

CHANGE REPLICATION SOURCE TO
  SOURCE_HOST='<DB_SYSTEM_IP>',
  SOURCE_PORT=3306,
  SOURCE_USER='repl_user',
  SOURCE_PASSWORD='SecurePassword123!',
  SOURCE_SSL=1,
  SOURCE_LOG_FILE='mysql-bin.000003',
  SOURCE_LOG_POS=154;

Starting Replication

START REPLICA;

Or for older MySQL versions:

START SLAVE;

Managing Replication

Checking Replication Status

SHOW REPLICA STATUS\G

Key Metrics:

Replica_IO_Running: Should be "Yes"
Replica_SQL_Running: Should be "Yes"
Seconds_Behind_Source: Replication lag
Last_IO_Error: IO thread errors
Last_SQL_Error: SQL thread errors

Stopping Replication

STOP REPLICA;

Resetting Replication

Warning: This clears all replication position data.

STOP REPLICA;
RESET REPLICA ALL;

Reconfiguring the Channel

To change replication settings:

STOP REPLICA;

CHANGE REPLICATION SOURCE TO
  SOURCE_HOST='<NEW_IP>',
  SOURCE_USER='<NEW_USER>',
  -- other parameters
  SOURCE_AUTO_POSITION=1;

START REPLICA;

Monitoring Replication

Using Performance Schema

Query replication metrics:

-- Connection status
SELECT * FROM performance_schema.replication_connection_status\G

-- Applier status
SELECT * FROM performance_schema.replication_applier_status\G

-- Worker threads
SELECT * FROM performance_schema.replication_applier_status_by_worker\G

Setting Up Alerts

Monitor these critical metrics:

Replication lag exceeding threshold (e.g., 60 seconds)
IO or SQL thread stopped
Replication errors
Connection failures

Using Database Management

If your replica is also a DB system on OCI, use Database Management to monitor:

Active connections
Statement count
Fetch state
Apply state
Time behind source
GTID executed

Common Outbound Replication Issues

Issue 1: Replica Cannot Connect to Source

Error: "Can't connect to MySQL server on ''"

Causes:

Network Load Balancer not configured
Security lists blocking traffic
VPN/FastConnect not established
Wrong IP or hostname

Solutions:

Verify Network Load Balancer configuration
Check security lists allow port 3306
Test connectivity: telnet <DB_SYSTEM_IP> 3306
Verify VPN/FastConnect is active

Issue 2: Authentication Failures

Error: "Access denied for user 'repl_user'@''"

Causes:

Replication user not created on source
Wrong credentials
User not allowed from replica IP

Solutions:

Verify user exists on source: SELECT User, Host FROM mysql.user WHERE User = 'repl_user';
Check credentials are correct
Verify user grants: SHOW GRANTS FOR 'repl_user'@'%';

Issue 3: Replication Lag Increasing

Symptoms: Seconds_Behind_Source growing

Causes:

High transaction volume on source
Replica underpowered
Network bandwidth limitations
Large transactions

Solutions:

Increase replica resources
Optimize slow queries
Check network bandwidth
Break large transactions into batches

Issue 4: SSL Connection Failures

Error: SSL-related errors in replication status

Causes:

SSL not configured on source
Certificate issues
Wrong SSL mode

Solutions:

Verify source supports SSL
Use SOURCE_SSL=1 for encrypted connection
Check certificate validity if using certificate verification

Best Practices

1. Always Use GTID Mode

Enable GTID on both source and replica for simplified management:

Easier failover
Automatic position tracking
Simplified reconfiguration

2. Enforce SSL/TLS Encryption

Always encrypt replication traffic:

CREATE USER 'repl_user'@'%' 
  IDENTIFIED BY 'SecurePassword!' 
  REQUIRE SSL;

3. Monitor Replication Lag

Set up alerts for:

Lag > 60 seconds
IO or SQL thread stopped
Replication errors

4. Use Network Load Balancer

For production external replicas:

Provides stable endpoint
Enables health checks
Supports SSL termination
Better security controls

5. Plan for Network Interruptions

Use appropriate binary log retention on source
Monitor relay log space on replica
Test reconnection scenarios

6. Regular Health Checks

Implement automated checks:

#!/bin/bash
# Check replication health
REPLICA_STATUS=$(mysql -e "SHOW REPLICA STATUS\G" | grep "Replica_.*_Running:" | awk '{print $2}')
if [[ "$REPLICA_STATUS" != "Yes" ]]; then
  echo "Replication issue detected"
  # Send alert
fi

7. Test Failover Procedures

Regularly test:

Stopping and starting replication
Switching to different source
Replica promotion scenarios
Application behavior during failures

8. Document Configuration

Maintain documentation of:

Source and replica connection details
Replication user credentials
Network paths used
Failover procedures

Limitations for Outbound Replication

Lakehouse Table Restrictions

Lakehouse tables can only replicate to replica DB systems. Replicating Lakehouse tables to external replicas (on-premises MySQL) causes replication to break with errors.

Workaround: Exclude Lakehouse tables from replication or only replicate to DB system replicas.

Prior to MySQL 8.3.0-u2

Cannot perform outbound replication from a DB system with HeatWave Lakehouse enabled
Cannot enable Lakehouse on HA DB system with HeatWave cluster

Read Replicas as Source

You cannot configure a read replica as the source for outbound replication. Only the primary DB system can serve as the replication source.

Outbound replication from MySQL HeatWave enables flexible data distribution strategies, supporting hybrid cloud architectures, disaster recovery, and read scaling scenarios. By understanding configuration requirements, network connectivity options, and best practices, you can implement reliable outbound replication for your specific needs.

Key takeaways:

Source is HeatWave: DB system acts as replication source
Channel on replica: Always configure replication on the replica side
Use GTID: Simplifies setup and management
Encrypt traffic: Always use SSL/TLS
Network connectivity: Use NLB, VPN, or FastConnect for external replicas
Monitor continuously: Track lag, errors, and thread status
Version compatibility: Replica must be same or higher MySQL version

Whether replicating to on-premises servers, cloud compute instances, or other DB systems, MySQL HeatWave outbound replication provides the flexibility needed for modern distributed architectures.

MySQL HeatWave: Replication Channels and Troubleshooting

Fri, 21 Nov 2025 16:28:02 +0000

Replication channels are the cornerstone of MySQL HeatWave inbound replication, establishing the connection between source databases and replica DB systems. Understanding channel states, operations, and troubleshooting techniques is essential for maintaining reliable data replication. This guide covers channel management, lifecycle states, common errors, and proven troubleshooting strategies.

Understanding Replication Channels

A replication channel represents a path through which changes produced by transactions on a source MySQL server or DB system are replicated to a target DB system.

What Channels Do

Each channel identifies:

A replication source (hostname, port, credentials)
Configuration required to connect to that source
How to copy transactions from source to replica
Replication positioning (GTID or binary log)

Important: Channels have their own lifecycle and can be enabled, disabled, reset, and resumed independently.

Channel States

MySQL HeatWave channels exist in either final states (stable) or transient states (temporary during operations).

Final States

1. ACTIVE

Channel is replicating data successfully.

Characteristics:

Both IO and SQL threads running
Transactions flowing from source to replica
May have slight replication lag (normal)
No errors reported

Console Indicator: Green icon with "Active" text

2. INACTIVE

Channel is not replicating; replication has been stopped.

Common Reasons:

Channel manually disabled by administrator
Created with "Enable automatically" unchecked
DB system is stopped

Console Indicator: Gray icon with "Inactive" text

To Resume: Enable the channel through console or CLI

3. NEEDS_ATTENTION

Channel is not replicating due to an error.

Common Causes:

Source server unreachable
Authentication failure
Binary log files purged on source
SSL/TLS configuration mismatch
Incompatible transactions
DB system is in INACTIVE state

Console Indicator: Red/orange icon with "Needs Attention" text

Action Required: Check channel details page for error messages, resolve issue, then resume channel

4. DELETED

Channel has been permanently removed.

Characteristics:

Appears briefly before removal from list
Cannot be recovered
All channel configuration deleted

Transient States

These are temporary states during channel operations:

1. CREATING

Channel is being created and configured.

Duration: Typically 1-3 minutes

Next State: ACTIVE (if auto-enabled) or INACTIVE

2. UPDATING

Channel is performing an update operation.

Occurs During:

Enabling or disabling channel
Modifying channel configuration
Resuming after error resolution
DB system operations affecting channel

Duration: Typically 30 seconds to 2 minutes

3. DELETING

Channel deletion is in progress.

Duration: Typically 1-2 minutes

Next State: DELETED

Channel Operations

Enabling a Channel

Starts replication from the source if no errors exist.

Console:

Navigate to Databases → HeatWave MySQL → Channels
Select channel → Actions → Enable
Channel transitions: INACTIVE → UPDATING → ACTIVE

CLI:

oci mysql channel update \
  --channel-id <CHANNEL_OCID> \
  --is-enabled true

Disabling a Channel

Stops replication and puts channel into INACTIVE state.

When to Disable:

Maintenance on source server
Temporary halt of replication
Troubleshooting issues
Before configuration changes

Console: Channel → Actions → Disable

CLI:

oci mysql channel update \
  --channel-id <CHANNEL_OCID> \
  --is-enabled false

Resuming a Channel

Restarts replication after error is resolved. Used when channel is in NEEDS_ATTENTION state.

Console: Channel → Actions → Resume

CLI:

oci mysql channel resume \
  --channel-id <CHANNEL_OCID>

Behavior: Channel transitions to UPDATING, then ACTIVE if error resolved

Resetting a Channel

Removes all replication position data except channel configuration. Channel must be INACTIVE first.

Warning: Resets replication position - will re-replicate transactions

Console: Channel → Actions → Reset

CLI:

oci mysql channel reset \
  --channel-id <CHANNEL_OCID> \
  --from-json '{"resetType": "POSITION"}'

Use Cases:

Recovering from corrupted replication position
Starting replication from beginning
After major source database changes

Deleting a Channel

Permanently removes channel and all associated configuration.

Console: Channel details → Delete

CLI:

oci mysql channel delete \
  --channel-id <CHANNEL_OCID> \
  --force

Troubleshooting Inbound Replication

Step 1: Identify Source of Failure

When channel enters NEEDS_ATTENTION state, determine the root cause:

Check Channel Details Page

Navigate to channel in console
Click channel name to view details
Review Error section for messages
Check Last Error timestamp
Note error code and description

Check Replication Status on Replica

Connect to replica DB system and query:

SHOW REPLICA STATUS\G

Key Fields:

Replica_IO_Running: Should be "Yes" (checks source connection)
Replica_SQL_Running: Should be "Yes" (checks transaction application)
Last_IO_Errno: IO thread error code
Last_IO_Error: IO thread error message
Last_SQL_Errno: SQL thread error code
Last_SQL_Error: SQL thread error message
Seconds_Behind_Source: Replication lag

Performance Schema Query:

SELECT * FROM performance_schema.replication_connection_status\G

Step 2: Identify Communication Failures

Verify connectivity between replica and source:

Test Network Connectivity

From replica DB system or compute instance in same VCN:

# Test if source is reachable
telnet <source_hostname> 3306

# Or use netcat
nc -zv <source_hostname> 3306

# Check DNS resolution
nslookup <source_hostname>

Test Replication User Authentication

mysql -h <source_hostname> \
      -P 3306 \
      -u <repl_user> \
      -p

If successful, authentication works; issue is elsewhere.

Common Error Codes and Solutions

Error 2003: Can't Connect to MySQL Server

Error Message: "Can't connect to MySQL server on '' (error_code)"

Meaning: Replica cannot establish network connection to source

Common Causes:

Source MySQL server not running
Firewall blocking port 3306
Network path doesn't exist
Wrong hostname or IP address
Source server not listening on expected port

Solutions:

Verify source is running:

-- On source server
SHOW GLOBAL STATUS LIKE 'Uptime';

Check firewall rules:

# On source server (Linux)
sudo iptables -L | grep 3306

# Or for firewalld
sudo firewall-cmd --list-all | grep 3306

Verify MySQL listening on port:

# On source server
sudo netstat -tlnp | grep 3306
# Or
sudo ss -tlnp | grep 3306

Check bind-address:

-- On source server
SHOW VARIABLES LIKE 'bind_address';

Should be 0.0.0.0 or specific IP (not 127.0.0.1 for remote connections)

Test from replica network:

telnet <source_ip> 3306

Error 1045: Access Denied

Error Message: "Access denied for user ''@'' (using password: YES/NO)"

Meaning: Authentication failed - wrong credentials or user doesn't have access from replica IP

Common Causes:

Incorrect username or password
User not granted access from replica IP
User exists but lacks REPLICATION SLAVE privilege
Blank users on source conflicting

Solutions:

Verify replication user exists:

-- On source server
SELECT User, Host FROM mysql.user WHERE User = 'repl_user';

Check user privileges:

SHOW GRANTS FOR 'repl_user'@'%';

Should include: GRANT REPLICATION SLAVE ON *.*

Verify replica IP is allowed:

-- On source
SELECT User, Host FROM mysql.user WHERE User = 'repl_user';

Host should be %, replica subnet (e.g., 10.0.1.%), or specific IP

Remove blank users (common issue):

-- On source
SELECT User, Host FROM mysql.user WHERE User = '';
DROP USER ''@'localhost';
DROP USER ''@'<hostname>';
FLUSH PRIVILEGES;

Test authentication:

mysql -h <source> -u repl_user -p

Error 3159: Insecure Connection

Error Message: "Connections using insecure transport are prohibited while --require_secure_transport=ON"

Meaning: Source requires SSL/TLS but channel not configured for it

Solutions:

Option 1: Configure SSL in channel (recommended)

Edit channel configuration
Set SSL mode to "REQUIRED" or higher
Provide SSL certificates if using VERIFY_CA or VERIFY_IDENTITY

Option 2: Disable require_secure_transport on source (not recommended)

-- On source server
SET GLOBAL require_secure_transport = OFF;

Verify SSL configuration:

-- On source
SHOW VARIABLES LIKE 'require_secure_transport';
SHOW VARIABLES LIKE 'have_ssl';

Error 13114: Binary Log Files Purged

Error Message: "Got fatal error 1236 from master when reading data from binary log: 'Could not find first log file name in binary log index file'" or "The slave is connecting using CHANGE MASTER TO MASTER_AUTO_POSITION = 1, but the master has purged binary logs containing GTIDs that the slave requires"

Meaning: Binary log files needed by replica have been deleted from source

Common Causes:

GTID not supported or configured
Binary log retention too short
Long replication outage
Binary logs manually purged

Solution 1: Fix GTID-Purged (For GTID-based replication)

Step 1: Retrieve gtid_executed from logical dump

When exporting source data with MySQL Shell:

mysqlsh user@source -- util dump-instance /path/to/dump \
  --ocimds=true

Capture gtid_executed value from dump metadata.

Or query source directly:

-- On source
SHOW GLOBAL VARIABLES LIKE 'gtid_executed';

Step 2: Set gtid_purged on replica

Connect to replica and execute:

-- On replica
CALL sys.SET_GTID_PURGED('<gtid_executed_value>');

Example:

CALL sys.SET_GTID_PURGED('3E11FA47-71CA-11E1-9E33-C80AA9429562:1-5');

Step 3: Resume channel

After setting gtid_purged, resume replication:

oci mysql channel resume --channel-id <CHANNEL_OCID>

Solution 2: Rebuild Replication (If GTID fix fails)

For severe cases:

Step 1: Delete channel and replica DB system

oci mysql channel delete --channel-id <CHANNEL_OCID> --force
oci mysql db-system delete --db-system-id <DB_SYSTEM_OCID> --force

Step 2: Take new snapshot of source

mysqlsh user@source -- util dump-instance /path/to/new-dump \
  --excludeSchemas=mysql,sys,performance_schema,information_schema \
  --ocimds=true

Step 3: Create new DB system

Create DB system and import data during creation or after.

Step 4: Create and enable channel

Create new channel with proper GTID positioning from new dump.

Best Practices for Channel Management

1. Monitor Channel Health

Set up alerts:

Channel state changes to NEEDS_ATTENTION
Replication lag exceeds threshold (e.g., 60 seconds)
IO or SQL thread stops

Regular checks:

SHOW REPLICA STATUS\G

2. Maintain Adequate Binary Log Retention

On source server:

-- Set binary log expiration (seconds)
SET GLOBAL binlog_expire_logs_seconds = 172800; -- 48 hours

For AWS RDS:

CALL mysql.rds_set_configuration('binlog retention hours', 48);

3. Document Channel Configuration

Maintain records of:

Source connection details
Replication user credentials
SSL configuration
GTID settings
Network paths used

4. Test Before Production

Create test channel in non-production
Verify connectivity
Test failover scenarios
Document lessons learned

5. Plan for Maintenance Windows

Before source maintenance:

Verify replication lag is minimal
Optionally disable channel
Perform maintenance
Re-enable channel
Monitor for catch-up

6. Use GTID When Possible

Benefits:

Auto-positioning simplifies management
Easier recovery from failures
Better failover capabilities

Advanced Troubleshooting

Replication Lag Increasing

Symptoms: Seconds_Behind_Source continuously growing

Diagnose:

-- Check what's being executed
SHOW REPLICA STATUS\G

-- Check running queries on replica
SHOW PROCESSLIST;

-- Check for lock waits
SELECT * FROM sys.innodb_lock_waits;

Solutions:

Increase replica resources (upgrade shape)
Optimize slow queries on source
Break large transactions into smaller batches
Check for long-running transactions

Channel Constantly Failing

Symptoms: Channel repeatedly enters NEEDS_ATTENTION

Diagnose:

Check error patterns (same error repeating?)
Review source server logs
Check network stability
Verify source server load

Solutions:

Fix root cause (network, authentication, binary logs)
Consider increasing connection timeouts
Upgrade source server resources if overloaded

MySQL Schema Changes Causing Failures

Remember: Changes to mysql schema are not replicated and cause replication to stop

Solution:

Manage user accounts independently

Skip problematic transaction:

SET GLOBAL sql_replica_skip_counter = 1;
START REPLICA;

Better: Avoid mysql schema changes during active replication

Effective channel management and troubleshooting are crucial for reliable MySQL HeatWave inbound replication. By understanding channel states, recognizing common error codes, and following best practices, you can maintain healthy replication and quickly resolve issues.

Key takeaways:

Monitor channel states: ACTIVE, INACTIVE, NEEDS_ATTENTION are key indicators
Check channel details: Error messages provide crucial troubleshooting information
Common errors have common fixes: Network (2003), authentication (1045), SSL (3159), binary logs (13114)
Use GTID: Simplifies management and recovery
Maintain binary log retention: Prevent purged log issues
Test thoroughly: Validate connectivity and authentication before production

MySQL HeatWave: Replicating to DB System (Inbound Replication)

Tue, 18 Nov 2025 09:39:47 +0000

Inbound replication enables data replication from external MySQL sources into HeatWave DB systems, supporting migrations, disaster recovery, and hybrid cloud architectures.

What is Inbound Replication?

Uses a replication channel to copy transactions from a MySQL source to a HeatWave DB system replica.

Key Characteristics:

Asynchronous: Replicas don't need permanent connection; can reconnect to receive updates
Single Source: One source per replica (multi-source not supported)
Source Independence: Source doesn't control replica; replica connects using replication credentials
Not Managed: You configure and maintain the channel and network connectivity

Supported Scenarios

On-Premises to DB System: Cloud migrations, hybrid architectures, disaster recovery
Cloud Compute to DB System: Cross-cloud replication, migrating self-managed to HeatWave
DB System to DB System: Cross-region replication, geographic redundancy

Replication Architecture

Source Database:

MySQL 5.7+ (on-premises, OCI compute, HeatWave DB System, or other clouds)
Binary logging enabled
Row-based replication format
GTID mode enabled (recommended)

Replica Database:

MySQL HeatWave DB System in private subnet
Applies transactions from source asynchronously

Flow: Source commits to binary log → Channel retrieves transactions → Replica receives and applies

Limitations

Row-Based Only: Statement-based and mixed replication not supported
Asynchronous Only: Semi-synchronous not supported
Single Source: Multi-source replication not supported
MySQL Schema Not Replicated: User/privilege changes don't replicate; manage separately
GTID Tags: Pre-May 2024 systems need upgrade for GTID tag support
Applier Privileges: Replication fails if applier lacks privileges for statements
HeatWave Cluster: ALTER TABLE SECONDARY_LOAD/UNLOAD don't affect target cluster
High Availability: Channel suspends during HA upgrades, resumes after completion

Prerequisites

Source Requirements

1. Enable GTID (Recommended):

SET GLOBAL enforce_gtid_consistency = ON;
SET GLOBAL gtid_mode = OFF_PERMISSIVE;
SET GLOBAL gtid_mode = ON_PERMISSIVE;
SET GLOBAL gtid_mode = ON;

2. Enable Binary Logging:

SET GLOBAL log_bin = ON;
SET GLOBAL binlog_format = 'ROW';

3. Extend Binary Log Retention:

-- For AWS RDS:
CALL mysql.rds_set_configuration('binlog retention hours', 24);

4. Create Replication User:

CREATE USER 'repl_user'@'%' 
  IDENTIFIED BY 'StrongPassword123!' 
  REQUIRE SSL;
GRANT REPLICATION SLAVE ON *.* TO 'repl_user'@'%';
FLUSH PRIVILEGES;

5. Network Connectivity: Allow replica access to source MySQL port (3306) with SSL/TLS

Replica Requirements

Active HeatWave DB system
Network path to source (VPN, FastConnect, PrivateLink)
Matching lower_case_table_names value with source

Connectivity Options

On-Premises to HeatWave:

FastConnect: Up to 10 Gbps, low latency, production workloads
Site-to-Site VPN: 250 Mbps per tunnel, cost-effective, dev/test

Cloud to HeatWave:

AWS: PrivateLink or VPN
Azure: OCI-Azure interconnect or VPN
GCP: Partner Interconnect or VPN

Dedicated Replication User Benefits

Minimum Privileges: Grant only REPLICATION SLAVE
Restrict Host: Limit to replica IP/subnet
Enforce Encryption: Use REQUIRE SSL

Example:

CREATE USER 'repl_user'@'10.0.1.%' 
  IDENTIFIED BY 'SecurePassword!' 
  REQUIRE SSL;
GRANT REPLICATION SLAVE ON *.* TO 'repl_user'@'10.0.1.%';

Loading Initial Data

Export from Source:

mysqlsh user@source_host:3306 -- util dump-instance /path/to/dump \
  --excludeSchemas=mysql,sys,performance_schema,information_schema \
  --ocimds=true

Import to Replica:

mysqlsh admin@replica_host:3306 -- util load-dump /path/to/dump \
  --progressFile=/path/to/progress.json

Creating Replication Channel

Using OCI Console

Navigate to DB System → Resources → Channels
Click Create channel
Configure settings (source hostname, port, credentials, SSL mode)
Click Create

Using OCI CLI

oci mysql channel create-from-mysql --generate-full-command-json-input > channel-config.json
# Edit channel-config.json with your settings
oci mysql channel create-from-mysql --from-json file://channel-config.json

Positioning Options

GTID Auto-Positioning (Recommended):

Automatic transaction tracking
No manual position needed
Requires gtid_mode=ON on source

Binary Log Positioning:

-- Get position on source:
SHOW MASTER STATUS;

Use File and Position values when creating channel

Managing Channels

Start: oci mysql channel update --channel-id <ID> --is-enabled true

Stop: oci mysql channel update --channel-id <ID> --is-enabled false

Reset: oci mysql channel reset --channel-id <ID> --from-json '{"resetType": "POSITION"}'

Delete: oci mysql channel delete --channel-id <ID> --force

Monitoring Replication

Check Status:

SHOW REPLICA STATUS\G

Key Metrics:

Replica_IO_Running: Should be "Yes"
Replica_SQL_Running: Should be "Yes"
Seconds_Behind_Source: Replication lag
Last_Error: Error messages

Channel States:

ACTIVE: Running normally
NEEDS_ATTENTION: Requires intervention
INACTIVE: Stopped

Best Practices

Enable GTID: Simplifies management
Use SSL/TLS: Encrypt replication traffic
Monitor Lag: Alert on >60 seconds lag
Test Failover: Regular failover testing
Binary Log Retention: 24-48 hours recommended
Health Checks: Automate monitoring
Gradual Migration: Phased approach for migrations

Troubleshooting

NEEDS_ATTENTION State:

Check error logs in channel details
Verify connectivity and credentials
Confirm SSL configuration and firewall rules

Increasing Lag:

Increase replica resources
Check network bandwidth
Optimize source queries
Break large transactions

MySQL Schema Errors:

Manage users independently on replica
Avoid mysql schema changes during replication

SSL Failures:

Verify source SSL enabled
Use "REQUIRED" mode
Check certificate validity

Key Takeaways

Asynchronous replication tolerates disconnections
Use GTID mode for simplified management
Always encrypt replication traffic
Monitor lag, errors, and channel health continuously
Understand limitations: row-based only, single source, mysql schema restrictions
Test thoroughly before production cutover

Inbound replication provides flexible, reliable data migration and hybrid cloud capabilities for MySQL HeatWave workloads.

Additional Resources

MySQL HeatWave: Backing Up and Restoring DB Systems

Sat, 15 Nov 2025 09:29:28 +0000

MySQL HeatWave provides comprehensive backup and restore capabilities to safeguard data against accidental loss, corruption, cyberattacks, and hardware failures. This guide covers backup types, creation methods, management operations, and point-in-time recovery strategies.

Understanding Backup Types

Full Backup

A full backup backs up all data in the DB system, creating a complete snapshot at a specific point in time.

Characteristics:

Captures all data in the DB system
Backup size usually smaller than DB storage (unused space not backed up)
Self-contained and can be restored independently
Takes longer than incremental backups
Consumes more storage space

Incremental Backup

An incremental backup backs up only data added or changed since the last backup.

Characteristics:

First incremental backup is always a full backup
Subsequent backups capture only changes (deltas)
Faster to create than full backups
Default backup type for MySQL HeatWave

Important: For data recovery, there's no functional difference between incremental and full backups. You can restore all data from any backup.

Backup Merging

When you delete a backup, it merges into the following incremental backup. The latest backup is not merged as no subsequent backup exists.

Backup Creation Types

1. Manual Backup

Created by users via OCI Console, CLI, or REST API.

Characteristics:

Retention: 1-365 days
Can be created on inactive DB systems
Not automatically deleted with DB system
Cannot be created on Always Free systems

CLI Example:

oci mysql backup create \
  --db-system-id <DB_SYSTEM_OCID> \
  --backup-type INCREMENTAL \
  --display-name "Pre-Upgrade-Backup" \
  --retention-in-days 30

2. Automatic Backup

Created automatically at scheduled time.

Characteristics:

Retention: 1-35 days (default: 7 days)
Created even when DB system is inactive
Binary logs backed up every 5 minutes
For HA systems: cannot be disabled
Deleted with DB system (unless retention configured)

Configure via CLI:

oci mysql db-system update \
  --db-system-id <DB_SYSTEM_OCID> \
  --backup-policy '{
    "isEnabled": true,
    "retentionInDays": 14,
    "pitrPolicy": {"isEnabled": true}
  }'

3. Final Backup

Created before deleting a DB system.

Characteristics:

Retention: 365 days (fixed)
Acts as safety net for inadvertent deletions
Cannot be modified after creation

4. Operator Backup

Created by MySQL Support team for troubleshooting.

Characteristics:

Always full backups
Deleted automatically after investigation
Do not affect service limits
Managed by Oracle Support

Managing Backups

Listing Backups

CLI:

# List all backups
oci mysql backup list --compartment-id <COMPARTMENT_OCID>

# List for specific DB system
oci mysql backup list \
  --compartment-id <COMPARTMENT_OCID> \
  --db-system-id <DB_SYSTEM_OCID>

Updating Backups

CLI:

oci mysql backup update \
  --backup-id <BACKUP_OCID> \
  --display-name "Updated-Name" \
  --retention-in-days 20

Moving Backups

CLI:

oci mysql backup change-compartment \
  --backup-id <BACKUP_OCID> \
  --compartment-id <TARGET_COMPARTMENT_OCID>

Deleting Backups

Soft Delete: When enabled, backups are retained for additional 7 days in DELETE_SCHEDULED state before permanent deletion.

CLI:

oci mysql backup delete --backup-id <BACKUP_OCID> --force

Validating Backups

Validation restores backup to a temporary standalone DB system (invisible to customer), then immediately deletes it.

Validation States: NOT_VALIDATED, VALIDATED, NEEDS_VALIDATION, FAILED

Point-in-Time Recovery (PITR)

PITR allows restoring a DB system to a specific point in time, achieving ~5-minute RPO.

How PITR Works

MySQL HeatWave takes initial full backup, then incremental backups
Binary logs backed up every 5 minutes
On PITR initiation, identifies latest backup before selected point
Restores from backup, applies binary log changes
Creates new DB system with data at exact point

Benefits

PITR safeguards against:

Accidental data loss: Dropped tables, mistaken DELETE/UPDATE
Cyberattacks: Ransomware, SQL injection, malicious modifications
Equipment failure: Hardware corruption, storage failures
Human error: Incorrect batch operations, failed deployments

Enabling PITR

Requirements:

Automatic backups must be enabled
Prior to MySQL 8.3.0-u2, cannot enable with HeatWave Lakehouse

CLI:

oci mysql db-system update \
  --db-system-id <DB_SYSTEM_OCID> \
  --backup-policy '{
    "isEnabled": true,
    "retentionInDays": 14,
    "pitrPolicy": {"isEnabled": true}
  }'

PITR Recovery Window

Earliest restore point: When PITR enabled or first backup time (whichever is later)
Latest restore point: Current time (within last 5 minutes)

Supported Scenarios

Standalone → Standalone or HA
HA → Standalone or HA

Requirements for restoring to HA:

Primary keys on every table
MySQL 8.0.24 or higher

Restoring from Backups

Console Method

Navigate to DB System details → Resources → Backups
Select backup → Click Restore backup
Configure new DB system settings
No admin credentials requested (uses original credentials)
Click Restore

PITR Restore (CLI)

oci mysql db-system create-from-backup \
  --admin-username admin \
  --admin-password <PASSWORD> \
  --compartment-id <COMPARTMENT_OCID> \
  --shape-name MySQL.8 \
  --subnet-id <SUBNET_OCID> \
  --display-name "PITR-Restored-System" \
  --source-type PITR \
  --source-backup-id <BACKUP_OCID> \
  --point-in-time-recovery-details '{
    "pointInTime": "2024-11-15T14:30:00.000Z"
  }'

Restoration Process

New DB system provisioned
Backup restored
Binary logs replayed (for PITR)
DB system transitions to ACTIVE
Original DB system unchanged

Duration: Typically 15-45 minutes for moderate databases (excludes PITR replay and additional features like HA, HeatWave cluster)

Best Practices

1. Implement 3-2-1 Rule

3 copies of data
2 different storage types
1 off-site copy

2. Enable Automatic Backups

Always enable for production
Set retention based on requirements (7-35 days)
Schedule during low-traffic periods
Enable PITR for critical systems

3. Create Manual Backups

Before major deployments
Prior to upgrades
For long-term archival (up to 365 days)

4. Test Regularly

Validate backup integrity
Practice restoration procedures
Verify RTO and RPO metrics

5. Use Regional Copies

Enable for mission-critical systems
Protects against regional disasters
Available on OCI only

6. Monitor Backup Health

Set up failure alerts
Monitor backup size trends
Track backup duration
Review logs regularly

7. Document Procedures

Maintain restoration runbooks
Document PITR decision criteria
Track retention policies

8. Optimize Costs

Right-size retention periods
Use incremental backups (default)
Clean up obsolete backups
Leverage soft delete

Platform-Specific Differences

MySQL HeatWave on OCI:

Full backup and PITR support
Regional backup copies available
All backup types supported

MySQL HeatWave on AWS:

Backups are Amazon EBS snapshots saved to S3
HeatWave cluster provisioning not supported when restoring to HA
Long periods between backups may slow operations

Always Free Systems:

Cannot create manual backups
Automatic backup retention fixed at 1 day
PITR not supported

Troubleshooting

Backup Creation Failures

Causes: Insufficient storage, long-running transactions, connectivity issues

Solutions:

Check DB system is ACTIVE
Verify sufficient storage
Review error messages
Check long-running transactions: SHOW PROCESSLIST;
Retry backup creation

PITR Restore Too Long

Causes: Large binary log replay, slow storage, complex database

Solutions:

Choose earlier restore point
Use larger shape
Monitor work request progress
Consider regular backup instead

Validation Failures

Causes: Backup corruption, compatibility issues

Solutions:

Retry validation
Test restore to actual DB system
Create new backup and validate
Contact Oracle Support

Effective backup and recovery strategies are fundamental for data integrity and business continuity in MySQL HeatWave. Key takeaways:

Use both backup types: Full and incremental work seamlessly
Enable automatic backups: Essential for production
Leverage PITR: Provides ~5-minute RPO
Test regularly: Validate backups and practice restoration
Plan for disasters: Use regional copies and off-site backups
Monitor and maintain: Track health and optimize retention

MySQL HeatWave's comprehensive backup and restore capabilities ensure quick, reliable data recovery from accidental deletions, system failures, or security incidents.

MySQL HeatWave: Connecting and Managing DB Systems

Wed, 12 Nov 2025 09:12:42 +0000

MySQL HeatWave: Connecting and Managing DB Systems

Introduction

MySQL HeatWave provides a fully managed database service with robust connectivity options and comprehensive management capabilities. This guide covers DB system endpoints, secure connections, and lifecycle management through the OCI console and CLI tools.

Understanding DB System Endpoints

Endpoint Components

A MySQL HeatWave DB system endpoint consists of three key elements:

1. Private IP Address

Assigned from the VCN subnet CIDR range
Remains constant throughout the DB system's lifetime
Not directly accessible from the public internet
Provides secure, isolated connectivity

2. MySQL Port Number (3306)

Port 3306 is the default port for MySQL protocol connections (used by MySQL Client and as backup by MySQL Shell). You can specify a port between 1024-65535 when creating the DB system.

3. X Protocol Port Number (33060)

Port 33060 is the default for X Protocol connections (used by MySQL Shell). Supports asynchronous operations, CRUD operations, document store functionality, and better performance for modern applications.

Platform-Specific Differences

MySQL HeatWave on OCI: Private IP addresses within VCN, requires network path through VCN/VPN/bastion

MySQL HeatWave on AWS: Public endpoint with FQDN, requires allowed client IPs in CIDR format, TLS 1.2 encryption

MySQL HeatWave for Azure: Hybrid networking between Azure VNet and OCI VCN with private connectivity

Network Path Requirements

Because DB systems use private IP addresses, you need one of these network paths:

Option 1: Same VCN

Connect from resources within the same VCN. Lowest latency, highest security, no additional networking costs.

Option 2: Same Region (Different VCN)

Use VCN Peering or Dynamic Routing Gateway (DRG) to connect VCNs in the same region.

Option 3: On-Premises Network

Site-to-Site VPN: IPSec VPN tunnel, up to 250 Mbps per tunnel
FastConnect: Dedicated physical connection, up to 10 Gbps bandwidth

Option 4: Bastion Host

Use OCI Bastion service or self-managed bastion host in public subnet for secure administrative access.

Option 5: OCI Cloud Shell

Browser-based terminal with pre-installed MySQL Shell. Free, no local setup required.

Connecting to MySQL HeatWave

Using OCI Cloud Shell

# Launch Cloud Shell from OCI Console
mysqlsh admin@10.0.1.100:3306 --sql

# Using X Protocol
mysqlsh admin@10.0.1.100:33060

# With database name
mysqlsh admin@10.0.1.100:3306/myapp_db --sql

Using Compute Instance

Install MySQL Shell (Oracle Linux/RHEL):

sudo yum install -y mysql-shell

Connect:

mysqlsh --host <DB_PRIVATE_IP> --port 3306 --user admin --password --sql

Application Connections

Python:

import mysql.connector
config = {
    'user': 'admin',
    'password': 'your_password',
    'host': '10.0.1.100',
    'port': 3306,
    'database': 'myapp_db',
    'ssl_disabled': False
}
connection = mysql.connector.connect(**config)

Node.js:

const mysql = require('mysql2');
const connection = mysql.createConnection({
  host: '10.0.1.100',
  port: 3306,
  user: 'admin',
  password: 'your_password',
  database: 'myapp_db',
  ssl: { rejectUnauthorized: true }
});

Managing DB Systems

Accessing Management Console

Log in to OCI Console at cloud.oracle.com
Navigate: Menu → Databases → HeatWave MySQL → DB Systems
Select compartment and click DB system name

Lifecycle Operations

Stop a DB System

Purpose: Stops compute billing (storage billing continues)

When to use: Non-production environments, cost optimization during idle periods

Impact: Connections terminated, DB unavailable, HeatWave cluster also stops and reloads data on restart

Console: DB System details → Stop → Select shutdown type (Fast/Slow/Immediate) → Confirm

CLI:

oci mysql db-system stop --db-system-id <OCID> --shutdown-type FAST

Start an Inactive DB System

Resume a stopped system. Billing resumes, HeatWave cluster automatically restarts and reloads data.

CLI:

oci mysql db-system start --db-system-id <OCID>

Startup times: Fast shutdown (2-5 min), Slow shutdown (1-3 min), Immediate shutdown (5-15 min)

Restart an Active DB System

Shuts down then restarts immediately. Brief downtime (3-10 minutes), all connections terminated, uncommitted transactions rolled back.

CLI:

oci mysql db-system restart --db-system-id <OCID> --shutdown-type FAST

Delete a DB System

Warning: Permanently deletes DB system and all data. Cannot be undone.

Before deleting: Create final backup, export critical data, check dependencies, review deletion policy

Console: DB System details → Delete → Choose backup retention → Type DB system name → Confirm

CLI:

oci mysql db-system delete --db-system-id <OCID> --force

Understanding DB System States

State Indicators

Green (Active/Available):

ACTIVE: Fully operational, accepting connections
UPDATING: Configuration changes in progress

Yellow (Transitional):

CREATING: Being provisioned (15-30 minutes)
STARTING: Booting up after stop
STOPPING: Shutting down
RESTARTING: Shutdown + automatic startup

Red (Failed/Error):

FAILED: Operation failed, may require recreation
DELETING: Deletion in progress

Gray (Inactive/Deleted):

INACTIVE: Stopped, no compute billing
DELETED: Permanently removed

Monitoring States

CLI:

oci mysql db-system get --db-system-id <OCID> --query 'data."lifecycle-state"'

Expected transition times: Creating (15-30 min), Stopping (2-5 min), Starting (3-10 min), Restarting (5-15 min), Deleting (10-20 min)

DB System Details Page

General Information: Name, OCID, compartment, availability domain, MySQL version, HA status

Configuration: Shape, CPU/memory, storage capacity, configuration applied, endpoints/ports

Networking: VCN/subnet, private IP, security lists/NSGs, DNS hostname

HeatWave Cluster (if attached): State, nodes, shape, memory, Lakehouse status

Backups: Policy settings, retention period, last backup time, PITR window

Security Best Practices

1. Network Security

Deploy in private subnets only
Security lists: Allow only ports 3306/33060 from known sources
Use Network Security Groups for granular control

2. Encryption

In Transit: Always use SSL/TLS, TLS 1.2+
At Rest: Enabled by default, Oracle-managed keys or BYOK

3. Credential Management

Store passwords in OCI Vault
Use principle of least privilege
Create separate accounts per application
Enable automatic password rotation

4. Access Monitoring

Enable MySQL Enterprise Audit
Enable VCN flow logs
Monitor failed login attempts
Regular audit log reviews

Troubleshooting Common Issues

Cannot Connect - "Connection Refused"

Causes: Security list not allowing MySQL ports, DB stopped, incorrect IP/port, no network path

Solutions: Verify DB is ACTIVE, check security lists (3306/33060), confirm private IP, test connectivity

"Access Denied" Errors

Causes: Incorrect credentials, user doesn't exist, insufficient privileges

Solutions: Verify credentials, check user exists: SELECT User, Host FROM mysql.user;, grant privileges

"Too Many Connections"

Causes: Application connection leaks, max_connections limit reached

Solutions: Check connections: SHOW PROCESSLIST;, increase max_connections, fix leaks, implement connection pooling

Slow Connection Establishment

Causes: DNS resolution delays, network latency

Solutions: Use IP instead of hostname, disable reverse DNS lookups, verify VCN routing

Conclusion

Connecting to and managing MySQL HeatWave DB systems requires understanding networking concepts, endpoint configurations, and lifecycle operations. This guide covered:

DB system endpoints and network path requirements
Multiple connection methods (Cloud Shell, Compute Instance, applications)
Essential lifecycle operations (stop, start, restart, delete)
DB system states and monitoring
Security best practices
Common troubleshooting scenarios

Whether connecting from Cloud Shell for quick administration or setting up production connectivity, MySQL HeatWave provides flexible, secure options for all your database needs.

MySQL HeatWave: Identifying and Testing Failover in High Availability DB Systems

Mon, 10 Nov 2025 10:01:57 +0000

In a high availability MySQL HeatWave DB system, automatic failover is a critical mechanism that ensures business continuity when the primary instance becomes unavailable. Understanding how to identify when a failover has occurred and how to properly test failover scenarios is essential for database administrators managing mission-critical applications.

This guide walks you through recognizing failover events through error messages, understanding the failover process, and implementing best practices for testing failover scenarios without disrupting production systems.

Understanding Failover in High Availability

What is Failover?

Failover is the automatic process where one of the secondary MySQL instances is promoted to become the new primary instance when the current primary fails or becomes unavailable. The promotion happens automatically without manual intervention, minimizing downtime and ensuring continuous availability.

Common Causes of Failover

Failover can be triggered by several scenarios:

Availability Domain Failure: Complete outage of the data center hosting the primary instance
Primary Database Instance Failure: Hardware failure, software crash, or critical system error on the primary instance
Network Connectivity Issues: Loss of network connectivity between the primary instance and the MySQL HeatWave Service management layer
Block Storage Issues: Persistent storage failures affecting the primary instance's ability to read or write data
Maintenance Activities: Certain planned maintenance operations that affect the primary instance

The Failover Process

When failover occurs:

MySQL HeatWave Service detects the primary instance failure
One of the secondary instances is automatically selected for promotion
The selected secondary is promoted to primary status
The DB system endpoint is reassigned to the newly promoted primary instance
The IP address remains the same, but all existing connections are closed
Client applications must reconnect to resume operations
If a HeatWave cluster was attached, it's automatically recreated and data is reloaded

Identifying a Failover: Error Messages and Indicators

Primary Indicators of Failover

1. Console Notification

The most straightforward way to verify if a failover has occurred is through the OCI Console:

Steps to Check:

Open the navigation menu
Select Databases → HeatWave MySQL → DB Systems
Choose your compartment from the List Scope
Click the name of your DB system to open the DB System Details page

If a failover has occurred, a message is displayed stating: "Current placement () differs from preferred placement, due to failover or maintenance activity"

This message indicates that the current primary instance is not in your originally preferred location due to automatic failover.

2. MySQL Event Notifications

When a failover happens, a MySQL - Automatic Recovery event is emitted on the DB system with the additionalDetails.isFailover property set to true

Configure event rules to receive notifications when these events occur.

Error Messages During Failover

When a failover is in progress or has just occurred, applications may encounter various error messages. Understanding these errors helps you identify failover events and implement appropriate retry logic.

Error 1290 (HY000): Read-Only Mode Error

Error Message:

ERROR 1290 (HY000): The MySQL server is running with the --super-read-only option 
so it cannot execute this statement

What It Means:
This error occurs when attempting to execute write operations (INSERT, UPDATE, DELETE, DDL statements) on an instance that is in super-read-only mode

When You'll See It:

During the brief window when a primary is being demoted to secondary
If your application still has connections to the old primary after failover
When attempting writes to a secondary instance

What to Do:

Implement connection retry logic in your application
Close existing connections and reconnect to get routed to the new primary
Ensure your application uses the DB system endpoint, not instance-specific IPs

Error 3100 (HY000): Replication Hook Error

Error Message:

ERROR 3100 (HY000): Error on observer while running replication hook 'before_commit'

What It Means:
This error indicates the system is starting the failover process, and transaction commits are being blocked or rolled back.

Context:
This error can also indicate transaction size errors when the message includes "Transaction of size X exceeds specified limit"

When You'll See It:

At the beginning of a failover event
When large transactions exceed the group replication transaction size limit
During the brief period when the primary instance is being transitioned

What to Do:

Transactions in progress will be rolled back automatically
Implement exponential backoff retry logic
Reconnect and retry the transaction
For transaction size errors, break large transactions into smaller batches

Error 2013 (HY000): Connection Lost During Query

Error Message:

ERROR 2013 (HY000): Lost connection to MySQL server during query

What It Means:
The connection to the MySQL server was lost while executing a query, typically because the primary instance became unavailable during failover.

When You'll See It:

During active failover when the primary instance goes offline
When long-running queries are interrupted by failover
During network disruptions affecting the primary

What to Do:

Close the broken connection
Establish a new connection to the DB system endpoint
Retry the query on the new primary
Implement idempotent query patterns where possible

Error Log Indicators

MySQL error logs provide detailed information about failover events. You can query the Performance Schema to view these logs.

Viewing Error Logs

Connect to your DB system using MySQL Shell or MySQL Client and run:

SELECT * FROM performance_schema.error_log 
WHERE SUBSYSTEM = 'Repl' 
ORDER BY LOGGED DESC 
LIMIT 50;

Key Error Codes Indicating Failover

MY-013213 and MY-011507: Primary Election Change

These errors indicate that a failover has occurred and a secondary instance has been promoted to primary through a Group Replication primary election change

MY-011608: Transaction Size Error

This error indicates a transaction size error where the transaction exceeds the group_replication_transaction_size_limit

MY-011566: Super Read-Only Mode Change

This error indicates that super_read_only has been switched off when an instance is promoted from secondary to primary, or switched on when demoted from primary to secondary.

What It Indicates:

An instance role change has occurred
The instance is transitioning between primary and secondary status
Part of the normal failover process

Additional Failover Indicators

Binary Log Position Differences

After a failover, the current binary log file name and position of the new primary may be different from the old primary

This is because binary logs of each instance are managed independently. Each transaction may be written to different binary log files and positions across instances.

To Check Binary Log Position:

SHOW MASTER STATUS;

Compare before and after failover to identify changes.

Connection String Behavior

While the DB system endpoint IP address remains constant during failover, all existing connections are terminated and must be re-established. Monitor your application's connection pool for sudden spikes in connection closures and new connection attempts.

Testing Failover: Process and Best Practices

Why Test Failover?

Regular failover testing is essential for:

Validating your disaster recovery procedures
Ensuring applications handle failovers gracefully
Training operations teams on recovery processes
Identifying configuration issues before real failures occur
Meeting compliance and audit requirements
Building confidence in your high availability architecture

Switchover vs. Failover

Failover: Automatic, unplanned promotion of a secondary to primary due to primary instance failure.

Switchover: Manual, planned promotion of a secondary to primary for testing, maintenance, or optimization purposes.

When you perform a switchover, the preferred placement and current placement change to the newly selected placement of the primary instance, and the DB system endpoint IP address does not change

Use switchover to safely test failover scenarios without actually causing a failure.

Step-by-Step Failover Testing Process

Phase 1: Set Up a Test DB System

Never test failover on production systems during business hours.

Create a Non-Production HA DB System
- Use a development or staging environment
- Configure it identically to production (same shape, configuration, HA settings)
- Ensure the system has high availability enabled
Import Representative Data
- Load a production-like dataset
- Include tables with primary keys (HA requirement)
- Consider data sensitivity and masking requirements
Deploy a Sample Application
- Use a test application that mimics production workload
- Implement connection pooling and retry logic
- Add monitoring and logging capabilities
- Configure the application to use the DB system endpoint

Phase 2: Establish Baseline Monitoring

Before triggering switchover, establish baseline metrics:

Application Performance Metrics
- Query response times
- Transaction throughput
- Connection pool statistics
- Error rates
Database Metrics
- Active connections
- Current binary log position
- Replication lag (should be minimal in HA systems)
- Current primary instance location
System Health Indicators
- CPU and memory utilization
- Storage I/O metrics
- Network latency between availability domains

Phase 3: Execute Switchover

Switchover causes a short period of downtime while the primary instance is redirected to the newly promoted instance, requiring all database connections to be reopened

Using the Console:

Navigate to your DB System details page
Click the Actions menu
Select Switchover
Choose the target availability domain or fault domain (different from current)
Confirm the switchover operation

Using OCI CLI:

oci mysql db-system update \
  --db-system-id <YOUR_DB_SYSTEM_OCID> \
  --availability-domain <TARGET_AD> \
  --force

Modern Switchover Behavior (for systems created/upgraded after February 25, 2025):

New transactions are blocked when switchover is initiated, running transactions are allowed to complete, and uncommitted transactions are rolled back when the connection to the previous primary is broken

Phase 4: Monitor the Switchover

During Switchover (typically 30-60 seconds):

Watch Application Behavior
- Monitor for connection errors (Error 1290, 2013, 3100)
- Observe connection pool drain and refill
- Track any failed transactions
- Verify automatic retry mechanisms

Check Error Logs

SELECT * FROM performance_schema.error_log 
WHERE LOGGED > DATE_SUB(NOW(), INTERVAL 5 MINUTE)
ORDER BY LOGGED DESC;

Verify HeatWave Cluster Status (if applicable)
When current placement changes in a switchover, HeatWave cluster is detached from the previous primary instance and either reused (same AD) or deleted and recreated (different AD)
```
SHOW GLOBAL STATUS LIKE 'rapid_plugin_bootstrapped';
```

After Switchover:

Verify New Primary Location
- Check OCI Console for current placement
- Confirm switchover to target AD/FD succeeded
- Verify "Current placement" matches target
Validate Application Recovery
- All connections successfully re-established
- Transaction throughput returns to baseline
- Query performance is consistent
- No persistent error conditions
Check Data Consistency
- Verify recent transactions completed successfully
- Confirm no data loss occurred
- Test read and write operations
- Validate application state consistency

Phase 5: Document and Analyze

Key Metrics to Record:

Metric	Before	During	After
Total Downtime	N/A	___ sec	N/A
Connections Dropped	0	___	0
Failed Transactions	0	___	0
Recovery Time	N/A	___ sec	N/A

Analysis Questions:

Did the application handle failover gracefully?
Were retry mechanisms effective?
Did any transactions fail permanently?
How quickly did the system return to normal operation?
Were there any unexpected issues?

Switchover to Return to Preferred Placement

After testing, you can switch back to the original preferred placement:

Option 1: Switchover Back
Perform another switchover to return the primary to the original AD/FD.

Option 2: Update Preferred Placement
You can switchover to the existing primary instance to change the preferred placement to the current placement with no downtime

Best Practices for Handling Failover

1. Test on Non-Production Systems First

Critical Rule: Never test failover on production systems during business hours or peak traffic periods.

Why This Matters:

Switchover causes brief downtime (30-60 seconds)
Connections are forcibly closed
Uncommitted transactions are rolled back
Long-running queries may be aborted

Best Practice:

Maintain a dedicated test environment that mirrors production
Perform regular failover drills quarterly or bi-annually
Test during maintenance windows for production validation
Document all test results and improvements made

2. Use Switchover to Simulate Failover

Recommendation: Use manual switchover operations to test disaster recovery procedures rather than artificially creating failures.

Benefits:

Controlled, predictable process
Ability to schedule during low-traffic periods
Minimizes risk of actual data loss
Provides clean before/after states for analysis

Testing Scenarios:

Same AD, Different Fault Domain: Minimal impact, tests intra-datacenter failover
Different AD: Simulates datacenter failure, tests cross-AD latency and behavior
With Active HeatWave Cluster: Tests analytics workload recovery
Under Load: Tests failover during peak transaction volume

3. Implement Robust Application-Level Retry Logic

Connection Retry Pattern:

import time
import mysql.connector
from mysql.connector import Error

def connect_with_retry(max_retries=5, base_delay=1):
    for attempt in range(max_retries):
        try:
            connection = mysql.connector.connect(
                host='<DB_SYSTEM_ENDPOINT>',
                database='your_database',
                user='admin',
                password='your_password',
                connect_timeout=10
            )
            return connection
        except Error as e:
            if attempt == max_retries - 1:
                raise
            delay = base_delay * (2 ** attempt)
            print(f"Connection failed: {e}. Retrying in {delay}s...")
            time.sleep(delay)

Key Principles:

Always use the DB system endpoint, never instance-specific IPs
Implement exponential backoff (1s, 2s, 4s, 8s, 16s)
Limit retry attempts to avoid infinite loops
Make transactions idempotent when possible
Log all retry attempts for monitoring

4. Move Primary Back After Unplanned Failover

After an automatic failover occurs in production:

The current placement differs from the preferred placement, and a message is displayed indicating "Current placement differs from preferred placement, due to failover or maintenance activity"

Decision Points:

Option A: Leave in New Location

System is stable and operational
Performance is acceptable
No immediate reason to change

Option B: Switchover Back to Preferred Location

Original location has specific advantages
Schedule switchover during next maintenance window
Perform during low-traffic period

Best Practice: Don't rush to move back immediately after failover. Validate the stability of the current state first, then plan a controlled switchover during a maintenance window.

5. Configure Connection Timeouts Appropriately

Recommended Settings:

SET GLOBAL connect_timeout = 10;
SET GLOBAL interactive_timeout = 28800;
SET GLOBAL wait_timeout = 28800;

For Applications Using PrivateLink:

Set shorter connect_timeout values (5 seconds)
Implement aggressive retry logic
Failovers and switchovers take longer when using Query PrivateLink endpoints

6. Test with HeatWave Cluster

If your DB system has a HeatWave cluster attached:

Additional Considerations:

Cluster Behavior During Switchover
- If new primary is in the same AD as previous primary, the existing HeatWave cluster is reused and reattached
- If new primary is in a different AD, the existing HeatWave cluster is deleted and a new one created
Data Reload
- Tables are automatically reloaded from HeatWave Storage Layer
- Monitor reload progress: SHOW GLOBAL STATUS LIKE 'rapid%';
Long-Running Queries
- When current placement changes in a switchover, long-running queries offloaded to HeatWave cluster can get aborted
- Reconnect and rerun queries after cluster is ready

Key Takeaways

Identifying and testing failover in MySQL HeatWave high availability DB systems is a critical skill for database administrators and DevOps teams. By understanding the error messages that indicate failover, implementing proper monitoring, and regularly testing switchover scenarios in non-production environments, you can ensure your applications handle failover gracefully with minimal disruption.

Key takeaways:

Recognize failover indicators: Watch for specific error codes (1290, 3100, 2013) and console messages
Use switchover for testing: Never test on production; use manual switchover to simulate failover safely
Monitor thoroughly: Track error logs, transaction status, and application behavior
Implement retry logic: Applications must handle connection failures and transaction rollbacks gracefully
Plan recovery: After automatic failover, assess whether to switch back during a maintenance window

MySQL HeatWave: Creating High Availability DB Systems and Understanding Limitations

Fri, 07 Nov 2025 14:01:41 +0000

MySQL HeatWave: Creating High Availability DB Systems and Understanding Limitations

Introduction

MySQL HeatWave offers enterprise-grade high availability features that ensure your database remains operational even during hardware failures or maintenance events. Understanding how to properly configure high availability and its associated limitations is crucial for designing resilient database architectures.

This guide walks you through creating a high availability DB system and explores the important limitations you should consider when planning your deployment.

Creating a High Availability DB System

Overview of the Creation Process

Creating a high availability DB system in MySQL HeatWave follows a structured workflow with three main phases:

Set Up Basic DB System Configuration
Specify Database and Infrastructure Details
Configure Backup and Advanced Options

Let's explore each phase in detail.

Phase 1: Set Up Basic DB System Configuration

Step 1: Access the Console

Step 2: Navigate to DB Systems

Open the navigation menu
Under Databases, select HeatWave MySQL (OCI) or MySQL HeatWave (AWS)
Click DB Systems
Click Create DB System

Step 3: Choose Deployment Template

Select the Production template, which automatically configures your system for high availability with recommended settings for production workloads.

Alternatively, you can start with Development or Testing and manually enable high availability later.

Step 4: Basic Configuration

Compartment

Select the compartment where the DB system will reside
Compartments help organize resources and control access

Name Your DB System

Display Name: Provide a meaningful name (e.g., "prod-mysql-ha-primary")
Description: (Optional) Add details about the system's purpose

Database System Type

Ensure "High Availability" is selected if using the Production template
For manual setup, you can enable this option here

Phase 2: Specify Database Information

Administrator Credentials

Username Configuration

Specify the administrator username (default is typically "admin")
Avoid reserved usernames like "root", "mysql", "sys"
Username can be up to 32 characters

Password Requirements

Must be 8-32 characters long
Include at least one:
- Uppercase letter
- Lowercase letter
- Number
- Special character from: ,.~-+*;:_!
Store credentials securely (consider OCI Vault or AWS Secrets Manager)

Network Configuration

Virtual Cloud Network (VCN)

Select an existing VCN or create a new one
Subnet: Choose a private subnet for enhanced security
Public subnets are not recommended for production databases

Availability Domain Selection

Choose the preferred availability domain for your primary instance
This becomes your preferred placement
The system will automatically distribute secondary instances across different ADs or fault domains

Security Configuration

Ensure security lists or network security groups allow:
- Port 3306 for MySQL connections
- Port 33060 for MySQL X Protocol
Restrict source CIDR blocks to known application IP ranges

Hardware Configuration (Shape)

Select DB System Shape

Choose a shape that meets your performance requirements:

ECPU Shapes (Recommended)

Architecture-agnostic and future-proof
Examples: MySQL.8, MySQL.16, MySQL.32, MySQL.64, MySQL.128, MySQL.256
Can be changed after deployment
Support high availability and HeatWave clusters

Shape Characteristics

Defines CPU count, memory, and network bandwidth
Higher shapes provide better performance but cost more
Use Autopilot Shape Advisor for recommendations based on workload

Important: All OCPU shapes are deprecated and unavailable to new users created after September 5, 2024. Existing OCPU shapes will be unavailable after March 13, 2026.

Storage Configuration

Data Storage Size

Minimum: 50 GB
Maximum: 131,072 GB (128 TB)
Plan for current data plus growth projections
Storage can only be increased, never decreased

Automatic Storage Expansion

Recommended: Enable this feature
Automatically increases storage when usage approaches limits
Prevents downtime due to storage exhaustion
Based on Health Monitor read-only thresholds

Storage Performance

Higher storage allocations provide better IOPS
Storage is backed by high-performance block storage

Phase 3: Configure Backups

Enable Automatic Backups

Why Backups Matter

Protects against data loss
Enables point-in-time recovery
Required for disaster recovery planning

Backup Configuration Options

Enable Automatic Backups

Toggle: ON (highly recommended for HA systems)
Backups run automatically at scheduled times

Backup Retention Period

Range: 1 to 35 days
Recommended: At least 7 days for production
Longer retention = higher storage costs but better recovery options

Backup Window Selection

Choose a time when database activity is lowest
Typically during off-peak hours (e.g., 2:00 AM - 4:00 AM)
Avoid peak transaction periods to minimize performance impact

Point-in-Time Recovery (PITR)

Enable to restore to any point within retention period
Uses binary logs for granular recovery
Essential for recovering from accidental data modifications

Regional Backup Copy (OCI only)

Schedule automatic backup copies to another OCI region
Provides geographic redundancy
Protects against regional disasters

Advanced Backup Options

Backup Validation

Verifies backup integrity
Ensures backups are restorable
Available on supported MySQL versions

Soft Delete for Backups

Enabled by default
Backups remain in DELETED state before permanent deletion
Provides grace period for recovery of accidentally deleted backups

Additional Configuration Options

HeatWave Cluster Configuration (Optional)

If you need analytics capabilities:

Enable HeatWave Cluster

Checkbox: Select to add analytics processing
Choose HeatWave node shape (HeatWave.512GB, HeatWave.256GB, HeatWave.128GB)
Specify number of nodes (1 to 512 depending on shape and region)

Important HA Consideration: On OCI, HeatWave clusters are fully supported with high availability. On AWS, HeatWave features (including Lakehouse, GenAI, and AutoML) are currently not supported on HA DB systems.

MySQL Configuration

Configuration Selection

Use default HA-compatible configuration (recommended)
Configurations with "HA" suffix support high availability
Custom configurations must be based on HA-compatible templates

Configuration Characteristics

Defines system variables and initialization parameters
Can be changed after deployment with rolling restarts
Affects memory allocation, connection limits, and performance tuning

Tagging (Optional)

Resource Tags

Add metadata for organization
Enable cost tracking and allocation
Support automation and bulk operations

Tag Types

Defined Tags: Created by administrators with controlled vocabularies
Free-form Tags: Simple key-value pairs for flexible labeling

Finalizing Creation

Review Configuration

Before creating:

Verify all settings are correct
Check estimated monthly cost
Ensure network connectivity is properly configured
Confirm backup settings meet compliance requirements

Create DB System

Click Create to provision your high availability DB system.

Provisioning Timeline

DB system enters CREATING state
Typical creation time: 15-30 minutes for HA systems
Three instances are provisioned across availability/fault domains
Status changes to ACTIVE when ready

What Happens During Creation

Primary instance is provisioned in preferred placement
Two secondary instances created in different domains
MySQL Group Replication is configured automatically
Initial backup is taken (if enabled)
Monitoring and health checks are activated

Understanding High Availability Limitations

While MySQL HeatWave high availability provides robust fault tolerance, it's important to understand its limitations to design systems appropriately.

1. Rolling Upgrades and Brief Downtime

Characteristic: High availability DB systems require rolling upgrades for maintenance, during which there is a brief period of downtime before the newly promoted MySQL instance resumes connections.

What This Means:

During maintenance, instances are upgraded one at a time
When the primary instance is upgraded, a failover occurs
Connection strings remain the same, but applications must reconnect
Downtime is typically measured in seconds, not minutes

Best Practice:

Implement connection retry logic in applications
Use connection pooling with automatic reconnection
Schedule maintenance during low-traffic periods
Test application behavior during failovers

2. MySQL Version Requirements

Limitation: High availability is only supported for MySQL version 8.0.24 or higher.

Implications:

Older MySQL versions cannot use HA features
Must upgrade to 8.0.24+ before enabling high availability
Consider upgrading to MySQL 8.4.x (LTS) or 9.x (Innovation) for latest features

Migration Path:

Upgrade standalone DB system to 8.0.24 or higher
Ensure all prerequisites are met (primary keys, etc.)
Enable high availability after upgrade completes

3. Direct Access Limitations

Limitation: You can only access the primary MySQL instance directly; the two secondary instances cannot be accessed using MySQL Shell or any other client.

Why This Matters:

Cannot distribute read load across secondary instances for HA systems
All connections go through the primary instance
Read replicas are separate feature for read scaling (not part of HA)

Alternative Solutions:

Use Read Replicas for read scaling (separate from HA)
Read replicas can be created on HA or standalone systems
Offload analytics queries to HeatWave cluster instead

4. Transaction Size Limits Based on Shape Memory

Limitation: The maximum transaction size depends on the amount of memory of the shape.

Transaction Size Limits by Shape:

Smaller shapes have lower transaction size limits
Larger transactions require more memory for Group Replication
Exceeding limits causes transaction failure

Example Limits (approximate):

MySQL.8 (8 ECPU, 128 GB RAM): ~2 GB transaction limit
MySQL.16 (16 ECPU, 256 GB RAM): ~4 GB transaction limit
MySQL.32 (32 ECPU, 512 GB RAM): ~8 GB transaction limit

Workarounds:

Break large transactions into smaller batches
Use bulk loading features with smaller commit sizes
Select appropriate shape for your transaction patterns
Consider upgrading shape if hitting limits regularly

5. HeatWave Cluster Operations and HA Changes

Limitation: If a HeatWave cluster load or reload operation is ongoing, enabling or disabling high availability can fail.

Why This Happens:

HeatWave cluster operations lock resources during data loading
HA changes require cluster reconfiguration
Concurrent operations can cause conflicts

Best Practice:

Wait for HeatWave operations to complete before HA changes
Monitor HeatWave operation status
Schedule HA changes during maintenance windows
Avoid simultaneous infrastructure and data operations

6. HeatWave Lakehouse Compatibility

Previous Limitation: Prior to MySQL 8.3.0-u2, you could not enable high availability on a DB system with HeatWave Lakehouse enabled.

Current Status: This limitation has been lifted in MySQL 8.3.0-u2 and later versions.

Upgrade Path:

Systems on older versions must upgrade to use HA with Lakehouse
After upgrade, both features can coexist
On OCI, full support is available
On AWS, HeatWave features remain unsupported on HA systems

7. Platform-Specific Limitations

MySQL HeatWave on AWS

On AWS, MySQL HeatWave Clusters and features requiring a cluster (MySQL HeatWave Lakehouse, GenAI, AutoML) are not currently supported on high availability DB systems.

Impact:

HA systems on AWS are limited to OLTP workloads
Analytics features require standalone systems
Must choose between HA and advanced analytics on AWS

Query PrivateLink Considerations

When using Query PrivateLink endpoints, failovers and switchovers take longer than on public endpoints.

Mitigation:

Set connect_timeout to small value (e.g., 5 seconds)
Implement aggressive connection retry logic
Monitor failover times
Consider public endpoints for latency-sensitive applications

8. Configuration and Infrastructure Limitations

Bulk Ingest: The Bulk Ingest feature is not supported on high availability DB systems.

Alternative: Use standard data loading methods like LOAD DATA INFILE or MySQL Shell utilities.

PrivateLink Management: High availability cannot be enabled or disabled on a DB system with PrivateLink(s) configured.

Workaround: Delete PrivateLinks, make HA changes, then recreate PrivateLinks.

9. Rolling Restart for Configuration Changes

Behavior: When updating configuration of a high availability DB system that changes only dynamic variables, a rolling restart is performed, which shortens downtime and causes a change in current primary placement.

What Happens:

Configuration updated on primary instance
Controlled failover to secondary instance
Original primary updated and becomes secondary
System automatically rebalances if needed
HeatWave data reloaded automatically if cluster attached

Implications:

Brief connection interruption during rolling restart
Current placement may differ from preferred placement
Applications must handle reconnections
HeatWave queries may be briefly unavailable during reload

Best Practices for HA DB Systems

1. Application Design

Connection Management

Implement exponential backoff for reconnection attempts
Use connection pooling with health checks
Set appropriate timeout values
Handle transient errors gracefully

Transaction Design

Keep transactions small and fast
Avoid long-running transactions that span minutes
Break large data modifications into batches
Monitor transaction sizes against shape limits

2. Monitoring and Alerting

Key Metrics to Monitor

Replication lag between primary and secondary instances
Storage utilization and growth trends
Transaction throughput and latency
Failover events and frequency
Backup success rates

Set Up Alerts For

Failover events
High replication lag
Storage approaching limits
Backup failures
Configuration change events

3. Testing and Validation

Regular Testing

Perform manual switchovers during maintenance windows
Test application behavior during failovers
Validate backup and restore procedures
Verify monitoring and alerting systems

Chaos Engineering

Simulate instance failures
Test network partition scenarios
Validate automatic recovery mechanisms
Document recovery procedures

4. Maintenance Planning

Scheduling

Plan configuration changes during low-traffic periods
Coordinate with application teams for maintenance windows
Avoid changes during critical business periods
Document rollback procedures

Pre-Change Validation

Test configuration changes in non-production first
Verify compatibility with HA requirements
Ensure backups are current
Have rollback plan ready

5. Cost Optimization

Resource Planning

Start with appropriate shape size
Monitor utilization with Autopilot Shape Advisor
Adjust shapes based on actual workload patterns
Balance performance needs with cost constraints

Backup Management

Set retention periods based on compliance requirements
Use regional backup copies only when needed
Enable backup validation selectively
Clean up old backups per policy

When NOT to Use High Availability

While high availability is excellent for many use cases, it may not be appropriate when:

Development/Testing: Standalone systems are more cost-effective
Analytics-Heavy on AWS: HA doesn't support HeatWave features on AWS
Very Large Transactions: Shape memory limits may be restrictive
Budget Constraints: HA requires 3x compute resources vs standalone
Read-Heavy Workloads: Read replicas might be more appropriate than HA

Alternative High Availability Strategies

If MySQL HeatWave HA limitations are too restrictive:

Read Replicas for Read Scaling

Distribute read traffic across multiple instances
Works with both standalone and HA systems
Up to 10 read replicas per DB system
Better for read-heavy workloads

Multi-Region Deployment

Deploy separate DB systems in different regions
Use application-level failover logic
Replicate data using MySQL replication channels
Provides geographic redundancy

Hybrid Architectures

Use HA for OLTP workloads
Separate standalone system with HeatWave for analytics (AWS)
Replicate data between systems using channels
Best of both worlds but adds complexity

Conclusion

MySQL HeatWave high availability provides enterprise-grade fault tolerance with automatic failover, data protection, and minimal downtime. Understanding the limitations—particularly around rolling upgrades, transaction sizes, and platform-specific constraints—is essential for designing robust database architectures.

By following best practices for application design, monitoring, and maintenance, you can build resilient systems that meet stringent uptime requirements while staying within the boundaries of high availability limitations.

The key is matching your requirements to the capabilities: if you need maximum uptime for OLTP workloads with automatic failover, MySQL HeatWave HA is an excellent choice. If you need advanced analytics features on AWS or have very specific requirements, consider alternative architectures that better fit your needs.

Additional Resources:

MySQL HeatWave: Creating DB Systems and High Availability Setup

Tue, 04 Nov 2025 11:52:49 +0000

MySQL HeatWave: Creating DB Systems and High Availability Setup

Introduction

MySQL HeatWave is Oracle's fully managed database service that combines transactional and analytical workloads in a single platform. This guide walks you through creating a MySQL HeatWave DB System and implementing high availability for enterprise-grade reliability.

Creating a MySQL HeatWave DB System

Step 1: Access OCI Console

Step 2: Navigate to DB Systems

Open the navigation menu (hamburger icon)
Under Databases, select HeatWave MySQL
Click DB Systems from the menu

Step 3: Initiate DB System Creation

Click the Create DB System button. This opens the Create DB System dialog with several configuration options.

Step 4: Choose a Template

Oracle provides three templates to streamline setup:

Production: Creates a high availability DB system with recommended settings for production environments
Development or Testing: Creates a standalone DB system optimized for dev/test workloads
Always Free: Creates a standalone system with limited resources (50GB storage, MySQL.Free shape) suitable for learning and small projects

Step 5: Configure Basic Information

Compartment Selection

Choose the compartment where you want to create the DB system
Compartments help organize and manage access to resources

System Identification

Name: Provide a display name for your DB system (doesn't need to be unique)
Description: (Optional) Add details about the system's purpose
OCID: Oracle automatically generates a unique identifier

Step 6: Configure Database Credentials

Administrator Credentials

Username: Specify the administrator username (avoid reserved usernames)
Password: Create a strong password that meets security requirements
Important: Save these credentials securely – you'll need them to connect to the database

Step 7: Configure Networking

Virtual Cloud Network (VCN)

Select an existing VCN or create a new one
Choose the private subnet for the DB system (recommended for security)
Availability Domain: Select the AD where the primary instance will run
Fault Domain: Oracle automatically selects an appropriate fault domain

Configure Security List

Ensure your VCN's security list allows MySQL traffic:

Source CIDR: 0.0.0.0/0 (or restrict to specific IP ranges)
Destination Ports: 3306 (MySQL) and 33060 (MySQL X Protocol)

Step 8: Select Shape

Choose a shape that defines your system's compute resources:

ECPU Shapes (Recommended as of 2024): Architecture-agnostic shapes that can be changed after deployment
OCPU Shapes: Being deprecated for new users (sunset date: March 13, 2026)
Examples: MySQL.8, MySQL.16, MySQL.32, MySQL.HeatWave.VM.Standard

The shape determines CPU count, memory, and network bandwidth. You can scale up or down later based on workload demands.

Step 9: Configure Storage

Initial Data Storage Size

Specify storage in GB (minimum: 50 GB, maximum: 131,072 GB)
Consider your current data size plus growth projections
Storage can be increased later but not decreased

Automatic Storage Expansion

Enable this feature to automatically grow storage when space usage approaches the Health Monitor read-only limits
Helps prevent storage-related outages

Step 10: Configure HeatWave Cluster (Optional)

Enable HeatWave Cluster

Select this option to add analytics capabilities
Configure the number of HeatWave nodes (minimum: 1, maximum: depends on shape and region)
Choose the HeatWave node shape

MySQL HeatWave Lakehouse

Enable to query data in Object Storage (CSV, Parquet, Aurora/Redshift backups)
Note: Prior to MySQL 8.3.0-u2, enabling Lakehouse required disabling HA, point-in-time recovery, and read replicas

Step 11: Configure Backup Plan

Backup Settings

Enable automatic backups: Recommended for production
Backup retention period: Choose from 1 to 35 days
Point-in-time recovery: Enables restoration to any point within the retention window
Backup window: Select a time when database activity is lowest

Step 12: Advanced Options (Optional)

Configuration

Apply a custom configuration or use the default
High availability requires an HA-compatible configuration (configurations with "HA" suffix)

Hostname Label

Define a DNS hostname for your DB system (requires DNS hostnames enabled in VCN)

Tags

Add metadata for organization and cost tracking

Step 13: Create the DB System

Review all settings and click Create. The DB system enters the CREATING state and typically becomes available in 15-30 minutes.

Understanding High Availability in MySQL HeatWave

High availability is a critical feature for production workloads that require maximum uptime and fault tolerance.

What is High Availability?

A high availability DB system consists of three MySQL instances: a primary instance and two secondary instances, with the primary functioning as a read/write endpoint and secondary instances receiving asynchronous data replication from the primary.

Key Benefits

Business Continuity: Three MySQL instances distributed across different availability or fault domains
Minimal Downtime: Automatic failover promotes a secondary instance to primary when the primary fails, resuming availability with no data loss
Flexible Placement: Choose your preferred availability domain and fault domain for the primary instance
Uptime SLA: Supports 99.99% availability requirements
HeatWave Compatible: You can enable HeatWave cluster on high availability DB systems

Architecture Components

Primary Instance

Handles all write operations and read queries
Located in your chosen availability domain
Provides the read/write endpoint with a consistent IP address

Secondary Instances (2)

Receive replicated data from the primary asynchronously
Placed in different availability or fault domains from the primary
Ready to be promoted to primary during failover

Placement Models

There are two deployment models:

Multiple Availability Domains with Regional Subnet: Three instances spread across three different ADs (AD1, AD2, AD3)
Single Availability Domain with Multiple Fault Domains: All three instances in one AD but different fault domains

Replication Mechanism

MySQL HeatWave uses MySQL Group Replication for high availability, with primary keys required as a prerequisite for the group replication used by high availability. Data is copied from the primary to secondary instances over a secure, managed internal network.

Important: The binary logs of each MySQL instance are managed independently, meaning even though instances have the same data, they can have different binary log files with different names and sizes.

Prerequisites for Enabling High Availability

Before enabling high availability on a DB system, ensure the following requirements are met:

1. MySQL Version Requirement

MySQL version must be 8.0.24 or higher to enable high availability. If using an older version, upgrade before enabling HA.

2. Primary Keys on All Tables

If you are migrating data to MySQL HeatWave and have not defined primary keys on your tables, you must add them.

Why Primary Keys Matter

Required for MySQL Group Replication
Ensure data consistency across instances
Impact load performance if composite or non-INTEGER

How to Check and Add Primary Keys

Use MySQL Shell to identify tables without primary keys:

SELECT DISTINCT
    table_schema,
    table_name
FROM information_schema.columns
WHERE table_schema NOT IN ('information_schema', 'mysql', 'performance_schema', 'sys')
AND table_name NOT IN (
    SELECT table_name
    FROM information_schema.table_constraints
    WHERE constraint_type = 'PRIMARY KEY'
);

Adding Primary Keys Using Invisible Columns

Using invisible columns to add primary keys is a low-impact way to update existing data for use with a high availability DB system:

ALTER TABLE your_table 
ADD COLUMN id BIGINT UNSIGNED AUTO_INCREMENT PRIMARY KEY INVISIBLE;

3. Crash Recovery Enabled

Verify crash recovery is enabled on your DB system. This is typically enabled by default but can be checked in the configuration settings.

4. Inbound Replication Stopped

If you intend to configure an inbound replication channel on this DB system, you must import data before enabling high availability and configure your channel after high availability is enabled.

5. Compatible Configuration

All default configurations that are compatible with high availability have the HA suffix in the configuration name. If using a custom configuration, copy it from an existing HA-compatible configuration.

6. HeatWave Cluster Considerations

If a HeatWave cluster load or reload operation is ongoing, enabling or disabling high availability can fail. Wait for these operations to complete before making HA changes.

Enabling High Availability on an Existing DB System

Using the Console

Navigate to your DB System details page
From the Actions menu, select Enable high availability
Review the prerequisites dialog
Click Enable high availability

The process creates two secondary instances by cloning from the primary instance and does not cause any downtime.

Using OCI CLI

oci mysql db-system update \
  --db-system-id <DB_SYSTEM_OCID> \
  --is-highly-available true

Note: If the configuration is not compatible with high availability, you must first change the DB system to use a high availability compatible configuration before enabling it.

Understanding Failover and Switchover

Automatic Failover

When the primary instance becomes unavailable due to hardware failure, network issues, or other problems, MySQL HeatWave Service automatically initiates a failover.

Failover Process:

Service detects primary instance failure
One secondary instance is automatically promoted to primary
The IP address of the read/write endpoint does not change, regardless of the placement of the primary instance
Applications must reconnect, but the endpoint remains the same
No data loss occurs

For a high availability DB system with HeatWave cluster, if the primary instance fails, HeatWave Service deletes the HeatWave cluster attached to the failed instance and adds a new one to the new primary instance, with tables automatically loaded.

Post-Failover State:

The current placement changes to the new primary's location
The preferred placement (your original choice) remains unchanged
A message appears: "Current placement differs from preferred placement, due to failover"

Manual Switchover

You can manually select one of the MySQL instances as the primary instance through switchover, which changes both the preferred placement and current placement to the newly selected placement.

When to Use Switchover:

Planned maintenance on the current primary's availability domain
Testing disaster recovery procedures
Rebalancing after a failover to restore preferred placement

Switchover Process (for systems created/upgraded after February 25, 2025):

New transactions are blocked when switchover is initiated, running transactions are allowed to complete, and the switchover process waits until all running transactions finish and commit.

Navigate to DB System details page
Click Switchover from the Actions menu
Select the target availability domain or fault domain
Confirm the switchover

When the current placement changes in a switchover, it results in a short period of downtime while the primary instance is redirected to the newly promoted instance, and all database connections must be reopened.

Using CLI:

oci mysql db-system update \
  --db-system-id <DB_SYSTEM_OCID> \
  --availability-domain <TARGET_AD>

Important Notes:

Uncommitted transactions are rolled back when the connection to the previous primary instance is broken
Long-running queries offloaded to HeatWave cluster may be aborted
After a switchover, the current binary log file name and position of the new primary may be different from the old primary

HeatWave Cluster with High Availability

You can enable HeatWave cluster on a high availability DB system for combined OLTP and analytics capabilities.

HeatWave Behavior During Placement Changes

Same Availability Domain:
If both the previous primary instance and the new primary instance are located in the same availability domain, the existing HeatWave cluster can be reused by detaching from the previous primary and reattaching to the new primary.

Different Availability Domains:
If the previous primary and new primary are in different availability domains, the existing HeatWave cluster must be detached and deleted, with a new HeatWave cluster created in the same AD as the new primary instance.

Data Recovery

The data in the HeatWave cluster is automatically recovered from:

The HeatWave Storage Layer (fastest option)
Reloaded from the DB system
Reloaded from Lakehouse Object Storage

This ensures minimal disruption to analytics workloads even during failover scenarios.

Disabling High Availability

If you need to convert a high availability DB system back to standalone:

Disabling high availability creates a standalone DB system from the instance running in the preferred primary placement, with no downtime if the primary is already in the preferred placement, but a controlled failover and short downtime if it is not.

Console Method:

Navigate to DB System details
Actions menu → Disable high availability
Confirm the action

CLI Method:

oci mysql db-system update \
  --db-system-id <DB_SYSTEM_OCID> \
  --is-highly-available false

Tip: You can avoid downtime by switching the preferred primary placement to the current primary instance before disabling high availability.

Monitoring and Observability

Health Monitoring

MySQL HeatWave provides built-in monitoring:

Instance health status
Storage utilization
Connection metrics
Replication lag (for HA systems)

Event Notifications

When a failover happens, a MySQL - Automatic Recovery event is emitted on the DB system with the additionalDetails.isFailover property set to true.

Configure operational notifications to receive alerts for:

Failover events
Storage threshold warnings
Backup completion/failure
Maintenance notifications

Integration with Monitoring Services

OCI Monitoring: Native integration with OCI Monitoring service
AWS CloudWatch: Available for MySQL HeatWave on AWS
Custom Metrics: Export metrics to third-party monitoring tools

Best Practices

Planning for High Availability

Always Use HA for Production: Any workload requiring uptime guarantees should use high availability
Test Failover Procedures: Regularly perform switchovers to verify your disaster recovery processes
Monitor Replication Lag: Though minimal, monitor lag between primary and secondary instances
Configure Application Retry Logic: Ensure applications can reconnect automatically after failover

Storage Planning

Oversize Storage Initially: Leave room for growth to avoid frequent expansions
Enable Automatic Expansion: Prevents storage-related outages
Monitor Storage Trends: Track usage patterns to predict future needs

Security Considerations

Use Private Subnets: Keep DB systems in private subnets
Restrict Security Lists: Limit access to known IP ranges
Secure Credentials: Use OCI Vault for secrets management
Regular Patches: Enable automatic security updates

Cost Optimization

Right-Size Shapes: Start with appropriate shapes and adjust based on metrics
Optimize Backup Retention: Balance compliance needs with storage costs
Use Development Templates: For non-production workloads, use lower-cost configurations
Monitor Resource Utilization: Use Autopilot Shape Advisor for recommendations

Conclusion

MySQL HeatWave provides a comprehensive, fully managed database service with enterprise-grade high availability features. By following this guide, you can create robust DB systems that meet demanding uptime requirements while benefiting from automatic failover, integrated analytics, and seamless scalability.

The combination of automated management, built-in high availability, and HeatWave analytics capabilities makes MySQL HeatWave an excellent choice for modern applications requiring both transactional processing and real-time analytics without the complexity of managing multiple database systems.

For the latest information, consult the official MySQL HeatWave documentation.

MySQL HeatWave Architecture: A Complete Guide

Sun, 02 Nov 2025 09:22:57 +0000

Introduction

MySQL HeatWave is Oracle's fully managed database service that uniquely combines Online Transaction Processing (OLTP) and Online Analytical Processing (OLAP) workloads in a single platform. Built on MySQL Enterprise Edition, it delivers real-time analytics on transactional data without requiring ETL duplication to a separate analytics database.

DB System Architecture Overview

A MySQL HeatWave DB System consists of several key components:

Core Components

Compute Instance: The foundational cloud-based compute resource
MySQL Server: MySQL Enterprise Edition with advanced security features including encryption, data masking, authentication, and a database firewall
Operating System: Oracle Linux OS
Virtual Network Interface Card (VNIC): Enables network connectivity
Block Storage: Persistent storage for data and transaction logs

Prerequisites for Deployment

Before creating a DB System, you need:

Tenancy: Your Oracle Cloud Infrastructure account
Compartment: Logical container for organizing resources
Policies: IAM policies for access control
Virtual Cloud Network (VCN): Network infrastructure for connectivity

DB System Provisioning

Shape Configuration

Shapes specify CPU, memory, and network bandwidth. As of 2024, you can provision MySQL HeatWave DB systems using ECPU shapes, which are architecture-agnostic. Shapes can be changed after deployment to scale your resources up or down based on workload demands.

Storage Capacity

Storage stores both data and logs and can be increased as your database grows. The system automatically manages storage allocation for optimal performance.

Endpoint Configuration

The endpoint comprises private IP addresses and port numbers that accept client connections. This provides secure, private connectivity to your database instance.

Data Security in HeatWave

MySQL HeatWave implements comprehensive security measures: data is encrypted and compressed both at rest and in transit. Client applications connect securely to the HeatWave DB System through encrypted channels.

Data Masking is commonly implemented for protecting sensitive information such as:

Social Security Numbers (SSNs)
Credit card numbers
Phone numbers
Personally Identifiable Information (PII)

High Availability Architecture

Standalone vs. High Availability DB Systems

A High Availability (HA) DB System provides enterprise-grade reliability through:

Three MySQL instances: one primary instance and two secondary instances, with all data written to the primary instance automatically copied to the secondary instances.

Distribution Across Availability Domains

The three instances are deployed across different availability domains (AD1, AD2, AD3) and fault domains. This geographic distribution ensures resilience against localized failures.

Automatic Failover Mechanism

In case the primary instance fails, MySQL HeatWave Service automatically promotes one of the secondary instances to function as the primary instance, resuming availability with no data loss.

Important Note: MySQL HeatWave uses MySQL Group Replication to replicate data from the primary to secondary instances over a secure, managed internal network.

HeatWave Cluster Architecture

Components and Features

The HeatWave cluster extends your DB System with powerful analytics capabilities:

HeatWave Plugin: Manages cluster operations and query routing
HeatWave Nodes: Store data in memory and process analytics queries, with each node containing an instance of HeatWave
Data Loading: Tables are loaded from MySQL into the HeatWave cluster
Query Offloading: Analytics queries are automatically offloaded to the cluster

Scalability

HeatWave's scale-out architecture enables massive parallelism, allowing the cluster to scale up to 512 nodes for provisioning, loading data, and processing queries.

Query Processing

MySQL HeatWave Cluster stores data in main memory using a hybrid columnar format that achieves the benefits of columnar processing while avoiding the materialization and update costs of pure columnar format.

Queries that meet specific prerequisites are automatically offloaded from the MySQL DB System to the HeatWave cluster for accelerated processing, with results returned to the client through the MySQL DB System.

HeatWave Storage Layer

The HeatWave Storage Layer is a critical component that provides data persistence and rapid recovery:

Key Features

Data loaded into MySQL HeatWave Cluster is automatically persisted to network-attached block storage, known as the MySQL HeatWave Storage Layer, which allows data to be reloaded quickly when the cluster resumes after a pause or recovers from a node or cluster failure.

Storage Layer Capabilities

Automated Persistence: Stores a copy of in-memory data automatically
Transparent Updates: Changes are propagated without manual intervention
Fast Recovery: Enables rapid restoration after failures
Data Reload: Supports efficient data restoration to memory
Automatic Expansion: On AWS, the storage layer is built on S3, allowing data to be reloaded to each MySQL HeatWave node independently and in parallel

Self-Managing Architecture

This automated, self-managing storage layer scales to the size required for your MySQL HeatWave Cluster and operates independently in the background.

HeatWave Lakehouse

MySQL HeatWave Lakehouse allows enterprises to process and query data across file formats such as CSV and Parquet, as well as Aurora and Redshift backups from AWS, even when data is not stored inside a MySQL database.

The Lakehouse capability enables organizations to:

Query up to 400TB of data in object storage
Combine OLTP data with data lake information
Eliminate ETL complexity and costs
Process structured, semi-structured, and unstructured data

Performance and Capabilities

AutoML Integration

MySQL HeatWave includes built-in machine learning capabilities through HeatWave AutoML, which:

Automates model training and selection
Eliminates the need for separate ML services
Processes data in-database for security and performance

GenAI and Vector Store

Recent additions include:

In-database Large Language Models (LLMs)
Automated vector store for AI applications
Natural language query capabilities
Contextual conversations with your data

Performance Benchmarks

Oracle claims HeatWave provides 1,400x faster analytics performance than Amazon RDS Aurora and 6-7x faster than AWS Redshift and Snowflake, based on their benchmarks.

Multicloud Availability

MySQL HeatWave is available across multiple cloud platforms:

Oracle Cloud Infrastructure (OCI): Native availability in all OCI regions
Amazon Web Services (AWS): Native deployment within AWS
Microsoft Azure: Available through Oracle Database Service for Azure

Best Practices

Enable High Availability: For production workloads requiring 99.99% uptime
Monitor Shape Performance: Use Autopilot Shape Advisor for optimization
Implement Data Masking: Protect sensitive information at the column level
Leverage Storage Layer: Trust automatic persistence for recovery scenarios
Plan Node Capacity: Use the Estimate Node Count feature for proper cluster sizing

Conclusion

MySQL HeatWave represents a significant advancement in database technology by unifying OLTP and OLAP workloads in a single, fully managed service. Its innovative architecture, featuring automated storage management, high availability, and integrated machine learning capabilities, makes it a compelling choice for organizations seeking to eliminate data duplication, reduce costs, and accelerate insights from their data.

For more detailed information, consult the official MySQL HeatWave documentation.

MySQL HeatWave: The Fully Managed Multi-Cloud Database with Integrated AI

Thu, 30 Oct 2025 11:04:20 +0000

MySQL HeatWave represents a revolutionary approach to database management, combining transaction processing, real-time analytics, machine learning, and generative AI in a single, fully managed service. Developed and supported by the MySQL team at Oracle, HeatWave transforms how organizations build modern data applications across multiple cloud platforms.

What is MySQL HeatWave?

Unified Database Platform

MySQL HeatWave is a fully managed database service that runs MySQL, providing a single platform that eliminates the complexity of managing separate systems for different workloads.

Integrated Capabilities:
MySQL HeatWave is the only MySQL database service to combine Online Transaction Processing (OLTP), Online Analytical Processing (OLAP), Machine Learning (ML), and Generative AI workloads within a single MySQL Database—without requiring ETL duplication or separate analytics databases.

Development and Support:
Developed and supported by the MySQL team at Oracle, ensuring deep MySQL expertise, community alignment, and continuous innovation.

Multi-Cloud Availability

Supported Cloud Platforms:

Oracle Cloud Infrastructure (OCI): Native deployment with full feature set
Amazon Web Services (AWS): Complete HeatWave capabilities on AWS
Microsoft Azure: Integrated with Azure cloud services

Multi-Cloud Benefits:

Deploy where your applications and data reside
Avoid cloud vendor lock-in
Consistent experience across cloud providers
Flexible hybrid and multi-cloud architectures

Key Advantages Over On-Premises MySQL

Rapid Provisioning and Deployment

Infrastructure Automation:
Rapid provisioning of database systems and all underlying infrastructure eliminates weeks of manual setup and configuration.

Deployment Speed:

Database instances available in minutes
Automatic compute, storage, and network configuration
Pre-configured high availability and security
Instant scaling without downtime

Automated Management Operations

Comprehensive Automation:
Automated tasks including patching, upgrading, and backups free database administrators from routine maintenance.

Managed Operations:

Automatic security patches and updates
Zero-downtime upgrades to new versions
Continuous automated backups
Disaster recovery configuration

Advanced Data Protection

Security Features:
Comprehensive data protection including encryption, firewall, and data masking ensures enterprise-grade security.

Protection Layers:

Encryption: TLS/SSL for data in transit, encryption at rest
Firewall: Network-level access controls
Data Masking: Sensitive data protection for non-production environments
Audit Logging: Comprehensive activity tracking

Always Current Features

Latest Enterprise Features:
Always up-to-date with the latest enterprise features and security enhancements without manual intervention.

Continuous Innovation:

Quarterly feature releases
Security updates applied automatically
Performance improvements continuously deployed
New AI and ML capabilities added regularly

Shared Responsibility Model

Customer Responsibilities

Application-Level Management:

1. Deployment Initiation:
Customers initiate the deployment of MySQL database systems through the cloud console or APIs.

2. User Account Management:
Create and manage database user accounts, roles, and privileges according to application requirements.

3. Database Object Creation:
Design and create database objects including schemas, tables, indexes, views, and stored procedures.

4. Data Management:
Load initial data, manage ongoing data operations, and maintain data quality and integrity.

Oracle Responsibilities

Infrastructure and Platform Management:

1. Infrastructure Provisioning:
Oracle handles provisioning of MySQL database systems and all underlying resources including compute, storage, and networking.

2. Patching and Updates:
Oracle patches the database software and operating system automatically with zero downtime.

3. Automated Backups:
Oracle performs automatic backups according to configured retention policies.

4. Disaster Recovery:
Oracle restores database systems from backups when needed and maintains high availability infrastructure.

Extreme Performance

100x Faster Than Standard MySQL

Query Acceleration:
MySQL HeatWave is extremely performant, delivering 100 times faster performance than standard MySQL database systems without HeatWave for analytical queries.

Performance Architecture:

In-memory columnar processing
Massively parallel execution
Intelligent query optimization
Hardware acceleration

Benchmark Results:
Independent benchmarks demonstrate HeatWave's superior performance compared to cloud data warehouse alternatives including Snowflake, Databricks, and Google BigQuery.

Application Compatibility

Zero Changes Required:
All existing MySQL applications work without any changes, ensuring seamless migration and adoption.

Compatibility Features:

Standard MySQL wire protocol
MySQL SQL dialect support
Existing driver compatibility
Familiar tools and interfaces

HeatWave Components and Capabilities

HeatWave MySQL: Transaction Processing

Enterprise-Grade OLTP:
HeatWave MySQL enables OLTP workloads to leverage Enterprise Edition features of MySQL Database and delivers unique capabilities such as Auto Shape Prediction, Auto Thread Pooling, Autopilot Indexing, and in-database JavaScript.

OLTP Enhancements:

Auto Shape Prediction: Intelligent right-sizing recommendations
Auto Thread Pooling: Optimized connection management
Autopilot Indexing: Automatic index recommendations
Native JavaScript: Write stored procedures in JavaScript
Bulk Ingest: Load data up to 5X faster

New in MySQL HeatWave 9.4:

Hypergraph Optimizer: True cost-based join optimization for complex queries
OCI Ops Insights Integration: ML-based performance analytics and capacity planning
Enhanced Concurrency: Process multiple queries simultaneously across sessions

HeatWave Lakehouse: Object Storage Analytics

Query Data in Place:
HeatWave Lakehouse enables querying structured, semi-structured, and unstructured data directly in object storage without data movement.

Supported Storage:

Amazon S3
Oracle Object Storage
Azure Blob Storage
Amazon Redshift
Amazon Aurora

Supported File Formats:

CSV: Comma-separated values
JSON: JavaScript Object Notation
XML: Extensible Markup Language
Avro: Row-based binary format
Parquet: Columnar storage format
ORC: Optimized Row Columnar

Lakehouse Features:

Query object storage at database speed
Write results back to object storage
Automatic change propagation from object storage
MapReduce application support
Guided Load for optimized data loading

HeatWave GenAI: Integrated Generative AI

In-Database Large Language Models:
MySQL HeatWave GenAI provides integrated, automated, and secure generative AI with in-database large language models (LLMs), an automated vector store, scale-out vector processing, and natural language conversation capabilities.

GenAI Features:

In-Database LLMs:
Use large language models running directly inside the database without external API calls or data movement.

Supported In-Database Models (as of MySQL 9.4):

llama3.2-3b-instruct-v1: Default LLM, multilingual support (8 languages)
llama3.2-1b-instruct-v1: Lightweight model, multilingual support
llama3.1-8b-instruct-v1: Advanced model, multilingual support
mistral-7b-instruct-v3: Efficient model, multilingual support

External LLM Support:

OCI Generative AI Service: Access Cohere and Meta models on OCI
OCI Dedicated Clusters: Enhanced flexibility and capabilities
Amazon Bedrock: Access foundation models on AWS
Vision Language Models (VLMs): Process vision-language tasks

HeatWave Vector Store:
MySQL HeatWave Vector Store houses proprietary documents in various formats, acting as the knowledge base for Retrieval-Augmented Generation (RAG) to deliver more accurate and contextually relevant answers.

Vector Store Features:

Automated Embedding Generation: No manual vector creation required
27 Language Support: Multilingual document processing
OCR Support: Optical character recognition for images and PDFs
Auto Parallel Load: Efficient document ingestion
Automatic Updates: Vector store synchronization with source changes

Performance Advantages:
For similarity search, MySQL HeatWave GenAI is 15X faster than Databricks, 18X faster than Google BigQuery, and 30X faster than Snowflake, while being less expensive.

Natural Language Conversations:
Have contextual conversations in natural language with HeatWave Chat, a chatbot that enables multiple follow-up questions about specific topics within a single session.

NL2ML - Natural Language to Machine Learning:
MySQL HeatWave now supports NL2ML, providing an intuitive natural-language interface to machine learning, making ML accessible without coding expertise.

NL2SQL - Natural Language to SQL:
Generate SQL queries from natural-language statements using the ML_NL_SQL routine, making database interactions easier for non-technical users.

Document Summarization:
Advanced document summarization capabilities for unstructured documents stored in Object Storage, enabling quick insights from large document collections.

HeatWave AutoML: Automated Machine Learning

Simplified Machine Learning:
HeatWave AutoML enables advanced machine learning capabilities directly within MySQL without data movement, separate ML services, or AI expertise.

AutoML Capabilities:

Automated Model Building: Algorithm selection, feature engineering, hyperparameter tuning
Model Training: In-database training on MySQL data
Model Deployment: Seamless deployment for predictions
Model Explanation: Understand model decisions and feature importance
Hyperparameter Recording: Enhanced transparency and reproducibility

New AutoML Features:

Enhanced Data Preparation: NL2ML-powered data preparation workflows
TRAIN_TEST_SPLIT: Automated dataset splitting for training and testing
Improved Concurrency: Process multiple ML queries simultaneously
Enhanced Performance: Faster predictions with ML_PREDICT_TABLE and ML_EXPLAIN_TABLE

Use Cases:

Predictive maintenance
Fraud detection
Personalized recommendations
Customer churn prediction
Demand forecasting

HeatWave Autopilot: Intelligent Automation

Automated Performance at Scale:
HeatWave Autopilot automates processes to improve performance at scale, handling complex optimization tasks automatically.

Autopilot Capabilities:

Auto Provisioning: Intelligent resource sizing
Auto Parallel Load: Optimized data loading
Auto Query Plan Improvement: Automatic query optimization
Auto Scheduling: Intelligent workload scheduling
Auto Encoding: Automatic data compression
Auto Data Placement: Optimal data distribution
Auto Change Propagation: Sync changes from object storage
Auto Indexing: Automatic index recommendations and management

Getting Started with MySQL HeatWave

Provisioning a HeatWave Instance

1. Access Cloud Console:
Navigate to MySQL HeatWave service in OCI, AWS, or Azure console

2. Create DB System:

Select cloud region
Choose compute shape (HeatWave.Free, HeatWave.32GB, HeatWave.512GB)
Configure storage
Set up networking (VCN, subnet)
Configure backups

3. Enable HeatWave Cluster:

Specify number of HeatWave nodes
Select appropriate shape for workload
Enable AutoML and GenAI features

4. Configure Security:

Set up users and privileges
Configure network access controls
Enable encryption
Set up audit logging

Loading Data

From MySQL Database:

-- Load table into HeatWave
ALTER TABLE sales SECONDARY_LOAD;

-- Load all tables in schema
CALL sys.heatwave_load(JSON_OBJECT('schema', 'sales_db'));

From Object Storage (Lakehouse):

-- Create external table
CREATE TABLE sales_external
ENGINE=RAPID
COMMENT='lakehouse:parquet'
AS SELECT * FROM performance_schema.file_locations
WHERE path = 's3://bucket/sales/*.parquet';

-- Load into HeatWave
ALTER TABLE sales_external SECONDARY_LOAD;

Using HeatWave GenAI

Generate Content:

-- Generate text content
SELECT ML_GENERATE(
    'Write a product description for a smartphone',
    JSON_OBJECT('task', 'generation')
);

Semantic Search with Vector Store:

-- Load documents into vector store
CALL sys.VECTOR_STORE_LOAD(
    's3://bucket/documents/',
    JSON_OBJECT('table_name', 'my_documents')
);

-- Perform RAG query
SELECT ML_RAG(
    'What are our return policies?',
    'my_documents',
    JSON_OBJECT('top_k', 5)
);

Natural Language Conversation:

-- Chat with your data
CALL sys.HEATWAVE_CHAT(
    'Show me top selling products last quarter',
    JSON_OBJECT('store_name', 'sales_vectors')
);

Using HeatWave AutoML

Train Model:

-- Train predictive model
CALL sys.ML_TRAIN(
    'customer_data',
    'churn_prediction',
    'churn_flag',
    JSON_OBJECT('task', 'classification')
);

Make Predictions:

-- Score new data
CALL sys.ML_PREDICT_TABLE(
    'customer_data',
    'churn_prediction',
    'predictions',
    NULL
);

Real-World Success Stories

SmarterD: Fast-Tracked Development

SmarterD fast-tracked its roadmap by 12 months and went from development to production in only one month using Oracle MySQL HeatWave GenAI.

NTT Solmare: Revenue Growth

NTT Solmare improved marketing campaigns and uncovered new revenue opportunities using HeatWave analytics and machine learning capabilities.

Fortune 100 Applications

Fortune 100 companies use HeatWave AutoML for predictive maintenance, fraud detection, and personalized recommendations without machine learning expertise while keeping data secure within the database.

Best Practices

Performance Optimization

Enable HeatWave cluster for analytical workloads
Use AUTO SECONDARY_LOAD for automatic data loading
Leverage Autopilot recommendations
Monitor query performance with Performance Schema
Use bulk ingest for large data loads

Cost Management

Right-size compute shapes using Auto Shape Prediction
Use HeatWave only when needed (start/stop capability)
Leverage object storage for cold data
Monitor usage with OCI Ops Insights integration
Take advantage of HeatWave.Free tier for development

Security

Enable encryption at rest and in transit
Configure network access controls appropriately
Use principle of least privilege for user accounts
Enable audit logging for compliance
Regularly review security configurations

GenAI Applications

Start with in-database LLMs for prototyping
Use RAG with Vector Store for domain-specific accuracy
Leverage Lakehouse Navigator for cost-effective searches
Monitor LLM usage and response quality
Implement guardrails for production deployments

Conclusion

MySQL HeatWave represents a paradigm shift in database technology, delivering a unified platform for transaction processing, analytics, machine learning, and generative AI across multiple clouds. Developed and supported by the MySQL team at Oracle, HeatWave provides 100x faster analytics performance than standard MySQL while maintaining complete application compatibility.

Key Value Propositions:

Unified Platform:

Single service for OLTP, OLAP, ML, and GenAI
No ETL between separate systems
Simplified architecture and operations
Reduced total cost of ownership

Multi-Cloud Flexibility:

Deploy on OCI, AWS, or Azure
Consistent experience across clouds
Avoid vendor lock-in
Support hybrid architectures

Integrated AI:

In-database LLMs without data movement
Automated vector store for RAG
AutoML without AI expertise
Natural language interfaces (NL2ML, NL2SQL)

Extreme Performance:

100x faster than MySQL without HeatWave
Outperforms cloud data warehouses
15-30X faster vector search than competitors
Real-time analytics on transactional data

Fully Managed:

Automated provisioning, patching, backups
Zero-downtime upgrades
Intelligent automation with Autopilot
Enterprise-grade security and compliance

Whether building transaction-heavy applications, performing real-time analytics, developing machine learning models, or creating generative AI applications, MySQL HeatWave provides the comprehensive, integrated, and automated platform necessary for modern data-driven applications—all while maintaining the simplicity, familiarity, and compatibility of MySQL.

Foreign Keys in MySQL for Serious System Design

Fri, 03 Oct 2025 21:33:56 +0000

Let’s face it: encountering foreign keys in MySQL is a classic developer rite of passage. At first, it's about linking tables—but swiftly turns into crafting a bulletproof web of relationships and rules that underpin entire data-driven systems. But at scale, these seemingly simple constraints form the backbone of reliable, maintainable, and secure architectures.

This isn’t just a guide to syntax. This is your comprehensive field manual for wielding foreign keys: strategies, subtle caveats, best practices, and even architectural patterns favored by pros.

Quick Refresher: What are Foreign Keys?

A foreign key is a column or set of columns in one table that references the primary key in another table, enforcing relationship integrity between records. Without them, you risk “orphan” data—think: orders without matching customers, employees reporting to non-existent managers, or posts referencing gone users.

Example:

CREATE TABLE orders (
    order_id INT PRIMARY KEY,
    customer_id INT,
    FOREIGN KEY (customer_id) REFERENCES customers(customer_id)
);

Advanced Foreign Key Tactics

1. Cascade Operations: Controlling Your Data’s Life Cycle

The magic of MySQL’s foreign keys isn’t just the linking—it's about controlling what happens on updates or deletes. Choose from:

CASCADE: Mods or deletes in the parent auto-propagate to children. Delete a customer? Their orders vanish too.
SET NULL: Leaves child reference as NULL if parent goes away—if your business logic allows “detached” records.
RESTRICT/NO ACTION: Block changes if dependencies remain. Most conservative.

When to Use What?

Operation	Use Case Example	Beware...
CASCADE	Orders deleted when customer is deleted	Unintended mass deletions, data loss
SET NULL	Product discontinued, but not deleting sale history	Nullability: Foreign key columns must allow it
RESTRICT	Employees can’t be deleted if they manage teams	Can hinder batch deletes/updates

2. Many-to-Many & Junction Tables

Not all relationships are one-to-many. Many-to-many needs a junction (associative) table with two or more foreign keys.

Example: Users and Roles

CREATE TABLE user_roles (
    user_id INT,
    role_id INT,
    PRIMARY KEY(user_id, role_id),
    FOREIGN KEY (user_id) REFERENCES users(user_id),
    FOREIGN KEY (role_id) REFERENCES roles(role_id)
);

This prevents a user from having invalid roles and vice-versa.

3. Self-Referential & Hierarchical Relationships

Foreign keys can reference the same table (think: management hierarchy or tree categories):

CREATE TABLE employees (
    employee_id INT PRIMARY KEY,
    manager_id INT,
    FOREIGN KEY (manager_id) REFERENCES employees(employee_id)
);

This enables recursive queries and elegant modeling of organizational trees.

4. Deferrable Constraints (Not Native in MySQL Yet)

Some databases let you defer foreign key checks till commit. MySQL doesn't natively support this (as of v8.0), but it’s crucial to know if you’re porting designs—sometimes, you need to temporarily violate constraints in multi-step transactions.

Workaround: Drop constraints, perform bulk actions, re-add and validate—dangerous if not careful.

5. Foreign Key Indexing for Performance

Every foreign key column, especially in large tables, must be indexed! MySQL often creates indexes automatically, but manual indexing ensures optimal joins, cascades, and validation efficiency.

CREATE INDEX idx_orders_customer_id ON orders(customer_id);

Potential Pitfalls & Best Practices

Pitfall	Impact/Example	Best Practices
Mismatched Datatypes	INT in one, BIGINT in another = constraint fails	Match PK and FK datatypes exactly
Circular Dependencies	Table A FK B, B FK A --> deadlocks, migrations pain	Avoid unless model strictly requires; batch disables
Large Cascading Deletes	Deletes millions unintentionally	Use `RESTRICT` or disable, and log deletes
Orphaned Data	Data in child table with no valid parent	Enforce referential integrity via FKs
Bulk Data Loads	INSERTs/LOADs slower due to constraint checks	Temporarily disable FKs during trusted imports

Beyond Basics: Multi-Table and Polymorphic Relations

Composite Foreign Keys

Sometimes, relationships are defined by multiple columns:

CREATE TABLE shipments (
    order_id INT,
    item_id INT,
    ...
    FOREIGN KEY (order_id, item_id) REFERENCES order_items(order_id, item_id)
);

Polymorphic Association (the Anti-pattern)

If you need a comment to belong to either a blog post or a photo—MySQL won’t let you set a FK to multiple tables. This is called a polymorphic association and is discouraged in strict relational design—better to have a dedicated join table per association, or use inheritance-like strategies.

Enterprise Patterns: Audit Trails, Soft Deletes, and Data Warehousing

Soft Deletes: Instead of deleting, mark as deleted_at. Don’t cascade deletes on these columns, handle logically.
Audit Trails: Log every change; foreign keys can help tie logs to users/actions.
Data Warehousing / ETL: Foreign keys are often dropped in warehouse schemas for performance—integrity must be enforced during loading (ETL).

Real-World Example: E-commerce Data Model—Expanded

Tables:

Customers (customer_id, name, email)
Orders (order_id, customer_id, order_date, status)
Products (product_id, name, category_id, price, stock_quantity)
Order_Items (order_item_id, order_id, product_id, quantity, subtotal)
Categories (category_id, name, parent_category_id)
Reviews (review_id, product_id, customer_id, stars, comment, created_at)
Coupons (coupon_id, code, discount_percent, expires_at)
Order_Coupons (order_id, coupon_id)

Sample Foreign Key Usage:

ALTER TABLE orders
    ADD CONSTRAINT fk_customer FOREIGN KEY (customer_id)
        REFERENCES customers(customer_id)
        ON DELETE RESTRICT;

ALTER TABLE order_items
    ADD CONSTRAINT fk_order FOREIGN KEY(order_id)
        REFERENCES orders(order_id)
        ON DELETE CASCADE, 
    ADD CONSTRAINT fk_product FOREIGN KEY(product_id)
        REFERENCES products(product_id)
        ON DELETE RESTRICT;

ALTER TABLE products
    ADD CONSTRAINT fk_category FOREIGN KEY(category_id)
        REFERENCES categories(category_id)
        ON DELETE SET NULL;

ALTER TABLE reviews
    ADD CONSTRAINT fk_review_product
        FOREIGN KEY (product_id) REFERENCES products(product_id)
        ON DELETE CASCADE,
    ADD CONSTRAINT fk_review_customer
        FOREIGN KEY (customer_id) REFERENCES customers(customer_id)
        ON DELETE SET NULL;

Features Achieved:

Orders tied strictly to real customers; orphans impossible.
Deleting a product deletes its reviews, but not historical order records.
Deleting a category doesn’t nuke the products—only unclassifies them.

Foreign Keys and ORM Mappings (Advanced)

Using ORMs like SQLAlchemy, Hibernate, or Django ORM? Foreign keys are the link between your domain models. Properly defined, they:

Enable cascading saves/deletes in code.
Enable reverse lookups and eager/lazy loading.
Ensure constraints enforced at both DB and code layer.

Pro tip: Always mirror your DB constraints in your ORM models, and don’t rely solely on ORM-level validation.

For The Data Architects: When NOT to Use Foreign Keys

High-Volume Event Logs: Can slow inserts; consider only if strong integrity is a must.
Denormalized Warehouses: Use constraints during ETL, not at warehouse runtime.
Third-Party/Cross-DB Data: Can’t enforce with foreign keys; use programmatic or scheduled validations.
Microservices: Sometimes, each service owns its data: cross-database FK constraints can break scaling and autonomy.

Checklist: Production-Ready Foreign Key Design

All FKs indexed?
Datatypes exactly match?
All cascade options reviewed and intentional?
No accidental circular references?
Soft deletes and TTL data not cascaded on delete?
Batch loads/disables FKs as needed?

Resources for Further Mastery

MySQL Foreign Key Docs: Deep-dive with syntax and real-world use cases.
Normalization and ER Diagrams: Strengthen foundation in data modeling.
Books: “SQL Antipatterns” (Bill Karwin), “Designing Data-Intensive Applications” (Martin Kleppmann).

In summary: Foreign keys are your shield and your sword—they safeguard data integrity, enable powerful querying, and make your system maintainable at scale—but demand planning, discipline, and regular review.

MySQL HeatWave and Oracle NoSQL: Modern Database Solutions for Enterprise Applications

Wed, 17 Sep 2025 19:37:20 +0000

Modern enterprises require database solutions that can handle diverse workloads, from traditional transactions to real-time analytics and flexible data models. MySQL HeatWave and Oracle NoSQL Database Service represent Oracle's comprehensive approach to addressing these varied requirements through innovative, cloud-native database technologies.

MySQL HeatWave: Unified OLTP and OLAP Platform

The Traditional Database Challenge

Traditional MySQL deployments are optimized for Online Transaction Processing (OLTP) workloads but struggle with analytical queries. This limitation has historically forced organizations to implement complex architectures:

Complex ETL Processes: Data must be extracted, transformed, and loaded into separate analytical systems
No Real-Time Analytics: Batch processing creates delays between transactions and insights
Security and Compliance Risks: Data duplication across multiple systems increases vulnerability surface
Increased Operational Costs: Managing multiple database platforms requires additional resources and expertise

MySQL HeatWave: Revolutionary Integration

MySQL HeatWave provides automated and integrated generative AI and machine learning (ML) in one cloud service for transactions and lakehouse scale analytics, fundamentally changing how organizations approach data management.

Core Value Propositions:

Unified Service: Single platform handling OLTP, OLAP, Machine Learning, and Lakehouse analytics
Elimination of ETL Duplication: Real-time analytics on data in MySQL Database and object storage without extract, transform, and load (ETL) duplication
Unmatched Performance: Significant performance improvements at reduced costs compared to traditional approaches
Real-Time Analytics: Analytics queries access the most current information as updates from transactions automatically replicate in real time
Enhanced Security: Consolidated security model reduces complexity and risk
High Application Availability: Built-in redundancy and failover capabilities

MySQL HeatWave Architecture and Features

HeatWave Lakehouse

HeatWave Lakehouse enables organizations to analyze massive datasets stored in various file formats without traditional data warehouse limitations.

Supported File Formats:
MySQL HeatWave Lakehouse supports a diverse range of object store file formats, inclusive of Parquet, CSV, and database exports, providing unprecedented flexibility for data integration.

Key Capabilities:

Query petabyte-scale datasets directly from object storage
Support for structured, semi-structured, and unstructured data
Automatic data discovery and schema inference
Cost-effective storage for historical and archival data

HeatWave AutoML

HeatWave AutoML democratizes machine learning by providing comprehensive ML capabilities integrated directly into the database platform.

AutoML Features:

Complete ML Lifecycle: Build, train, deploy, and explain ML models without additional costs
Popular Integration: Integrated with popular notebooks such as Jupyter notebooks and Apache Zeppelin
Automated Model Selection: Intelligent algorithm selection based on data characteristics
In-Database Processing: Models execute directly within the database for maximum performance

MySQL Autopilot

MySQL Autopilot represents Oracle's commitment to autonomous database operations, automating critical performance optimization tasks.

Automation Capabilities:

Intelligent Provisioning: Automatic resource allocation based on workload patterns
Optimized Data Loading: Automated data distribution and indexing strategies
Query Execution Optimization: Real-time query plan optimization and tuning
Predictive Scaling: Proactive resource scaling based on usage patterns

MySQL HeatWave Service Characteristics

Ease of Use and Management

Fully Managed Service: Oracle handles infrastructure management, patching, and maintenance
Instant Provisioning: Rapid deployment with latest features and capabilities
Intuitive Interface: Simplified administration through Oracle Cloud Console
Automated Operations: Self-tuning and self-optimizing capabilities

Security and Compliance

Enterprise-Grade Security:

Built on Oracle's Generation 2 Cloud Infrastructure
Comprehensive encryption for data at rest and in transit
Advanced access controls and audit capabilities
Regulatory Compliance: Meets GDPR, PCI DSS, and HIPAA requirements

Enterprise Readiness

High Reliability: 99.95% availability SLA with built-in redundancy
24/7 Support: Oracle's global support organization
Oracle Integration: Seamless integration with Oracle's technology stack
100% MySQL Compatibility: Full compatibility with on-premises MySQL applications

Multi-Cloud Deployment

You can deploy HeatWave on OCI, AWS, or Azure. You can replicate data from on-premises OLTP applications to HeatWave to get near real-time analytics, providing unprecedented deployment flexibility.

Oracle NoSQL Database Service

Modern Application Challenges

Contemporary applications face unprecedented demands that traditional relational databases struggle to address:

High-Volume, High-Velocity Data: Applications generate massive amounts of data at extreme speeds
Rapid Innovation Cycles: Development teams require database flexibility to support fast feature delivery
Elastic Scaling Requirements: Workloads demand dynamic throughput and storage scaling
Always-On Availability: Users expect 24/7 service availability across global deployments

Oracle NoSQL Database Service Solution

The service offers active-active regional replication, ACID transactions, serverless scaling, comprehensive security, and low pay-per-use pricing for both on-demand and provisioned capacity modes.

Core Service Characteristics:

Fully Managed: Zero-administration database service
Elastic Scaling: Automatic scaling based on demand
High Performance: Predictable low-latency responses
Developer Friendly: Multiple programming language support and intuitive APIs
Hybrid Cloud: Seamless integration across cloud and on-premises environments
Data Model Flexibility: Support for multiple data models within single service
Enterprise Security: Comprehensive encryption, authentication, and access controls
Cost Optimization: Pay-per-use pricing model with reserved capacity options

Oracle NoSQL Database Use Cases

Modern Application Patterns

Mobile and IoT Applications:

High-frequency data ingestion from sensors and devices
Real-time user interaction tracking
Location-based services and geospatial queries

Customer Experience Platforms:

360-degree customer view aggregation
User profile management and personalization
Real-time recommendation engines

Digital Commerce:

Product catalog management with flexible schemas
Shopping cart and session management
Inventory tracking and real-time updates

Content and Media:

Content management systems with multimedia support
Social network activity feeds and notifications
Real-time messaging and communication platforms

Enterprise Applications:

Online advertising campaign management
Real-time big data analytics and processing
Gaming leaderboards and player statistics

Developer Experience

Multi-Cloud Deployment:
Applications can run consistently across any cloud environment, providing deployment flexibility and vendor independence.

Flexible Data Models:
Supports JSON, Table and Key-Value datatypes, all with flexible transaction guarantees:

Document Model: Schema-less JSON document storage
Key-Value Model: Simple key-value pair storage for caching and sessions
Columnar Schema: Traditional structured data with fixed schemas

Security Integration:

Encryption: Data protection at rest and in transit
Authentication: Integration with enterprise identity systems
Access Control: Fine-grained permissions and role-based access

Oracle NoSQL Database Features

Scalability and Performance

Elastic Scaling:

Table-Level Scaling: Independent scaling for different application components
Instant Elasticity: Automatic resource adjustment based on demand
Predictable Performance: Consistent low-latency responses across scale levels

High Availability:
Oracle NoSQL Database Cloud Service (NDCS) is a fully HA service designed for highly demanding applications that require low latency response times

Development Flexibility

Time-to-Live (TTL) Support:
Automatic data expiration for temporary data and compliance requirements.

API and SQL Access:

RESTful APIs for modern application integration
SQL query support for familiar development patterns
Multiple SDK support for popular programming languages

Advanced Indexing:

Secondary Indexes: Support for simple scalars, non-scalars, composite, and JSON indexes
Rich Query Support: Complex query capabilities with SQL-like syntax
Geospatial Queries: Location-based query support for mobile applications

Latest Enhancements

Developer Tooling:
Enhanced Visual Studio Code and IntelliJ plugins with form-based column entry and DDL statement support

Migration Support:
NoSQL Database Migrator supports session token-based authentication and OKE Workload Identity Authentication

SDK Expansion:
Oracle NoSQL Database Cloud Service supports Rust SDK for comprehensive application development

Integration and Management

Oracle Cloud Console Access

Both MySQL HeatWave and Oracle NoSQL Database Service provide comprehensive management through the Oracle Cloud Console:

Unified Dashboard: Single interface for monitoring and management
Performance Metrics: Real-time performance monitoring and alerting
Security Management: Centralized security policy and access control
Cost Optimization: Usage tracking and cost management tools

Enterprise Integration

Oracle Technology Stack:
Seamless integration with Oracle's comprehensive technology portfolio including:

Oracle Analytics Cloud for advanced analytics
Oracle Integration Cloud for application connectivity
Oracle Identity Cloud Service for authentication and authorization
Oracle Management Cloud for monitoring and operations

Choosing the Right Solution

MySQL HeatWave Ideal For:

Transactional Applications: Applications requiring both OLTP and real-time analytics
Data Consolidation: Organizations looking to eliminate data silos
ML-Driven Applications: Applications requiring integrated machine learning capabilities
Cost Optimization: Organizations seeking to reduce infrastructure complexity

Oracle NoSQL Database Ideal For:

Modern Applications: Applications requiring flexible data models and elastic scaling
High-Volume Scenarios: Applications with extreme throughput requirements
Global Applications: Applications requiring multi-region deployment
Developer Productivity: Teams requiring rapid development and deployment cycles

Future Considerations

Emerging Trends

AI and ML Integration:
Both platforms continue evolving to provide deeper AI/ML integration, enabling intelligent applications with minimal development complexity.

Edge Computing:
Support for edge deployment scenarios where applications require local data processing capabilities.

Multi-Cloud Strategy:
Enhanced support for hybrid and multi-cloud deployments, providing maximum deployment flexibility.

Conclusion

MySQL HeatWave and Oracle NoSQL Database Service represent Oracle's commitment to providing comprehensive database solutions for modern enterprise requirements. MySQL HeatWave's unique integration of OLTP and OLAP in a single system, combined with built-in Data Lakehouse and advanced machine learning capabilities, addresses the complexity of traditional analytical architectures.

Similarly, Oracle NoSQL Database Service provides the flexibility and scale required for modern application development, with active-active regional replication, ACID transactions, and serverless scaling capabilities.

Key Decision Factors:

Data Consistency Requirements: ACID transactions vs. eventual consistency models
Query Complexity: SQL analytics vs. simple key-value operations
Scaling Patterns: Vertical vs. horizontal scaling requirements
Development Timeline: Existing MySQL expertise vs. modern application patterns

Both solutions provide enterprise-grade security, reliability, and Oracle's comprehensive support, enabling organizations to choose the optimal database platform based on specific application requirements and architectural preferences.

Whether implementing real-time analytics with MySQL HeatWave or building flexible, scalable applications with Oracle NoSQL Database Service, organizations can leverage Oracle's cloud-native database technologies to accelerate innovation and reduce operational complexity.