Apache Seatunnel断点续传方式

原创已于 2026-08-31 22:04:55 修改 · 218 阅读

6 ·

本内容遵循CC 4.0 BY-SA版权协议

MrZhou

GEO检测

收录于

数据库运维案例分享

于 2026-08-31 21:52:14 首次发布

Apache Seatunnel断点续传方式

首先需要创建一个savecheckpoint,执行以下命令会创建一个savecheckpoint并停止任务

–创建一个保存 savepoint
/data/apache-seatunnel-2.3.12/bin/seatunnel.sh
-s 1146757990657818625
-cn seatunnel

# /data/apache-seatunnel-2.3.12/bin/seatunnel.sh \
>   -s 1146757990657818625 \
>   -cn seatunnel
2026-08-31 21:41:48,125 INFO  [c.h.i.c.AbstractConfigLocator ] [main] - Loading configuration '/data/apache-seatunnel-2.3.12/config/seatunnel.yaml' from System property 'seatunnel.config'
2026-08-31 21:41:48,129 INFO  [c.h.i.c.AbstractConfigLocator ] [main] - Using configuration file at /data/apache-seatunnel-2.3.12/config/seatunnel.yaml
2026-08-31 21:41:48,131 INFO  [o.a.s.e.c.c.SeaTunnelConfig   ] [main] - seatunnel.home is /root
2026-08-31 21:41:48,229 INFO  [amlSeaTunnelDomConfigProcessor] [main] - Dynamic slot is enabled, the schedule strategy is set to REJECT
2026-08-31 21:41:48,230 INFO  [c.h.i.c.AbstractConfigLocator ] [main] - Loading configuration '/data/apache-seatunnel-2.3.12/config/hazelcast.yaml' from System property 'hazelcast.config'
2026-08-31 21:41:48,230 INFO  [c.h.i.c.AbstractConfigLocator ] [main] - Using configuration file at /data/apache-seatunnel-2.3.12/config/hazelcast.yaml
2026-08-31 21:41:48,566 INFO  [c.h.i.c.AbstractConfigLocator ] [main] - Loading configuration '/data/apache-seatunnel-2.3.12/config/hazelcast-client.yaml' from System property 'hazelcast.client.config'
2026-08-31 21:41:48,567 INFO  [c.h.i.c.AbstractConfigLocator ] [main] - Using configuration file at /data/apache-seatunnel-2.3.12/config/hazelcast-client.yaml
2026-08-31 21:41:48,811 INFO  [.c.i.s.ClientInvocationService] [main] - hz.client_1 [seatunnel] [5.1] Running with 2 response threads, dynamic=true
2026-08-31 21:41:48,869 INFO  [c.h.c.LifecycleService        ] [main] - hz.client_1 [seatunnel] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is STARTING
2026-08-31 21:41:48,869 INFO  [c.h.c.LifecycleService        ] [main] - hz.client_1 [seatunnel] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is STARTED
2026-08-31 21:41:48,891 INFO  [.c.i.c.ClientConnectionManager] [main] - hz.client_1 [seatunnel] [5.1] Trying to connect to cluster: seatunnel
2026-08-31 21:41:48,895 INFO  [.c.i.c.ClientConnectionManager] [main] - hz.client_1 [seatunnel] [5.1] Trying to connect to [localhost]:5801
2026-08-31 21:41:48,923 INFO  [c.h.c.LifecycleService        ] [main] - hz.client_1 [seatunnel] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is CLIENT_CONNECTED
2026-08-31 21:41:48,924 INFO  [.c.i.c.ClientConnectionManager] [main] - hz.client_1 [seatunnel] [5.1] Authenticated with server [localhost]:5801:b987bf06-e3c1-4e92-9522-c1d685264412, server version: 5.1, local address: /127.0.0.1:29903
2026-08-31 21:41:48,926 INFO  [c.h.i.d.Diagnostics           ] [main] - hz.client_1 [seatunnel] [5.1] Diagnostics disabled. To enable add -Dhazelcast.diagnostics.enabled=true to the JVM arguments.
2026-08-31 21:41:48,933 INFO  [c.h.c.i.s.ClientClusterService] [hz.client_1.event-3] - hz.client_1 [seatunnel] [5.1] 

Members [1] {
        Member [localhost]:5801 - b987bf06-e3c1-4e92-9522-c1d685264412 [master node]
}

2026-08-31 21:41:48,958 INFO  [.c.i.s.ClientStatisticsService] [main] - Client statistics is enabled with period 5 seconds.
2026-08-31 21:41:49,609 INFO  [c.h.c.LifecycleService        ] [main] - hz.client_1 [seatunnel] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is SHUTTING_DOWN
2026-08-31 21:41:49,612 INFO  [.c.i.c.ClientConnectionManager] [main] - hz.client_1 [seatunnel] [5.1] Removed connection to endpoint: [localhost]:5801:b987bf06-e3c1-4e92-9522-c1d685264412, connection: ClientConnection{alive=false, connectionId=1, channel=NioChannel{/127.0.0.1:29903->localhost/127.0.0.1:5801}, remoteAddress=[localhost]:5801, lastReadTime=2026-08-31 21:41:49.607, lastWriteTime=2026-08-31 21:41:49.088, closedTime=2026-08-31 21:41:49.610, connected server version=5.1}
2026-08-31 21:41:49,612 INFO  [c.h.c.LifecycleService        ] [main] - hz.client_1 [seatunnel] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is CLIENT_DISCONNECTED
2026-08-31 21:41:49,614 INFO  [c.h.c.LifecycleService        ] [main] - hz.client_1 [seatunnel] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is SHUTDOWN
2026-08-31 21:41:49,615 INFO  [s.c.s.s.c.ClientExecuteCommand] [main] - Closed SeaTunnel client......
[root@oggres1948 ~]# 
[1]+  Done                    /data/apache-seatunnel-2.3.12/bin/seatunnel.sh -c /data/apache-seatunnel-2.3.12/config/demo_2tables_mysql.conf -r 1146757990657818625 -cn seatunnel -n MySQL_CDC_2Tables > /data/apache-seatunnel-2.3.12/logs/demo_2tables_mysql_cluster.out 2>&1

–查看JOB状态
/data/apache-seatunnel-2.3.12/bin/seatunnel.sh -l -cn seatunnel

# /data/apache-seatunnel-2.3.12/bin/seatunnel.sh  -l  -cn seatunnel
2026-08-31 21:41:55,210 INFO  [c.h.i.c.AbstractConfigLocator ] [main] - Loading configuration '/data/apache-seatunnel-2.3.12/config/seatunnel.yaml' from System property 'seatunnel.config'
2026-08-31 21:41:55,214 INFO  [c.h.i.c.AbstractConfigLocator ] [main] - Using configuration file at /data/apache-seatunnel-2.3.12/config/seatunnel.yaml
2026-08-31 21:41:55,217 INFO  [o.a.s.e.c.c.SeaTunnelConfig   ] [main] - seatunnel.home is /root
2026-08-31 21:41:55,319 INFO  [amlSeaTunnelDomConfigProcessor] [main] - Dynamic slot is enabled, the schedule strategy is set to REJECT
2026-08-31 21:41:55,319 INFO  [c.h.i.c.AbstractConfigLocator ] [main] - Loading configuration '/data/apache-seatunnel-2.3.12/config/hazelcast.yaml' from System property 'hazelcast.config'
2026-08-31 21:41:55,319 INFO  [c.h.i.c.AbstractConfigLocator ] [main] - Using configuration file at /data/apache-seatunnel-2.3.12/config/hazelcast.yaml
2026-08-31 21:41:55,655 INFO  [c.h.i.c.AbstractConfigLocator ] [main] - Loading configuration '/data/apache-seatunnel-2.3.12/config/hazelcast-client.yaml' from System property 'hazelcast.client.config'
2026-08-31 21:41:55,655 INFO  [c.h.i.c.AbstractConfigLocator ] [main] - Using configuration file at /data/apache-seatunnel-2.3.12/config/hazelcast-client.yaml
2026-08-31 21:41:55,901 INFO  [.c.i.s.ClientInvocationService] [main] - hz.client_1 [seatunnel] [5.1] Running with 2 response threads, dynamic=true
2026-08-31 21:41:55,956 INFO  [c.h.c.LifecycleService        ] [main] - hz.client_1 [seatunnel] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is STARTING
2026-08-31 21:41:55,957 INFO  [c.h.c.LifecycleService        ] [main] - hz.client_1 [seatunnel] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is STARTED
2026-08-31 21:41:55,979 INFO  [.c.i.c.ClientConnectionManager] [main] - hz.client_1 [seatunnel] [5.1] Trying to connect to cluster: seatunnel
2026-08-31 21:41:55,982 INFO  [.c.i.c.ClientConnectionManager] [main] - hz.client_1 [seatunnel] [5.1] Trying to connect to [localhost]:5801
2026-08-31 21:41:56,011 INFO  [c.h.c.LifecycleService        ] [main] - hz.client_1 [seatunnel] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is CLIENT_CONNECTED
2026-08-31 21:41:56,011 INFO  [.c.i.c.ClientConnectionManager] [main] - hz.client_1 [seatunnel] [5.1] Authenticated with server [localhost]:5801:b987bf06-e3c1-4e92-9522-c1d685264412, server version: 5.1, local address: /127.0.0.1:58649
2026-08-31 21:41:56,012 INFO  [c.h.i.d.Diagnostics           ] [main] - hz.client_1 [seatunnel] [5.1] Diagnostics disabled. To enable add -Dhazelcast.diagnostics.enabled=true to the JVM arguments.
2026-08-31 21:41:56,020 INFO  [c.h.c.i.s.ClientClusterService] [hz.client_1.event-3] - hz.client_1 [seatunnel] [5.1] 

Members [1] {
        Member [localhost]:5801 - b987bf06-e3c1-4e92-9522-c1d685264412 [master node]
}

2026-08-31 21:41:56,044 INFO  [.c.i.s.ClientStatisticsService] [main] - Client statistics is enabled with period 5 seconds.
Job ID               Job Name           Job Status      Submit Time              Start Time               Finished Time            
-------------------  -----------------  --------------  -----------------------  -----------------------  -----------------------  
1146757990657818625  MySQL_CDC_2Tables  SAVEPOINT_DONE  2026-08-31 21:38:33.65   2026-08-31 21:38:33.721  2026-08-31 21:41:49.606
1146724631827185665  SeaTunnel_Job      CANCELED        2026-08-31 16:34:19.68   2026-08-31 16:34:20.312  2026-08-31 16:46:17.122

2026-08-31 21:41:56,239 INFO  [c.h.c.LifecycleService        ] [main] - hz.client_1 [seatunnel] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is SHUTTING_DOWN
2026-08-31 21:41:56,242 INFO  [.c.i.c.ClientConnectionManager] [main] - hz.client_1 [seatunnel] [5.1] Removed connection to endpoint: [localhost]:5801:b987bf06-e3c1-4e92-9522-c1d685264412, connection: ClientConnection{alive=false, connectionId=1, channel=NioChannel{/127.0.0.1:58649->localhost/127.0.0.1:5801}, remoteAddress=[localhost]:5801, lastReadTime=2026-08-31 21:41:56.184, lastWriteTime=2026-08-31 21:41:56.182, closedTime=2026-08-31 21:41:56.240, connected server version=5.1}
2026-08-31 21:41:56,242 INFO  [c.h.c.LifecycleService        ] [main] - hz.client_1 [seatunnel] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is CLIENT_DISCONNECTED
2026-08-31 21:41:56,244 INFO  [c.h.c.LifecycleService        ] [main] - hz.client_1 [seatunnel] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is SHUTDOWN
2026-08-31 21:41:56,244 INFO  [s.c.s.s.c.ClientExecuteCommand] [main] - Closed SeaTunnel client......

根据savecheckpoint重启任务

/data/apache-seatunnel-2.3.12/bin/seatunnel.sh \
  -c /data/apache-seatunnel-2.3.12/config/demo_2tables_mysql.conf \
  -r 1146757990657818625 \
  -cn seatunnel \
  -n MySQL_CDC_2Tables \
  > /data/apache-seatunnel-2.3.12/logs/demo_2tables_mysql_cluster.out 2>&1 &

重启日志

# /data/apache-seatunnel-2.3.12/bin/seatunnel.sh \
>   -c /data/apache-seatunnel-2.3.12/config/demo_2tables_mysql.conf \
>   -r 1146757990657818625 \
>   -cn seatunnel \
>   -n MySQL_CDC_2Tables
2026-08-31 21:42:13,987 INFO  [c.h.i.c.AbstractConfigLocator ] [main] - Loading configuration '/data/apache-seatunnel-2.3.12/config/seatunnel.yaml' from System property 'seatunnel.config'
2026-08-31 21:42:13,991 INFO  [c.h.i.c.AbstractConfigLocator ] [main] - Using configuration file at /data/apache-seatunnel-2.3.12/config/seatunnel.yaml
2026-08-31 21:42:13,993 INFO  [o.a.s.e.c.c.SeaTunnelConfig   ] [main] - seatunnel.home is /root
2026-08-31 21:42:14,093 INFO  [amlSeaTunnelDomConfigProcessor] [main] - Dynamic slot is enabled, the schedule strategy is set to REJECT
2026-08-31 21:42:14,093 INFO  [c.h.i.c.AbstractConfigLocator ] [main] - Loading configuration '/data/apache-seatunnel-2.3.12/config/hazelcast.yaml' from System property 'hazelcast.config'
2026-08-31 21:42:14,093 INFO  [c.h.i.c.AbstractConfigLocator ] [main] - Using configuration file at /data/apache-seatunnel-2.3.12/config/hazelcast.yaml
2026-08-31 21:42:14,440 INFO  [c.h.i.c.AbstractConfigLocator ] [main] - Loading configuration '/data/apache-seatunnel-2.3.12/config/hazelcast-client.yaml' from System property 'hazelcast.client.config'
2026-08-31 21:42:14,440 INFO  [c.h.i.c.AbstractConfigLocator ] [main] - Using configuration file at /data/apache-seatunnel-2.3.12/config/hazelcast-client.yaml
2026-08-31 21:42:14,707 INFO  [.c.i.s.ClientInvocationService] [main] - hz.client_1 [seatunnel] [5.1] Running with 2 response threads, dynamic=true
2026-08-31 21:42:14,764 INFO  [c.h.c.LifecycleService        ] [main] - hz.client_1 [seatunnel] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is STARTING
2026-08-31 21:42:14,765 INFO  [c.h.c.LifecycleService        ] [main] - hz.client_1 [seatunnel] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is STARTED
2026-08-31 21:42:14,786 INFO  [.c.i.c.ClientConnectionManager] [main] - hz.client_1 [seatunnel] [5.1] Trying to connect to cluster: seatunnel
2026-08-31 21:42:14,790 INFO  [.c.i.c.ClientConnectionManager] [main] - hz.client_1 [seatunnel] [5.1] Trying to connect to [localhost]:5801
2026-08-31 21:42:14,818 INFO  [c.h.c.LifecycleService        ] [main] - hz.client_1 [seatunnel] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is CLIENT_CONNECTED
2026-08-31 21:42:14,819 INFO  [.c.i.c.ClientConnectionManager] [main] - hz.client_1 [seatunnel] [5.1] Authenticated with server [localhost]:5801:b987bf06-e3c1-4e92-9522-c1d685264412, server version: 5.1, local address: /127.0.0.1:35308
2026-08-31 21:42:14,820 INFO  [c.h.i.d.Diagnostics           ] [main] - hz.client_1 [seatunnel] [5.1] Diagnostics disabled. To enable add -Dhazelcast.diagnostics.enabled=true to the JVM arguments.
2026-08-31 21:42:14,828 INFO  [c.h.c.i.s.ClientClusterService] [hz.client_1.event-3] - hz.client_1 [seatunnel] [5.1] 

Members [1] {
        Member [localhost]:5801 - b987bf06-e3c1-4e92-9522-c1d685264412 [master node]
}

2026-08-31 21:42:14,851 INFO  [.c.i.s.ClientStatisticsService] [main] - Client statistics is enabled with period 5 seconds.
2026-08-31 21:42:14,980 INFO  [.ClientJobExecutionEnvironment] [main] - Start with savepoint, load checkpoint state from job client <<<<<<<<<<<<
2026-08-31 21:42:15,041 INFO  [o.a.s.c.s.u.ConfigBuilder     ] [main] - Loading config file from path: /data/apache-seatunnel-2.3.12/config/demo_2tables_mysql.conf
2026-08-31 21:42:15,129 INFO  [o.a.s.c.s.u.ConfigShadeUtils  ] [main] - Load config shade spi: [base64]
2026-08-31 21:42:15,161 INFO  [o.a.s.c.s.u.ConfigBuilder     ] [main] - Parsed config file: 
{
    "env" : {
        "execution.parallelism" : 2,
        "job.mode" : "STREAMING",
        "checkpoint.interval" : 10000
    },
    "source" : [
        {
            "base-url" : "jdbc:mysql://127.0.0.1:6033/bigdata",
            "username" : "******",
            "password" : "******",
            "database-names" : [
                "bigdata"
            ],
            "table-names" : [
                "bigdata.test_orders",
                "bigdata.t_8_100w"
            ],
            "startup.mode" : "initial",
            "server-id" : 5400,
            "server-time-zone" : "Asia/Shanghai",
            "plugin_name" : "MySQL-CDC"
        }
    ],
    "transform" : [],
    "sink" : [
        {
            "url" : "jdbc:mysql://127.0.0.1:6033",
            "driver" : "com.mysql.cj.jdbc.Driver",
            "user" : "cdcadmin",
            "password" : "******",
            "generate_sink_sql" : true,
            "database" : "targetdb",
            "table" : "${table_name}",
            "primary_keys" : [
                "${primary_key}"
            ],
            "schema_save_mode" : "CREATE_SCHEMA_WHEN_NOT_EXIST",
            "data_save_mode" : "APPEND_DATA",
            "batch_size" : 5000,
            "max_retries" : 3,
            "connection_check_timeout_sec" : 300,
            "properties" : {
                "useUnicode" : true,
                "characterEncoding" : "utf8",
                "serverTimezone" : "Asia/Shanghai",
                "rewriteBatchedStatements" : "true",
                "useCompression" : "true",
                "useServerPrepStmts" : "false"
            },
            "plugin_name" : "jdbc"
        }
    ]
}

2026-08-31 21:42:15,169 INFO  [p.MultipleTableJobConfigParser] [main] - add common jar in plugins :[]
2026-08-31 21:42:15,191 INFO  [.s.p.d.AbstractPluginDiscovery] [main] - Load SeaTunnelSink Plugin from /data/apache-seatunnel-2.3.12/connectors
2026-08-31 21:42:15,196 INFO  [.s.p.d.AbstractPluginDiscovery] [main] - Discovery plugin jar for: PluginIdentifier{engineType='seatunnel', pluginType='source', pluginName='MySQL-CDC'} at: [file:/data/apache-seatunnel-2.3.12/connectors/connector-cdc-base-2.3.12.jar, file:/data/apache-seatunnel-2.3.12/connectors/connector-cdc-mysql-2.3.12.jar]
2026-08-31 21:42:15,196 INFO  [.s.p.d.AbstractPluginDiscovery] [main] - find connector jar and dependency for PluginIdentifier{engineType='seatunnel', pluginType='source', pluginName='MySQL-CDC'}: [file:/data/apache-seatunnel-2.3.12/connectors/connector-cdc-base-2.3.12.jar, file:/data/apache-seatunnel-2.3.12/connectors/connector-cdc-mysql-2.3.12.jar]
2026-08-31 21:42:15,201 INFO  [.s.p.d.AbstractPluginDiscovery] [main] - Load SeaTunnelSink Plugin from /data/apache-seatunnel-2.3.12/connectors
2026-08-31 21:42:15,206 INFO  [.s.p.d.AbstractPluginDiscovery] [main] - Load SeaTunnelSink Plugin from /data/apache-seatunnel-2.3.12/connectors
2026-08-31 21:42:15,208 INFO  [.s.p.d.AbstractPluginDiscovery] [main] - Discovery plugin jar for: PluginIdentifier{engineType='seatunnel', pluginType='sink', pluginName='jdbc'} at: [file:/data/apache-seatunnel-2.3.12/connectors/connector-jdbc-2.3.12.jar]
2026-08-31 21:42:15,208 INFO  [.s.p.d.AbstractPluginDiscovery] [main] - find connector jar and dependency for PluginIdentifier{engineType='seatunnel', pluginType='sink', pluginName='jdbc'}: [file:/data/apache-seatunnel-2.3.12/connectors/connector-jdbc-2.3.12.jar]
2026-08-31 21:42:15,211 INFO  [p.MultipleTableJobConfigParser] [main] - start generating all sources.
2026-08-31 21:42:15,266 WARN  [o.a.s.a.c.ReadonlyConfig      ] [main] - Please use the new key 'url' instead of the deprecated key 'base-url'.
2026-08-31 21:42:15,268 INFO  [o.a.s.a.t.f.FactoryUtil       ] [main] - Restore create source from checkpoint state:  <<<<<<<
ChangeStreamTableSourceState(enumeratorState=HybridPendingSplitsState(snapshotPhaseState=SnapshotPhaseState(remainingTables=[], alreadyProcessedTables=[bigdata.t_8_100w, bigdata.test_orders], remainingSplits=[], assignedSplits={}, splitCompletedOffsets={}, isAssignerCompleted=true, isTableIdCaseSensitive=false, isRemainingTablesCheckpointed=true), incrementalPhaseState=IncrementalPhaseState()), splits=[[IncrementalSplit(tableIds=[bigdata.t_8_100w, bigdata.test_orders], startupOffset={transaction_id=null, file=mysql-bin.000014, pos=22333411, gtids=743d8fd4-a38f-11f1-aeb1-0cda411d29c9:1-1426, server_id=102}, stopOffset={ts_sec=0, file=, pos=-9223372036854775808, row=0, event=0}, completedSnapshotSplitInfos=[], checkpointDataType=null, checkpointTables=[CatalogTable{tableId=MySQL.bigdata.test_orders, tableSchema=TableSchema(primaryKey=PrimaryKey(primaryKey=PRIMARY, columnNames=[id], enableAutoId=null), constraintKeys=[]), options={connector=jdbc, table-name=bigdata.test_orders, url=jdbc:mysql://127.0.0.1:6033/bigdata}, partitionKeys=[], comment='', catalogName='MySQL', metadata=MetadataSchema()}, CatalogTable{tableId=MySQL.bigdata.t_8_100w, tableSchema=TableSchema(primaryKey=PrimaryKey(primaryKey=PRIMARY, columnNames=[id], enableAutoId=null), constraintKeys=[]), options={connector=jdbc, table-name=bigdata.t_8_100w, url=jdbc:mysql://127.0.0.1:6033/bigdata}, partitionKeys=[], comment='', catalogName='MySQL', metadata=MetadataSchema()}], historyTableChanges={bigdata.t_8_100w=[B@2eb917d0, bigdata.test_orders=[B@c6b2dd9})]])
2026-08-31 21:42:15,269 INFO  [ChangeStreamTableSourceFactory] [main] - Restore source using checkpoint tables: [CatalogTable{tableId=MySQL.bigdata.test_orders, tableSchema=TableSchema(primaryKey=PrimaryKey(primaryKey=PRIMARY, columnNames=[id], enableAutoId=null), constraintKeys=[]), options={connector=jdbc, table-name=bigdata.test_orders, url=jdbc:mysql://127.0.0.1:6033/bigdata}, partitionKeys=[], comment='', catalogName='MySQL', metadata=MetadataSchema()}, CatalogTable{tableId=MySQL.bigdata.t_8_100w, tableSchema=TableSchema(primaryKey=PrimaryKey(primaryKey=PRIMARY, columnNames=[id], enableAutoId=null), constraintKeys=[]), options={connector=jdbc, table-name=bigdata.t_8_100w, url=jdbc:mysql://127.0.0.1:6033/bigdata}, partitionKeys=[], comment='', catalogName='MySQL', metadata=MetadataSchema()}]
2026-08-31 21:42:15,272 WARN  [o.a.s.a.c.ReadonlyConfig      ] [main] - Please use the new key 'url' instead of the deprecated key 'base-url'.
2026-08-31 21:42:15,683 INFO  [.s.c.s.j.c.AbstractJdbcCatalog] [main] - Catalog MySQL established connection to jdbc:mysql://127.0.0.1:6033/bigdata
2026-08-31 21:42:15,722 INFO  [o.a.s.a.t.c.CatalogTableUtil  ] [main] - Get catalog tables, cost time: 39 ms
2026-08-31 21:42:15,723 INFO  [.s.c.s.j.c.AbstractJdbcCatalog] [main] - Catalog MySQL closing
2026-08-31 21:42:15,732 WARN  [o.a.s.a.c.ReadonlyConfig      ] [main] - Please use the new key 'url' instead of the deprecated key 'base-url'.
2026-08-31 21:42:15,783 WARN  [i.d.c.m.MySqlConnection       ] [main] - Database configuration option 'serverTimezone' is set but is obsolete, please use 'connectionTimeZone' instead
2026-08-31 21:42:15,793 INFO  [i.d.c.m.MySqlConnection       ] [main] - Using 'SHOW BINARY LOG STATUS' to get binary log status
2026-08-31 21:42:15,813 INFO  [.c.s.c.m.u.TableDiscoveryUtils] [main] - Read list of available databases
2026-08-31 21:42:15,815 INFO  [.c.s.c.m.u.TableDiscoveryUtils] [main] -          list of available databases is: [bigdata]
2026-08-31 21:42:15,815 INFO  [.c.s.c.m.u.TableDiscoveryUtils] [main] - Read list of available tables in each database
2026-08-31 21:42:15,820 INFO  [i.d.j.JdbcConnection          ] [pool-2-thread-1] - Connection gracefully closed
2026-08-31 21:42:15,824 WARN  [i.d.c.m.MySqlConnection       ] [main] - Database configuration option 'serverTimezone' is set but is obsolete, please use 'connectionTimeZone' instead
2026-08-31 21:42:15,827 INFO  [i.d.c.m.MySqlConnection       ] [main] - Using 'SHOW BINARY LOG STATUS' to get binary log status
2026-08-31 21:42:16,477 INFO  [.a.s.c.c.b.u.CatalogTableUtils] [main] - Override primary key([id]) for catalog table bigdata.t_8_100w
2026-08-31 21:42:16,497 INFO  [.a.s.c.c.b.u.CatalogTableUtils] [main] - Override primary key([id]) for catalog table bigdata.test_orders
2026-08-31 21:42:16,498 INFO  [i.d.j.JdbcConnection          ] [pool-3-thread-1] - Connection gracefully closed
2026-08-31 21:42:16,511 WARN  [o.a.s.a.c.ReadonlyConfig      ] [main] - Please use the new key 'url' instead of the deprecated key 'base-url'.
2026-08-31 21:42:16,654 INFO  [o.a.k.c.j.JsonConverterConfig ] [main] - JsonConverterConfig values: 
        converter.type = value
        decimal.format = BASE64
        schemas.cache.size = 1000
        schemas.enable = true

2026-08-31 21:42:16,678 INFO  [o.a.s.a.t.f.FactoryUtil       ] [main] - get the CatalogTable from source MySQL-CDC: MySQL.bigdata.test_orders,MySQL.bigdata.t_8_100w
2026-08-31 21:42:16,690 INFO  [.s.p.d.AbstractPluginDiscovery] [main] - Load SeaTunnelSource Plugin from /data/apache-seatunnel-2.3.12/connectors
2026-08-31 21:42:16,691 INFO  [.s.p.d.AbstractPluginDiscovery] [main] - Discovery plugin jar for: PluginIdentifier{engineType='seatunnel', pluginType='source', pluginName='MySQL-CDC'} at: [file:/data/apache-seatunnel-2.3.12/connectors/connector-cdc-base-2.3.12.jar, file:/data/apache-seatunnel-2.3.12/connectors/connector-cdc-mysql-2.3.12.jar]
2026-08-31 21:42:16,691 INFO  [.s.p.d.AbstractPluginDiscovery] [main] - find connector jar and dependency for PluginIdentifier{engineType='seatunnel', pluginType='source', pluginName='MySQL-CDC'}: [file:/data/apache-seatunnel-2.3.12/connectors/connector-cdc-base-2.3.12.jar, file:/data/apache-seatunnel-2.3.12/connectors/connector-cdc-mysql-2.3.12.jar]
2026-08-31 21:42:16,693 INFO  [p.MultipleTableJobConfigParser] [main] - start generating all transforms.
2026-08-31 21:42:16,693 INFO  [p.MultipleTableJobConfigParser] [main] - start generating all sinks.
2026-08-31 21:42:16,698 INFO  [.s.p.d.AbstractPluginDiscovery] [main] - Load SeaTunnelSink Plugin from /data/apache-seatunnel-2.3.12/connectors
2026-08-31 21:42:16,699 INFO  [.s.p.d.AbstractPluginDiscovery] [main] - Discovery plugin jar for: PluginIdentifier{engineType='seatunnel', pluginType='sink', pluginName='jdbc'} at: [file:/data/apache-seatunnel-2.3.12/connectors/connector-jdbc-2.3.12.jar]
2026-08-31 21:42:16,699 INFO  [.s.p.d.AbstractPluginDiscovery] [main] - find connector jar and dependency for PluginIdentifier{engineType='seatunnel', pluginType='sink', pluginName='jdbc'}: [file:/data/apache-seatunnel-2.3.12/connectors/connector-jdbc-2.3.12.jar]
2026-08-31 21:42:16,716 INFO  [o.a.s.a.t.f.FactoryUtil       ] [main] - Create sink 'jdbc' with upstream input catalog-table[database: bigdata, schema: null, table: test_orders]
2026-08-31 21:42:16,720 WARN  [o.a.s.a.c.ReadonlyConfig      ] [main] - Please use the new key 'username' instead of the deprecated key 'user'.
2026-08-31 21:42:16,745 INFO  [o.a.s.a.t.f.FactoryUtil       ] [main] - Create sink 'jdbc' with upstream input catalog-table[database: bigdata, schema: null, table: t_8_100w]
2026-08-31 21:42:16,746 WARN  [o.a.s.a.c.ReadonlyConfig      ] [main] - Please use the new key 'username' instead of the deprecated key 'user'.
2026-08-31 21:42:16,811 INFO  [o.a.s.e.c.j.ClientJobProxy    ] [main] - Start submit job, job id: 1146757990657818625, with plugin jar [file:/data/apache-seatunnel-2.3.12/connectors/connector-cdc-mysql-2.3.12.jar, file:/data/apache-seatunnel-2.3.12/connectors/connector-jdbc-2.3.12.jar, file:/data/apache-seatunnel-2.3.12/connectors/connector-cdc-base-2.3.12.jar]
2026-08-31 21:42:16,826 INFO  [o.a.s.e.c.j.ClientJobProxy    ] [main] - Submit job finished, job id: 1146757990657818625, job name: MySQL_CDC_2Tables
2026-08-31 21:42:16,841 WARN  [o.a.s.e.c.j.JobMetricsRunner  ] [job-metrics-runner-1146757990657818625] - Failed to get job metrics summary, it maybe first-run
2026-08-31 21:43:16,842 INFO  [o.a.s.e.c.j.JobMetricsRunner  ] [job-metrics-runner-1146757990657818625] -

日志分析：

-r 确实加载了之前的状态

Start with savepoint, load checkpoint state from job client

最关键：Snapshot 已经完成

SnapshotPhaseState(
    remainingTables=[],
    alreadyProcessedTables=[
        bigdata.t_8_100w,
        bigdata.test_orders
    ],
    remainingSplits=[],
    assignedSplits={},
    splitCompletedOffsets={},
    isAssignerCompleted=true
)

remainingTables=[]
alreadyProcessedTables=[
    bigdata.t_8_100w,
    bigdata.test_orders
]
isAssignerCompleted=true

这说明之前的全量 Snapshot 已经完成。

所以恢复的时候不会重新对这两张表做全量 Snapshot。

更关键：已经恢复出了 Binlog 位点

startupOffset={
    transaction_id=null,
    file=mysql-bin.000014,
    pos=22333411,
    gtids=743d8fd4-a38f-11f1-aeb1-0cda411d29c9:1-1426,
    server_id=102
}

也就是说，这个 Job 恢复后的 MySQL CDC 起点是：

Binlog File : mysql-bin.000014
Binlog Pos  : 22333411
GTID        : 743d8fd4-a38f-11f1-aeb1-0cda411d29c9:1-1426

不是重新从 startup.mode = “initial” 开始 Snapshot，而是从 Checkpoint 保存的 Binlog 位点继续。

这次恢复后的 Job ID 没有改变


Start submit job, job id: 1146757990657818625
Submit job finished, job id: 1146757990657818625,
job name: MySQL_CDC_2Tables

也就是说：

原 Job ID
1146757990657818625

恢复后 Job ID
1146757990657818625

再次查看状态恢复RUNNING


# /data/apache-seatunnel-2.3.12/bin/seatunnel.sh  -l  -cn seatunnel
2026-08-31 21:51:13,783 INFO  [c.h.i.c.AbstractConfigLocator ] [main] - Loading configuration '/data/apache-seatunnel-2.3.12/config/seatunnel.yaml' from System property 'seatunnel.config'
2026-08-31 21:51:13,787 INFO  [c.h.i.c.AbstractConfigLocator ] [main] - Using configuration file at /data/apache-seatunnel-2.3.12/config/seatunnel.yaml
2026-08-31 21:51:13,789 INFO  [o.a.s.e.c.c.SeaTunnelConfig   ] [main] - seatunnel.home is /root
2026-08-31 21:51:13,890 INFO  [amlSeaTunnelDomConfigProcessor] [main] - Dynamic slot is enabled, the schedule strategy is set to REJECT
2026-08-31 21:51:13,891 INFO  [c.h.i.c.AbstractConfigLocator ] [main] - Loading configuration '/data/apache-seatunnel-2.3.12/config/hazelcast.yaml' from System property 'hazelcast.config'
2026-08-31 21:51:13,891 INFO  [c.h.i.c.AbstractConfigLocator ] [main] - Using configuration file at /data/apache-seatunnel-2.3.12/config/hazelcast.yaml
2026-08-31 21:51:14,244 INFO  [c.h.i.c.AbstractConfigLocator ] [main] - Loading configuration '/data/apache-seatunnel-2.3.12/config/hazelcast-client.yaml' from System property 'hazelcast.client.config'
2026-08-31 21:51:14,245 INFO  [c.h.i.c.AbstractConfigLocator ] [main] - Using configuration file at /data/apache-seatunnel-2.3.12/config/hazelcast-client.yaml
2026-08-31 21:51:14,503 INFO  [.c.i.s.ClientInvocationService] [main] - hz.client_1 [seatunnel] [5.1] Running with 2 response threads, dynamic=true
2026-08-31 21:51:14,557 INFO  [c.h.c.LifecycleService        ] [main] - hz.client_1 [seatunnel] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is STARTING
2026-08-31 21:51:14,558 INFO  [c.h.c.LifecycleService        ] [main] - hz.client_1 [seatunnel] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is STARTED
2026-08-31 21:51:14,578 INFO  [.c.i.c.ClientConnectionManager] [main] - hz.client_1 [seatunnel] [5.1] Trying to connect to cluster: seatunnel
2026-08-31 21:51:14,582 INFO  [.c.i.c.ClientConnectionManager] [main] - hz.client_1 [seatunnel] [5.1] Trying to connect to [localhost]:5801
2026-08-31 21:51:14,610 INFO  [c.h.c.LifecycleService        ] [main] - hz.client_1 [seatunnel] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is CLIENT_CONNECTED
2026-08-31 21:51:14,610 INFO  [.c.i.c.ClientConnectionManager] [main] - hz.client_1 [seatunnel] [5.1] Authenticated with server [localhost]:5801:b987bf06-e3c1-4e92-9522-c1d685264412, server version: 5.1, local address: /127.0.0.1:57766
2026-08-31 21:51:14,611 INFO  [c.h.i.d.Diagnostics           ] [main] - hz.client_1 [seatunnel] [5.1] Diagnostics disabled. To enable add -Dhazelcast.diagnostics.enabled=true to the JVM arguments.
2026-08-31 21:51:14,619 INFO  [c.h.c.i.s.ClientClusterService] [hz.client_1.event-3] - hz.client_1 [seatunnel] [5.1] 

Members [1] {
        Member [localhost]:5801 - b987bf06-e3c1-4e92-9522-c1d685264412 [master node]
}

2026-08-31 21:51:14,642 INFO  [.c.i.s.ClientStatisticsService] [main] - Client statistics is enabled with period 5 seconds.
Job ID               Job Name           Job Status  Submit Time              Start Time               Finished Time            
-------------------  -----------------  ----------  -----------------------  -----------------------  -----------------------  
1146757990657818625  MySQL_CDC_2Tables  RUNNING     2026-08-31 21:50:37.535  2026-08-31 21:50:37.588  
1146724631827185665  SeaTunnel_Job      CANCELED    2026-08-31 16:34:19.68   2026-08-31 16:34:20.312  2026-08-31 16:46:17.122

2026-08-31 21:51:14,816 INFO  [c.h.c.LifecycleService        ] [main] - hz.client_1 [seatunnel] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is SHUTTING_DOWN
2026-08-31 21:51:14,819 INFO  [.c.i.c.ClientConnectionManager] [main] - hz.client_1 [seatunnel] [5.1] Removed connection to endpoint: [localhost]:5801:b987bf06-e3c1-4e92-9522-c1d685264412, connection: ClientConnection{alive=false, connectionId=1, channel=NioChannel{/127.0.0.1:57766->localhost/127.0.0.1:5801}, remoteAddress=[localhost]:5801, lastReadTime=2026-08-31 21:51:14.765, lastWriteTime=2026-08-31 21:51:14.763, closedTime=2026-08-31 21:51:14.817, connected server version=5.1}
2026-08-31 21:51:14,819 INFO  [c.h.c.LifecycleService        ] [main] - hz.client_1 [seatunnel] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is CLIENT_DISCONNECTED
2026-08-31 21:51:14,822 INFO  [c.h.c.LifecycleService        ] [main] - hz.client_1 [seatunnel] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is SHUTDOWN
2026-08-31 21:51:14,822 INFO  [s.c.s.s.c.ClientExecuteCommand] [main] - Closed SeaTunnel client......

结论

你这次实验已经成功验证了：

MySQL CDC
    │
    ├── startup.mode = initial
    │
    ├── 首次运行
    │      ├── Snapshot test_orders
    │      ├── Snapshot t_8_100w
    │      └── 持续读取 Binlog
    │
    ├── Checkpoint
    │      └── 保存 Snapshot 完成状态 + Binlog Offset
    │
    ├── Job 停止
    │
    └── -r 1146757990657818625
             │
             ├── 加载 Checkpoint
             ├── Snapshot remainingTables = []
             ├── alreadyProcessedTables = 两张表
             ├── isAssignerCompleted = true
             │
             └── 从 mysql-bin.000014:22333411
                 继续 CDC

标签

#apache #seatunnel #断点续传