Rebase auto increment not executed after second restore

Bug Report

In the restore process, if the first restore fails but leaves partial table structures in the cluster, and the second restore uses OnExistIgnore, those pre-existing tables will be skipped.

As a result, row_id rebase is not executed for these tables, leading to incorrect auto-increment behavior.

The root cause is that batchCreateTables is not atomic.

tidb/pkg/ddl/table.go

Line 231 in 287f27e

tbInfo, err := createTable(d, t, stubJob, fkCheck)

Some table could become public while the whole Create Tables job still on going.

1. Minimal reproduce step (Required)

doing a restore with batch tables.
random kill restore process, try to make createTables partial succeed.

2. What did you expect to see? (Required)

All tables in the createTables batch should either be created successfully together or fail entirely — there should be no partial success.

3. What did you see instead (Required)

After the restore process exited, some tables from the batch were still present, indicating a partial creation.

4. What is your TiDB version? (Required)

v7.5.5

suppose we are creating 2 tables using batchCreateTables, we create the metakv of first table success on owner, but the second table already exists, we will mark the whole job as cancelled

tidb/pkg/ddl/create_table.go

Lines 80 to 86 in f030b63

    
           retryable, err := checkTableForeignKeyValidInOwner(jobCtx, job, tbInfo, fkCheck) 
        
           if err != nil { 
        
           	if !retryable { 
        
           		job.State = model.JobStateCancelled 
        
           	} 
        
           	return tbInfo, errors.Trace(err) 
        
           }

and then we commit the transaction, so metakv of the first table is commited

tidb/pkg/ddl/job_worker.go

Lines 622 to 626 in f030b63

    
           if job.IsCancelled() { 
        
           	defer jobCtx.unlockSchemaVersion(jobCtx, job.ID) 
        
           	w.sess.Reset() 
        
           	return 0, w.handleJobDone(jobCtx, job) 
        
           }

then on executor part, we return directly, without running createTableWithInfoPost for any table

tidb/pkg/ddl/executor.go

Lines 1303 to 1317 in f030b63

    
           err = e.DoDDLJobWrapper(ctx, jobW) 
        
           if err != nil { 
        
           	// table exists, but if_not_exists flags is true, so we ignore this error. 
        
           	if c.OnExist == OnExistIgnore && infoschema.ErrTableExists.Equal(err) { 
        
           		ctx.GetSessionVars().StmtCtx.AppendNote(err) 
        
           		err = nil 
        
           	} 
        
           	return errors.Trace(err) 
        
           } 
        
           for _, tblArgs := range args.Tables { 
        
           	if err = e.createTableWithInfoPost(ctx, tblArgs.TableInfo, jobW.SchemaID); err != nil { 
        
           		return errors.Trace(err) 
        
           	} 
        
           }

batch create tables is added before 6.1
/label affects-8.5
/label affects-8.1
/label affects-7.5
/label affects-7.1
/label affects-6.5
/label affects-6.1

@joechenrh

@joechenrh: The label(s) affects-6.1 cannot be applied. These labels are supported: fuzz/sqlancer, fuzz/comp, challenge-program, compatibility-breaker, first-time-contributor, contribution, good first issue, correctness, duplicate, proposal, security, needs-more-info, needs-cherry-pick-release-6.5, needs-cherry-pick-release-7.1, needs-cherry-pick-release-7.5, needs-cherry-pick-release-8.1, needs-cherry-pick-release-8.5, needs-cherry-pick-release-9.0-beta.2, affects-6.5, affects-7.1, affects-7.5, affects-8.1, affects-8.5, affects-9.0, may-affects-6.5, may-affects-7.1, may-affects-7.5, may-affects-8.1, may-affects-8.5.

Details

In response to this:

batch create tables is added before 6.1
/label affects-8.5
/label affects-8.1
/label affects-7.5
/label affects-7.1
/label affects-6.5
/label affects-6.1

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

/remove-label may-affects-6.1

@joechenrh

@joechenrh: The label(s) may-affects-6.1 cannot be applied. These labels are supported: fuzz/sqlancer, fuzz/comp, challenge-program, compatibility-breaker, first-time-contributor, contribution, good first issue, correctness, duplicate, proposal, security, needs-more-info, needs-cherry-pick-release-6.5, needs-cherry-pick-release-7.1, needs-cherry-pick-release-7.5, needs-cherry-pick-release-8.1, needs-cherry-pick-release-8.5, needs-cherry-pick-release-9.0-beta.2, affects-6.5, affects-7.1, affects-7.5, affects-8.1, affects-8.5, affects-9.0, may-affects-6.5, may-affects-7.1, may-affects-7.5, may-affects-8.1, may-affects-8.5.

Details

In response to this:

/remove-label may-affects-6.1

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

ddl: move auto id handling from submitter to executor (pingcap#62449)

ddl: add auto ID rebase create table submitted by BR (#64356)

ddl: move auto id handling from submitter to executor (pingcap#62449)

ddl: move auto id handling from submitter to executor (#62449) (#64761)

ddl: add auto ID rebase create table submitted by BR (pingcap#64356)

ddl: add auto ID rebase create table submitted by BR (#64356) (#64762)

Navigation Menu

Repository navigation

Rebase auto increment not executed after second restore #60804

Bug Report

1. Minimal reproduce step (Required)

2. What did you expect to see? (Required)

3. What did you see instead (Required)

4. What is your TiDB version? (Required)

5 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	retryable, err := checkTableForeignKeyValidInOwner(jobCtx, job, tbInfo, fkCheck)
	if err != nil {
	if !retryable {
	job.State = model.JobStateCancelled
	}
	return tbInfo, errors.Trace(err)
	}

	if job.IsCancelled() {
	defer jobCtx.unlockSchemaVersion(jobCtx, job.ID)
	w.sess.Reset()
	return 0, w.handleJobDone(jobCtx, job)
	}

	err = e.DoDDLJobWrapper(ctx, jobW)
	if err != nil {
	// table exists, but if_not_exists flags is true, so we ignore this error.
	if c.OnExist == OnExistIgnore && infoschema.ErrTableExists.Equal(err) {
	ctx.GetSessionVars().StmtCtx.AppendNote(err)
	err = nil
	}
	return errors.Trace(err)
	}

	for _, tblArgs := range args.Tables {
	if err = e.createTableWithInfoPost(ctx, tblArgs.TableInfo, jobW.SchemaID); err != nil {
	return errors.Trace(err)
	}
	}

Rebase auto increment not executed after second restore #60804

Description

Bug Report

1. Minimal reproduce step (Required)

2. What did you expect to see? (Required)

3. What did you see instead (Required)

4. What is your TiDB version? (Required)

Activity

D3Hunter commented on May 8, 2025

joechenrh commented on Jul 18, 2025

5 remaining items

ti-chi-bot commented on Jul 18, 2025

joechenrh commented on Jul 18, 2025

ti-chi-bot commented on Jul 18, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions