Skip to content

Fix pg_dump for hash partitioning on enum columns #1176

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 5 commits into
base: main
Choose a base branch
from

Conversation

ZTE-EBASE
Copy link

Fixes #ENUM_HASH_PARTITION_ISSUE
Fixes #PARALLEL_RESTORE_DEADLOCK

What does this PR do?

  1. Automatic partition handling: Auto-enables --load-via-partition-root when dumping partitioned tables with hash partitioning on enum types to prevent partition constraint violations after dump/reload cycles
  2. Parallel restore safety: Modifies pg_restore to skip TRUNCATE operations when partition root loading is detected, eliminating deadlocks and data loss risks
  3. Dump metadata enhancement: Adds special comment markers for TABLE DATA items using partition root loading mode
  4. Documentation update: Removes obsolete warning about --load-via-partition-root with parallel restore from pg_dump documentation

Type of Change

  • Bug fix (non-breaking change)
  • New feature (non-breaking change)

Breaking Changes

None

Test Plan

Impact

Performance:

User-facing changes:

No significant change in dump performance for partitioned tables
Parallel restore stability improves overall throughput by eliminating deadlock points

Dependencies:

Users no longer need to manually specify --load-via-partition-root for enum hash partitions
Eliminates data corruption risks during pg_upgrade of partitioned tables
Provides clearer dump file annotation structure

Checklist

Additional Context

Compatibility note​​:
For existing dumps using --inserts + --load-via-partition-root:

Parallel restore might still trigger deadlocks (legacy file limitation)
Mitigation: Use single-threaded restore or regenerate dump files

CI Skip Instructions


@my-ship-it my-ship-it requested a review from yjhjstz June 20, 2025 02:54
@ZTE-EBASE ZTE-EBASE closed this Jul 1, 2025
@ZTE-EBASE ZTE-EBASE deleted the fix_pg_dump branch July 1, 2025 01:19
@ZTE-EBASE ZTE-EBASE restored the fix_pg_dump branch July 1, 2025 01:40
@ZTE-EBASE
Copy link
Author

Automatic partition handling: Auto-enables --load-via-partition-root when dumping partitioned tables with hash partitioning on enum types to prevent partition constraint violations after dump/reload cycles
Parallel restore safety: Modifies pg_restore to skip TRUNCATE operations when partition root loading is detected, eliminating deadlocks and data loss risks
Dump metadata enhancement: Adds special comment markers for TABLE DATA items using partition root loading mode
Documentation update: Removes obsolete warning about --load-via-partition-root with parallel restore from pg_dump documentation

@ZTE-EBASE ZTE-EBASE reopened this Jul 1, 2025
@@ -341,6 +341,7 @@ typedef struct _tableInfo
bool dummy_view; /* view's real definition must be postponed */
bool postponed_def; /* matview must be postponed into post-data */
bool ispartition; /* is table a partition? */
bool unsafe_partitions; /* is it an unsafe partitioned table? */
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

align please

@@ -2808,6 +2847,7 @@ dumpTableData(Archive *fout, const TableDataInfo *tdinfo)
.owner = tbinfo->rolname,
.description = "TABLE DATA",
.section = SECTION_DATA,
.createStmt = tdDefn,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you add some test to ~src/bin/pg_dump ?

@tuhaihe tuhaihe self-requested a review July 1, 2025 07:26

parentTbinfo = getRootTableInfo(tbinfo);
copyFrom = fmtQualifiedDumpable(parentTbinfo);
printfPQExpBuffer(copyBuf, "-- load via partition root %s",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not --load-via-partition-root ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants