Skip to content

Enhance backup script with pigz support #9

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Cristy94 wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Enhance backup script with pigz support #9

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
6b6b093
Enhance backup script with pigz support
Cristy94 Nov 5, 2025
ad90a9b
Fix pigz decompression, and suppress file-changed warnings while comp…
Cristy94 Nov 8, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 27 additions & 7 deletions migrate.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -84,12 +84,24 @@ if [ ! -f "$backupFileName" ]; then
echo "🚸 Docker not stopped, continuing with the backup"
fi

# Choose compressor
if command -v pigz >/dev/null 2>&1; then
echo "✅ Using pigz for parallel gzip"
compressor="pigz -p$(nproc)"
Copy link

Copilot AI Nov 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nitpick] The -p$(nproc) flag uses all available CPU cores, which could overwhelm the system during migration. Consider limiting the number of threads (e.g., pigz -p$(($(nproc) - 1))) to leave at least one core available for other processes, or make it configurable.

Copilot uses AI. Check for mistakes.
Copy link

Copilot AI Nov 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The -p$(nproc) option may fail on systems where nproc is not available (e.g., some BSD systems or minimal environments). Consider adding a fallback:

compressor="pigz -p${NPROC:-$(nproc 2>/dev/null || echo 4)}"

Or simply use pigz without the -p flag, as it will automatically use all available cores by default.

Suggested change
compressor="pigz -p$(nproc)"
compressor="pigz -p${NPROC:-$(nproc 2>/dev/null || echo 4)}"

Copilot uses AI. Check for mistakes.
else
echo "ℹ️ pigz not found, using gzip"
compressor="gzip"
fi

# shellcheck disable=SC2086
if ! tar --exclude='*.sock' -Pczf $backupFileName -C / $backupSourceDir $HOME/.ssh/authorized_keys $volumePaths; then
tar --exclude='*.sock' --warning=no-file-changed -I "$compressor" -Pcf "$backupFileName" \
-C / $backupSourceDir $HOME/.ssh/authorized_keys $volumePaths
rc=$?
Copy link

Copilot AI Dec 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The tar exit code handling allows exit code 1 to pass (which indicates files changed during archival). While this is likely intentional given the --warning=no-file-changed flag, the condition should be "if [ $rc -ne 0 ] && [ $rc -ne 1 ]; then" or "if [ $rc -gt 1 ]; then" to be explicit about which codes are acceptable. The current implementation correctly uses -gt 1, but a comment explaining that exit code 1 is acceptable (files changed during read) would improve clarity.

Suggested change
rc=$?
rc=$?
# tar exit code 0 = success, 1 = files changed during read (acceptable with --warning=no-file-changed)

Copilot uses AI. Check for mistakes.
if [ $rc -gt 1 ]; then
echo "❌ Backup file creation failed"
exit 1
fi
echo "✅ Backup file created"
echo "✅ Backup file created (with change warnings suppressed)"
else
echo "🚸 Backup file already exists, skipping creation"
fi
Expand Down Expand Up @@ -156,12 +168,20 @@ remoteCommands="
echo '🚸 Saving existing authorized keys...';
cp ~/.ssh/authorized_keys ~/.ssh/authorized_keys_backup;

echo '🚸 Extracting backup file...';
if ! tar -Pxzf - -C /; then
echo '❌ Backup file extraction failed';
exit 1;
echo '🚸 Extracting backup file...'
Copy link

Copilot AI Nov 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing semicolon at the end of the statement on line 171. The script's existing style consistently uses semicolons. Add one for consistency:

echo '🚸 Extracting backup file...';
Suggested change
echo '🚸 Extracting backup file...'
echo '🚸 Extracting backup file...';

Copilot uses AI. Check for mistakes.
if command -v pigz >/dev/null 2>&1; then
echo '✅ Using pigz for parallel decompression'
Copy link

Copilot AI Nov 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing semicolon at the end of the statement on line 173. The script's existing style consistently uses semicolons. Add one for consistency:

echo '✅ Using pigz for parallel decompression';
Suggested change
echo '✅ Using pigz for parallel decompression'
echo '✅ Using pigz for parallel decompression';

Copilot uses AI. Check for mistakes.
if ! tar -I pigz -Pxf - -C /; then
Copy link

Copilot AI Dec 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The pigz compression command includes the number of processors with -p$(nproc), but during decompression, pigz is used without the -p flag. While pigz will auto-detect the number of cores by default, it would be more consistent and explicit to also specify the thread count during decompression, especially since it was explicitly set during compression.

Suggested change
if ! tar -I pigz -Pxf - -C /; then
if ! tar -I "pigz -p\$(nproc)" -Pxf - -C /; then

Copilot uses AI. Check for mistakes.
echo '❌ Backup file extraction failed'
exit 1
fi
else
if ! tar -Pzxf - -C /; then
echo '❌ Backup file extraction failed'
exit 1
Comment on lines +180 to +181
Copy link

Copilot AI Nov 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing semicolons at the end of statements. The script's existing style (visible in the surrounding remote commands) consistently uses semicolons. Add them for consistency:

else
  if ! tar -Pzxf - -C /; then
    echo '❌ Backup file extraction failed';
    exit 1;
  fi
fi
Suggested change
echo '❌ Backup file extraction failed'
exit 1
echo '❌ Backup file extraction failed';
exit 1;

Copilot uses AI. Check for mistakes.
Comment on lines +175 to +181
Copy link

Copilot AI Nov 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing semicolons at the end of statements on lines 175, 176, and 181. While bash doesn't strictly require semicolons before fi, the script's existing style (visible in the surrounding remote commands) consistently uses semicolons. This inconsistency could lead to maintainability issues.

Add semicolons for consistency:

if ! tar -I pigz -Pxf - -C /; then
  echo '❌ Backup file extraction failed';
  exit 1;
fi
Suggested change
echo '❌ Backup file extraction failed'
exit 1
fi
else
if ! tar -Pzxf - -C /; then
echo '❌ Backup file extraction failed'
exit 1
echo '❌ Backup file extraction failed';
exit 1;
fi
else
if ! tar -Pzxf - -C /; then
echo '❌ Backup file extraction failed';
exit 1;

Copilot uses AI. Check for mistakes.
fi
fi
Comment on lines +172 to 183
Copy link

Copilot AI Dec 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The extraction logic duplicates error handling code across both branches (pigz and gzip). The conditional could be simplified by determining the decompressor flags first, then calling tar once. This would reduce code duplication and make the logic easier to maintain.

Suggested change
if command -v pigz >/dev/null 2>&1; then
echo '✅ Using pigz for parallel decompression'
if ! tar -I pigz -Pxf - -C /; then
echo '❌ Backup file extraction failed'
exit 1
fi
else
if ! tar -Pzxf - -C /; then
echo '❌ Backup file extraction failed'
exit 1
fi
fi
tar_opts="-Pzxf"
if command -v pigz >/dev/null 2>&1; then
echo '✅ Using pigz for parallel decompression'
tar_opts="-I pigz -Pxf"
fi
if ! tar \$tar_opts - -C /; then
echo '❌ Backup file extraction failed'
exit 1
fi

Copilot uses AI. Check for mistakes.
echo '✅ Backup file extracted';
echo '✅ Backup file extracted'
Copy link

Copilot AI Nov 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing semicolon at the end of the statement. The script's existing style consistently uses semicolons. Add one for consistency:

echo '✅ Backup file extracted';
Suggested change
echo '✅ Backup file extracted'
echo '✅ Backup file extracted';

Copilot uses AI. Check for mistakes.

echo '🚸 Merging authorized keys...';
cat ~/.ssh/authorized_keys_backup ~/.ssh/authorized_keys | sort | uniq > ~/.ssh/authorized_keys_temp;
Expand Down