Integrate fiemap support and fixes v2#11
Closed
kakra wants to merge 4 commits into
Closed
Conversation
I think this is a bug so I fix this: It seems that this means to calculate the ideal position of this file just before the last file in the order. To do this correctly, we need to subtract y->size and not y->blocks, as z->start is in bytes and not in blocks.
The hinting currently doesn't handle backup and restore phase differently which can be improved. This patch tells the kernel that the data won't be reused in the copy function, thus it can discard it from the page cache after it was read. Next, it will instead tell the kernel that the accused file and the tmp file both will be needed for reading next. The first call to WILLNEED should be put up further in the call chain to also support reading the allocation map but this would invole tracking DONTNEED differently, so it is not implemented here. In the end, the page caches will be discarded explicitly by telling the kernel that the data is no longer needed. Testing shows that the active page cache during shaking now is released after each file operation and system responsiveness improves during shaking.
This patch adds support for FIEMAP and leaves the current FIBMAP approach in place as a fallback. Instead of trying to guess extents from FIBMAP block maps, it directly uses extents as fragments. The logic is similar as it merges adjecent extents as one fragment. It also detects hole extents and inlined extents and doesn't account those as fragments similar to the FIBMAP approach. It optimizes handling filesystems with tail support (e.g., reiserfs) by not accounting tailed extents as fragments, as the rewrite process probably would tail the last extent again. Similarily it tries to optimize for file systems that support shared extents (xfs, btrfs) by not accounting such extents as fragments. This helps preventing unsharing accused files and thus increasing space usage. I pretend that keeping shared extents is more important than continuous extents, as the operation of deduplicating those extents in the first place is known to fragment those files. However, if the heuristics decide to shake the file, extents still become unshared. I have no idea currently how to optimize for that, thus it is not part of this patch.
Contributor
Author
|
Closed by #17 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This obsoletes PR #8 by removing the merge commits and putting the patches into correct order, rebased on unbrice/master.
This change is