feat: Recusively scan folders [#120]#124
Open
JicLotus wants to merge 4 commits intoVirusTotal:masterfrom
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
#120
Note on permissions: If recursion is disabled, the function behaves like the previous implementation and will only fail if the top-level directory is unreadable. With recursion enabled, permission errors can occur on subdirectories, as WalkDir attempts to access them.
Note on OS compatibility: We are using filepath.Clean and filepath.WalkDir, which are cross-platform aware, so depth calculation using strings.Split(path, string(os.PathSeparator)) should work on Windows/Unix.
This PR refactors NewFileDirReader to use Go’s standard
filepath.WalkDir(here) for directory traversal, and avoid custom recursive implementations. The function efficiently reads all files under a directory, optionally traversing subdirectories up to a configurable maxDepth. Traversal is stopped immediately for directories beyond the allowed depth or when recursion is disabled usingfs.SkipDir, avoiding unnecessary filesystem reads and reducing resource usage. This approach is safe, efficient, and reduces the risk of errors compared to manual recursion, while remaining idiomatic and easy to maintain.Usage:
vt scan file YOUR_DIRECTORY(default recursive disabled + maxDepth == 1)vt scan file YOUR_DIRECTORY -r(recursive enabled with default maxDepth == 1)vt scan file YOUR_DIRECTORY -r -d 3(recursive enabled + setting up maxDepth to read more subtrees until reaching depth == 3 on each subdirectory)Testing
Unit testing: Added comprehensive unit tests covering scenarios with recursion enabled and disabled, as well as multiple depth levels, both with and without subdirectories.
Manual build testing(mac-os). Successfully built+installed the binary and verified normal usage. Note we scanning multiple directories now
make all