You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository was archived by the owner on Apr 2, 2024. It is now read-only.
The LogstreamerInput uses filepath.Walk() for locating files to compare to the file_match regexes.
This is normally fine in a mostly sane directory structure.
In the case of ridiculously large/complex filesystems with numerous levels containing millions of files, this can be somewhat inefficient.
The pattern matching is useful so not suggesting a replacement for regexing.
This change should still follow the same rules for log_directory and file_match.
I propose adding an additional config option to use globbing specifically for the filepath.Walk portion. https://github.com/mozilla-services/heka/blob/dev/logstreamer/filehandling.go#L280
The LogstreamerInput uses
filepath.Walk()for locating files to compare to the file_match regexes.This is normally fine in a mostly sane directory structure.
In the case of ridiculously large/complex filesystems with numerous levels containing millions of files, this can be somewhat inefficient.
The pattern matching is useful so not suggesting a replacement for regexing.
This change should still follow the same rules for log_directory and file_match.
I propose adding an additional config option to use globbing specifically for the
filepath.Walkportion.https://github.com/mozilla-services/heka/blob/dev/logstreamer/filehandling.go#L280
glob_pattern = "/this/*/path*/would/*take/*/*/*/forever/to/*/walk/*/in/a*/reasonable/*/*/*/timeframe/*"