refactor(file): composable Lines iterator; deprecate channel-based ReadFile helpers

[ChrisJr404]

Closes #719.

Supersedes the original ReadFileSplit / ReadFileWithReaderSplit proposal. Per review discussion: rather than add yet another ReadFile* variant (each new option adds a new combinatorial helper), introduce a single composable iterator and reduce the existing helpers to thin deprecated wrappers over it.

What

New file file/lines.go exposes one primitive per source, returning a Go 1.23+ iterator:

func Lines(filename string, opts ...LineOption) iter.Seq2[string, error]
func LinesReader(r io.Reader, opts ...LineOption) iter.Seq2[string, error]

Composable options (applied in order scan → split → trim → skip-empty → filter):

WithBufferSize(n int)            // scanner buffer
WithSplit(separators ...rune)    // split each line on any rune (FieldsFunc semantics)
WithTrimSpace()                  // TrimSpace on each emitted value
WithSkipEmpty()                  // drop empty values (post-trim)
WithFilter(func(string) bool)    // arbitrary predicate, e.g. skip "# comment" lines

The resolver-file use case from #719 / httpx#2351 becomes:

for v, err := range fileutil.Lines(path,
    fileutil.WithSplit(','),
    fileutil.WithTrimSpace(),
    fileutil.WithSkipEmpty(),
) {
    if err != nil { return err }
    // use v
}

Why this shape

• No combinatorial explosion. The old surface was already trending toward ReadFile, ReadFileWithBufferSize, ReadFileSplit, ReadFileSplitWithBufferSize, …WithReaderAnd…, etc. One iterator plus N orthogonal options replaces all of them and any future variants (WithFilter, WithComment, …) compose for free.
• Errors are surfaced. The channel-based helpers silently swallowed os.Open failures inside the goroutine and scanner.Err() entirely. iter.Seq2[string, error] makes both visible: the iterator yields a final ("", err) pair and stops.
• No goroutine leaks. iter.Seq2 cancels naturally on break; the file is closed via defer when iteration ends. The old channel pattern leaked a goroutine forever if the consumer stopped reading early.
• Idiomatic Go 1.24. Mirrors the new stdlib strings.Lines precedent.
• Lazy. The file isn't opened until the first range step.

Deprecation

The four existing exported helpers keep their (chan string, error) signatures (zero source-level breakage) and are marked // Deprecated: with the migration recipe. Their bodies now pump from the new iterator into the channel, so behaviour is byte-for-byte identical and the existing TestReadFile* tests pass unmodified.

Deprecated	Replacement
ReadFile(path)	Lines(path)
ReadFileWithBufferSize(path, n)	Lines(path, WithBufferSize(n))
ReadFileWithReader(r)	LinesReader(r)
ReadFileWithReaderAndBufferSize(r, n)	LinesReader(r, WithBufferSize(n))

The ReadFileSplit / ReadFileWithReaderSplit / splitLineByRunes helpers from the earlier revision of this PR are removed; their functionality is now expressed as Lines(path, WithSplit(','), WithTrimSpace(), WithSkipEmpty()).

Tests

• New file/lines_test.go (14 tests): each option in isolation, the resolver-file end-to-end scenario, missing-file error path, early-break behaviour, scanner-error propagation through LinesReader, and reader-side equivalents.
• The original file/file_split_test.go was removed along with the helpers it covered.
• Existing TestReadFile, TestReadFileWithBufferSize, TestReadFileWithReader, TestReadFileWithReaderAndBufferSize are unchanged and still pass — they validate that the deprecated wrappers preserve exact prior behaviour (including emitting empty lines and not trimming).

$ go vet ./file/...
$ go test ./file/... -count=1
ok  	github.com/projectdiscovery/utils/file	19.1s

Notes for downstream tools

After this lands, httpx / nuclei / subfinder etc. can migrate at their own pace:

1. Replace for line := range fileutil.ReadFile(path) → for line, err := range fileutil.Lines(path).
2. For resolver-file parsing, drop any local "split on comma + trim" helper in favour of Lines(path, WithSplit(','), WithTrimSpace(), WithSkipEmpty()).

No flag-day required; the deprecated channel functions stay until consumers have moved.

[neo-by-projectdiscovery-dev]

Neo - PR Security Review

No security issues found

_{Comment @pdneo help for available commands. · Open in Neo}