Memory leak when calling WithSubstitution multiple times

[Anton-Pronkin]

Description

When WithSubstitution is called multiple times on the same IConfigurationBuilder, it causes recursive accumulation of configuration sources, leading to excessive memory usage.

Root Cause

In SubstitutingConfigurationSource.Build(), when building valueRoot, the code iterates through ALL sources in builder.Sources except itself and creates a new ConfigurationBuilder with copies of those sources. Each subsequent call to WithSubstitution adds a new SubstitutingConfigurationSource that, when built, will copy all previously added sources again.

Demo

To see allocated memory depending on the number of added WithSubstitution calls, you can try:

const int iterations = 5;

// Emulate configurations
var configurations = Enumerable.Range(0, 500)
    .Select(key => new KeyValuePair<string, string>(key.ToString(), null));

// Emulate multiple sources
var configuration = new ConfigurationBuilder();
for (var i = 0; i < iterations; i++)
{
    configuration.WithSubstitution(c => c.AddInMemoryCollection(configurations));
}

configuration.Build();

// Check memory usage
var currentProcess = Process.GetCurrentProcess();
var usageInMb = currentProcess.WorkingSet64  / 1024 / 1024;

On my machine, the results were:

Sources	Memory usage in MB
2	48
3	49
4	52
5	61
6	139
7	661
8	4947
9	> 32 GB, OOM

In real projects, there could be many sources, and this behavior may cause memory problems.

Suggestions

1. First of all, I would recommend adding a notification in the README to use a single WithSubstitution if possible.

Important

Avoid the following pattern to prevent memory leaks:

builder
    .WithSubstitution(c => c.AddJsonFile("appsettings1.json"))
    .WithSubstitution(c => c.AddJsonFile("appsettings2.json"))
    .WithSubstitution(c => c.AddJsonFile("appsettings3.json"));

Use a single WithSubstitution with multiple sources instead:

builder.WithSubstitution(c => 
{
    c.AddJsonFile("appsettings1.json");
    c.AddJsonFile("appsettings2.json");
    c.AddJsonFile("appsettings3.json");
});

Also, I would suggest to add configurable option to prevent multiple registrations:

public class SubstitutingSettings
{
    /// <summary>
    /// Use single substituting source to prevent multiple registrations, which can cause memory leaks.
    /// Default value: false
    /// </summary>
    public bool SingleSubstitutingSource { get; set; } = false;
}

internal class SubstitutingConfigurationSource : IConfigurationSource
{
    private readonly SubstitutingSettings _settings;

    public SubstitutingConfigurationSource(..., SubstitutingSettings settings)
    {
        _settings = settings;
    }

    public IConfigurationProvider Build(IConfigurationBuilder builder)
    {
        // ...

        var valueBuilder = new ConfigurationBuilder();
        for (var i = 0; i < builder.Sources.Count; i++)
        {
            var source = builder.Sources[i];
            if (ReferenceEquals(source, this))
            {
                continue;
            }

            // Check if SingleSubstitutingSource is enabled
            if (_settings.SingleSubstitutingSource && source is SubstitutingConfigurationSource)
            {
                throw new InvalidOperationException("Multiple substitution sources were detected when SingleSubstitutingSource setting is enabled.");
            }

            valueBuilder.Add(source);
        }

        // ...
    }
}

Usage:

var configuration = new ConfigurationBuilder()
    .WithSubstitution(
        c => ...,
        new SubstitutingSettings { SingleSubstitutingSource = true }
    )
    .WithSubstitution(
        c => ...,
        new SubstitutingSettings { SingleSubstitutingSource = true }
    )
    .Build();

SingleSubstitutingSource could be set to true by default to prevent such memory leaks, but this might break existing behavior. A safer approach is to keep the default as false and recommend enabling it in new projects, or introduce this change in a major version update.

Discussion

What do you think? Are there more elegant solutions – for example, adding a CycleDetector at build time to prevent recursive accumulation?

I'm open to suggestions. Would a pull request be welcome?