Problem
The downloader makes HTTP requests to the ERCOT document list API and download endpoint with no retry logic. A transient 5xx response, TCP reset, or DNS hiccup causes the entire run to fail. Since the downloader is invoked by cron, a single transient failure means that day's files are not downloaded until the next scheduled run.
Expected behavior
Wrap ERCOT HTTP calls with exponential backoff retry (e.g., 3 attempts, starting at 1s). Only persist failure to esiid_documents after all retries are exhausted. Use a crate like backoff or tokio-retry. Network errors and 5xx responses should be retried; 4xx responses (except 429) should fail immediately.
Problem
The downloader makes HTTP requests to the ERCOT document list API and download endpoint with no retry logic. A transient 5xx response, TCP reset, or DNS hiccup causes the entire run to fail. Since the downloader is invoked by cron, a single transient failure means that day's files are not downloaded until the next scheduled run.
Expected behavior
Wrap ERCOT HTTP calls with exponential backoff retry (e.g., 3 attempts, starting at 1s). Only persist failure to esiid_documents after all retries are exhausted. Use a crate like backoff or tokio-retry. Network errors and 5xx responses should be retried; 4xx responses (except 429) should fail immediately.