chore: improve error logging in ingestion handler

[nikhilsinhaparseable]

add stream name to the error log on ingestion failure

Summary by CodeRabbit

• Bug Fixes
  • Error messages during ingestion now include stream names for clearer troubleshooting.
  • OpenTelemetry log/metrics/trace ingestion now logs parse and validation failures with stream context.
  • OTEL unsupported formats and invalid content types produce clearer, contextual error responses.
  • Rejected OTEL log sources return a descriptive client error with the stream name included.

[coderabbitai]

Walkthrough

Reject OTEL log sources and add pervasive stream-contextual error logging across OTEL ingestion and general ingest paths. Change public error enum variant PostError::OtelNotSupported to carry the stream name (String).

Changes

Cohort / File(s)	Summary
Ingest & OTEL handlers src/handlers/http/ingest.rs	Added stream-contextual logging throughout ingest and OTEL processing (handle_otel_* handlers, post_event, ingest_internal_stream). Converted PostError::OtelNotSupported from a unit variant to OtelNotSupported(String). Wrapped async calls with map_err to log stream-specific failures, added explicit JSON parse error logging, and updated ResponseError mapping for the new variant.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Possibly related PRs

• restrict otel ingestion #1263: Modifies OTEL ingestion error handling and PostError usages in src/handlers/http/ingest.rs.
• Protobuf ingestion #1391: Touches OTEL handlers and content-processing logic in the same ingest paths.
• refactor: specialized flatten happens at the same semantic location #1177: Changes OTEL ingestion control flow and how OTEL sources/errors are handled in ingest.rs.

Suggested labels

for next release

Suggested reviewers

• nitisht

Poem

🐰 I hopped through logs with nose so twitchy,
Now errors call out the stream, bold and pitchy,
OtelNotSupported carries the name,
Helping hunters find where faults became,
A tiny hop for code, a giant debug for me 🥕

🚥 Pre-merge checks | ✅ 1 | ❌ 2

❌ Failed checks (2 warnings)

Check name	Status	Explanation	Resolution
Description check	⚠️ Warning	The description is minimal and does not follow the provided template structure; it lacks detailed explanation of the goal, rationale, key changes, and required testing/documentation checkboxes.	Expand the description to include the goal, rationale for changes, key modifications made, and complete the checklist items (testing, comments, documentation).
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (1 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title 'chore: improve error logging in ingestion handler' accurately reflects the main change of adding stream context to error logs throughout the ingestion path.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing touches

• 📝 Generate docstrings

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 1

🤖 Fix all issues with AI agents

In `@src/handlers/http/ingest.rs`:
- Around line 143-155: The error logging currently prints the full
serde_json::Error (which includes JSON snippets) when flatten_and_push_logs
fails; update ingest_internal_stream and handle_raw_logs_internal so
serde_json::Error is redacted before being logged: either detect/match
PostError::SerdeError (or the inner serde_json::Error returned by
serde_json::from_slice) and log a generic message like "malformed JSON" (and, if
needed, emit the full serde error at debug level only), or wrap
serde_json::Error in a custom error type with a Display impl that returns a
redacted string; apply this handling where flatten_and_push_logs is awaited and
where serde_json::from_slice(&body) is called so no JSON snippets are output by
error!().

[coderabbitai]

Actionable comments posted: 1

🤖 Fix all issues with AI agents

In `@src/handlers/http/ingest.rs`:
- Around line 554-557: The PostError enum variant OtelNotSupported was changed
from a unit variant to OtelNotSupported(String), so update every site that
constructs or matches against OtelNotSupported to provide and bind the stream
name: replace bare OtelNotSupported with
OtelNotSupported(stream_name.to_string() or an appropriate &str->String
conversion) where the error is created, and update match arms like
`OtelNotSupported => ...` to `OtelNotSupported(name) => ...` (or `_` if the name
is unused); ensure functions that return PostError::OtelNotSupported now pass
the stream identifier (use the same identifier used in that context, e.g.,
`stream` or `stream_name`) and adjust any tests/expectations to accept the tuple
variant.

🧹 Nitpick comments (3)

src/handlers/http/ingest.rs (3)

309-321: Consider extracting duplicated error messages.

The error message is duplicated between the error!() macro and the PostError::Invalid anyhow message. This creates maintenance overhead if the message needs to change.

♻️ Optional refactor to reduce duplication

             } else if content_type == CONTENT_TYPE_PROTOBUF {
+                let msg = format!(
+                    "Ingestion failed for stream {stream_name}: Protobuf ingestion is not supported in Parseable OSS"
+                );
-                error!(
-                    "Ingestion failed for stream {stream_name}: Protobuf ingestion is not supported in Parseable OSS"
-                );
-                return Err(PostError::Invalid(anyhow::anyhow!(
-                    "Ingestion failed for stream {stream_name}: Protobuf ingestion is not supported in Parseable OSS"
-                )));
+                error!("{msg}");
+                return Err(PostError::Invalid(anyhow::anyhow!("{msg}")));
             } else {
+                let msg = format!(
+                    "Ingestion failed for stream {stream_name}: Unsupported Content-Type: {content_type}. Expected application/json or application/x-protobuf"
+                );
-                error!(
-                    "Ingestion failed for stream {stream_name}: Unsupported Content-Type: {content_type}. Expected application/json or application/x-protobuf"
-                );
-                return Err(PostError::Invalid(anyhow::anyhow!(
-                    "Ingestion failed for stream {stream_name}: Unsupported Content-Type: {content_type}. Expected application/json or application/x-protobuf"
-                )));
+                error!("{msg}");
+                return Err(PostError::Invalid(anyhow::anyhow!("{msg}")));
             }

---

325-330: Same duplication pattern as above.

Consider applying the same refactoring approach to reduce maintenance overhead.

---

350-354: Stream name unavailable in setup error logs.

When setup_otel_stream fails, the stream name may have already been extracted but isn't included in the error log since it's part of the returned tuple. For failures that occur after stream name extraction (e.g., during create_stream_if_not_exists), consider logging the stream name within setup_otel_stream itself, or restructuring to return partial results on certain failure paths.

♻️ Alternative: Add internal logging to setup_otel_stream

// In setup_otel_stream, after extracting stream_name, wrap operations with error logging:
let stream_name = stream_name.to_str().unwrap().to_owned();

PARSEABLE
    .create_stream_if_not_exists(/* ... */)
    .await
    .map_err(|e| {
        error!("OTEL ingestion setup failed for stream {stream_name}: {e}");
        e
    })?;