-
Notifications
You must be signed in to change notification settings - Fork 0
feat: add HTTP transport layer with logging and OpenTelemetry support #16
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
9 commits
Select commit
Hold shift + click to select a range
d9545b9
Add HTTP transport layer with logging and OpenTelemetry support
musale bb9a039
Enhance GavaConnect SDK with HTTP logging and telemetry features
musale 1a9a697
Fix type hint for keyword arguments and ensure float return in jitter…
musale b62dc40
Make this line readable
musale 348eabe
Catch specific exceptions
musale a58d018
Refactor HTTP transport tests for improved readability and consistenc…
musale 9f5b82a
Refactor test_log_request_with_authorization_header for improved read…
musale d3be993
Refactor request method signature to improve type hinting for keyword…
musale f26720e
Add security comment to jitter function to suppress security warnings
musale File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
Binary file not shown.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,13 @@ | ||
| """HTTP transport layer for the GavaConnect SDK.""" | ||
|
|
||
| from .logging import log_request, log_response | ||
| from .telemetry import otel_request_span, otel_response_span | ||
| from .transport import AsyncTransport | ||
|
|
||
| __all__ = [ | ||
| "log_request", | ||
| "log_response", | ||
| "otel_request_span", | ||
| "otel_response_span", | ||
| "AsyncTransport", | ||
| ] |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,35 @@ | ||
| """HTTP request and response logging utilities.""" | ||
|
|
||
| import logging | ||
| import time | ||
|
|
||
| import httpx | ||
|
|
||
| logger = logging.getLogger("gavaconnect") | ||
|
|
||
|
|
||
| async def log_request(req: httpx.Request) -> None: | ||
| """Log an HTTP request with sanitized headers. | ||
|
|
||
| Args: | ||
| req: The HTTP request to log. | ||
|
|
||
| """ | ||
| req.extensions["start_time"] = time.perf_counter() | ||
| hdrs = dict(req.headers) | ||
| hdrs.pop("authorization", None) | ||
| logger.debug(f"HTTP {req.method} {req.url} headers={hdrs}") | ||
|
|
||
|
|
||
| async def log_response(req: httpx.Request, resp: httpx.Response) -> None: | ||
| """Log an HTTP response with timing information. | ||
|
|
||
| Args: | ||
| req: The HTTP request. | ||
| resp: The HTTP response to log. | ||
|
|
||
| """ | ||
| dur = time.perf_counter() - req.extensions.get("start_time", time.perf_counter()) | ||
| logger.info( | ||
| f"HTTP {req.method} {req.url} -> {resp.status_code} in {dur:.3f}s request_id={resp.headers.get('x-request-id')}" | ||
| ) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,36 @@ | ||
| """OpenTelemetry tracing utilities for HTTP requests.""" | ||
|
|
||
| import httpx | ||
| from opentelemetry import trace | ||
|
|
||
| tracer = trace.get_tracer("gavaconnect") | ||
|
|
||
|
|
||
| async def otel_request_span(req: httpx.Request) -> None: | ||
| """Start an OpenTelemetry span for an HTTP request. | ||
|
|
||
| Args: | ||
| req: The HTTP request to trace. | ||
|
|
||
| """ | ||
| span = tracer.start_span( | ||
| "http.client", attributes={"http.method": req.method, "http.url": str(req.url)} | ||
| ) | ||
| req.extensions["otel_span"] = span | ||
|
|
||
|
|
||
| async def otel_response_span(req: httpx.Request, resp: httpx.Response) -> None: | ||
| """Complete an OpenTelemetry span for an HTTP response. | ||
|
|
||
| Args: | ||
| req: The HTTP request. | ||
| resp: The HTTP response. | ||
|
|
||
| """ | ||
| span = req.extensions.pop("otel_span", None) | ||
| if span: | ||
| span.set_attribute("http.status_code", resp.status_code) | ||
| rid = resp.headers.get("x-request-id") | ||
| if rid: | ||
| span.set_attribute("http.response.request_id", rid) | ||
| span.end() |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,138 @@ | ||
| """HTTP transport implementation with retry logic and error handling.""" | ||
|
|
||
| from __future__ import annotations | ||
|
|
||
| import asyncio | ||
| import json | ||
| import random | ||
| from typing import Any | ||
|
|
||
| import httpx | ||
|
|
||
| from gavaconnect.auth import AuthPolicy | ||
| from gavaconnect.config import SDKConfig | ||
| from gavaconnect.errors import APIError, RateLimitError, TransportError | ||
|
|
||
|
|
||
| def _jitter(base: float, attempt: int) -> float: | ||
| return float(base * (2 ** (attempt - 1)) * (1 + random.random() * 0.2)) # nosec B311 | ||
|
|
||
|
|
||
| class AsyncTransport: | ||
| """Async HTTP transport with retry logic and authentication support.""" | ||
|
|
||
| def __init__(self, cfg: SDKConfig) -> None: | ||
| """Initialize the async transport. | ||
|
|
||
| Args: | ||
| cfg: SDK configuration containing timeout and retry settings. | ||
|
|
||
| """ | ||
| self.cfg = cfg | ||
| self.client = httpx.AsyncClient( | ||
| base_url=cfg.base_url, | ||
| http2=True, | ||
| timeout=httpx.Timeout( | ||
| cfg.total_timeout_s, | ||
| read=cfg.read_timeout_s, | ||
| connect=cfg.connect_timeout_s, | ||
| ), | ||
| headers={"user-agent": cfg.user_agent, "x-client-version": cfg.user_agent}, | ||
| ) | ||
|
|
||
| async def close(self) -> None: | ||
| """Close the underlying HTTP client.""" | ||
| await self.client.aclose() | ||
|
|
||
| async def request( | ||
| self, | ||
| method: str, | ||
| url: str, | ||
| *, | ||
| auth: AuthPolicy | None = None, | ||
| **kw: Any, # noqa: ANN401 | ||
| ) -> httpx.Response: | ||
| """Make an HTTP request with retry logic and authentication. | ||
|
|
||
| Args: | ||
| method: HTTP method (GET, POST, etc.). | ||
| url: Request URL. | ||
| auth: Optional authentication policy. | ||
| **kw: Additional keyword arguments for the request. | ||
|
|
||
| Returns: | ||
| The HTTP response. | ||
|
|
||
| Raises: | ||
| TransportError: If the request fails after all retries. | ||
|
|
||
| """ | ||
| req = self.client.build_request(method, url, **kw) | ||
| if auth: | ||
| await auth.authorize(req) | ||
| attempt = 1 | ||
| while True: | ||
| try: | ||
| resp = await self.client.send(req, stream=False) | ||
| except httpx.HTTPError as e: | ||
| if attempt > self.cfg.retry.max_attempts: | ||
| raise TransportError(str(e)) from e | ||
| await asyncio.sleep(_jitter(self.cfg.retry.base_backoff_s, attempt)) | ||
| attempt += 1 | ||
| continue | ||
| if resp.status_code == 401 and auth and await auth.on_unauthorized(): | ||
| req = self.client.build_request(method, url, **kw) | ||
| await auth.authorize(req) | ||
| resp = await self.client.send(req, stream=False) | ||
| if ( | ||
| resp.status_code in self.cfg.retry.retry_on_status | ||
| and attempt <= self.cfg.retry.max_attempts | ||
| ): | ||
| ra = resp.headers.get("retry-after") | ||
| backoff = ( | ||
| float(ra) | ||
| if ra and ra.isdigit() | ||
| else _jitter(self.cfg.retry.base_backoff_s, attempt) | ||
| ) | ||
| await asyncio.sleep(backoff) | ||
| attempt += 1 | ||
| continue | ||
| return resp | ||
|
|
||
| @staticmethod | ||
| def raise_for_api_error(resp: httpx.Response) -> None: | ||
| """Raise appropriate API error based on response status and content. | ||
|
|
||
| Args: | ||
| resp: HTTP response to check for errors. | ||
|
|
||
| Raises: | ||
| APIError: For general API errors. | ||
| RateLimitError: For rate limit errors (status 429). | ||
|
|
||
| """ | ||
| if resp.status_code < 400: | ||
| return | ||
| try: | ||
| b = resp.json() | ||
| err = b.get("error", {}) | ||
| except (json.JSONDecodeError, ValueError) as e: | ||
| raise APIError( | ||
| resp.status_code, | ||
| "api_error", | ||
| resp.text, | ||
| None, | ||
| resp.headers.get("x-request-id"), | ||
| None, | ||
| resp.content, | ||
| ) from e | ||
| type_ = err.get("type") or "api_error" | ||
| msg = err.get("message") or resp.text | ||
| code = err.get("code") | ||
| rid = resp.headers.get("x-request-id") | ||
| ra = err.get("retry_after") | ||
| if resp.status_code == 429: | ||
| raise RateLimitError( | ||
| resp.status_code, type_, msg, code, rid, ra, resp.content | ||
| ) | ||
| raise APIError(resp.status_code, type_, msg, code, rid, ra, resp.content) | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.