Skip to content

fix: make TDgpt model downloads fallback#35355

Open
tomchon wants to merge 5 commits into
mainfrom
fix-tdgpt-download-fallback
Open

fix: make TDgpt model downloads fallback#35355
tomchon wants to merge 5 commits into
mainfrom
fix-tdgpt-download-fallback

Conversation

@tomchon
Copy link
Copy Markdown
Contributor

@tomchon tomchon commented May 21, 2026

Summary

  • Add a shared TDgpt Hugging Face download helper with correct boolean parsing and endpoint override support.
  • Fall back from non-official endpoints such as hf-mirror.com to official Hugging Face when the first download attempt fails.
  • Wire model_downloader.py and non-tdtsfm tsfmservice model servers to the helper, with focused tests for endpoint/fallback behavior.

Test Plan

  • python3 -m pytest tools/tdgpt/tests/hf_download_test.py tools/tdgpt/tests/model_downloader_test.py -q
  • python3 -m py_compile tools/tdgpt/taosanalytics/misc/hf_download.py tools/tdgpt/taosanalytics/misc/model_downloader.py tools/tdgpt/taosanalytics/tsfmservice/chronos-server.py tools/tdgpt/taosanalytics/tsfmservice/timemoe-server.py tools/tdgpt/taosanalytics/tsfmservice/timesfm-server.py tools/tdgpt/taosanalytics/tsfmservice/moirai-server.py tools/tdgpt/taosanalytics/tsfmservice/moment-server.py
  • git diff --check

chenhaoran and others added 4 commits May 21, 2026 14:34
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@tomchon tomchon requested a review from guanshengliang as a code owner May 21, 2026 06:35
Copilot AI review requested due to automatic review settings May 21, 2026 06:35
@tomchon tomchon requested a review from zitsen as a code owner May 21, 2026 06:35
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a centralized Hugging Face model download utility with a fallback mechanism that reverts to the official endpoint if a mirror fails. It updates several service scripts and the model downloader CLI to utilize this new utility, while also improving boolean argument parsing and error handling. Review feedback highlights a logic error in is_official_endpoint regarding environment variable precedence and suggests removing endpoint from kwargs to avoid potential TypeError during the download call.

Comment thread tools/tdgpt/taosanalytics/misc/hf_download.py Outdated
Comment thread tools/tdgpt/taosanalytics/misc/hf_download.py Outdated
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR improves reliability of TDgpt model downloads by introducing a shared Hugging Face download helper that supports endpoint overrides and automatic fallback to the official Hugging Face endpoint when a mirror/custom endpoint fails.

Changes:

  • Added hf_download.py helper with robust boolean parsing, endpoint resolution, and mirror→official fallback logic.
  • Updated model_downloader.py and multiple tsfmservice/*-server.py scripts to use the shared helper instead of directly calling huggingface_hub.snapshot_download.
  • Added focused pytest coverage for endpoint resolution/fallback behavior and invalid CLI boolean handling.

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
tools/tdgpt/taosanalytics/misc/hf_download.py New shared helper for endpoint selection and fallback download behavior.
tools/tdgpt/taosanalytics/misc/model_downloader.py Uses shared helper; fixes CLI boolean parsing + exit code.
tools/tdgpt/taosanalytics/tsfmservice/chronos-server.py Switches model download to helper with fallback support.
tools/tdgpt/taosanalytics/tsfmservice/moirai-server.py Switches model download to helper with fallback support.
tools/tdgpt/taosanalytics/tsfmservice/moment-server.py Switches model download to helper with fallback support.
tools/tdgpt/taosanalytics/tsfmservice/timemoe-server.py Switches model download to helper with fallback support.
tools/tdgpt/taosanalytics/tsfmservice/timesfm-server.py Switches model download to helper with fallback support.
tools/tdgpt/tests/hf_download_test.py New unit tests validating endpoint precedence and fallback behavior.
tools/tdgpt/tests/model_downloader_test.py New test ensuring invalid boolean CLI arg exits cleanly.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread tools/tdgpt/taosanalytics/misc/hf_download.py
Comment thread tools/tdgpt/taosanalytics/tsfmservice/timesfm-server.py Outdated
Comment thread tools/tdgpt/taosanalytics/tsfmservice/timemoe-server.py Outdated
Comment thread tools/tdgpt/taosanalytics/tsfmservice/moment-server.py Outdated
Comment thread tools/tdgpt/taosanalytics/tsfmservice/moirai-server.py
Comment thread tools/tdgpt/taosanalytics/tsfmservice/chronos-server.py Outdated
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants