Skip to content

Conversation

@darshdinger
Copy link

Short description of the changes:

Removed the obsolete JSON data format option for publishing plot data. The server now only accepts HTML format data. The data_type field remains in the database model for backward compatibility but is effectively always set to 1 (HTML).

Long description of the changes:

This PR addresses issues with the fragile JSON format detection logic (the <div string check in get_data_type_from_data() method) that caused problems with REF_M data processing.

Changes made:

  • Simplified DATA_TYPES constant from {"json": 0, "html": 1} to {"html": 1}
  • Removed JSON upload endpoint (/update/json/)
  • Removed user data endpoints (/upload_user_data/ and /list/)
  • Removed get_data_type_from_data() and get_data_type_from_string() helper methods
  • Simplified _store() function to always use HTML data type
  • Removed store_user_data() from view utilities
  • Updated tests to remove JSON and user data test cases
  • Cleaned up documentation references to JSON format

The database schema remains unchanged - the data_type field is preserved to avoid migration complexity and maintain backward compatibility with existing data.

Check list for the pull request

  • I have read the [CONTRIBUTING]
  • I have read the [CODE_OF_CONDUCT]
  • I have added tests for my changes
  • I have updated the documentation accordingly

Check list for the reviewer

  • I have read the [CONTRIBUTING]
  • I have verified the proposed changes
  • best software practices
    • all internal functions have an underbar, as is python standard
    • clearly named variables (better to be verbose in variable names)
    • code comments explaining the intent of code blocks
  • All the tests are passing
  • The documentation is up to date
  • code comments added when explaining intent

Manual test for the reviewer

  1. Start the Docker environment: docker compose up -d
  2. Verify the Django container starts successfully
  3. Run the test suite in the pixi environment: pixi shell then pytest
  4. All 9 tests should pass (2 enum tests, 2 expiration tests, 5 post/get tests)
  5. Test uploading HTML plot data:
    curl -X POST http://localhost/plots/TEST_INST/12345/upload_plot_data/ \
      -F "file=@tests/data/reflectivity.html" \
      -F "data_id=test_plot" \
      -H "Authorization: <test_key>"
  6. Verify the old JSON endpoint returns 404:
    curl http://localhost/plots/TEST_INST/12345/update/json/

References

  • Related to REF_M data processing issues caused by fragile <div string detection
  • EWM Work Item: EWM # 14160

@codecov
Copy link

codecov bot commented Dec 23, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 85.15%. Comparing base (efdf308) to head (ba8edb0).

Additional details and impacted files
@@            Coverage Diff             @@
##             next      #57      +/-   ##
==========================================
- Coverage   89.97%   85.15%   -4.82%     
==========================================
  Files          12       12              
  Lines         359      283      -76     
==========================================
- Hits          323      241      -82     
- Misses         36       42       +6     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants