Skip to content

move backoff settings as well as other functions out of utils#1209

Open
AlbertSnows wants to merge 23 commits into
masterfrom
fix/move_backoff_settings_v2
Open

move backoff settings as well as other functions out of utils#1209
AlbertSnows wants to merge 23 commits into
masterfrom
fix/move_backoff_settings_v2

Conversation

@AlbertSnows
Copy link
Copy Markdown
Contributor

@AlbertSnows AlbertSnows commented Jan 10, 2025

Link to JIRA ticket if there is one: https://ctds-planx.atlassian.net/browse/PXP-11392

This PR moves config properties and functions out of utils into config. Utils should be low-level, and thus not reference other namespaces for sake of not inducing circular dependencies.
redesign of #1181

New Features

Breaking Changes

Bug Fixes

Improvements

Dependency updates

  • moved properties and functions out of utils to config or models to resolve cyclic dependencies

Deployment changes

@AlbertSnows AlbertSnows changed the title move backoff settings to config move backoff settings as well as other functions out of utils Jan 10, 2025
@coveralls
Copy link
Copy Markdown

coveralls commented Jan 10, 2025

Pull Request Test Coverage Report for Build 13727398361

Details

  • 0 of 0 changed or added relevant lines in 0 files are covered.
  • 402 unchanged lines in 9 files lost coverage.
  • Overall coverage increased (+0.2%) to 75.427%

Files with Coverage Reduction New Missed Lines %
resources/audit/client.py 11 73.42%
config.py 13 78.46%
resources/openid/ras_oauth2.py 16 58.96%
init.py 20 90.76%
models.py 21 86.68%
utils.py 40 67.59%
resources/google/utils.py 51 77.11%
scripting/google_monitor.py 87 40.5%
scripting/fence_create.py 143 60.5%
Totals Coverage Status
Change from base Build 13573029339: 0.2%
Covered Lines: 7855
Relevant Lines: 10414

💛 - Coveralls

@github-actions
Copy link
Copy Markdown

Please find the ci env pod logs here

@github-actions
Copy link
Copy Markdown

filepath $$\textcolor{#23d18b}{\tt{passed}}$$ $$\textcolor{#f14c4c}{\tt{failed}}$$ $$\textcolor{#ffa500}{\tt{skipped}}$$ SUBTOTAL
$$\textcolor{#23d18b}{\tt{tests/test\_oauth2.py}}$$ $$\textcolor{#23d18b}{\tt{15}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#23d18b}{\tt{15}}$$
$$\textcolor{#23d18b}{\tt{tests/test\_centralized\_auth.py}}$$ $$\textcolor{#23d18b}{\tt{16}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#23d18b}{\tt{16}}$$
$$\textcolor{#ffa500}{\tt{tests/test\_data\_upload.py}}$$ $$\textcolor{#23d18b}{\tt{8}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#ffa500}{\tt{1}}$$ $$\textcolor{#ffa500}{\tt{9}}$$
$$\textcolor{#23d18b}{\tt{tests/test\_presigned\_url.py}}$$ $$\textcolor{#23d18b}{\tt{7}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#23d18b}{\tt{7}}$$
$$\textcolor{#23d18b}{\tt{tests/test\_user\_token.py}}$$ $$\textcolor{#23d18b}{\tt{5}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#23d18b}{\tt{5}}$$
$$\textcolor{#f14c4c}{\tt{tests/test\_dbgap.py}}$$ $$\textcolor{#23d18b}{\tt{3}}$$ $$\textcolor{#f14c4c}{\tt{1}}$$ $$\textcolor{#ffa500}{\tt{1}}$$ $$\textcolor{#f14c4c}{\tt{5}}$$
$$\textcolor{#23d18b}{\tt{tests/test\_drs\_endpoint.py}}$$ $$\textcolor{#23d18b}{\tt{4}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#23d18b}{\tt{4}}$$
$$\textcolor{#23d18b}{\tt{tests/test\_ras\_authn.py}}$$ $$\textcolor{#23d18b}{\tt{3}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#23d18b}{\tt{3}}$$
$$\textcolor{#23d18b}{\tt{tests/test\_oidc\_client.py}}$$ $$\textcolor{#23d18b}{\tt{2}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#23d18b}{\tt{2}}$$
$$\textcolor{#23d18b}{\tt{tests/test\_google\_data\_access.py}}$$ $$\textcolor{#23d18b}{\tt{1}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#23d18b}{\tt{tests/test\_audit\_service.py}}$$ $$\textcolor{#23d18b}{\tt{1}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#23d18b}{\tt{tests/test\_client\_credentials.py}}$$ $$\textcolor{#23d18b}{\tt{1}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#ffa500}{\tt{tests/test\_register\_user.py}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#ffa500}{\tt{2}}$$ $$\textcolor{#ffa500}{\tt{2}}$$
$$\textcolor{#f14c4c}{\tt{TOTAL}}$$ $$\textcolor{#23d18b}{\tt{66}}$$ $$\textcolor{#f14c4c}{\tt{1}}$$ $$\textcolor{#ffa500}{\tt{4}}$$ $$\textcolor{#f14c4c}{\tt{71}}$$

Please find the detailed integration test report here

Please find the ci env pod logs here

@github-actions
Copy link
Copy Markdown

filepath $$\textcolor{#23d18b}{\tt{passed}}$$ $$\textcolor{#f14c4c}{\tt{failed}}$$ $$\textcolor{#ffa500}{\tt{skipped}}$$ SUBTOTAL
$$\textcolor{#23d18b}{\tt{tests/test\_oauth2.py}}$$ $$\textcolor{#23d18b}{\tt{15}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#23d18b}{\tt{15}}$$
$$\textcolor{#23d18b}{\tt{tests/test\_centralized\_auth.py}}$$ $$\textcolor{#23d18b}{\tt{16}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#23d18b}{\tt{16}}$$
$$\textcolor{#f14c4c}{\tt{tests/test\_data\_upload.py}}$$ $$\textcolor{#23d18b}{\tt{7}}$$ $$\textcolor{#f14c4c}{\tt{1}}$$ $$\textcolor{#ffa500}{\tt{1}}$$ $$\textcolor{#f14c4c}{\tt{9}}$$
$$\textcolor{#23d18b}{\tt{tests/test\_presigned\_url.py}}$$ $$\textcolor{#23d18b}{\tt{7}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#23d18b}{\tt{7}}$$
$$\textcolor{#f14c4c}{\tt{tests/test\_dbgap.py}}$$ $$\textcolor{#23d18b}{\tt{3}}$$ $$\textcolor{#f14c4c}{\tt{1}}$$ $$\textcolor{#ffa500}{\tt{1}}$$ $$\textcolor{#f14c4c}{\tt{5}}$$
$$\textcolor{#23d18b}{\tt{tests/test\_user\_token.py}}$$ $$\textcolor{#23d18b}{\tt{5}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#23d18b}{\tt{5}}$$
$$\textcolor{#23d18b}{\tt{tests/test\_drs\_endpoint.py}}$$ $$\textcolor{#23d18b}{\tt{4}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#23d18b}{\tt{4}}$$
$$\textcolor{#23d18b}{\tt{tests/test\_ras\_authn.py}}$$ $$\textcolor{#23d18b}{\tt{3}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#23d18b}{\tt{3}}$$
$$\textcolor{#23d18b}{\tt{tests/test\_oidc\_client.py}}$$ $$\textcolor{#23d18b}{\tt{2}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#23d18b}{\tt{2}}$$
$$\textcolor{#23d18b}{\tt{tests/test\_google\_data\_access.py}}$$ $$\textcolor{#23d18b}{\tt{1}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#23d18b}{\tt{tests/test\_audit\_service.py}}$$ $$\textcolor{#23d18b}{\tt{1}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#23d18b}{\tt{tests/test\_client\_credentials.py}}$$ $$\textcolor{#23d18b}{\tt{1}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#ffa500}{\tt{tests/test\_register\_user.py}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#ffa500}{\tt{2}}$$ $$\textcolor{#ffa500}{\tt{2}}$$
$$\textcolor{#f14c4c}{\tt{TOTAL}}$$ $$\textcolor{#23d18b}{\tt{65}}$$ $$\textcolor{#f14c4c}{\tt{2}}$$ $$\textcolor{#ffa500}{\tt{4}}$$ $$\textcolor{#f14c4c}{\tt{71}}$$

Please find the detailed integration test report here

Please find the ci env pod logs here

@github-actions
Copy link
Copy Markdown

Please find the ci env pod logs here

@piotrsenkow piotrsenkow self-requested a review February 28, 2025 22:06
piotrsenkow
piotrsenkow previously approved these changes Feb 28, 2025
Copy link
Copy Markdown
Contributor

@piotrsenkow piotrsenkow left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Everything looks good to me, nice refactoring!

@github-actions
Copy link
Copy Markdown

filepath $$\textcolor{#23d18b}{\tt{passed}}$$ $$\textcolor{#f14c4c}{\tt{failed}}$$ $$\textcolor{#f14c4c}{\tt{error}}$$ $$\textcolor{#ffa500}{\tt{skipped}}$$ SUBTOTAL
$$\textcolor{#f14c4c}{\tt{tests/test\_centralized\_auth.py}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#f14c4c}{\tt{16}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#f14c4c}{\tt{16}}$$
$$\textcolor{#f14c4c}{\tt{tests/test\_data\_upload.py}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#f14c4c}{\tt{9}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#f14c4c}{\tt{9}}$$
$$\textcolor{#f14c4c}{\tt{tests/test\_presigned\_url.py}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#f14c4c}{\tt{7}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#f14c4c}{\tt{7}}$$
$$\textcolor{#f14c4c}{\tt{tests/test\_dbgap.py}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#f14c4c}{\tt{4}}$$ $$\textcolor{#ffa500}{\tt{1}}$$ $$\textcolor{#f14c4c}{\tt{5}}$$
$$\textcolor{#f14c4c}{\tt{tests/test\_drs\_endpoint.py}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#f14c4c}{\tt{4}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#f14c4c}{\tt{4}}$$
$$\textcolor{#f14c4c}{\tt{tests/test\_google\_data\_access.py}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#f14c4c}{\tt{1}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#f14c4c}{\tt{1}}$$
$$\textcolor{#23d18b}{\tt{tests/test\_oauth2.py}}$$ $$\textcolor{#23d18b}{\tt{15}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#23d18b}{\tt{15}}$$
$$\textcolor{#23d18b}{\tt{tests/test\_user\_token.py}}$$ $$\textcolor{#23d18b}{\tt{5}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#23d18b}{\tt{5}}$$
$$\textcolor{#23d18b}{\tt{tests/test\_ras\_authn.py}}$$ $$\textcolor{#23d18b}{\tt{3}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#23d18b}{\tt{3}}$$
$$\textcolor{#f14c4c}{\tt{tests/test\_oidc\_client.py}}$$ $$\textcolor{#23d18b}{\tt{1}}$$ $$\textcolor{#f14c4c}{\tt{1}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#f14c4c}{\tt{2}}$$
$$\textcolor{#ffa500}{\tt{tests/test\_register\_user.py}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#ffa500}{\tt{2}}$$ $$\textcolor{#ffa500}{\tt{2}}$$
$$\textcolor{#f14c4c}{\tt{tests/test\_audit\_service.py}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#f14c4c}{\tt{1}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#f14c4c}{\tt{1}}$$
$$\textcolor{#f14c4c}{\tt{tests/test\_client\_credentials.py}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#f14c4c}{\tt{1}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#f14c4c}{\tt{1}}$$
$$\textcolor{#f14c4c}{\tt{TOTAL}}$$ $$\textcolor{#23d18b}{\tt{24}}$$ $$\textcolor{#f14c4c}{\tt{3}}$$ $$\textcolor{#f14c4c}{\tt{41}}$$ $$\textcolor{#ffa500}{\tt{3}}$$ $$\textcolor{#f14c4c}{\tt{71}}$$

Please find the detailed integration test report here

Please find the ci env pod logs here

Comment thread fence/config.py Outdated
@github-actions
Copy link
Copy Markdown

filepath $$\textcolor{#23d18b}{\tt{passed}}$$ $$\textcolor{#f14c4c}{\tt{failed}}$$ $$\textcolor{#ffa500}{\tt{skipped}}$$ SUBTOTAL
$$\textcolor{#23d18b}{\tt{tests/test\_oauth2.py}}$$ $$\textcolor{#23d18b}{\tt{15}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#23d18b}{\tt{15}}$$
$$\textcolor{#23d18b}{\tt{tests/test\_centralized\_auth.py}}$$ $$\textcolor{#23d18b}{\tt{16}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#23d18b}{\tt{16}}$$
$$\textcolor{#ffa500}{\tt{tests/test\_data\_upload.py}}$$ $$\textcolor{#23d18b}{\tt{8}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#ffa500}{\tt{1}}$$ $$\textcolor{#ffa500}{\tt{9}}$$
$$\textcolor{#23d18b}{\tt{tests/test\_presigned\_url.py}}$$ $$\textcolor{#23d18b}{\tt{7}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#23d18b}{\tt{7}}$$
$$\textcolor{#23d18b}{\tt{tests/test\_user\_token.py}}$$ $$\textcolor{#23d18b}{\tt{5}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#23d18b}{\tt{5}}$$
$$\textcolor{#ffa500}{\tt{tests/test\_dbgap.py}}$$ $$\textcolor{#23d18b}{\tt{4}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#ffa500}{\tt{1}}$$ $$\textcolor{#ffa500}{\tt{5}}$$
$$\textcolor{#23d18b}{\tt{tests/test\_drs\_endpoint.py}}$$ $$\textcolor{#23d18b}{\tt{4}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#23d18b}{\tt{4}}$$
$$\textcolor{#23d18b}{\tt{tests/test\_ras\_authn.py}}$$ $$\textcolor{#23d18b}{\tt{3}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#23d18b}{\tt{3}}$$
$$\textcolor{#23d18b}{\tt{tests/test\_oidc\_client.py}}$$ $$\textcolor{#23d18b}{\tt{2}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#23d18b}{\tt{2}}$$
$$\textcolor{#23d18b}{\tt{tests/test\_google\_data\_access.py}}$$ $$\textcolor{#23d18b}{\tt{1}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#23d18b}{\tt{tests/test\_client\_credentials.py}}$$ $$\textcolor{#23d18b}{\tt{1}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#ffa500}{\tt{tests/test\_register\_user.py}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#ffa500}{\tt{2}}$$ $$\textcolor{#ffa500}{\tt{2}}$$
$$\textcolor{#f14c4c}{\tt{tests/test\_audit\_service.py}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#f14c4c}{\tt{1}}$$ $$\textcolor{#666666}{\tt{0}}$$ $$\textcolor{#f14c4c}{\tt{1}}$$
$$\textcolor{#f14c4c}{\tt{TOTAL}}$$ $$\textcolor{#23d18b}{\tt{66}}$$ $$\textcolor{#f14c4c}{\tt{1}}$$ $$\textcolor{#ffa500}{\tt{4}}$$ $$\textcolor{#f14c4c}{\tt{71}}$$

Please find the detailed integration test report here

Please find the ci env pod logs here

Comment thread fence/scripting/google_monitor.py Outdated
@github-actions
Copy link
Copy Markdown

github-actions Bot commented Mar 7, 2025

Please find the ci env pod logs here

@AlbertSnows
Copy link
Copy Markdown
Contributor Author

PR blocked until ongoing audit issue is resolved

@github-actions
Copy link
Copy Markdown

github-actions Bot commented Mar 7, 2025

filepath $$\textcolor{#666666}{\tt{SUBTOTAL}}$$
$$\textcolor{#23d18b}{\tt{TOTAL}}$$ $$\textcolor{#666666}{\tt{0}}$$

Please find the detailed integration test report here

Please find the ci env pod logs here

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants