Update project dependencies by paulapreuss · Pull Request #432 · open-plan-tool/gui

paulapreuss · 2026-01-15T14:46:20Z

OpenPlan is currently based on python 3.9, which has already reached it's EoL, and Django 4.2, which has its long term support (LTS) phase also ending at the end of the year. Additionally, most project dependencies have not been updated in a long time, posing possible security issues, among other things.

This PR aims to upgrade the project to python 3.12 and Django 5.2 (as a compromise between a jump to all latest versions and moving to versions that have at least two years of long term support ahead). Other project dependencies will also be checked for upgrades.

Within this PR, oemof packages are also upgraded to oemof-solph==0.5.7 and oemof-thermal==0.0.8. Will have to do some testing. Likely these versions were pinned to be in sync with MVS, but since we are moving away from MVS we will need to update anyway. The upgrade was necessary, since the package dependencies were not compatible with python 3.12.

Closes #399

TODO:

Bachibouzouk

I just though yesterday that we were due to update django. Shouldn't we unpin the dependencies ? I would keep a file with pinned dependencies but use the one with unpinned such that we always get the latest updates when we deploy (we could setup a scheduled job to build using the unpinned requirements and be notifyed when it fails). If it fails and it is a friday or one day before a workshop and one has no time, we can fallback on the pinned dependencies and leave it as TODO to fix the dependencies later.

app/requirements/base.txt

Caused from upgrade: `ValueError: Model instances passed to related filters must be saved.`

Only raw html displayed after update

Depends on new environment variable `PROD_ENV` being set to true

github-actions · 2026-03-09T09:54:23Z

✅dependabot config looks good 👍

Should only be used to serve frontend assets and was causing issues with migrations on CI test build

paulapreuss · 2026-03-09T14:28:46Z

Hi @Bachibouzouk, I think this is about ready. What I did:

Upgrade Django to 5.2
Upgrade python to 3.12
Upgrade all other packages, only pinning the ones triggering version conflicts or causing other issues in the code
Differentiate between docker-compose-production and docker-compose-local and update local deployment description
Add build test on CI (builds local-compose-local to check that the requirements are working and there are no migration files are missing)
Add a basic dependabot configuration (we will have to see if this is catching all we are interested in, it is hard to test locally)
Add a github action to automatically tag releases when they get merged into prod
Update the pre-commit config to be quite a bit stricter (now uses ruff and a few other hooks) - this has also caused the changed files to be so huge, since I ran it once on the repo. It should mostly be code formatting, though. Locally, you should update your pre-commit config with pre-commit install.

Let me know what you think. I did change quite a few things so I assume this will take quite a bit of noodling around on staging to make sure that everything really still works as intended.

One thing I have not yet done is move from Django-q to celery beat. I will do this when I have a bit of time again.

Also, I have now included the changes from your other closed PRs to the Changelog here, as they were still missing.

Bachibouzouk · 2026-03-11T09:53:07Z

app/epa/settings.py

+# https://docs.djangoproject.com/en/dev/ref/settings/#csrf-cookie-secure
+CSRF_COOKIE_SECURE = True
+# https://docs.djangoproject.com/en/dev/ref/settings/#secure-hsts-seconds
+# TODO: set this to 60 seconds first and then to 518400 once you prove the former works


NB: make sure this is addressed in a later commit

Bachibouzouk · 2026-03-11T11:58:30Z

app/epa/settings.py

+    # https://docs.djangoproject.com/en/dev/ref/settings/#csrf-cookie-secure
+    CSRF_COOKIE_SECURE = True
+    # https://docs.djangoproject.com/en/dev/ref/settings/#secure-hsts-seconds
+    # TODO: set this to 60 seconds first and then to 518400 once you prove the former works


Shouldn't we set it to 518400 ?

@Bachibouzouk I would keep it as is for now and then gradually increase before moving from staging to production? We can also set it a bit higher already, but I wouldn't go to 518400 right away.

Bachibouzouk

This is so far I got for the moment, testing the production on the sandbox server

Bachibouzouk · 2026-03-12T22:12:18Z

app/dashboard/helpers.py

 from django.templatetags.static import static
 from numbers import Number

+from epa.settings import RESOURCES_DIR


Did you test if the path is accessible in production?

Bachibouzouk · 2026-03-12T23:10:44Z

app/epa/settings.py

    "orm": "default",
 }
+
+if PROD_ENV:


If I set prod env to False I get the sandbox to work, if not I makes and endless ssl redirect and ends up with a ns-error-redirect-loop error

Likely due to https://docs.djangoproject.com/en/dev/ref/settings/#secure-ssl-redirect

When one sets PROD_ENV to True while setting DJANGO_SECURE_SSL_REDIRECT to False, then it works :)

@Bachibouzouk I added SECURE_PROXY_SSL_HEADER to the settings as recommended by the documentation, now the DJANGO_SECURE_SSL_REDIRECT setting also seems to work without endless redirects.

Should fix SECURE_SSL_REDIRECT causing infinite loops

paulapreuss · 2026-03-18T14:01:30Z

@Bachibouzouk I also toyed around a bit on the sandbox server with this and it looks to be working. Are you good with merging it into main/staging?

paulapreuss force-pushed the fix/security branch 2 times, most recently from 1a0275b to f412f35 Compare January 15, 2026 15:05

Bachibouzouk reviewed Feb 13, 2026

View reviewed changes

app/requirements/base.txt Outdated Show resolved Hide resolved

app/requirements/base.txt Outdated Show resolved Hide resolved

paulapreuss added 10 commits March 5, 2026 10:08

Upgrade django

4a1fa19

Add crispy bootstrap requirement

5b2b27d

Fix model instance filter error

fbebaa1

Caused from upgrade: `ValueError: Model instances passed to related filters must be saved.`

Remove deprecated USE_L10N setting

71449c6

Mark label strings as safe before render

f7a8900

Only raw html displayed after update

Pin new package versions

1e0e315

Delete unused pipfile for dependency management

c2a4fa5

Update python version in github actions

26d2b91

Add security settings flagged by manage.py check --deploy

b5c3417

Fix oemof-datapackage branch and pin toml package

ec3ab2b

paulapreuss force-pushed the fix/security branch from 25435f6 to ec3ab2b Compare March 5, 2026 09:17

Enable security settings only when in production environment

fdf4e93

Depends on new environment variable `PROD_ENV` being set to true

paulapreuss force-pushed the fix/security branch 2 times, most recently from 4136233 to b772c7c Compare March 9, 2026 09:43

paulapreuss force-pushed the fix/security branch from c68a703 to 4b8eefb Compare March 9, 2026 13:46

paulapreuss added 11 commits March 9, 2026 15:18

Update pre-commit config

4d0eb41

Update CI workflow

a4160bf

Run pre-commit on all files

e62eb89

Set up dependabot for package upgrades

9117810

Update build context

37bb007

Update pre-commit config location

92cd284

Update TRUSTED_HOST env

2777da3

Add missing migration file

4e0e17b

Move csv resources out of static folder

f93a19d

Should only be used to serve frontend assets and was causing issues with migrations on CI test build

Add tag on prod workflow

1f38d13

Refactor docker-compose files into production and local

c2c729b

paulapreuss added 3 commits March 9, 2026 15:18

Fix docker-compose file reference in deploy script

8830160

Update README

1035ad8

Specify python version in README

5d5a7d8

paulapreuss force-pushed the fix/security branch from 45b935d to 5d5a7d8 Compare March 9, 2026 14:18

paulapreuss added 2 commits March 9, 2026 15:29

Include pre-commit setup in readme

dd27cb4

Update changelog

6ffc1ec

paulapreuss marked this pull request as ready for review March 9, 2026 14:40

Bachibouzouk reviewed Mar 11, 2026

View reviewed changes

Bachibouzouk reviewed Mar 12, 2026

View reviewed changes

Bachibouzouk and others added 2 commits March 13, 2026 00:20

Add gunicorn to prod dependencies

d39b4d2

Add SECURE_PROXY_SSL_HEADER setting

abf27f8

Should fix SECURE_SSL_REDIRECT causing infinite loops

Bachibouzouk approved these changes Mar 18, 2026

View reviewed changes

paulapreuss merged commit 2222a53 into main Mar 19, 2026
2 checks passed

paulapreuss deleted the fix/security branch March 19, 2026 07:52

paulapreuss mentioned this pull request Mar 19, 2026

Fix remaining security warnings #445

Merged

Conversation

paulapreuss commented Jan 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Bachibouzouk left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Mar 9, 2026

Uh oh!

paulapreuss commented Mar 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Bachibouzouk Mar 11, 2026

Choose a reason for hiding this comment

Uh oh!

Bachibouzouk Mar 11, 2026

Choose a reason for hiding this comment

Uh oh!

paulapreuss Mar 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Bachibouzouk left a comment

Choose a reason for hiding this comment

Uh oh!

Bachibouzouk Mar 12, 2026

Choose a reason for hiding this comment

Uh oh!

Bachibouzouk Mar 12, 2026

Choose a reason for hiding this comment

Uh oh!

Bachibouzouk Mar 12, 2026

Choose a reason for hiding this comment

Uh oh!

Bachibouzouk Mar 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

paulapreuss Mar 18, 2026

Choose a reason for hiding this comment

Uh oh!

paulapreuss commented Mar 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

paulapreuss commented Jan 15, 2026 •

edited

Loading

paulapreuss commented Mar 9, 2026 •

edited

Loading

paulapreuss Mar 12, 2026 •

edited

Loading

Bachibouzouk Mar 12, 2026 •

edited

Loading

paulapreuss commented Mar 18, 2026 •

edited

Loading