Skip to content

Conversation

@devsjc
Copy link
Contributor

@devsjc devsjc commented Jan 19, 2026

Contribution Checklist

  • Have you followed the Open Climate Fix Contribution Guidelines?
  • Have you referenced the Issue this PR addresses, where applicable?
  • Have you checked to ensure there aren't other open Pull Requests for the same change?
  • Have you added a summary of the changes?
  • Have you written new tests for your changes, where applicable?
  • Have you successfully run make lint with your changes locally?
  • Have you successfully run make test with your changes locally?

Warning

PRs may be closed if all the above boxes are not checked.

Changes in this Pull Request

A number of speedups/optimizations:

  • Updates the forecast table index and LatestForecasts query such that they are used correctly

@github-actions
Copy link

github-actions bot commented Jan 19, 2026

Benchmark Results

Benchmark results
?   	github.com/openclimatefix/data-platform/cmd	[no test files]
?   	github.com/openclimatefix/data-platform/internal/gen/ocf/dp	[no test files]
?   	github.com/openclimatefix/data-platform/internal/interceptors	[no test files]
PASS
ok  	github.com/openclimatefix/data-platform/internal/server/dummy	0.005s
{"level":"debug","time":"2026-01-19T16:45:06Z","message":"Running migrations"}
goos: linux
goarch: amd64
pkg: github.com/openclimatefix/data-platform/internal/server/postgres
cpu: AMD EPYC 7763 64-Core Processor                
BenchmarkPostgresClient/6144000/GetForecastAsTimeseries-4         	     172	   6400149 ns/op
BenchmarkPostgresClient/6144000/GetForecastAtTimestamp-4          	     108	  10788355 ns/op
BenchmarkPostgresClient/6144000/GetObservationsAsTimeseries-4     	     962	   1197873 ns/op
BenchmarkPostgresClient/6144000/CreateForecast-4                  	     276	   4079176 ns/op
PASS
ok  	github.com/openclimatefix/data-platform/internal/server/postgres	184.830s
?   	github.com/openclimatefix/data-platform/internal/server/postgres/gen	[no test files]
Benchmark vs base branch
goos: linux
goarch: amd64
pkg: github.com/openclimatefix/data-platform/internal/server/postgres
cpu: AMD EPYC 7763 64-Core Processor                
                                                     │ bench-main.txt │ bench-devsjc-forecast-partitioning.txt │
                                                     │     sec/op     │    sec/op      vs base                 │
PostgresClient/6144000/GetForecastAsTimeseries-4         4.826m ± ∞ ¹    6.400m ± ∞ ¹        ~ (p=1.000 n=1) ²
PostgresClient/6144000/GetForecastAtTimestamp-4         105.00m ± ∞ ¹    10.79m ± ∞ ¹        ~ (p=1.000 n=1) ²
PostgresClient/6144000/GetObservationsAsTimeseries-4     1.307m ± ∞ ¹    1.198m ± ∞ ¹        ~ (p=1.000 n=1) ²
PostgresClient/6144000/CreateForecast-4                  2.612m ± ∞ ¹    4.079m ± ∞ ¹        ~ (p=1.000 n=1) ²
geomean                                                  6.449m          4.286m        -33.54%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

@devsjc devsjc force-pushed the devsjc/forecast-partitioning branch 2 times, most recently from 42a8eca to 711b10b Compare January 19, 2026 16:23
@devsjc devsjc changed the title feat(sql): Speed up LatestForecast via better indexing feat(sql): Speed up LatestForecast, enable pg_cron Jan 19, 2026
@devsjc devsjc force-pushed the devsjc/forecast-partitioning branch from c07712d to 89315bc Compare January 19, 2026 16:37
@devsjc devsjc merged commit 9a4ec71 into main Jan 19, 2026
3 checks passed
@devsjc devsjc deleted the devsjc/forecast-partitioning branch January 19, 2026 16:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants