Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion .github/workflows/build-and-deploy-job.yml
Original file line number Diff line number Diff line change
Expand Up @@ -32,14 +32,15 @@ jobs:
echo "CONTAINER_INSTANCE_BASE_NAME=aci-${APP_NAME}" >> ${GITHUB_ENV}
echo "RESOURCE_GROUP_BASE_NAME=rg-${APP_NAME}" >> ${GITHUB_ENV}
echo "STORAGE_ACCOUNT_NAME=sa${APP_NAME//-/}$TARGET_ENVIRONMENT" >> ${GITHUB_ENV}
echo "APP_NAME=${APP_NAME}" >> ${GITHUB_ENV}

- name: 'Print calculated environment variables'
run: |
echo $TARGET_ENVIRONMENT_UPPER
echo $CONTAINER_INSTANCE_BASE_NAME
echo $RESOURCE_GROUP_BASE_NAME
echo $STORAGE_ACCOUNT_NAME

echo $APP_NAME
- name: 'Checkout GitHub Action'
uses: actions/checkout@v4

Expand Down Expand Up @@ -105,6 +106,7 @@ jobs:
LOG_WORKSPACE_KEY: ${{ secrets[format('{0}_{1}', env.TARGET_ENVIRONMENT_UPPER, 'LOG_WORKSPACE_KEY')] }}

# Variables which configure the app
AZURE_SERVICE_BUS_DATASET_CHECK_RESULTS_TOPIC_NAME: ${{ vars[format('{0}_{1}', env.TARGET_ENVIRONMENT_UPPER, 'AZURE_SERVICE_BUS_DATASET_CHECK_RESULTS_TOPIC_NAME')] }}
AZURE_SERVICE_BUS_REGISTRY_SUB_NAME: ${{ vars[format('{0}_{1}', env.TARGET_ENVIRONMENT_UPPER, 'AZURE_SERVICE_BUS_REGISTRY_SUB_NAME')] }}
AZURE_SERVICE_BUS_REGISTRY_TOPIC_NAME: ${{ vars[format('{0}_{1}', env.TARGET_ENVIRONMENT_UPPER, 'AZURE_SERVICE_BUS_REGISTRY_TOPIC_NAME')] }}
AZURE_SERVICE_BUS_WAIT_TIME: ${{ vars[format('{0}_{1}', env.TARGET_ENVIRONMENT_UPPER, 'AZURE_SERVICE_BUS_WAIT_TIME')] }}
Expand Down Expand Up @@ -137,6 +139,8 @@ jobs:
az -v
az container create --debug \
--resource-group "${{ env.RESOURCE_GROUP_BASE_NAME }}-${{ env.TARGET_ENVIRONMENT }}" \
--vnet "${{ env.APP_NAME }}-${{ env.TARGET_ENVIRONMENT }}-vnet" \
--subnet "${{ env.APP_NAME }}-${{ env.TARGET_ENVIRONMENT }}-subnet" \
--file ./azure-deployment/azure-resource-manager-deployment-manifest.yml

- name: 'Re-generate the website links'
Expand Down
10 changes: 10 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,16 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).

### Removed

## [1.4.6] - 2026-05-11

### Changed

- Updated deploy to use dedicated vnet & subnet

### Fixed

- Added env var for the MQ topic name to the GitHub workflow.

## [1.4.5] - 2026-05-06

### Changed
Expand Down
12 changes: 12 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -239,6 +239,8 @@ pytest-watcher .

### Initial Provisioning

#### Bulk Data Service App

You can create an Azure-based instance of Bulk Data Service using the `azure-create-resources.sh` script. It must be run from the root of the repository, and it requires (i) the environment variable `BDS_DB_ADMIN_PASSWORD` to be set with the password for the database, and (ii) a single parameter which is the name of the environment/instance. For instance, the following command will create a dev instance:

```bash
Expand All @@ -249,6 +251,16 @@ This will create a resource group on Azure called `rg-bulk-data-service-dev`, an

At the end of its run, the `azure-create-resources.sh` script will print out various secrets which need to be added to Github Actions.

**NOTE**: This is only really useful for temporary deployment or initial setup; once you're setup with CI/CD, the GitHub action does all this.

#### Bulk Data Service Network and Public IP

The Bulk Data Service is deployed to a dedicated vnet with subnet and attached NAT Gateway which has a public IP. To ensure the IP remains, these are not destroyed and re-created on every release (like the Azure Container Instances are). To create the networks and public IPs for dev and production, run:

```bash
./azure-provision/create-vnets-public-ips.sh
```

### Deployment - Versioning

The app version is set in `pyproject.toml`, and this is read by the app to use in the `User-Agent` header. When making a new release, set the version here to the appropriate value. Then, when releasing the app using the normal IATI Python app deployment process, choose the tag name to match the version chosen.
Expand Down
45 changes: 45 additions & 0 deletions azure-provision/create-vnets-public-ips.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
#!/usr/bin/env bash

set -o errexit # abort on nonzero exitstatus
set -o nounset # abort on unbound variable
set -o pipefail # don't hide errors within pipes

# This script creates the virtual networks, subnets and public IPs for the bulk data service.

RESOURCE_GROUP_NAME="rg-bulk-data-service-vnets"
LOCATION="uksouth"

az group create --name "$RESOURCE_GROUP_NAME" --location "$LOCATION"

for ENV in dev prod; do
az network vnet create --resource-group "$RESOURCE_GROUP_NAME" \
--name "bulk-data-service-${ENV}-vnet" \
--address-prefix 10.0.0.0/16 \
--subnet-name "bulk-data-service-${ENV}-subnet" \
--subnet-prefix 10.0.1.0/24

az network vnet subnet update --resource-group "$RESOURCE_GROUP_NAME" \
--vnet-name "bulk-data-service-${ENV}-vnet" \
--name "bulk-data-service-${ENV}-subnet" \
--delegation Microsoft.ContainerInstance/containerGroups

az network public-ip create \
--resource-group "$RESOURCE_GROUP_NAME" \
--name "bulk-data-service-${ENV}-public-ip" \
--sku Standard \
--allocation-method Static \
--location "$LOCATION"

az network nat gateway create \
--resource-group "$RESOURCE_GROUP_NAME" \
--name "bulk-data-service-${ENV}-nat-gateway" \
--location "$LOCATION" \
--public-ip-addresses "bulk-data-service-${ENV}-public-ip" \
--idle-timeout 10

az network vnet subnet update \
--resource-group "$RESOURCE_GROUP_NAME" \
--vnet-name "bulk-data-service-${ENV}-vnet" \
--name "bulk-data-service-${ENV}-subnet" \
--nat-gateway "bulk-data-service-${ENV}-nat-gateway"
done
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[project]
name = "bulk-data-service"
version = "1.4.5"
version = "1.4.6"
requires-python = ">= 3.12.6"
readme = "README.md"
dependencies = [
Expand Down
Loading