diff --git a/README.md b/README.md new file mode 100644 index 0000000..299d952 --- /dev/null +++ b/README.md @@ -0,0 +1,107 @@ +# Pythia + +[![Build](https://github.com/jonjau/pythia/actions/workflows/rust.yml/badge.svg)](https://github.com/jonjau/pythia/actions/workflows/rust.yml) +[![License:MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT) + +Pythia is a novel 'state change explorer' tool built with Rust and [Scryer Prolog](https://www.scryer.pl/). + +Run unit tests with database changes tracked in Pythia to help answer questions like: +- How many unique database records of a certain table does Test `T1` change? +- Which tests cover the mutation of field `F`, for example from 'ordered', 'dispatched', then 'delivered'? +- Is state `S` is reachable from state `S'` in the context of Test `T1` or Test `T2`? +- If field `F` changes, do any other fields tend to change with it? +- How many database modifications would it take, to take a database record from state `S` to state `S'`? +- If we were to run the functionality covered by Test `T1` followed by that of Test `T2` on the same database record, what would be the resulting state? + +Run completely locally with `docker` or `podman`, or try out the public demo at [pythia.jonjauhari.com](https://pythia.jonjauhari.com) (I don't always keep it running). + +## Built with + +- Rust and Axum: server for HTML and exposing a REST API, with cookie-based anonymous sessions. +- Scryer Prolog: underlying logic engine for graph reachability calculations. +- Askama: Prolog code generation and HTML templating. +- HTMX, Alpine.js, and TailwindCSS: lightweight web-based UI with barely any JavaScript. +- DynamoDB: single-table database to persist all application data. +- AWS (Fargate), Docker, Terraform, GitHub Actions: continuous deployment workflow. +- Cloudflare: DNS provider. + +## How to run + +### Run locally with Docker/Podman + +At the project root: + +```bash +docker compose up +``` + +Pythia will be running locally and listening on port `3000`. All data will be locally stored in a `.db` file under `dynamodb/`. + +### Run locally without Docker/Podman + +Linux/Windows binaries for Pythia are available in 'Releases' in this repo. + +Some useful environment variables that can be set: + +- `PYTHIA_RUN_MODE`: `local` (default if unspecified) or `remote`. +- `AWS_ENDPOINT_URL`: the DynamoDB endpoint to use for persistence. This is required for the `local` run mode, and ignored for `remote` run mode. The regional DynamoDB endpoint is used for the `remote` run mode. +- `RUST_LOG`: the log level; `INFO` is good. + +For example you can run the Pythia executable by pointing to a separately running DynamoDB instance on port `8000`: + +```bash +AWS_ENDPOINT_URL="http://localhost:8000" RUST_LOG=info ./path/to/executable +``` + +Or build it directly if you have Cargo installed: + +```bash +cargo build && AWS_ENDPOINT_URL="http://localhost:8000" RUST_LOG=info ./target/debug/pythia +``` + +### Run on AWS + +Ensure you have authenticated with the AWS CLI with enough permissions, a Cloudflare API token, and have Terraform installed. + +In `infra/bootstrap/`, run `terraform init` and `terraform apply`. This project will setup an S3 bucket that can be used as a backend for the other Terraform projects (and also an IAM role for the GitHub Actions deployment automation). + +In `infra/ecr/`, init terraform with a backend (ideally a remote location like S3) then `terraform apply`. This project will set up the ECR repository. + +In `infra/`, init terraform with a backend (ideally a remote location as well) then `terraform apply`. This project will set up the actual Pythia application deployment, including: + +- ECS: cluster, Fargate service, task definition +- IAM roles: for ECS and the ECS Task +- Networking: AWS Subnets, gateways, route tables, the ALB. The EC2-based [`fck-nat`](https://registry.terraform.io/modules/RaJiska/fck-nat/aws/latest) module is used in place of AWS's managed NAT gateway in order to cut down on running costs. +- DNS: ACM certificate, Cloudflare DNS records + +## Usage + +1. Start a session: + + + +2. Add record types (either via web UI or the REST API): + + + +3. Add facts for the records (either via the web UI or the REST API): + + + +4. Calculate state change paths! + + + + +There's a few things we can loosely infer: +- `Test_FailOrder` tests 2 unique 'order' records in total. +- Both tests cover the mutation of `Status` from 'ordered' to 'dispatched'. +- Starting from the `Status` of 'ordered' we can reach the `Status` of 'cancelled' or 'delivered'. +- When the `Status` changes to 'dispatched', the `DispatchDate` is set to some date, likewise for 'delivered' and `DeliveryDate`. +- It takes 2 steps to get from `Status` of 'ordered' to 'delivered'. +- We could probably cancel 'Ord2' before it goes to the 'delivered' `Status`. + +## License + +Pythia is currently licensed under the terms of both the MIT license and the Apache License (Version 2.0). See [`LICENSE-MIT`](/LICENSE-MIT) and [`LICENSE-APACHE`](/LICENSE-APACHE) for more details. + diff --git a/data/dimlink.pl b/data/dimlink.pl deleted file mode 100644 index 302a197..0000000 --- a/data/dimlink.pl +++ /dev/null @@ -1,13 +0,0 @@ -:- discontiguous(record/7). -:- dynamic(dimlink/9). -dimlink("M5", "ID1", "J3", "2023-02-08", "2023-02-09", "Test1", "2024-02-18 09:17:11", "V", "1"). -dimlink("M1", "ID1", "J1", "2023-02-08", "2023-02-10", "Test1", "2024-02-18 08:16:11", "D", "0"). -dimlink("M4", "ID1", "J2", "2023-02-08", "2023-02-09", "Test1", "2024-02-18 09:17:11", "O", "0"). -dimlink("M2", "ID2", "J3", "2023-02-08", "2023-02-11", "Test1", "2024-02-18 08:20:15", "E", "4"). -dimlink("M5", "ID1", "J3", "2023-02-08", "2023-02-09", "Test1", "2024-02-18 09:17:11", "D", "0"). -dimlink("M2", "ID2", "J1", "2023-02-08", "2023-02-11", "Test1", "2024-02-18 08:20:13", "D", "2"). -dimlink("M1", "ID1", "J1", "2023-02-09", "2023-02-10", "Test1", "2024-02-18 08:17:11", "E", "1"). -dimlink("M2", "ID2", "J1", "2023-02-08", "2023-02-11", "Test1", "2024-02-18 08:20:12", "D", "1"). -dimlink("M2", "ID2", "J1", "2023-02-08", "2023-02-11", "Test1", "2024-02-18 08:20:11", "D", "0"). -dimlink("M3", "ID2", "J2", "2023-02-08", "2023-02-09", "Test1", "2024-02-18 08:20:14", "O", "0"). -dimlink("M2", "ID2", "J2", "2023-02-09", "2023-02-11", "Test1", "2024-02-18 08:20:14", "D", "3"). diff --git a/data/internal/pythia.pl b/data/internal/pythia.pl deleted file mode 100644 index a752fa2..0000000 --- a/data/internal/pythia.pl +++ /dev/null @@ -1,52 +0,0 @@ -:- use_module('data/dimlink.pl'). -:- use_module('data/transaction.pl'). -:- use_module(library(clpz)). -:- use_module(library(lists)). - -:- discontiguous(change_step/5). -:- discontiguous(change_path/6). - -change_step(RType, Ctx, Id, Vals1, Vals2) :- - record(RType, Ctx, _, _, SeqNum1, Id, Vals1), - record(RType, Ctx, _, _, SeqNum2, Id, Vals2), - number_chars(Num1, SeqNum1), - number_chars(Num2, SeqNum2), - Num2 #= Num1 + 1, - Vals1 \= Vals2. - -change_path(RType, Ctx, Id, Vals, Vals, []) :- - record(RType, Ctx, _, _, _, Id, Vals). - -change_path(RType, Ctx, Id, Vals1, Vals2, [Step|Steps]) :- - % Enforce step exists before constructing step term - change_step(RType, Ctx, Id, Vals1, ValsMid), - Step = [Vals1, ValsMid], - change_path(RType, Ctx, Id, ValsMid, Vals2, Steps). - - -record( - "dimlink", - Context, - EditTime, - RecStatus, - SeqNum, - [Id], - [DRef, IRef, BegPeriod, EndPeriod] -) :- -dimlink( - Id, DRef, IRef, BegPeriod, EndPeriod, Context, EditTime, RecStatus, SeqNum -). - -record( - "transaction", - Context, - EditTime, - RecStatus, - SeqNum, - [Id1, Id2], - [DRef, IRef, BegPeriod, EndPeriod] -) :- -transaction( - Id1, Id2, DRef, IRef, BegPeriod, EndPeriod, Context, EditTime, RecStatus, SeqNum -). - diff --git a/data/transaction.pl b/data/transaction.pl deleted file mode 100644 index e12b235..0000000 --- a/data/transaction.pl +++ /dev/null @@ -1,12 +0,0 @@ -:- discontiguous(record/7). -:- dynamic(transaction/10). -transaction("T4", "M4", "ID1", "J2", "2023-02-08", "2023-02-09", "Test1", "2024-02-18 09:17:11", "O", "0"). -transaction("T2", "M2", "ID2", "J1", "2023-02-08", "2023-02-11", "Test1", "2024-02-18 08:20:11", "D", "0"). -transaction("T1", "M2", "ID1", "J1", "2023-02-09", "2023-02-10", "Test1", "2024-02-18 08:17:11", "E", "1"). -transaction("T2", "M2", "ID2", "J1", "2023-02-08", "2023-02-11", "Test1", "2024-02-18 08:20:12", "D", "1"). -transaction("T5", "M5", "ID1", "J3", "2023-02-08", "2023-02-09", "Test1", "2024-02-18 09:17:11", "D", "0"). -transaction("T1", "M1", "ID1", "J1", "2023-02-08", "2023-02-10", "Test1", "2024-02-18 08:16:11", "D", "0"). -transaction("T2", "M2", "ID2", "J2", "2023-02-09", "2023-02-11", "Test1", "2024-02-18 08:20:14", "D", "3"). -transaction("T2", "M2", "ID2", "J3", "2023-02-08", "2023-02-11", "Test1", "2024-02-18 08:20:15", "E", "4"). -transaction("T3", "M3", "ID2", "J2", "2023-02-08", "2023-02-09", "Test1", "2024-02-18 08:20:14", "O", "0"). -transaction("T2", "M2", "ID2", "J1", "2023-02-08", "2023-02-11", "Test1", "2024-02-18 08:20:13", "D", "2"). diff --git a/data/types.json b/data/types.json deleted file mode 100644 index dd99523..0000000 --- a/data/types.json +++ /dev/null @@ -1,39 +0,0 @@ -[ - { - "name": "dimlink", - "id_fields": [ - "Id" - ], - "data_fields": [ - "DRef", - "IRef", - "BegPeriod", - "EndPeriod" - ], - "metadata_fields": [ - "Context", - "EditTime", - "RecStatus", - "SeqNum" - ] - }, - { - "name": "transaction", - "id_fields": [ - "Id1", - "Id2" - ], - "data_fields": [ - "DRef", - "IRef", - "BegPeriod", - "EndPeriod" - ], - "metadata_fields": [ - "Context", - "EditTime", - "RecStatus", - "SeqNum" - ] - } -] \ No newline at end of file diff --git a/doc/facts.png b/doc/facts.png new file mode 100644 index 0000000..24646e9 Binary files /dev/null and b/doc/facts.png differ diff --git a/doc/record-types.png b/doc/record-types.png new file mode 100644 index 0000000..42f5d3d Binary files /dev/null and b/doc/record-types.png differ diff --git a/doc/sessions.png b/doc/sessions.png new file mode 100644 index 0000000..f9ee7b4 Binary files /dev/null and b/doc/sessions.png differ diff --git a/doc/state-change-1.png b/doc/state-change-1.png new file mode 100644 index 0000000..f4561e4 Binary files /dev/null and b/doc/state-change-1.png differ diff --git a/doc/state-change-2.png b/doc/state-change-2.png new file mode 100644 index 0000000..a8136b2 Binary files /dev/null and b/doc/state-change-2.png differ diff --git a/templates/fact/facts-table.html b/templates/fact/facts-table.html index 3e82003..bfaf0bc 100644 --- a/templates/fact/facts-table.html +++ b/templates/fact/facts-table.html @@ -26,7 +26,7 @@ + class="cursor-pointer text-amber-200 hover:text-amber-400"> {{ value }} Copied! diff --git a/templates/how-to-use.html b/templates/how-to-use.html index 7ce4877..1987eb6 100644 --- a/templates/how-to-use.html +++ b/templates/how-to-use.html @@ -5,7 +5,7 @@

What is this?

Pythia is an open-source 'state change - explorer'. It's a tool to track how + explorer'. It's a tool to visualise and search how records change over time, based on recorded facts for those record types.

@@ -16,7 +16,7 @@

What is this?

- Pythia's intended use case is to aid in reverse-engineering the + Pythia's intended use case is to aid in inferring the rules that produced the facts which are recorded.

@@ -43,14 +43,14 @@

How do I use it?

  1. Get a user token, via 'Manage session' which will set a user_token + class="text-amber-200">user_token as a cookie.
    This token is displayed at the top right of the page.
  2. Define record types via 'Record types', or POST to the API route: -
    /api/record-types (the - user_token cookie must be set) +
    /api/record-types (the + user_token cookie must be set)
  3. Add facts for record types under 'Inquiries', or POST to the API - route:
    /api/{record_type}/facts.
    4. + route:
    /api/{record_type}/facts.
  4. Calculate state change paths under 'Inquiries' based on the recorded facts.
  5. Narrow it down with filters, and make conclusions about the data.
    6. diff --git a/templates/layout.html b/templates/layout.html index 356728f..9aa7d1a 100644 --- a/templates/layout.html +++ b/templates/layout.html @@ -38,7 +38,7 @@ {% endif %} + class="cursor-pointer font-monospace text-amber-200 hover:text-amber-400"> {{ user_token }} Copied to clipboard! diff --git a/terraform.tfstate b/terraform.tfstate deleted file mode 100644 index 480e3e3..0000000 --- a/terraform.tfstate +++ /dev/null @@ -1,9 +0,0 @@ -{ - "version": 4, - "terraform_version": "1.12.2", - "serial": 1, - "lineage": "301c5003-4ed2-60f6-e569-e60abd5a0e9e", - "outputs": {}, - "resources": [], - "check_results": null -}