Use sensemaker-tools as npm package rather than installing from repo

[digitalWestie]

Makes maintenance and upgrades easier. Will also help to simplify install / deploy script.

[digitalWestie]

Progress:

The library part of the repo has been published on NPM: https://www.npmjs.com/package/@cosla/sensemaking-tools

We can use this until we get the changes we need (e.g. using API keys) in the main Jigsaw repo.

[digitalWestie]

TODO:

• Publish web-ui as a separate package
• Update paths and package.json in rails app to use packages

[digitalWestie]

Published web-ui https://www.npmjs.com/package/@cosla/sensemaking-web-ui

[digitalWestie]

IT appears we need to update packages, from running: npm audit:

@angular/common  <19.2.16
Severity: high
Angular is Vulnerable to XSRF Token Leakage via Protocol-Relative URLs in Angular HTTP Client - https://github.com/advisories/GHSA-58c5-g7wp-6w37

...

  @cosla/sensemaking-web-ui  *
  Depends on vulnerable versions of @angular/common
  Depends on vulnerable versions of @angular/compiler
  Depends on vulnerable versions of @angular/forms
  Depends on vulnerable versions of @angular/platform-browser
  Depends on vulnerable versions of @angular/platform-browser-dynamic
  Depends on vulnerable versions of @angular/platform-server
  Depends on vulnerable versions of @angular/router
  Depends on vulnerable versions of @angular/ssr
  Depends on vulnerable versions of ngx-markdown
  node_modules/@cosla/sensemaking-web-ui

[digitalWestie]

PR1 is now based on using the packages: consuldemocracy#6179

However, npm audit identifies multiple vulnerable / outdated packages used by web-ui scripts-

npm audit
# npm audit report

@angular/common  <19.2.16
Severity: high
Angular is Vulnerable to XSRF Token Leakage via Protocol-Relative URLs in Angular HTTP Client - https://github.com/advisories/GHSA-58c5-g7wp-6w37
No fix available
node_modules/@angular/common
  @angular/forms  <=19.2.15
  Depends on vulnerable versions of @angular/common
  Depends on vulnerable versions of @angular/platform-browser
  node_modules/@angular/forms
  @angular/platform-browser  <=19.2.15
  Depends on vulnerable versions of @angular/common
  node_modules/@angular/platform-browser
    ngx-markdown  1.6.0 - 18.1.0
    Depends on vulnerable versions of @angular/common
    Depends on vulnerable versions of @angular/platform-browser
    node_modules/ngx-markdown
  @angular/platform-browser-dynamic  <=19.2.15
  Depends on vulnerable versions of @angular/common
  Depends on vulnerable versions of @angular/compiler
  Depends on vulnerable versions of @angular/platform-browser
  node_modules/@angular/platform-browser-dynamic
  @angular/platform-server  <=19.2.15
  Depends on vulnerable versions of @angular/common
  Depends on vulnerable versions of @angular/compiler
  Depends on vulnerable versions of @angular/platform-browser
  node_modules/@angular/platform-server
  @angular/router  <=0.0.0-ROUTERPLACEHOLDER || 2.0.0-rc.0 - 19.2.15
  Depends on vulnerable versions of @angular/common
  Depends on vulnerable versions of @angular/platform-browser
  node_modules/@angular/router
  @angular/ssr  <=18.2.21
  Depends on vulnerable versions of @angular/common
  node_modules/@angular/ssr
  @cosla/sensemaking-web-ui  *
  Depends on vulnerable versions of @angular/common
  Depends on vulnerable versions of @angular/compiler
  Depends on vulnerable versions of @angular/forms
  Depends on vulnerable versions of @angular/platform-browser
  Depends on vulnerable versions of @angular/platform-browser-dynamic
  Depends on vulnerable versions of @angular/platform-server
  Depends on vulnerable versions of @angular/router
  Depends on vulnerable versions of @angular/ssr
  Depends on vulnerable versions of esbuild
  Depends on vulnerable versions of ngx-markdown
  node_modules/@cosla/sensemaking-web-ui

@angular/compiler  <=18.2.14
Severity: high
Angular Stored XSS Vulnerability via SVG Animation, SVG URL and MathML Attributes - https://github.com/advisories/GHSA-v4hv-rgfq-gp49
fix available via `npm audit fix`
node_modules/@angular/compiler

esbuild  <=0.24.2
Severity: moderate
esbuild enables any website to send any requests to the development server and read the response - https://github.com/advisories/GHSA-67mh-4wv8-2f99
No fix available
node_modules/esbuild

11 vulnerabilities (1 moderate, 10 high)

To address issues that do not require attention, run:
  npm audit fix

Some issues need review, and may require choosing
a different dependency.

We'll need to update these to newer versions.

[digitalWestie]

Upgraded the packages in web-ui to safe versions, and added a PR to the Jigsaw project - Jigsaw-Code/sensemaking-tools#50

[digitalWestie]

Next:

Apply change to follow on PRs.

[digitalWestie]

Package change applied to all integration PRs.

Next:

Need to review the setup / cap rake tasks to see if they can be simplified and/or if necessary.

[digitalWestie]

Setup script

No change required.

Setup script has its use as it helps with dependencies, feature flag, and directories. We will however need to make changes as we move to using existing LLM settings #33 .

Setup Script Summary

• Checks system dependencies: Verifies Node.js, npm, and npx are installed
• Verifies package.json configuration: Checks that @cosla/sensemaking-tools and @cosla/sensemaking-web-ui are listed in package.json (raises an error with instructions if missing; does not modify the file)
• Ensures Angular build exists: If the build is missing, installs npm dependencies and builds the Angular web UI in development mode
• Creates directories:
  • Sensemaker data directory (from Sensemaker::Paths.sensemaker_folder, or defaults to vendor/sensemaking-tools)
  • Data directory (from Sensemaker::Paths.sensemaker_data_folder, or defaults to vendor/sensemaking-tools/data)
• Verifies CLI availability: Tests that the Sensemaker CLI tool can run
• Enables feature flag: Creates or updates the feature.sensemaker setting to "true" in the database
• Checks for service account key file: If missing, provides instructions for creating a Google Cloud service account key file