feat: In GTFS feed validation, only check consistency against disruption data

[jzimbel-mbta]

Summary of changes

Asana Ticket: 🏹 Prevent feed validation from blocking most application functionality

The meat of the change:

• The validation transaction now checks only the "external" FK constraints on the feed, those originating in Arrow disruption data tables. For example, shuttle_route_stops.gtfs_stop_id references gtfs_stops.id.
  • Feed data is imported only from the GTFS tables referenced by said "external" FKs.
  • While the transaction still locks certain application-critical tables, it runs much, much faster now so it's not really a problem.

In less abstract terms, this means validation only looks at lines.txt, routes.txt, and stops.txt now.

It completes in half a second locally now, should be about as fast in prod as well.

Reviewer Checklist

• Meets ticket's acceptance criteria
• Any new or changed functions have typespecs
• Tests were added for any new functionality (don't just rely on Codecov)
• This branch was deployed to the staging environment and is currently running with no unexpected increase in warnings, and no errors or crashes.

[rudiejd]

noting that this change, if merged, would get rid of the only way we have to validate that each checkpoint_id in stop_times shows up in checkpoints.txt. We should have a follow-up to add this validation to GTFS creator

[jzimbel-mbta]

noting that this change, if merged, would get rid of the only way we have to validate that each checkpoint_id in stop_times shows up in checkpoints.txt. We should have a follow-up to add this validation to GTFS creator

Thanks for calling that one out. Yeah, that's the only "feed-internal" consistency check by Arrow that doesn't duplicate what's already validated by other gtfs_creator CI steps.

I'll create a ticket.

[jzimbel-mbta]

Ticket to implement checkpoints<->stop_times validation in gtfs_creator.

[jzimbel-mbta]

I'm working on splitting the diff into meaningful commits to ease review. Will also try to add some integration tests for validation and import.

[jzimbel-mbta]

Related gtfs_creator PR to add stop_times.checkpoint_id <-> checkpoints.checkpoint_id validation
Now merged.