M2 Planning: Re-organize code and document the organization for future maintainers

[handrews]

Milestone 2 incudes cleanup, documentation, and unit testing of the code. With Milestone 1 successfully demo'd, it's clear that some code should be rearranged a bit, and unit tests written for the rearranged code.

Terminology: APID refers to API D(escription|efinition|ocument) in the (potentially) multi-document sense. An "OAS document" (probably small-d) refers to a single discrete element (file or network resource) within an OAS APID. Documentation should use "OAS APID" in most places, as there are other APID formats.

APID interface

• We may want to avoid the "API description" vs "API definition" debate and go with "APID" (e.g. oascomply.apid in place of oascomply.apidescription), particularly in code so we don't have to change it to align with any future consensus
• Document loading and caching should be factored out into its own module, instead of being attached to apidescription. This probably includes support for argparse-based CLI options, the structure of which is a bit unusual but seems to work, and is shared between the oascomply and oas30-schema tools (albeit with different option names)
• Clearly define the external interface for oascomply as a library, which should center on the APID concept

Naming Conventions and coding style

• Document and be consistent about naming. I seem to have settled into a "camelCase everything, including acronyms" approach as it keeps things distinct from various libraries using almost-but-not-quite-the-same names with all-caps acronyms (e.g. rfc3986.URIReference, jschon.URI, rdflib.URIRef)
• Determine and document what things can and should be abbreviated (Pointer vs Ptr, Relative vs Rel, etc.), balancing readability and name length (RelativeJsonPointerTemplate is really quite long, and RelativeJsonPointerTemplateEvaluationError is ridiculous - RelJsonPtrTemplateEvalError is probably clear enough an actually fits on a line)
• Consider an auto-formatter like black. It will irritate me but probably be better in the long run.
• Decide what to do with type hinting. It seems kind of nice for interface documentation, but probably not more than that

Error handling

• Define and document a base exception class and error management philosophy
• It should be entirely possible to determine how to handle an error from the exception type, without examining its message at all; exception classes should define properties to access the relevant structured data
• Ideally, exception class constructors would not take a message, but create one from its arguments (although allowing part of the message to be passed directly is fine as long as it is purely for human consumption)
• Error structures from libraries should be encapsulated like any other aspect of library usage; however, raw access to library errors and data structures should be available in the API and should be logged if verbosity is turned all of the way up

Document construction and access

• Need to decide whether and how much to encapsulate the usage of jschon's suite of JSON tools (JSON, JSONPointer, RelativeJSONPointer, and JSONPatch)
• Subclassing is not sufficient to prevent leakage via exception classes
• It's unclear if the OasJson class is the right approach; an alternative might be giving an extension of jschon.JSONSchema the notion of a schema root independent of a document root

URIs (and IRIs), [Relative] JSON Pointers, and Templates of all of these things

• Put the basic encapsulation of and user-friendly facade for the rfc3987 in its own module (which could be its own package)
• Put OAS-style URL templating in its own module
• If encapsulating jschon's JSON Pointer classes, put those in their own module
• Keep [R]JP Templates in their own module
• Put the JSON Pointer IRI-fragment-specific extensions in their own module, and decide whether .fragment should a a pointer instance or stay a string and have the pointer instance be .ptr or similar
• Decide whether making all of these things comparable to str is really the right idea

JSON Schema functionality

• Code that specifically extends jschon (potentially relying on unmerged upstream PRs that might change direction) to support OAS-related functionality should get its own module
• The jschon-based implementation of the OAS dialects should get its own module, probably including the OAS-defined formats (although maybe they get a separate module)
• Implementation of JSON Schema-defined formats for jschon should get its own module (which could potentially be contributed upstream if we wanted, although our correctness vs performance balance may not be ideal for general use)
• The old oastype4jschon module is not used and should either be removed or updated; as the vocabulary is still annotation-only, it's unclear if there's a use for an implementation module (other than to allow the vocabulary to be used in $vocabulary with a true value, which might be reason enough given how trivial it is to support)
• The framework for driving actions through annotations should be extracted from apidescription and probably generalized somewhat, in order to support future extensibility through additional annotation-driven actions

RDF Graph functionality

• The code that creates rdfs:labels based on oastype should be isolated and made easy to extend and/or override; it is one of the few places where understanding of the OAS structure appears in code, and therefore needs to be kept clearly separate from the generic code
• The annotation processing patterns, particularly around RJP Templates, can probably be condensed further. There may be a boundary to draw between annotation processing and graph construction that is not currently in the right place (see also the "framework for driving actions through annotations" comment in the previous section)
• The toml serialization should go in its own module; decide how aware of OAS it should be
• Need to figure out what, if any, graph construction should be handled by RDFS/OWL inferencing

more tbd...

[handrews]

Some updates:

• A lot of what was in oascomply.apidescription and oascomply.oas30dialect is being moved into a more comprehensive set of jschon extensions in a new oascomply.oasjson
• oas30dialect is now just the dialect/vocabulary/keyword support
• Much of the rest of oascomply.apidescription will move into an oascomply.cli library containing code that is not needed if using oascomply as a library
• oascomply.apidescription will probably be just the basic flow control for carrying out the parsing/vaidating/linting process, including registering and running extensions
• I'm probably not going to do too much more with the URI/IRI/JSONPointer things, as they work well enough and are separate enough to be easily improved later
• I'm still not sure what to do about type hints - I like them on interfaces, but getting the tools to be useful seems like a time sink perhaps best deferred for a future contributor who knows more about them
• Error handling improvement still needs to be done.

The above code organization changes are making the code a lot more testable and readable.