asdf and table support

[braingram]

The ASDF standard defines a core/table tag/schema.

Currently asdf does not support reading or writing this tag. The asdf-astropy extension supports reading core/table but does not support writing:
https://github.com/astropy/asdf-astropy/blob/b532105fea731cfe963c8015041002aa568fa29d/asdf_astropy/converters/table/table.py#L51
(an issue is open for asdf-astropy to investigate options for writing: astropy/asdf-astropy#209)

To improve interoperability with other languages (or non-astropy users) what options exist for reading and writing core/table tags?

@eslavich put together a prototype that enables asdf-astropy to writing core/table tags. astropy/asdf-astropy#211

Perhaps how asdf-astropy handles core/unit might provide a potential option:
https://github.com/astropy/asdf-astropy/blob/main/asdf_astropy/converters/unit/unit.py#L23
The select_tag method inspects the Unit, determines if it should work as a core/unit and then falls back to the more specific astropy unit.

Pinging @perrygreenfield @Cadair @nden for input.

[braingram]

To try and capture some of the discussion with @perrygreenfield and @eslavich (forgot to add you to the initial post for this discussion).

Some key questions that came up were:

• Should asdf implement a class that allows conversion to/from the core table tag? This would allow users who want a table stored with a core tag to convert their astropy table (or other more complex table) to the core table class first to signify to asdf that the core tag should be used. Is providing a table class something that makes asdf useful?
• Should we provide some api for users to manage tag conflicts between extensions? For instance if a user has both asdf-astropy and another unit providing library (possibly like in weldx) installed, do all unit classes round trip (I doubt that is the case at the moment)? Users could possibly reorder extensions to avoid these conflicts for a specific unit class but that would require advanced understanding of asdf internals.
• How much work would be involved in constructing a tag conflict (like as described above with unit but one more likely to be encountered by a user)? For instance, if we made a pandas table extension and allowed it to read/write core table tags (and added core table tag writing to asdf-astropy) we would have a set of extensions that would allow us to experiment with various conflicting scenarios to better understand what api is needed.
• Is the core table schema flexible enough to store all astropy tables? Even if this is the case now, what should be done if astropy adds new requirements/features that require schema updates. Since this would reintroduce astropy as a dependency (in a round-about way) what coordination and testing would be required to keep things in sync?
• Should table and column be part of the core?

@perrygreenfield mentioned seeking clarification from @Cadair about use of the core table tag. Some questions that came up are:

• Is the goal to make a non-astropy dependent data file?
• or a non-python dependent data file?
• Is 'round-tripping' of astropy tables important when using the core tag?