Draft: Rest api support

[kugel-]

Incomplete because I don't own a lot of devices for testing. I left out light bulbs and blinds for now, but also templates and triggers.

I hope the general architecture is alright. Retrofitting AHA-API was a slight pain but I think I got it. On the downside, the output may be different depending on the API, but I hope it's not too mich of a concern as long as you can force AHA for any given device. I did not attempt to fully unify output and class members because I anticipate that AHA API will be dropped eventually (in a few years). I wouldn't wand to severely cripple REST API support just because of ongoing AHA support.

However, I do try unify things where it's simple and worthwhile to do, especially for things that are used by HA.

The REST api does not expose flat devices anymore. Devices are comprised of one or more units, and units are comprised of one or more interfaces, although typically a device is also a (single) unit with numerous interfaces. I tried to reflect this in the code base to make future maintenance easier and less surprising. At the end of the day, most of the interesting stuff is in the interfaces.

So in a nutshell:
unchanged classes:
FritzhomeDevice* (except FritzhomeDeviceBase)
FritzhomeTrigger
FritzhomeTetmplate

modified classes:
FritzhomeDevice
FritzhomeDeviceBase
FritzhomeEntityBase

new classes
FritzhomeDeviceAHA
FritzhomeDeviceREST
FritzhomeInterface
FritzhomeMixin
FritzhomeUnit
FritzhomeUnitBase

For AHA the class hierarchy is largely unchanged, except it starts with FritzhomeDeviceAHA at the bottom.
For REST the class hierarchy does not include any of the FritzhomeDevice* classes anymore. Instead,
the hierarchy includes FritzhomeUnit and Fritzhome*Interface classes. I created the REST hierarchy first (basically from scratch). When reintegrating AHA support I figured it would be easiest to keep the related code unchanged (AHA API in FRITZ!OS is very unlikely to change and new endpoints are going to be REST-exclusive (see thermostat offset control).

Further complication comes from the two REST namespaces, overview and configuration. On the one hand, "overview" allows to get the big picture of all devicesin a single REST request. But on the other hand, only "configuration" gives full access. In the overview the "interfaces" nodes lack lack several fields (such as temperature offset).

As I said, this is very incomplete and many controls to not work yet. I want to get some general feedback on the general feedback before infesting more time.

[kugel-]

@hthiery do you have any kind of feedback? I work further on this and I'm also already testing together with HASS (in a development VM). It would be tragic if I'm nearing completion and would have to start over again based on your feedback.

[hthiery]

since @mib1185 is mainly working on this I would expect to get feedback from him.

[flabbamann]

Hi @kugel-,
I did not have time to take a closer look at your code, but I have some general thoughts in this topic I would like to share and also hear @mib1185's opinion.

I am not sure if it is a good Idea to fit both APIs into one library. Maybe it would be less complicated to create a new library for the REST-API. You would then also be free in how to structure the code.
But maybe this would bring us more complexity in the places where we use the library (home assistant) 🤔.

Another thing: Fritz provides an OpenAPI-Specification for their Rest-API. You can generate code from that. I use this a lot in the Java/Spring Boot world and it is really helpful, but I have no experience with Python code generation.

[kugel-]

Thanks for your insight. Actually, I have a working scheme that keeps the old API support effectively unchanged while also being able to implement REST support using a revamped structure which is close to the REST API objects itself.

On the other hand, it's an enormous help to have 2 clients (HA and the fritzhome cli command) that I can easily use against both APIs. In fact, for debugging and testing, I extend the cli interface and also retrofit these extensions to the AHA API. This is really helpful for A/B tests and I also learn better about the fine details where AHA and REST APIs mismatch.

So my plan is actually to transition the HA integration to a new pyfritzhome version but not switching to REST by default yet (interested early-adopters would be able to enable a flag), to iron out sharp edges, and then switch to REST by default in some later version of pyfritzhome.

Fritz provides an OpenAPI-Specification for their Rest-API.

I'm not an expert so I don't really know the benefits. Importing json responses is straightforward in Python (json.loads() returns a native dict that represents REST response) and from there you can use all the Python APIs and tooling. I wouldn't even know which of the new code I would want to have generated.

[mib1185]

Hi @kugel-
sorry for the late reply, but I was a bit out of time to get all these things sorted (to many projects and side-quests 🙈 )
In generell I would like to keep the interface of this lib as is (so the consuming application Home Assistant doesn't need to change or handle multiple libraries) and the lib should consider on its own, when to use the old AHA and when to use the new REST API (maybe it can be overruled by some parameter)
Further I think we should limit this PR to add the base implementation for the REST API and adopt device interface for the beginning, so the PR keeps to be reviewable and the adoption of the other methods can be done step-by-step in follow up PRs.

[kugel-]

We can consider splitting the PR once I'm (near) finished. Currently even the base isn't finished and subject to continuous refactoring so it doesn't make sense yet to review (and merge) only the base.

As I add support for more interfaces (as in the major objects in the REST responses) I also revisit previous interface and the very base.

But I would immensely appreciate if you could have a look at the overall design and give comments on that.

In generell I would like to keep the interface of this lib as is (so the consuming application Home Assistant doesn't need to change or handle multiple libraries) and the lib should consider on its own, when to use the old AHA and when to use the new REST API (maybe it can be overruled by some parameter)

I try very hard to achieve this, this is a primary goal. Hopefully it works out in the end. There are substantial differences in some darker corners between AHA and REST API device representations that are tricky to paper over, though. But even if HA needs slight adoptions: There should be one pyfritzhome version the fritzbox integrations runs against where AHA vs REST completely abstracted (and no regressions, especially if using the AHA API) so that HA doesn't really need to care.