Expose config and config updates through MQTT #634

mathieucarbou · 2023-05-09T10:56:02Z

No description provided.

mathieucarbou · 2023-05-09T10:58:00Z

@KipK : I've created this PR to monitor config and config changes with MQTT. This is useful to display dashboard components accordingly on HA, especially to get some config like power ratio, max_current_soft, etc.

Note: this introduces a code duplication that needs to be solved - just please let me know here would be the good candidate for putting the common code between web and mqtt to generate the json payload.

thanks ;-)

mathieucarbou · 2023-05-09T13:30:06Z

Example in HA

Ref: https://gist.github.com/mathieucarbou/92a3d5e0dc38d6b68aa1bdaf153a80c5

mathieucarbou · 2023-05-09T16:02:55Z

Note: when a RAPI command is sent through MQTT or debug mode (ie. $SA 190 0), the config does not update.

Which is OK IMO

src/web_server_config.cpp

src/mqtt.cpp

src/app_config.h

src/app_config.cpp

src/mqtt.cpp

mathieucarbou · 2023-05-10T10:37:39Z

PR rebased and updated to push the config each 30sec with the new time

src/mqtt.cpp

src/app_config.cpp

KipK · 2023-05-10T11:07:24Z

PR rebased and updated to push the config each 30sec with the new time

config should only be pushed at mqtt connection with retain flag, and thereafter only when there is an update ( i.e when config_version has changed ) . No need to push this each 30sec, just push it after a config change.

mathieucarbou · 2023-05-10T11:27:34Z

h 30sec, just push it aft

that is what I did initially. But once mqtt connects, I guess the evse manager is not ready yet ? Because all properties returnes from evse like firmware, max_current_soft, offset, etc are all equals to empty strings or 0.

So I did that to get a second push to ensue the config gets sent at each time update

mathieucarbou · 2023-05-12T09:01:30Z

(sorry - mistake - closed and reopened)

jeremypoulter

Looking good, just a few small changes

src/evse_monitor.cpp

src/app_config.cpp

LavredisG · 2023-05-14T20:53:41Z

I've read that RAPI will soon be removed (probably) from the MQTT and I have some questions on it:

Does this affect working with HTTP too, or in this case only http is used as api?
Is there any difference in general while working with either HTTP or MQTT?

jeremypoulter · 2023-05-14T21:22:22Z

Yeah the intention has always been to remove RAPI. I have created a ticket for this, will probably be part of the upcoming v5 release: #643

Let's move any discussion about removing RAPI there so we didn't lose any comments

mathieucarbou · 2023-05-14T22:18:34Z

Rebased!

glynhudson · 2023-05-16T11:08:54Z

@mathieucarbou could you add the new MQTT API to the docs as part of this PR? Thanks

Your HA integration looks great, have you published this?

mathieucarbou · 2023-05-16T13:16:26Z

@mathieucarbou could you add the new MQTT API to the docs as part of this PR? Thanks

done!

Your HA integration looks great, have you published this?

I am maintaining it there: https://gist.github.com/mathieucarbou/92a3d5e0dc38d6b68aa1bdaf153a80c5

I am always improving it.

Some notes (fyi @KipK and @jeremypoulter):

The integration still relies on some stuff not yet merged:

MQTT /config
MQTT /restart (for evse)
MQTT /config/set: in one of my branch, I didn't PR it yet because I know you guys fear that

HA is strongly integrated with MQTT and MQTT is pretty efficient. Going through HTTP API calls, even if this is towards what OpenEVSE is going, is highly inefficient with HA, especially when binding some UI components because, of the lags, http overhead and polling requirements. This is not a push system and HA does not have a great websocket support to bind components.

There are some config settings, like max current soft, min charging time, max power for shaper, divert settings etc that are worth exposing in read-write mode. The reason is that a Home Assistant software is supposed to centralise management and monitoring. And some settings are tightly coupled with improving the energy consumption of the charge (i.e. divert). For example, in summer or winter, I might not use the same settings. Also, since days now, I am tweaking the values to find the best calibration and numbers that work here for solar divert and also to compute a correct voltage because the EVSE does not have a voltmeter.

And to do that, I am also looking at some graph in Home Assistant.
I don't want to have to deal with many UIs if I can have everything I need and tweak tweak the values directly from HA.

/config/set is not about providing the ability to automate things. This is to provide the ability to mutate some key attributes that I think could be exposed from HA like I did in my integration.

jeremypoulter · 2023-05-21T21:20:15Z

src/mqtt.cpp

+  String fulltopic = mqtt_topic + "/config_version";
+  String payload = String(config_version());
+  mqttclient.publish(fulltopic, payload, true);


You don't need this, config_version is evented via event_send when the version number is incremented

You don't need this, config_version is evented via event_send when the version number is incremented

We need it when the first publication comes from the mqtt layer. Version is not upgraded at startup. I initially didn't have it but if we remove it, only the config is pushed not the version, and it takes a config change coming from the UI to get it published.

But with it here it is published twice. It may be better to handle explicitly in the MQTT connection code. I would also assume the same bug exists for all the xxx_version, that being said are these useful for MQTT if you are exposing the resources directly? They are meant so the version number is evented over MQTT or Websocket, and then you would fetch the resource over HTTP. With the config, etc being sent over MQTT directly the version is irrelevant.

But with it here it is published twice.

Yes I knew that...

It may be better to handle explicitly in the MQTT connection code.

Thought about that and no because the version number and config might not be published at mqtt connection time if evse is not started. So the initial publish will come from the loop.

I would also assume the same bug exists for all the xxx_version, that being said are these useful for MQTT if you are exposing the resources directly?

Not at all... There is no point in having have these *_version exposed in mqtt (IMO). Only exception is if someone wants to know hen a config change happened...

They are meant so the version number is evented over MQTT or Websocket, and then you would fetch the resource over HTTP. With the config, etc being sent over MQTT directly the version is irrelevant.

I agree.

So to keep the behaviour backward compatible but still avoid this duplicate publish, what I can do is check the version number is the mqtt publish function. If it is 1, it means it has to be published because this is the initial publish. If it is greater than 1, then there has been a config change, and the event will be responsible to publish.

I've just pushed ad95246 as a proposal to remove the duplication

mathieucarbou · 2023-05-21T21:46:46Z

rebased on master just now

src/mqtt.cpp

KipK reviewed May 9, 2023

View reviewed changes

src/web_server_config.cpp Outdated Show resolved Hide resolved

mathieucarbou force-pushed the config branch 3 times, most recently from 71fa0d9 to 7e393db Compare May 9, 2023 16:58

mathieucarbou requested a review from KipK May 9, 2023 16:59

jeremypoulter reviewed May 9, 2023

View reviewed changes

src/mqtt.cpp Outdated Show resolved Hide resolved

mathieucarbou force-pushed the config branch from 7e393db to 95dc691 Compare May 9, 2023 19:48