Feature Request: Add device config via loadable template specific file

[Jason2866]

Have you look for this feature in other issues and in the wiki?
Yes
Is your feature request related to a problem? Please describe.

Since the growing amount of devices not all devices can be added to Tasmota code
Describe the solution you'd like

Possibility to load a config file (template) which can be loaded via webfrontend to config Tasmota.
So everyone could share working device config templates for easy setting up his device

[blakadder]

As the voice of the average user I wholeheartedly support this feature and it would relieve a lot of stress on the support team (github and discord) as well as making wiki entries a simple image of the device and a link to the config file

[arendst]

Considering.

[arendst]

How about a command like this:

Template {"Name":"Sonoff Basic","Gpio":[17,255,255,255,255,0,0,0,21,56,255,0,0],"Flag":0x00}

With a name up to 14 characters, GPIO numbers as presented by the GPIO command and an optional flag as currently available in sonoff_template.h. The number 255 represents a user selectable configurable GPIO just as in sonoff_template.h.

No fancy upload process needed. Just a command.

[Jason2866]

Nice, for advanced users!
We have in Discord Chat many users with beginner skill. So we have to explain how to handle this command. What do you think about implementing this now and let us make some experience with.
If this isnt end user friendly enough, doing it additional/alternative via file upload?

[arendst]

Working on it. Might be ready by tomorrow.

[digiblur]

Awesome! It will definitely help with the flood of the new devices from Tuya convert. This is a great combination with the recent addition power monitoring pin selection via the GUI ! Easy to cover most of these plugs with Generic now and a simple Wiki page.

[arendst]

Takes some more time for testing as there are major changes needed to satisfy future user requests (I had a vision...).

Anyway I have it functional and am testing the final changes.

[arendst]

Give it a chance.

Command Template usage:

Template - Show current user template
Template 0 - Copy current module to user template
Template <module number> - Copy module to user template
Template {"NAME":"GenericTheo","GPIO":[17,243,29,244,7,201,255,255,138,255,139,255,255],"FLAG":0,"BASE":18} - Set a template. Any JSON field is optional.

Choose user template using command module 0 or use web GUI and select top module name

Usage:

• Load default Generic module config to user template by executing command template 18
• Using GUI select user module and Save
• Using GUI and make your GPIO choices and save
• To export template just execute command template and copy text to distribute.

Open for any minor changes suggestion.

[andrethomas]

Looks good @arendst :)

@jziolkowski feature request for Tasmota Device Manager - drop downs to push the JSON? :)

[netpok]

Looks great, but if I get it correctly you use this template with template jsonstring, but after you set a pin you cannot change it, because it wont be a user settable pin (USER_GPIO) anymore.

[arendst]

Did you try it?

[Jason2866]

@arendst Hope i will find time tomorrow to try!

[netpok]

@arendst Yes, I tried it, but now I found out the problem was on my side, I used Blitzwolf SHP as a base, because that was the only device in reach, with Generic device as base it can be changed.

[andrethomas]

I did a quick test setting a POW to generic and then template 6 - Should it behave like a POW after that?

Either that or I missed some step because its not showing the pin configs in the web ui...

[andrethomas]

Ah no I did something wrong... so false alarm :)

[andrethomas]

{"NAME":"Sonoff Pow","GPIO":[17,0,0,0,0,130,0,0,21,132,133,52,0],"FLAG":0,"BASE":6}

[Jason2866]

Works as designed. :-) Big Thx Real cool new feature!
If i had a wish ;-). Could you add Template loading and activating via "Upgrade by file upload" ?
Would be easiest way to share and activate ready made Templates (from the big Tasmota community)
for users with all skill levels. -> Hosting ready made Templates on a "central place" (Community will do!)
Once again thank you for this great "next big thing"

09:22:40 CMD: Template
09:22:40 MQT: stat/sonoff-D4533B3B/RESULT = {"NAME":"BlitzWolf SHP","GPIO":[57,255,56,255,0,134,0,0,131,17,132,21,0],"FLAG":0,"BASE":45}

[arendst]

Will add GUI upload too as suggested by @Jason2866

[arendst]

See https://github.com/arendst/Sonoff-Tasmota/blob/development/TEMPLATE.md for background information. Ready to be wikified by someone...

[jziolkowski]

@andrethomas

@jziolkowski feature request for Tasmota Device Manager - drop downs to push the JSON? :)

Definitely. I'm polling the firmware for available modules and gpios, so JSON Template output is easy-peasy.

[Jason2866]

@arendst Thx!
@jziolkowski @andrethomas @blakadder @digiblur Let us discuss in Discord how we can collect, provide templates and get the info to the users.

[meingraham]

@jziolkowski,

the whole concept opens the floodgates for people publishing templates everywhere

Isn't this basically already the case? The maker community has been adding new devices to the wiki without issue. Some have made their case for adding them to the official list of templates. But many have just added a new device page and then added a link in the right-hand menu bar.

It goes beyond that. For example, with the advent of Tuya OTA, many new devices are flooding in. Since those methods are using TASMOTA as the custom firmware of choice (at least for the initial OTA), there are many in the user base that are providing their configurations for others to leverage.

Of course it can all be abused. But my sense is that users are honorable and want to be helpful in providing their configurations. I think they will view adding templates in the same light.

people should be invited to share their verified and tested templates, so we can compile a list on the wiki which we can then provide support for

Who is we? With the volume of ESP based devices coming to market almost by the minute, it would be nigh impossible to have templates be moderated and yet keep up with new devices. If templates cannot be kept "up to date", the TEMPLATE feature becomes much less valuable.

Regards.

Mike

[balvant813]

Works great!
template {"NAME":"SP201","GPIO":[56,0,0,131,17,134,0,0,21,18,132,22,0],"FLAG":0,"BASE":18}

[Jason2866]

I like the suggestion from @andrethomas in compliance with the notes from @meingraham
Imho, configuration of device(s) should be possible without the need of internet access.
Complicates the set up process...

[netpok]

As I implied this would be an optional thing, by default the manual setup is always available, if the device is connected to the internet this should display as an additional field, where the user could select the template to use, if needed I could make a proof of concept solution on the weekend.

Not related to this, I suggest to make a base profile where IO pins are not settable and use that as base profile for any power meter device (As I saw the point of closing my ticket #5134), of course the users still can change the base profile to general, but it should discourage the use of those pins for those who lacks the basic understanding of the firmware.

[Jason2866]

@netpok PoC would be cool.

Important point!

Not related to this, I suggest to make a base profile where IO pins are not settable and use that as base profile for any power meter device (As I saw the point of closing my ticket #5134), of course the users still can change the base profile to general, but it should discourage the use of those pins for those who lacks the basic understanding of the firmware.

[arendst]

As templates can be uploaded using serial, http, mqtt and even GUI I see no reason to add overhead for a webpage allowing to upload a template.

The current version provides enough hooks for external tools like TasmoAdmin to change the template just like any other setting.

[arendst]

Oh and the already present templates will stay as removing any will defeat backward compatibility and some templates still need extra coding anyway. I like to stop adding templates that are just a deviation of the already available templates like many energy monitoring ones. They will need to use the new template feature.

I agree with @meingraham and @andrethomas to have an open repository of templates as there is just no possibility to test them all. Let the user decide what works best. The proposed layout is fine but it doesn't look good in a webpage I guess. Adding a picture will add a lot of space between single entries making the list a very long webpage.

[blakadder]

if it is a table layour then it is possible to use pictures, since using names is a fruitless endeavour with all the mushroom brands derived from tuya based products

like this

stolen from esphomeyaml docs

[netpok]

My idea of the repository includes an interface to add new devices with two features:

• When there are pins defined as power metering, it sets the base module to an unchangable one
• It doesn't allow adding new devices with the same pin configurations thus limits the infinite crap

[netpok]

@arendst The main goal of my proposal is to not add overhead to the project (okay there is a minimal overhead of including it like 25-50 characters in the html code), it should use already existing endpoints and any required elements are loaded externally, if there are network connection available on the browser, the "sonoff" doesn't even need to have access to the internet.

[arendst]

@netpok You'll get 52 bytes of code space ;-)

[meingraham]

Adding a picture will add a lot of space between single entries making the list a very long webpage.

I was proposing a link to a photo, not an img... because I agree that the page would get unruly quickly... Although this suggestion from @blakadder looks "tidy". Nevertheless, once you get dozens of devices, it'll probably be too unruly.

Mike

[jziolkowski]

@arendst minor detail: can the JSON reply for the command "template" have a less ambiguous name? I'd suggest "TemplateName" instead of "NAME". Makes it easier for developers of external tools ;)

[blakadder]

@arendst minor detail: can the JSON reply for the command "template" have a less ambiguous name? I'd suggest "TemplateName" instead of "NAME". Makes it easier for developers of external tools ;)

Is this being considered? Would be good to know before a lot of documentation is made!

[arendst]

I want to keep it as small as possible so for now I won't change it.

[jziolkowski]

TplName? :)

[arendst]

Kind questions: What is the need for this name change? When is it a problem it is called NAME instead of TPLNAME?

[jziolkowski]

Ok, meanwhile @blakadder over discord pointed me to template.md. I see that the keywords have meaning, so now it makes sense. As long as there will be no other /RESULT reply with JSON key NAME inside, then it's fine I guess. I'm just used to other key names in Tasmota to be more self-explanatory, that's all.

[arendst]

You can change the /RESULT with SetOption4 1 to, in this case /TEMPLATE if that makes sense.

[jziolkowski]

It makes, yes. But doesn't solve my "issue" because I assume 99% of users will have it the standard way. But ok, it's all clear now, thx.

[netpok]

As promised I created a proof of concept implementation of the device database integration. As it is not meant to be merged I did not create a pull request from it, should I?

It is currently available at my fork: https://github.com/netpok/Sonoff-Tasmota/tree/online-device-database
I have made it available as a precompiled version with ota update support at: http://netpok.ga/sonoff/sonoff.bin

Currently it doesn't have any fancy features like searchable selects and the database behind it, but it has two device configs for testing based on the blitzwolf SHP smartplugs.

Here's a screenshot:

And last but not least it currently does not comply with the 52 byte code limit, but it's managable I think.

[arendst]

As POC I see you managed to integrate a complete jQuery example with just one line of code! This opens up a whole new kind of client / server functionality (and can of worms).

Nice idea and could even lead to much smaller web code in the device but it will depend on a (local or remote) server.

For now I go for the new, local template menu but am open to this non-local server idea.

[netpok]

I never intended to offload any part of the existing code to an external server as it would lead to loss of functionality without internet. jQuery was included to speed up the development, in the final version it could be replaced with native code if required.

The can of worms of course should be locked away behind SRI and HSTS, so they can't come out :)

I like this local template menu, but I was not able to find it, is it still work in progress?

[arendst]

Yes it is WIP but I hope to release today.

I need to tune the Ajax part in the near future as currently it uses functionality based on my then simple knowledge of updating pages without rewrites.

[netpok]

Okay, also the required JS code on the device can be further reduced by using ES6, but this would lead to the requirement for modern web browsers. Here is a compatibility table (but probably the basic functions are supported in some of the earlier versions too):

[blakadder]

TBPH if the webui configuration is structures this way there's really no need to include the web downloading component. This is perfectly straightforward, it is in continuation with the existing gpio configuration so almost no learning curve and it leaves the user free to experiment and change the template according to his needs

[jziolkowski]

@arendst let's discuss the JSON template you have in mind in a client-server scenario. I could write a standalone one (to the tune of tasmota ota fw-server.py) or integrate it into https://github.com/jziolkowski/tdm

EDIT: or expand the fw-server to become ota-template-server.py

[arendst]

@jziolkowski too early. It was just a brainwave after seeing the possibilities provided by @netpok.

If regarding the current template functionality it would be just that. The released JSON message as provided by the template command.

[jziolkowski]

Got it. But in case you decide it's a plausible idea, I'm up for the task.