Gather serial number

[gtdiehl]

Home Assistant can remove single entities but the sensor needs to implement a unique ID. This would be helpful when an inverter is removed from the physical system but still reports as part of the envoy inventory.

Since inverter gathering is all or nothing, the filtering would be done through Home Assistant.

One way to retrieve the serial number is through an authenticated page /prov just not sure if all firmwares support this page.

[lnlp]

It's not exactly clear to me what the issue is. Can you elaborate?

One way to retrieve the serial number is through an authenticated page /prov just not sure if all firmwares support this page.

My firmware (R4.10.35 (6ed292)) reports it but requires installer permissions to read it (I can read the contents).

Serial number of what?
What exactly would /prov be needed for?
Could /inventory.json be an alternative?

[gtdiehl]

It's not exactly clear to me what the issue is. Can you elaborate?

The real issue is how can we filter out dead/removed inverters from Home Assistant. To do that we have to implement a unique_id() method in the Enphase Envoy sensor code in Home Assistant. Per the Unique ID requirements the ID (or any text that makes up the ID) should not be "possible for the user to change the unique ID".

The unique ID can be implemented right now for the Inverter sensor entities has we get the Serial Number for each of the inverters, but right now we don't get the Serial Number for the Envoy device itself (other then when we retrieve /info.xml when getting inverter data).

My thinking was one way is to get the serial number is from /prov but if the installer credentials are needed then this is probably not a viable solution. Another thought of mine was just to reuse /info.xml as the only use case (that I can think of when filtering entities in Home Assistant) is to filter out dead/removed inverters and if you want to filter out these inverters than credentials are already given or retrieved, so we can than create a unique ID for the Envoy device it self.

I just would want to find a cleaner solution to get the Envoy serial number to give each sensor in Home Assistant a unique ID regardless if your getting inverter data or not. The only way I can do this is by scraping the /home page, which I'm not really a fan of!

Serial number of what?

Serial number of Envoy device

What exactly would /prov be needed for?

To get the serial number, not sure what your output looks like but/provon my device gives the serial number. But not a viable solution if installer credentials are needed and we are already using /info.xml to get the serial number anyway.
{ "envoy_sn": "1213xxxxxxxx" ... }

Could /inventory.json be an alternative?

Older Envoys don't support this page 😞

[rct]

@gtdiehl - I'm having a hard time following the issues you are bring up. Let's try to separate a few things:

1. Is the inverter serial number "unique" enough?

From what I've seen the envoy and each inverter have a serial number set by Enphase which look like they come from the same pool of numbers. which seem to be currently in the 40 bit range, mostly due to a prefix in my case of 121, which is probably some sort of name space partitioning.

Since unique ID appears to be a string within Home Assistant, What would be the issue with just using the following to put Enphase serial numbers into a partition of their own?

unique_id = "enphase" + "-" + serial_number

Do you have a reason to believe Enphase might reuse the serial numbers?

Do you think the serial number might in some way be a property of the Envoy and might change if the Envoy was replaced?

---

2. Using the Envoy's serial number to make things more unique

If I follow correctly you were suggesting also using the Envoy's serial number to make things more probabilistically unique like:

unique_id = "enphase" + "-" +envoy_serial + "-" + serial_number

While this would certainly make things more unique, it would have the side effect of changing all of the inverter unique IDs if the Envoy ever had to be replaced. Off the top of my head, I don't think you'd want to disassociate any/all of the inverter history you have if you have to replace the Envoy.

---

3. Getting and using the Envoy's serial number

Separately, What are the issues with getting and using the Envoy serial number from info.xml?

IIRC info.xml is fetched unless the Envoy password is supplied by the user. Any reason not to always attempt to fetch info.xml when the Envoy reader is initialized?

(As I mentioned separately, the envoy sensor created by Envoy Reader, should publish the Envoy serial number, and firmware version info as attributes of the Envoy sensor, so you can see that info within Home Assistant.)

---

4. detecting dead inverters

A timestamp of the last time an inverter reported lastReportDate is part of the data returned by /api/v1/production/inverters. Wouldn't it be possible to filter out inverters that haven't reported within some appropriate interval (2 days, 7 days, ...)?

(As I mentioned in another issue, this data should be available within Home Assistant so that it's possible to do alerting, etc.)

[gtdiehl]

@rct I'll try and answer your points, but before I do maybe I can explain my rationale a different way.
What I'm looking for is a way to retrieve the Envoy serial number that works across all firmware versions and does not require user credentials.

Is the inverter serial number "unique" enough?

Yes the Inverter serial number is unique enough for each inverter, but not for the other entities that are created for the other monitored conditions such as production, daily_production, etc...

Do you have a reason to believe Enphase might reuse the serial numbers?
Do you think the serial number might in some way be a property of the Envoy and might change if the Envoy was replaced?

Currently the serial of the Envoy device itself is only retrieved when the inverter data is gathered. And when we do get the serial number we only save the last 6 digits. So currently we really are not storing the entire Envoy serial number. I'm not looking at changing Envoy serial numbers use case as I assumed this value will not change. Replacing the Envoy with another Envoy but with a different serial, well I would expect in that use case that historical data for the old Envoy will not be joined with the new Envoy data in Home Assistant.

Using the Envoy's serial number to make things more unique

My thought (after getting the Envoy serial number) is to really have two ways of creating a unique id. Here name is the name that you see in Home Assistant and type is the list of monitored conditions such as production, inverters, etc...

if type != 'inverters':
    unique_id = name + "_" + envoy_serial_number
else:
    unique_id = name

Or something along those lines, as the inverters already have a serial number. The other entities will just have the Envoy serial number appended to the name for the unique_id() method

Getting and using the Envoy's serial number

No issues, I just want to find a possible solution without using authentication. If it is not possible than it just maybe a known limitation.

I understand you have mentioned the firmware version and other attributes but as you know we are limited by what the Envoy pages return, and I would not want to scrape data or force the user to get basic information that is from a page that requires authentication. With some Envoys you are locked out of the device by Enphase as the default password is not the last 6 digits of the serial number, such as my device. Here is a link to a tool that I had to use to get my password; Enphase Password

detecting dead inverters
Rather than having the API act as an abstraction layer to filter out data, I believe that should be up to the application. Which is than either part of the sensor.py code in Home Assistant or to utilize features and functions within Home Assistant.

A notification implementation in Home Assistant would be nice but for the user to remove the dead inverter we still have to implement the unique_id() method in the sensor.py code, which means we need to solve the original problem of getting the Envoy device serial number.

If we don't have a clean way to get the Envoy serial number that would work across all firmwares than maybe we can have some limitations?

• Filtering entities in Home Assistant will only occur if the correct credentials are provided in the configuration.yaml
• Filtering entities in Home Assistant will only work with firmware >=3.9 (I think the firmwares <3.9 don't have the inverter page?)

[rct]

Yes the Inverter serial number is unique enough for each inverter, but not for the other entities that are created for the other monitored conditions such as production, daily_production, etc...

Ok, I think I have a better understanding now. This is probably easier to discuss setting aside the individual inverter discussion for now. Looking at the MQTT sensors I'm creating via rtl_433 and MQTT auto discovery, I see what you are talking about and how it should probably work.

In order to be able to use the Home Assistant UI to configure, customize, or delete entities (and devices), the entities must each have a unique ID that doesn't change. Currently, this gives the user the option of changing the name or icon, or disabling or deleting the entity from the Hass UI.

Additionally, there should be a unique ID for the Envoy device which will enable the HA entities created by Envoy Reader such as production, consumption, etc. sensors to be grouped in the UI as belonging to the parent Envoy device and enable some amount of customization.

By tying the entities together to the parent device, a device page is then available in the UI like this:

I understand you have mentioned the firmware version and other attributes but as you know we are limited by what the Envoy pages return, and I would not want to scrape data or force the user to get basic information that is from a page that requires authentication.

No scraping should be required. My Envoy IQ (and I'm assuming at least recent versions of the S) return the serial number and firmware version from /info.xml without any authentication.

I have to assume the availability of /info.xml is widespread enough that the code was written to guess the envoy password by obtaining the serial number from it. Envoy reader does not attempt HTTP auth for fetching /info.xml

Currently the serial of the Envoy device itself is only retrieved when the inverter data is gathered. And when we do get the serial number we only save the last 6 digits. So currently we really are not storing the entire Envoy serial number. I'm not looking at changing Envoy serial numbers use case as I assumed this value will not change.

Only 6 digits are saved because that is what is needed for the default password for the envoy account. That implementation looks like to be "private" to Envoy reader so storing the full serial number, and then only using the last 6 digits for the default password, shouldn't really have an effect outside of the envoy reader code unless I'm missing something. Does so would require envoy reader to attempt to fetch info.xml, during initialization whether inverter data is requested or not.

Some ideas:

A user configuration setting could be added to set the Envoy serial number if it can't be obtained.

A fallback if both of those options fail, if the user specifies the envoy IP address or hostname in the configuration, it could be assumed to be fairly static and used as part of the Envoy's unique ID string. Is hostname/IP address what is meant by name in the code you posted above?

If all of those fail, then just completely skip trying to create a unique ID for the entities, If that makes the code to complex, just create a static string. I'm sure the majority of installations will only have a single Envoy so it should be unique enough. It only needs to be unique within a Home Assistant instance not a globally unique identifier. (Does the current envoy code work if you wanted to have more than one Envoy polled in Home Assistant?)

With some Envoys you are locked out of the device by Enphase as the default password is not the last 6 digits of the serial number, such as my device. Here is a link to a tool that I had to use to get my password; Enphase Password

• So on your Envoy is HTTP auth required for /info.xml?
• For clarification, there seem to typically be two user accounts on most Envoys, username envoy and username installer. The Android APK you linked to guesses the default installer password which is based on the Envoy serial number through some possibly crypto manipulation.
• The code in Envoy reader only attempts to do HTTP auth for /api/v1/production/inverters. My understanding is that page is available to either the envoy account or the installer account. So the current envoy reader code tries to use the envoy account with the default serial number based password. There seems to be the option to change the envoy account password. I guess some installers might change that password as an attempt to lock customers into dealing with them, or security minded users might change it, but then they know to add it to their config.
• The /stream/meter page requires the installer account as does the /prov page. Neither are currently used by envoy reader.

[lnlp]

A fallback if both of those options fail, if the user specifies the envoy IP address or hostname in the configuration, it could be assumed to be fairly static and used as part of the Envoy's unique ID string

Relying on an assumption that IP address or hostname 'are fairly static' is a 'dangerous' assumption and bad practice.
I have changed both at least once already and probably will again in the future.

Neither IP address nor hostname should be used for creating a unique ID string (unless that ID can be changed at later moment, in a very simple manner, clearly visible to the user and without any further consequences).