Timeseries and Location Group/Category Naming and Standards

[krowvin]

A question was brought up about whether or not the Timeseries / Location groups/categories had standard naming conventions in the document.

Other talking points would be:

1. Should there be a CDA named group(s) and/or categories for every office that we can state here for public/others to know which timeseries they should use across the nation?
2. Should a standard be created for all names, not just a CDA name?
3. Should we add documentation here to discuss these, why you might use them, and which ones to use (assuming 1. is agreed upon)?

I.e.

curl -X 'GET' \
  'https://cwms-data.usace.army.mil/cwms-data/timeseries/group?office=NAE' \
  -H 'accept: application/json'

or

curl -X 'GET' \
'https://cwms-data.usace.army.mil/cwms-data/timeseries/group?office=SWT' \
-H 'accept: application/json'

[DanielTOsborne]

What's the use case for groups?
Aside from A2W, I don't know what I'd need them for, outside of CCP, which has its own groups.

The timeseries naming is specified in the "WMIST CWMS Standard Naming Conventions" document (even though that document has flaws).

The biggest issue CDA has IMO is that the catalog call returns TS that have no data (either never did, or isn't synced via goldengate).
You can get a catalog of all TS at a location, and 90% of them are empty. That's not useful.

[krowvin]

The biggest issue CDA has IMO is that the catalog call returns TS that have no data

Agreed, and in my opinion it should be a higher priority to remove these from all results. I understand this may be complicated, and so it is easier said than done.

Another discussion was brought up whether all groups/categories should be returned or if there should be filtering for those as well.

[rma-rripken]

Timeseries groups/categories and Location groups/categories powered the old cwms-mobile sites. Its a very versatile feature for building db backed apps. It allowed districts to organize the data displayed by cwms-mobile however they felt it made the most sense. One office could organize locations by Lakes, Rivers, Stream Gages, Weather. Another office could decide they wanted things organized by Basin. That the assignments to the groups could be ordered was huge. cwms-mobile linked a ts-group to a location-group assignment which made it possible for someone to setup the relevant and important ts for a location. The database then knows that for a specific site these x timeseries were the ones that actually had useful data or were the ts that should be displayed to the public or to users. The plotting config could also be stored in the database using the text/properties api. The website didn't have to change, the configuration was in the database. Someone could tweak the assignments and suddenly cwms-mobile had a new group or a new plot. After we had cwms-mobile working someone from SWT (If I remember correctly) saw how cwms-mobile was working and made their own Adobe Flex app by using their own groups/categories and streaming from the cwms-mobile server. That validated the RADAR/CDA concept.
I always had a worry with the cwms-mobile Category/Groups stuff that someone else would mess with the categories or groups and break the sites. It never happened but I was always concerned about it and made sure the groups were named with a "CWMS Mobile" prefix and I saved the scripts used to setup the categories/groups so that they could be easy re-run. But again, the site never broke in that way.

[rma-rripken]

The biggest issue CDA has IMO is that the catalog call returns TS that have no data (either never did, or isn't synced via goldengate).
You can get a catalog of all TS at a location, and 90% of them are empty. That's not useful.

If that is the biggest issue CDA is doing pretty well. If there isn't any data the ts catalog returns empty extents "[]"- right? Seems trivial to filter those out on the client side. A flag to filter them out on the server-side would be easy. As to why are they returned? They exist in the database and some api behave differently if the underlying ts exists or not so it seems like CDA has to return them.

[DanielTOsborne]

It's more of an issue when trying to document CDA to external users that are requesting data.

I can show them how to fetch locations, and fetch timeseries list, then fetch data, but there's no way to indicate which timeseries are actually relevant.
That makes the location/timeseries catalog calls useless, since if I have to tell the user which timeseries to fetch, then they don't need the catalog calls.

EDIT: What we need is a way to filter for Active and "In Upward Reporting" flags. Without those, there's no way to distinguish between a timeseries that is truly empty/unused (or not in upward reporting), and one that is just missing data in the requested range.