Consistent empty/missing behavior

[dmlloyd]

This issue is to document an ongoing discussion. See the links at the bottom for background before commenting with issues, because odds are good that the problem has been raised before.

Use cases

Force empty values

It is not possible (from a higher-priority configuration source) to force a configuration value (from a lower-priority configuration source) to be considered non-present or empty.

As an application user, I should be able to specify that a configuration property is explicitly empty, overriding any lower-priority value configured for that property.

The issue ID for this use case is #531.

Consistent handling

Empty and missing value handling is not consistent. The inconsistency is characterized by special treatment of certain configuration types, particularly array and String, which produce somewhat bizarre effects when retrieving empty values compared to missing values. Special cases in general APIs lead to confusion and frustrating limitations.

As an API user, I should have a consistent experience handling values that are empty or missing that does not need to vary depending on the conversion type.

The issue ID for this use case is #532.

Natural representation

Some value types, such as collections, arrays, and strings, have a "natural" empty representation. However some value types do not. In some cases (like primitive values), no such representation is possible, where in other cases, null is the only possible representation. There should be one value that can represent empty for any value type, whether it is a collection, simple object, or primitive type.

As an API user, I should be able to detect that a property is empty regardless of its type.

As an API user, I should be able to decode an empty value into the target type in a way that is appropriate for the use site.

As an API user, I should not need to be concerned with whether a collection with a given element type can or cannot contain an empty value.

As a type converter implementer, I need to be able to indicate whether a given input string corresponds to an empty value.

The issue ID for this use case is also #532.

Default values

Sometimes a property needs to have a value that is returned if no configuration source has explicitly defined what the value of the property should be. This is commonly referred to as a "default value".

As an application developer, I expect to be able to specify the default value for a configuration property relating to my application which is used in any API usage context (i.e. not just in CDI).

As an API user, I expect to receive the default value for a configuration property when no configuration source specifies a value for the property and when no configuration source explicitly clears the property.

The issue ID for this use case is #533.

Future: Property expansion

An ongoing, separate proposal for this specification relates to property expression expansion. Because of the currently inconsistent behavior relating to empty and missing values, a number of implementation problems arise when it comes to empty expressions, particularly around arrays and lists. It is therefore important that the handling of these use cases be "ready" for property expression support.

As a user, I expect that a configuration property whose value is an expression, where the expression is empty, would be treated identically to a property which has an empty value.

As a user, I expect that a list or array property value containing expressions which resolve to empty values not contain empty segments, so that I can conditionally supplement list or array values with expression values as needed.

Tasks

The use cases imply the following tasks:

• Specify that giving a property an empty value forces it to be considered "non-present"
• Provide one single value that can represent "non-present" for any type, even those types which do not have a natural "empty" representation (such as integer values)
• Clarify NoSuchElementException to indicate that non-present values were disallowed but a given value was not present (with no default value) or was explicitly forced to be non-present by being given an empty value
• Clarify Optional to indicate that the property value is allowed to be implicitly or explicitly non-present, and that non-present values correspond to Optional.empty(), and that the orElse method must be used to decode type-specific empty representations if that is desired
• Specify that Converter returns null to indicate that the converted value is empty
• Specify that all built-in converters return null when the given input string is an empty string
• Define "empty" for strings to mean "an empty string"
• Define "empty" for arrays and collections to mean "an array or collection consisting only of zero or more empty values"
• Clarify that array and collection converters must always return either non-empty collections with no empty values, or null to indicate that the overall value is empty

Mapping of behaviors from old to new

Id	Input string	Output type	Method	Legacy rules	Updated rules
1	missing	String[]	getValue	throws NoSuchElementException	throws NoSuchElementException
2	""	String[]	getValue	{ }	throws NoSuchElementException
3	","	String[]	getValue	{ "", "" }	throws NoSuchElementException
3(b)	"\,"	String[]	getValue	{ "," }	throws NoSuchElementException
4	",,"	String[]	getValue	{ "", "", "" }	throws NoSuchElementException
5	"foo,bar"	String[]	getValue	{ "foo", "bar" }	{ "foo", "bar" }
6	"foo,"	String[]	getValue	{ "foo", "" }	{ "foo" }
7	",bar"	String[]	getValue	{ "", "bar" }	{ "bar" }
8	" "	String[]	getValue	{ " " }	{ " " }
9	"foo"	String	getValue	"foo"	"foo"
10	""	String	getValue	""	throws NoSuchElementException
11	","	String	getValue	","	","
12	missing	String	getValue	throws NoSuchElementException	throws NoSuchElementException
13	"foo"	String	getOptionalValue	Optional.of("foo")	Optional.of("foo")
14	""	String	getOptionalValue	Optional.of("")	Optional.empty()
15	missing	String	getOptionalValue	Optional.empty()	Optional.empty()
16	""	String[]	getOptionalValue	Optional.of({})	Optional.empty()
17	","	String[]	getOptionalValue	Optional.of({ "", "" })	Optional.empty()
18	",,"	String[]	getOptionalValue	Optional.of({ "", "", "" })	Optional.empty()
19	missing	String[]	getOptionalValue	Optional.empty	Optional.empty()

For more information

For more information and background, see also #397, #407, and the weekly meeting minutes.

[dmlloyd]

I'm thinking of an "option 3" let's call it "option 2-prime" which addresses the concerns that have been raised about the confusing semantics of NoSuchElementException, the problems with unintuitive empty values, and the problems around default values. I'm not sure if it's feasible yet though and it will take me a little while to work up and test the actual change. But stay tuned...

[dmlloyd]

I haven't been able to make the "option 2-prime" idea work. It always introduces cases where the user of the configuration file can get confusing different behavior based on the preferences of the configuration API consumer. So I'm scrapping that idea for now. The #445 PR still represents the best available solution for this problem.

[dmlloyd]

I've updated this issue to very clearly delineate use cases and requirements with the hope that this change can get into 2.0.

The only way to move forward on this issue is to do so progressively: first, discuss each use case to ensure that they are understood and agreed upon. Second, agree that the use cases imply the given requirements, updating them accordingly per discussion. Third, ensure that there are no existing use cases that would be compromised by the change.

Only once these steps are done can proposed solution(s) be reviewed. If we have not yet agreed on use cases or requirements before reviewing the implementation, everyone's time will be wasted as we go in circles and this specification will continue to languish, so please, let's try to stay disciplined and organized to move this forward. These changes are necessary for Quarkus so I'm hoping we can successfully bring this to the greater specification community without the chaos that has plagued this specification thus far.