Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consider having field value sub-fields #51

Open
spbnick opened this issue Dec 9, 2016 · 2 comments
Open

Consider having field value sub-fields #51

spbnick opened this issue Dec 9, 2016 · 2 comments
Milestone

Comments

@spbnick
Copy link
Member

spbnick commented Dec 9, 2016

As Elasticsearch doesn't support arrays with values of different types, it will be a problem to store raw and interpreted values with different types in it, as currently planned.

We need to either have everything as strings, which wouldn't work for searching some quantitative field values, or have subfields for some interpreted fields.

@spbnick spbnick added this to the Schema proofed milestone Dec 9, 2016
@spbnick
Copy link
Member Author

spbnick commented Dec 9, 2016

We need to also try indexing them as Multi Field Types.

Say, have "keyword" and "long" subfields for each.

We need to balance the raw JSON size and increasing default querying difficulty of having subfields in source JSON, but more specific mapping vs index size increase of having subfields in Elasticsearch.

@spbnick
Copy link
Member Author

spbnick commented Dec 9, 2016

Another good idea can be to have one field subfield per value type. So, if the raw value is string then JSON would have "s" subfield, and XML would have "s" attribute. This way both schemas and mappings can be greatly simplified. We would still need to store the knowledge of what field of what record is what type, but at least the schemas and mappings would be simpler.

@spbnick spbnick modified the milestones: v1, Schema proofed Dec 16, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant