Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extend ElasticSearch proxy to filter out elements defined in the schema filters configurations #6869

Merged
merged 6 commits into from
Aug 30, 2023

Conversation

josegar74
Copy link
Member

Currently the metadata schemas allow to define filters defined in schema-ident.xml file, to remove certain information from the metadata documents if certain conditions apply.

For example in iso19139, if the user has no editing permission the elements with the attribute @gco:nilReason='withheld' are removed from the XML output and the full view formatter:

<filters xmlns:gco="http://www.isotc211.org/2005/gco">
<filter xpath=".//*[@gco:nilReason='withheld']"
ifNotOperation="editing">
<keepMarkedElement gco:nilReason="withheld"/>
</filter>

The attribute @gco:nilReason='withheld' should be added in update-fixed-info.xsl process of each metadata schema to mark which elements should be removed. A typical usage is to remove certain type of contacts for users that can't not edit the metadata record.

This pull request extends this support to filter out these elements from the ElasticSearch response. The indexing process has to be extended to handle the @gco:nilReason='withheld' for the related ElasticSearch fields (these fields should be object fields). An example for contacts is part of the pull request:

https://github.com/geonetwork/core-geonetwork/compare/withheld-es?expand=1#diff-e5443fc6e8489ecc9eb314b5c9f9a3e2e9c4bb0c85888bade22d731edf0f08ea

A new type of operation has been added (authenticated) to allow to remove information only for non-authenticated users. For example, if defined the following in schema-ident.xml file, the elements with @gco:nilReason='withheld' will be removed for non authenticated users that can view the metadata, but will be preserved for authenticated users that can view the metadata :

<filter xpath=".//*[@gco:nilReason='withheld']"
           jsonpath="$.*[?(@.nilReason == 'withheld')]"
           ifNotOperation="authenticated">
    <keepMarkedElement gco:nilReason="withheld"/>
</filter>

@josegar74 josegar74 added this to the 4.2.3 milestone Feb 27, 2023
@josegar74 josegar74 marked this pull request as draft February 27, 2023 15:16
@josegar74 josegar74 changed the title Extend ElasticSearch proxy to filtered out elements with the withheld attribute Extend ElasticSearch proxy to filter out elements with the withheld attribute Feb 27, 2023
@josegar74 josegar74 changed the title Extend ElasticSearch proxy to filter out elements with the withheld attribute Extend ElasticSearch proxy to filter out elements defined in the schema filters configurations Feb 27, 2023
@sonarcloud
Copy link

sonarcloud bot commented Mar 9, 2023

SonarCloud Quality Gate failed.    Quality Gate failed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 3 Code Smells

0.0% 0.0% Coverage
0.0% 0.0% Duplication

@fxprunayre fxprunayre modified the milestones: 4.2.4, 4.2.5 May 10, 2023
…he documentStandard field to check the schema config.
…Add nilReason for links (and not only contact).
@fxprunayre fxprunayre marked this pull request as ready for review June 6, 2023 08:46
@fxprunayre fxprunayre added the index structure change Indicate that this work introduces an index change. label Jun 6, 2023
@sonarcloud
Copy link

sonarcloud bot commented Jun 6, 2023

SonarCloud Quality Gate failed.    Quality Gate failed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 4 Code Smells

0.0% 0.0% Coverage
0.0% 0.0% Duplication

@fxprunayre fxprunayre modified the milestones: 4.2.5, 4.2.6 Jul 7, 2023
@josegar74 josegar74 modified the milestones: 4.2.6, 4.4.0 Aug 30, 2023
fxprunayre added a commit to geonetwork/geonetwork-microservices that referenced this pull request Aug 30, 2023
Added by geonetwork/core-geonetwork#6869 (review)


Error was:
```
29/8/2023 19:28:02com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException: Unrecognized field "nilReason" (class org.fao.geonet.index.model.gn.Link), not marked as ignorable (8 known properties: "protocol", "applicationProfile", "nameObject", "descriptionObject", "mimeType", "group", "function", "urlObject"])
29/8/2023 19:28:02 at [Source: (StringReader); line: 1, column: 34563] (through reference chain: org.fao.geonet.index.model.gn.IndexRecord["link"]->java.util.ArrayList[0]->org.fao.geonet.index.model.gn.Link["nilReason"])
```
@josegar74 josegar74 merged commit 9f43df8 into main Aug 30, 2023
juanluisrp pushed a commit to geonetwork/geonetwork-microservices that referenced this pull request Aug 30, 2023
Added by geonetwork/core-geonetwork#6869 (review)


Error was:
```
29/8/2023 19:28:02com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException: Unrecognized field "nilReason" (class org.fao.geonet.index.model.gn.Link), not marked as ignorable (8 known properties: "protocol", "applicationProfile", "nameObject", "descriptionObject", "mimeType", "group", "function", "urlObject"])
29/8/2023 19:28:02 at [Source: (StringReader); line: 1, column: 34563] (through reference chain: org.fao.geonet.index.model.gn.IndexRecord["link"]->java.util.ArrayList[0]->org.fao.geonet.index.model.gn.Link["nilReason"])
```
@fxprunayre fxprunayre deleted the withheld-es branch September 25, 2023 08:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
index structure change Indicate that this work introduces an index change.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants