Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BEAM-11322] Apache Beam Example to tokenize sensitive data #13995

Merged
merged 34 commits into from
Mar 31, 2021
Merged

[BEAM-11322] Apache Beam Example to tokenize sensitive data #13995

merged 34 commits into from
Mar 31, 2021

Commits on Jan 11, 2021

  1. [WIP] Transfer from DataflowTemplates to Beam

    Artur Khanin committed Jan 11, 2021
    Configuration menu
    Copy the full SHA
    45ee19f View commit details
    Browse the repository at this point in the history

  2. Move to beam repo

    daria-malkova committed Jan 11, 2021
    Configuration menu
    Copy the full SHA
    ba046cb View commit details
    Browse the repository at this point in the history

Commits on Jan 12, 2021

  1. moved convertors for GCSio

    daria-malkova committed Jan 12, 2021
    Configuration menu
    Copy the full SHA
    df5881f View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' into DataTokenizationExample

    @ilya-kozyrev
    ilya-kozyrev committed Jan 12, 2021
    Configuration menu
    Copy the full SHA
    e6652ab View commit details
    Browse the repository at this point in the history

  3. Renaming + readme

    @ilya-kozyrev
    ilya-kozyrev committed Jan 12, 2021
    Configuration menu
    Copy the full SHA
    1e9c336 View commit details
    Browse the repository at this point in the history

Commits on Jan 13, 2021

  1. build errors

    daria-malkova committed Jan 13, 2021
    Configuration menu
    Copy the full SHA
    76accdc View commit details
    Browse the repository at this point in the history

  2. Merge remote-tracking branch 'origin/DataTokenizationExample' into Da… 

    …taTokenizationExample

# Conflicts:
#	examples/java/src/main/java/org/apache/beam/examples/complete/datatokenization/DataTokenization.java
#	examples/java/src/main/java/org/apache/beam/examples/complete/datatokenization/transforms/DataProtectors.java
#	examples/java/src/main/java/org/apache/beam/examples/complete/datatokenization/utils/JavascriptTextTransformer.java
    daria-malkova committed Jan 13, 2021
    Configuration menu
    Copy the full SHA
    e464202 View commit details
    Browse the repository at this point in the history

  3. minimize suppress

    daria-malkova committed Jan 13, 2021
    Configuration menu
    Copy the full SHA
    8782cba View commit details
    Browse the repository at this point in the history

Commits on Jan 14, 2021

  1. remove UDF usages

    @ilya-kozyrev
    ilya-kozyrev committed Jan 14, 2021
    Configuration menu
    Copy the full SHA
    46805f3 View commit details
    Browse the repository at this point in the history

  2. Fixes for stylechecks

    Artur Khanin committed Jan 14, 2021
    Configuration menu
    Copy the full SHA
    d9a8e33 View commit details
    Browse the repository at this point in the history

  3. grooming for data protectors

    @ilya-kozyrev
    ilya-kozyrev committed Jan 14, 2021
    Configuration menu
    Copy the full SHA
    76af970 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'DataTokenizationExample' of github.com:akvelon/beam int… 

    …o DataTokenizationExample

� Conflicts:
�	examples/java/src/main/java/org/apache/beam/examples/complete/datatokenization/transforms/DataProtectors.java
    @ilya-kozyrev
    ilya-kozyrev committed Jan 14, 2021
    Configuration menu
    Copy the full SHA
    e556e37 View commit details
    Browse the repository at this point in the history

  5. grooming for data protectors

    @ilya-kozyrev
    ilya-kozyrev committed Jan 14, 2021
    Configuration menu
    Copy the full SHA
    bfcba7f View commit details
    Browse the repository at this point in the history

  6. fix javadoc

    @ilya-kozyrev
    ilya-kozyrev committed Jan 14, 2021
    Configuration menu
    Copy the full SHA
    cfa41a7 View commit details
    Browse the repository at this point in the history

Commits on Jan 15, 2021

  1. Added support for window writing; Fixed ordering in tokenization process

    Artur Khanin committed Jan 15, 2021
    Configuration menu
    Copy the full SHA
    c8200f4 View commit details
    Browse the repository at this point in the history

  2. supressed checkstyle errors for BigTableIO class

    @ramazan-yapparov
    ramazan-yapparov committed Jan 15, 2021
    Configuration menu
    Copy the full SHA
    6e8f62f View commit details
    Browse the repository at this point in the history

Commits on Jan 18, 2021

  1. add data tokenization tests

    daria-malkova committed Jan 18, 2021
    Configuration menu
    Copy the full SHA
    b489cc7 View commit details
    Browse the repository at this point in the history

  2. Changed GCS to FileSystem and removed redundant function from Schemas… 

    …Utils
    Artur Khanin committed Jan 18, 2021
    Configuration menu
    Copy the full SHA
    ee93d14 View commit details
    Browse the repository at this point in the history

Commits on Jan 19, 2021

  1. Updated README.md for local run with BigQuery sink

    Artur Khanin committed Jan 19, 2021
    Configuration menu
    Copy the full SHA
    4b59ed9 View commit details
    Browse the repository at this point in the history

Commits on Jan 20, 2021

  1. add docstring

    @ilya-kozyrev
    ilya-kozyrev committed Jan 20, 2021
    Configuration menu
    Copy the full SHA
    2e0efe0 View commit details
    Browse the repository at this point in the history

Commits on Jan 22, 2021

  1. Updated README.md

    Artur Khanin committed Jan 22, 2021
    Configuration menu
    Copy the full SHA
    3136249 View commit details
    Browse the repository at this point in the history

Commits on Jan 25, 2021

  1. Updated README.md and added javadoc for the main pipeline class

    Artur Khanin committed Jan 25, 2021
    Configuration menu
    Copy the full SHA
    606ecb2 View commit details
    Browse the repository at this point in the history

Commits on Jan 27, 2021

  1. remove unused test case

    daria-malkova committed Jan 27, 2021
    Configuration menu
    Copy the full SHA
    2b30240 View commit details
    Browse the repository at this point in the history

  2. Merge remote-tracking branch 'origin/DataTokenizationExample' into Da… 

    …taTokenizationExample
    daria-malkova committed Jan 27, 2021
    Configuration menu
    Copy the full SHA
    329c3e6 View commit details
    Browse the repository at this point in the history

Commits on Feb 16, 2021

  1. Merge branch 'master' into DataTokenizationExample 

    # Conflicts:
#	examples/java/build.gradle
    Artur Khanin committed Feb 16, 2021
    Configuration menu
    Copy the full SHA
    341ae1e View commit details
    Browse the repository at this point in the history

  2. Style fix

    Artur Khanin committed Feb 16, 2021
    Configuration menu
    Copy the full SHA
    3002f7c View commit details
    Browse the repository at this point in the history

  3. Whitespaces fix

    Artur Khanin committed Feb 16, 2021
    Configuration menu
    Copy the full SHA
    7b7664e View commit details
    Browse the repository at this point in the history

Commits on Feb 17, 2021

  1. Fixed undeclared dependencies and excluded .csv resource files from l… 

    …icense analysis
    Artur Khanin committed Feb 17, 2021
    Configuration menu
    Copy the full SHA
    aeb471e View commit details
    Browse the repository at this point in the history

Commits on Mar 18, 2021

  1. Fix for incorrect rpc url

    @MikhailMedvedevAkvelon
    MikhailMedvedevAkvelon committed Mar 18, 2021
    Configuration menu
    Copy the full SHA
    a923846 View commit details
    Browse the repository at this point in the history

  2. Fix for nullable types

    @MikhailMedvedevAkvelon
    MikhailMedvedevAkvelon committed Mar 18, 2021
    Configuration menu
    Copy the full SHA
    5a5d0c4 View commit details
    Browse the repository at this point in the history

Commits on Mar 22, 2021

  1. Data tokenization example group into batches (#11) 

    * GroupIntoBatches was used in the data tokenization pipeline

* io files were renamed for the data tokenization template
    @Nuzhdina-Elena
    Nuzhdina-Elena authored Mar 22, 2021
    Configuration menu
    Copy the full SHA
    6ac4193 View commit details
    Browse the repository at this point in the history

  2. code format fixed

    @Nuzhdina-Elena
    Nuzhdina-Elena committed Mar 22, 2021
    Configuration menu
    Copy the full SHA
    906e6e7 View commit details
    Browse the repository at this point in the history

Commits on Mar 23, 2021

  1. Getting value from environment variables for maxBufferingDurationMs. … 

    …Information about it added to README (#14)

Getting value from environment variables for maxBufferingDurationMs
    @Nuzhdina-Elena
    Nuzhdina-Elena authored Mar 23, 2021
    Configuration menu
    Copy the full SHA
    9ad7e12 View commit details
    Browse the repository at this point in the history

  2. [DATAFLOW-139] Incorrect DSG url lead to NPE (#13) 

    * Fix bug incorrect DSG url lead to NPE DATAFLOW-139
    @MikhailMedvedevAkvelon
    MikhailMedvedevAkvelon authored Mar 23, 2021
    Configuration menu
    Copy the full SHA
    3ab9289 View commit details
    Browse the repository at this point in the history