Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BCDA-8301: Fix large object S3 trigger + update logging #984

Merged
merged 14 commits into from
Aug 13, 2024
10 changes: 5 additions & 5 deletions .github/workflows/cclf-import-test-integration.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Integration test that triggers a CCLF import in the test environment and
# Integration test that triggers a CCLF import in the test environment and
# verifies the imported data in the test database for ACO A0001.
name: cclf-import test integration

Expand All @@ -7,7 +7,8 @@ on:
paths:
- .github/workflows/cclf-import-test-integration.yml
- .github/workflows/cclf-import-test-deploy.yml
- cclf/**
- bcda/cclf/**
- bcda/lambda/cclf/**
workflow_dispatch:

# Ensure we have only one integration test running at a time
Expand Down Expand Up @@ -72,7 +73,7 @@ jobs:
verify:
needs: trigger
runs-on: self-hosted
env:
env:
ACTIONS_ALLOW_USE_UNSECURE_NODE_VERSION: "true"
steps:
- uses: actions/checkout@v3
Expand All @@ -82,7 +83,7 @@ jobs:
role-to-assume: arn:aws:iam::${{ secrets.ACCOUNT_ID }}:role/delegatedadmin/developer/bcda-test-github-actions
- name: Install psql
run: |
sudo amazon-linux-extras install postgresql14
sudo amazon-linux-extras install postgresql14
- name: Get database credentials
uses: cmsgov/ab2d-bcda-dpc-platform/actions/aws-params-env-action@main
env:
Expand Down Expand Up @@ -116,4 +117,3 @@ jobs:
exit 1
fi
fi

13 changes: 7 additions & 6 deletions .github/workflows/opt-out-import-test-integration.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Integration test that triggers an Opt Out import in the test environment and
# Integration test that triggers an Opt Out import in the test environment and
# verifies the imported data in the test database.
name: opt-out-import test integration

Expand All @@ -8,14 +8,15 @@ on:
- .github/workflows/opt-out-import-test-integration.yml
- .github/workflows/opt-out-import-test-deploy.yml
- optout/**
- bcda/suppression/**
- bcda/lambda/optout/**
workflow_dispatch:

# Ensure we have only one integration test running at a time
concurrency:
group: opt-out-import-test-integration

jobs:

# Deploy first if triggered by pull_request
deploy:
if: ${{ github.event_name == 'pull_request' }}
Expand Down Expand Up @@ -54,11 +55,11 @@ jobs:
echo "FILENAME=$fname" >> "$GITHUB_OUTPUT"
aws s3 cp ../shared_files/synthetic1800MedicareFiles/test/T\#EFT.ON.ACO.NGD1800.DPRF.D181120.T1000009 \
s3://bfd-test-eft/bfdeft01/bcda/in/test/$fname
verify:
needs: trigger
runs-on: self-hosted
env:
env:
ACTIONS_ALLOW_USE_UNSECURE_NODE_VERSION: "true"
steps:
- uses: actions/checkout@v3
Expand All @@ -68,7 +69,7 @@ jobs:
role-to-assume: arn:aws:iam::${{ secrets.ACCOUNT_ID }}:role/delegatedadmin/developer/bcda-test-github-actions
- name: Install psql
run: |
sudo amazon-linux-extras install postgresql14
sudo amazon-linux-extras install postgresql14
- name: Get database credentials
uses: cmsgov/ab2d-bcda-dpc-platform/actions/aws-params-env-action@main
env:
Expand All @@ -94,4 +95,4 @@ jobs:
exit 1
fi
fi
1 change: 1 addition & 0 deletions bcda/bcdacli/cli.go
Original file line number Diff line number Diff line change
Expand Up @@ -398,6 +398,7 @@ func setUpApp() *cli.App {
}

importer := cclf.CclfImporter{
Logger: log.API,
FileProcessor: file_processor,
}

Expand Down
Loading
Loading