Skip to content

Commit

Permalink
Updates for v4.0.0
Browse files Browse the repository at this point in the history
  • Loading branch information
@eggoynes
eggoynes committed Jul 6, 2022
1 parent 8dccded commit 18ff983
Show file tree
Hide file tree
Showing 37 changed files with 33,480 additions and 14,364 deletions.
1 change: 1 addition & 0 deletions .gitignore
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ build/
dev/
global-s3-assets/
regional-s3-assets/
open-source/
.pnp
.pnp.js
.DS_Store
Expand Down
36 changes: 31 additions & 5 deletions CHANGELOG.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,38 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [3.1.2] - 2022-3-24
### Updated
- Pathval updated to 1.1.1
- Follow Redirects updated to 1.14.8
## [4.0.0] - 2022-7-28

### New
```
# Added cdk infrastructure in source/constructs directory
# Defined resources for cdk stack in source/constructs/lib/live-streaming.ts
# Added links to MediaLive and S3 consoles to CloudFormation Outputs
# Added links to metric dashboards for MediaLive and MediaPackage to CloudFormation Outputs
# Added SonarQube properties file: sonar-project.properties
# Added snapshot test to source/constructs/test directory
# Added cdk nag rule suppressions
# Added SolutionId tag to resources
```

### Changed
```
# Removed CloudFormation template live-streaming-on-aws.yaml
# Use CachePolicy instead of ForwardedValues(deprecated) for cloudfront distribution
# Use @aws-solutions-constructs/aws-cloudfront-s3 construct to deploy demo resources
# Updated deployment/run-unit-tests.sh to generate unit test coverage reports
# Updated deployment/build-s3-dist.sh to output cdk nag errors
# Updated source/custom-resource/lib/medialive/index.spec.js to increase unit test coverage
# Generate secret string for Cdn Secret resource using cdk instead of hard coding
# Upgrade path from old versions require a delete and re-deploy since moving to CDK
```

### Contributors
* @sandimciin
* @eggoynes

## [3.1.1] - 2022-1-24

### Updated
- Follow Redirects updated to 1.14.7
- AWS SDK updated to 2.814.0
Expand Down Expand Up @@ -48,7 +74,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [2.4.0] - 2020-07-30
### Added
- Origin headers and Custom Error 404 TTL for CloudFront
- CDN Autnorizarion for Elemental MediaPackage EndPoints
- CDN Authorization for Elemental MediaPackage EndPoints
### Updated
- MediaLive Encoding now supports QVBR and 4 second segment sizes
- MediaLive with a new set of outputs 1920x1080, 1280x720, 960x540, 768x432, 640x360, 512x288
Expand Down
95 changes: 45 additions & 50 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -40,85 +40,80 @@ The profiles are defined in JSON and and can be found in:
**source/custom-resources::**<br/>
A NodeJS based Lambda function used as a custom resource for deploying MediaLive and MediaPackage resources through CloudFormation.

## Creating a custom build
The solution can be deployed through the CloudFormation template available on the solution [home page](https://aws.amazon.com/solutions/implementations/live-streaming-on-aws/). The Following steps are required to customize and deploy your own version of the solution:

1. download or clone this repo,
2. update the source code with any changes you require
3. run the deployment/build-s3-dist.sh script to package the lambda source code and update the s3 bucket mappings in the CloudFormation template (see bellow).
4. upload lambda deployment package to Amazon S3
5. deploy the CloudFormation template.


## Example Custom Deployment.
The CloudFormation template is configured to pull the source code from Amazon S3 bucket in the same region the template is being launched in. The template includes the following mappings for the source code:

```
SourceCode:
General:
S3Bucket: CODE_BUCKET //This is the name of the S3 bucket
KeyPrefix: SOLUTION_NAME/CODE_VERSION //This is the path to the source code (eg: live-streaming-on-aws/v3.0.0)
```

The example bellow assumes the following:
* The solution is going to be deployed to us-east-1
* the bucket name is mybucket-us-east-1
* the solution name is live streaming-on-aws
* the version is v3.0.0
## Deployment
The solution can be deployed through the CloudFormation template available on the solution [home page](https://aws.amazon.com/solutions/implementations/live-streaming-on-aws/).

## Creating a custom build

### Prerequisites:
* [AWS Command Line Interface](https://aws.amazon.com/cli/)
* Node.js 12.x or later
* AWS CDK 1.125.0

### 1. Create an Amazon S3 Bucket
Create a bucket in us-east-1 region with the region appended to the name. Please consider using randomized bucket names, ensure your buckets are not public, and verify bucket ownership prior to uploading application code or templates.
The are 2 options for deploying the solution: using the CDK deployment tools or running the build script to generate a CFN template and the packaged lambda code.

```
aws s3 mb s3://mybucket-us-east-1
```
### Options 1: Deploying through the CDK.
This options simply flollows the standard CDK deployment process. You will need to run `cdk bootstrap` before you run cdk deploy the first time to setup the cdk resource in your AWS account. Details on using the CDK can be found [here].

### 2. Create the deployment packages
Run the build-s3-dist.sh script passing in 3 parameters for CODE_BUCKET, SOLUTION_NAME, CODE_VERSION:
1. Download or clone the repo and make the required changes to the source code.
2. installl the dependencies for the lambda function and construct.
3. then run `cdk deploy` (first time use you will need to run `cdk bootstrap` first)

```
cd deployment/ && chmod +x ./build-s3-dist.sh
./build-s3-dist.sh mybucket live-streaming-on-aws v3.0.0
cd ./source/custom-resource
npm install --production
cd ../constructs
npm install
cdk bootstrap //if you havent already done so.
cdk deploy
```

**note**
The template adds the -region suffix to any referrence to the bucket, this allows the same template to be deployed to multiple regions. For example the cusntom Resource Lambda function has the follwing:
### Option 2: Generate a CloudFormation template.
The CloudFormation template (generated by the CDK) includes a lambda backed custom resource to configure MediaLive and create a UUID. To launch the solution the Lambda source code has to be deployed to an Amazon S3 bucket in the region you intend to deploy the solution.

### 1. Clone the repo
Download or clone the repo and make the required changes to the source code.

### 2. Running unit tests for customization
Run unit tests to make sure added customization passes the tests:
```
S3Bucket: !Join ["-", [!FindInMap ["SourceCode", "General", "S3Bucket"], Ref: "AWS::Region"]]
cd ./deployment
chmod +x ./run-unit-tests.sh && ./run-unit-tests.sh
```

### 3. Deploy the source code to S3:
### 3. Create an Amazon S3 Bucket
The CloudFormation template is configured to pull the Lambda deployment packages from Amazon S3 bucket in the region the template is being launched in. Create a bucket in the desired region with the region name appended to the name of the bucket. eg: for us-east-1 create a bucket named: `my-bucket-us-east-1`
```
aws s3 mb s3://my-bucket-us-east-1
```

Ensure that you are owner of the AWS S3 bucket.
```
aws s3api head-bucket --bucket mybucket-us-east-1 --expected-bucket-owner YOUR-AWS-ACCOUNT-NUMBER
aws s3api head-bucket --bucket my-bucket-us-east-1 --expected-bucket-owner YOUR-AWS-ACCOUNT-NUMBER
```

Uploads the files to your S3 bucket.
### 4. Create the deployment packages
Build the distributable:
```
aws s3 sync ./regional-s3-assets/ s3://mybucket-us-east-1/live-streaming-on-aws/v3.0.0/
chmod +x ./build-s3-dist.sh
./build-s3-dist.sh <my-bucket> live-streaming-on-aws <version>
```

### 4. Launch the CloudFormation template.
The buid-s3-dist.sh script creates a copy of the template in deployment/global-assets/ with the Mappins section updated with you s3 details:

SourceCode:
General:
S3Bucket: mybucket
KeyPrefix: live-streaming-on-aws/v3.0.0
> **Notes**: The _build-s3-dist_ script expects the bucket name as one of its parameters. This value should not have the region suffix (remove the -us-east-1)
Launch the Template through the AWS Console in us-east-1.
Deploy the distributable to the Amazon S3 bucket in your account:
```
aws s3 sync ./regional-s3-assets/ s3://my-bucket-us-east-1/live-streaming-on-aws/<version>/
aws s3 sync ./global-s3-assets/ s3://my-bucket-us-east-1/live-streaming-on-aws/<version>/
```

### 5. Launch the CloudFormation template.
* Get the link of the live-streaming-on-aws.template uploaded to your Amazon S3 bucket.
* Deploy the solution.

## License

* This project is licensed under the terms of the Apache 2.0 license. See `LICENSE`.
* This project is licensed under the terms of the Apache 2.0 license. See here `LICENSE`.

This solution collects anonymous operational metrics to help AWS improve the
quality of features of the solution. For more information, including how to disable
Expand Down
119 changes: 88 additions & 31 deletions deployment/build-s3-dist.sh
View file
Original file line number Diff line number Diff line change
@@ -1,70 +1,125 @@
#!/bin/bash
#
# This assumes all of the OS-level configuration has been completed and git repo has already been cloned
# This script will perform the following tasks:
# 1. Remove any old dist files from previous runs.
# 2. Install dependencies for the cdk-solution-helper; responsible for
# converting standard 'cdk synth' output into solution assets.
# 3. Build and synthesize your CDK project.
# 4. Run the cdk-solution-helper on template outputs and organize
# those outputs into the /global-s3-assets folder.
# 5. Organize source code artifacts into the /regional-s3-assets folder.
# 6. Remove any temporary files used for staging.
#
# This script should be run from the repo's deployment directory
# cd deployment
# ./build-s3-dist.sh source-bucket-base-name trademarked-solution-name version-code
# ./build-s3-dist.sh source-bucket-base-name solution-name version-code template-bucket-name
#
# Paramenters:
# Parameters:
# - source-bucket-base-name: Name for the S3 bucket location where the template will source the Lambda
# code from. The template will append '-[region_name]' to this bucket name.
# For example: ./build-s3-dist.sh solutions my-solution v1.0.0
# The template will then expect the source code to be located in the solutions-[region_name] bucket
#
# - trademarked-solution-name: name of the solution for consistency
#
# - solution-name: name of the solution for consistency
# - version-code: version of the package
[ "$DEBUG" == 'true' ] && set -x
set -e

# Check to see if input has been provided:
if [ -z "$1" ] || [ -z "$2" ] || [ -z "$3" ]; then
echo "Please provide the base source bucket name, trademark approved solution name and version where the lambda code will eventually reside."
echo "Please provide all required parameters for the build script"
echo "For example: ./build-s3-dist.sh solutions trademarked-solution-name v1.0.0"
exit 1
fi

set -e
bucket_name="$1"
solution_name="$2"
solution_version="$3"

# Get reference for all important folders
template_dir="$PWD"
staging_dist_dir="$template_dir/staging"
template_dist_dir="$template_dir/global-s3-assets"
build_dist_dir="$template_dir/regional-s3-assets"
source_dir="$template_dir/../source"

echo "------------------------------------------------------------------------------"
echo "Rebuild distribution"
echo "[Init] Remove any old dist files from previous runs"
echo "------------------------------------------------------------------------------"
rm -rf $template_dist_dir
mkdir -p $template_dist_dir

rm -rf $build_dist_dir
mkdir -p $build_dist_dir

[ -e $template_dist_dir ] && rm -r $template_dist_dir
[ -e $build_dist_dir ] && rm -r $build_dist_dir
mkdir -p $template_dist_dir $build_dist_dir
rm -rf $staging_dist_dir
mkdir -p $staging_dist_dir

echo "------------------------------------------------------------------------------"
echo "[Init] Install dependencies for the cdk-solution-helper"
echo "------------------------------------------------------------------------------"
cd $template_dir/cdk-solution-helper
npm install --production

echo "------------------------------------------------------------------------------"
echo "[Synth] CDK Project"
echo "------------------------------------------------------------------------------"
# Make sure user has the newest CDK version
npm uninstall -g aws-cdk && npm install -g aws-cdk@1

cd $source_dir/constructs
npm install
cdk synth --output=$staging_dist_dir
if [ $? -ne 0 ]
then
echo "******************************************************************************"
echo "cdk-nag found errors"
echo "******************************************************************************"
exit 1
fi

cd $staging_dist_dir
rm tree.json manifest.json cdk.out

echo "------------------------------------------------------------------------------"
echo "Main CloudFormation Template"
echo "Run Cdk Helper and update template placeholders"
echo "------------------------------------------------------------------------------"
cp $template_dir/live-streaming-on-aws.yaml $template_dist_dir/live-streaming-on-aws.template
mv LiveStreaming.template.json $template_dist_dir/$solution_name.template

node $template_dir/cdk-solution-helper/index

for file in $template_dist_dir/*.template
do
replace="s/%%BUCKET_NAME%%/$bucket_name/g"
sed -i -e $replace $file

replace="s/CODE_BUCKET/$1/g"
echo "sed -i -e $replace"
sed -i -e $replace $template_dist_dir/live-streaming-on-aws.template
replace="s/SOLUTION_NAME/$2/g"
echo "sed -i -e $replace"
sed -i -e $replace $template_dist_dir/live-streaming-on-aws.template
replace="s/CODE_VERSION/$3/g"
echo "sed -i -e $replace"
sed -i -e $replace $template_dist_dir/live-streaming-on-aws.template
# remove tmp file for MACs
[ -e $template_dist_dir/live-streaming-on-aws.template-e ] && rm -r $template_dist_dir/live-streaming-on-aws.template-e
replace="s/%%SOLUTION_NAME%%/$solution_name/g"
sed -i -e $replace $file

replace="s/%%VERSION%%/$solution_version/g"
sed -i -e $replace $file
done

echo "------------------------------------------------------------------------------"
echo "Creating NODE custom-resource deployment package"
echo "[Packing] Source code artifacts"
echo "------------------------------------------------------------------------------"
cd $source_dir/custom-resource/
rm -rf node_modules/
npm ci --production
zip -q -r9 $build_dist_dir/custom-resource.zip *
# ... For each asset.* source code artifact in the temporary /staging folder...
cd $staging_dist_dir
for d in `find . -mindepth 1 -maxdepth 1 -type d`; do
# Rename the artifact, removing the period for handler compatibility
pfname="$(basename -- $d)"
fname="$(echo $pfname | sed -e 's/\.//g')"
mv $d $fname
# Zip artifacts from asset folder
cd $fname
rm -rf node_modules/
rm -rf coverage/
npm ci --production
zip -rq ../$fname.zip *
cd ..

# Copy the zipped artifact from /staging to /regional-s3-assets
mv $fname.zip $build_dist_dir
done

echo "------------------------------------------------------------------------------"
echo "Building console"
Expand All @@ -87,5 +142,7 @@ manifest_json=$(IFS=,;printf "%s" "${manifest[*]}")
echo "[\"$manifest_json\"]" | sed 's/,/","/g' > $build_dist_dir/console-manifest.json

echo "------------------------------------------------------------------------------"
echo "Build S3 Packaging Complete"
echo "[Cleanup] Remove temporary files"
echo "------------------------------------------------------------------------------"
rm -rf $staging_dist_dir
rm -f $template_dist_dir/*.template-e
Loading

0 comments on commit 18ff983

Please sign in to comment.