Skip to content

[AWSX-1559] feat: update Python version to 3.13 #924

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 7 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/aws_integration_test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,4 +28,4 @@ jobs:

- name: AWS Python integration tests
run: |
./aws/logs_monitoring/tools/integration_tests/integration_tests.sh --python-version=3.12
./aws/logs_monitoring/tools/integration_tests/integration_tests.sh --python-version=3.13
4 changes: 2 additions & 2 deletions .github/workflows/aws_lint.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ jobs:
- name: Setup Python
uses: actions/setup-python@v3
with:
python-version: "3.12"
python-version: "3.13"

- name: Install pip
run: |
Expand All @@ -33,7 +33,7 @@ jobs:

- name: Install cfn-lint print version and check formatting
run: |
pip install cfn-lint==1.11.1
pip install cfn-lint==1.19.0
cfn-lint --version
cfn-lint -t aws/logs_monitoring/template.yaml
cfn-lint -t aws/rds_enhanced_monitoring/rds-enhanced-sam-template.yaml
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/aws_unit_test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,10 +8,10 @@ jobs:
steps:
- name: Checkout source
uses: actions/checkout@v3
- name: Setup Python 3.12
- name: Setup Python 3.13
uses: actions/setup-python@v3
with:
python-version: 3.12
python-version: 3.13

- name: AWS Python unit tests
env:
Expand Down
6 changes: 3 additions & 3 deletions aws/logs_monitoring/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -115,7 +115,7 @@ resource "aws_cloudformation_stack" "datadog_forwarder" {

If you can't install the Forwarder using the provided CloudFormation template, you can install the Forwarder manually following the steps below. Feel free to open an issue or pull request to let us know if there is anything we can improve to make the template work for you.

1. Create a Python 3.12 Lambda function using `aws-dd-forwarder-<VERSION>.zip` from the latest [releases][101].
1. Create a Python 3.13 Lambda function using `aws-dd-forwarder-<VERSION>.zip` from the latest [releases][101].
2. Save your [Datadog API key][102] in AWS Secrets Manager, set environment variable `DD_API_KEY_SECRET_ARN` with the secret ARN on the Lambda function, and add the `secretsmanager:GetSecretValue` permission to the Lambda execution role.
3. If you need to forward logs from S3 buckets, add the `s3:GetObject` permission to the Lambda execution role.
4. Set the environment variable `DD_ENHANCED_METRICS` to `false` on the Forwarder. This stops the Forwarder from generating enhanced metrics itself, but it will still forward custom metrics from other lambdas.DdFetchLambdaTags
Expand Down Expand Up @@ -149,7 +149,7 @@ The <a href="#cloudformation-parameters">environment variables provided on this
If you encounter issues upgrading to the latest version, check the Troubleshooting section.

### Upgrade an older verison to 4.3.0+
Starting verison 4.3.0 Lambda forwarder will support a single python version only. The supported Python version of this release is 3.12.
Starting verison 4.3.0 Lambda forwarder will support a single python version only. The supported Python version of this release is 3.13.

### Upgrade an older version to +4.0.0
Starting version 4.0.0 `source`, `service` and `host` identification logic will be pulled out from the Lambda forwarder's code and set in directly in Datadog's backend. The first migrated log source is `RDS`.
Expand Down Expand Up @@ -748,7 +748,7 @@ The value of the `service` tag is determined based on multiple inputs. These inp
1. Log message custom tags: If the log message has a `ddtags` key which contains a `service` tag value, it will be used to override the `service` tag in the log event.
2. Lambda tags cache (applicable for Lambda logs only): Activating `DdFetchLambdaTags` will fetch and store all Lambda functions tags and will override the `service` tag if it wasn't set previously or was set to a default value i.e. `source` value.
3. Cloudwatch log group tags cache (applicable for Cloudwatch logs only): Activating `DdFetchLogGroupTags` will fetch and store all Cloudwatch log groups tags which are added to the `ddtags` entry in the log event. If `service` tag value was set in the tags cache it will be used to set the `service` tag for the log event.
4. Directly setting a `service` tag value in the forwarder's `ddtags` ENV var.
4. Directly setting a `service` tag value in the forwarder's `ddtags` ENV var.
5. Default value equal to the `source` tag.


Expand Down
2 changes: 1 addition & 1 deletion aws/logs_monitoring/requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ charset-normalizer
datadog-lambda==6.104.0
datadog==0.50.2
ddsketch==3.0.1
ddtrace==2.17.3
ddtrace==3.0.0
deprecated
envier
exceptiongroup
Expand Down
4 changes: 2 additions & 2 deletions aws/logs_monitoring/setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,10 +12,10 @@
author="Datadog, Inc.",
author_email="dev@datadoghq.com",
classifiers=[
"Programming Language :: Python :: 3.12",
"Programming Language :: Python :: 3.13",
],
keywords="datadog aws lambda layer",
python_requires=">=3.12, <3.13",
python_requires=">=3.13, <3.14",
extras_require={
"dev": ["nose2==0.9.1", "flake8==3.7.9", "requests==2.22.0", "boto3==1.10.33"]
},
Expand Down
4 changes: 2 additions & 2 deletions aws/logs_monitoring/template.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -432,7 +432,7 @@ Resources:
- {DdForwarderVersion: !FindInMap [Constants, DdForwarder, Version]}
- ZipFile: " "
MemorySize: !Ref MemorySize
Runtime: python3.12
Runtime: python3.13
Architectures:
- arm64
Timeout: !Ref Timeout
Expand Down Expand Up @@ -843,7 +843,7 @@ Resources:
Properties:
Description: Copies Datadog Forwarder zip to the destination S3 bucket
Handler: index.handler
Runtime: python3.12
Runtime: python3.13
Timeout: 600
Code:
ZipFile: |
Expand Down
2 changes: 1 addition & 1 deletion aws/logs_monitoring/tools/build_bundle.sh
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ else
VERSION=$1
fi

PYTHON_VERSION="${PYTHON_VERSION:-3.12}"
PYTHON_VERSION="${PYTHON_VERSION:-3.13}"
FORWARDER_PREFIX="aws-dd-forwarder"
FORWARDER_DIR="../.forwarder"

Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
service: integration-tests
provider:
name: aws
runtime: python3.12
runtime: python3.13

functions:
cache_test_lambda:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,9 @@

set -e

PYTHON_VERSION="python3.12"
PYTHON_VERSION_TAG="3.12"
PYTHON_IMAGE="python:3.12"
PYTHON_VERSION="python3.13"
PYTHON_VERSION_TAG="3.13"
PYTHON_IMAGE="python:3.13"
SKIP_FORWARDER_BUILD=false
UPDATE_SNAPSHOTS=false
LOG_LEVEL=info
Expand All @@ -36,7 +36,7 @@ for arg in "$@"; do
SKIP_FORWARDER_BUILD=true
shift
;;


# -u or --update
# Update the snapshots to reflect this test run
Expand Down
2 changes: 1 addition & 1 deletion aws/logs_monitoring/tools/publish_layers.sh
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,7 @@ user_confirm() {
# Makes sure any subprocesses will be terminated with this process
trap "pkill -P $$; exit 1;" INT

PYTHON_VERSIONS_FOR_AWS_CLI=("python3.12")
PYTHON_VERSIONS_FOR_AWS_CLI=("python3.13")
LAYER_PATHS=(".forwarder/aws-dd-forwarder-${FORWARDER_VERSION}-layer.zip")
AVAILABLE_LAYERS=("Datadog-Forwarder")
AVAILABLE_REGIONS=$(aws ec2 describe-regions | jq -r '.[] | .[] | .RegionName')
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ rm -rf ./bin
# between different python runtimes.

if [[ $(docker image ls | grep -c golang) -lt 1 ]]; then
docker buildx build --platform linux/arm64 -t golang . --no-cache --build-arg "runtime=python:3.12"
docker buildx build --platform linux/arm64 -t golang . --no-cache --build-arg "runtime=python:3.13"
fi

id=$(docker create --platform linux/arm64 golang)
Expand Down
2 changes: 1 addition & 1 deletion aws/logs_monitoring/trace_forwarder/scripts/run_tests.sh
Original file line number Diff line number Diff line change
Expand Up @@ -11,5 +11,5 @@ set -e
# Change to the parent of the directory this script is in
cd $(dirname "$0")/..

docker buildx build --platform linux/arm64 -t datadog-go-layer . --build-arg runtime=python:3.12
docker buildx build --platform linux/arm64 -t datadog-go-layer . --build-arg runtime=python:3.13
docker run --rm datadog-go-layer go test -v ./...
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ Resources:
KeyId: !Ref KMSKeyId
Architectures:
- arm64
Runtime: python3.12
Runtime: python3.13
Timeout: 10
KmsKeyArn:
!Sub
Expand Down
2 changes: 1 addition & 1 deletion aws/vpc_flow_log_monitoring/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ version, account, eni, source, destination, srcport, destport="22", protocol="6"

- Create a `lambda_execution` role and attach this policy

- Create a lambda function: Skip the blueprint, name it `functionname`, set the Runtime to `Python 3.12`, the Architecture to `arm64`,, the handle to `lambda_function.lambda_handler`, and the role to `lambda_execution`.
- Create a lambda function: Skip the blueprint, name it `functionname`, set the Runtime to `Python 3.13`, the Architecture to `arm64`,, the handle to `lambda_function.lambda_handler`, and the role to `lambda_execution`.

- Copy the content of `functionname/lambda_function.py` in the code section, make sure to update the `KMS_ENCRYPTED_KEYS` environment variable with the encrypted key generated in step 1

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ Resources:
Policies:
KMSDecryptPolicy:
KeyId: !Ref KMSKeyId
Runtime: python3.12
Runtime: python3.13
Architectures:
- arm64
Timeout: 10
Expand Down
Loading