diff --git a/docs.json b/docs.json index ab63b2d0..e4400081 100644 --- a/docs.json +++ b/docs.json @@ -320,7 +320,11 @@ }, { "group": "Cloud Platforms", - "pages": ["integrations/cloud/azure"] + "pages": [ + "integrations/cloud/azure", + "integrations/cloud/aws", + "integrations/cloud/aws-govcloud" + ] }, { "group": "Guardrails", diff --git a/integrations/cloud/aws-govcloud.mdx b/integrations/cloud/aws-govcloud.mdx new file mode 100644 index 00000000..b67f3d67 --- /dev/null +++ b/integrations/cloud/aws-govcloud.mdx @@ -0,0 +1,198 @@ +--- +title: "AWS GovCloud (US)" +description: "Enable secure AI operations in AWS GovCloud with Portkey's enterprise AI gateway platform" +--- + +> Unlock enterprise-grade AI orchestration for your AWS GovCloud environment + +## What is Portkey for AWS GovCloud (US)? + +Portkey is an **Enterprise AI Gateway and Observability Platform** designed to work seamlessly with AWS GovCloud (US), providing government agencies and regulated industries with the security, compliance, and control they need for AI operations. As a unified management layer, Portkey enables compliant AI deployment within AWS GovCloud's U.S. sovereign environment while maintaining the stringent security requirements necessary for handling sensitive and Controlled Unclassified Information (CUI). + +Our platform empowers organizations to transform their AWS GovCloud infrastructure into a complete enterprise AI platform with full visibility and control. Government agencies and organizations with elevated compliance needs can achieve **accelerated mission outcomes**, **enhanced governance**, and **optimized AI operations** while maintaining FedRAMP, ITAR, and DoD compliance. + + +**Enterprise advantages with Portkey in AWS GovCloud (US):** + +* Deploy Portkey within your existing AWS GovCloud environment +* Comprehensive cost attribution and optimization for your AI spend +* Enterprise-grade controls: SSO, RBAC, and access management +* Unified API gateway with observability across all AI models +* Seamless IAM role assumption for secure access management + +Schedule a 1:1 architecture review + +## Enterprise-Ready AWS GovCloud (US) Integration + + + + Deploy Portkey within your AWS GovCloud VPC for maximum security and control. + + + + Unified gateway for AI models available in AWS GovCloud with seamless integration. + + + + Implement granular budget controls, rate limits, and cost optimization with project/department-level attribution. + + + + Gain comprehensive visibility into your AI usage patterns, request performance, and spend across all applications. + + + + Maintain data sovereignty with private deployment options, audit logs, and advanced security features. + + + + Integrate with your existing identity providers for secure access control and automated user provisioning. + + + + Implement comprehensive organizational structures with role-based access control across teams and projects. + + + + Apply configurable PII detection and redaction to every AI request with no code changes required. + + + Bring your own encryption keys to Portkey AI to encrypt data at storage with AWS KMS support. + + + Secure access without API keys - Portkey temporarily assumes authorized IAM roles for Bedrock model access. + + + Connect AWS with guardrails, agentic frameworks, popular LLMs, and developer libraries for an extensible AI infrastructure. + + + + +## How Portkey Works in AWS GovCloud (US) + +Portkey acts as an intelligent middleware layer that can be deployed within your AWS GovCloud environment: + +1. **Unified Gateway**: Route all AI traffic through a single consistent interface +2. **Request Enrichment**: Automatically add metadata, tracking, and governance controls +3. **Intelligent Routing**: Direct requests to the optimal AI models based on cost, performance, and requirements +4. **Response Processing**: Apply caching, monitoring, and analytics to maximize performance and insights +5. **Comprehensive Observability**: Track usage, costs, and performance metrics across your entire organization +6. **Advanced Governance**: Implement role-based access control, workspace-level policies, and audit trails + +**Enterprise AI Features Compatible with AWS GovCloud** + + + + Centralize and streamline your fine-tuning workflows for AWS models with version control and deployment management. + + + + Process large volumes of prompts efficiently with batch processing capabilities, optimizing throughput and resource utilization for Bedrock models. + + + + Optimize performance and reduce costs with intelligent caching strategies specifically tuned for AWS models. + + + + With Portkey, you can upload files in a unified OpenAI format and Portkey will handle transforming them into the format required by Bedrock on the fly. + + + Route to your AWS SageMaker models through Portkey for comprehensive management, logging, and access control. + + + +**Deployment Architecture for AWS GovCloud** +```mermaid +flowchart LR + A[Your AWS GovCloud services] --> B[Portkey AI Gateway] + B --> C[Unified Observability\n& Cost Control] + B --> D[Centralized Guardrails\n& Security] + B --> E[Enhanced Reliability\n& Performance] + B --> F[Accelerated Development\nCycles] + C --> G[Actionable Insights &\nROI on AWS GovCloud services Spend] + D --> G + E --> G + F --> G +``` + +**Deployment Options in AWS GovCloud** + +1. **Full Air-Gapped Cloud Deployment** + * Run Portkey entirely within your AWS GovCloud VPC + * All data and traffic contained within your AWS GovCloud infrastructure + * Maximum security for sensitive environments + +2. **Hybrid Deployment** + * Control plane in Portkey Cloud, data plane in your AWS GovCloud environment + * Balance convenience with security requirements + * Flexible deployment to match your security posture + + +## Why Organizations Choose Portkey for AWS GovCloud + +### 1. Enhanced Security & Compliance + +Portkey's integration with AWS GovCloud preserves your security posture: + +* **AWS IAM Role Assumption**: No API keys exchanged - Portkey temporarily assumes authorized IAM roles +* **Data Sovereignty**: All data remains within your AWS GovCloud environment with Private Cloud deployment +* **End-to-End Encryption**: Data encrypted at rest and in transit with your KMS keys +* **Comprehensive Audit Logs**: Detailed audit trails for all administrative actions and requests + +### 2. Enterprise-Grade AI Governance + +Implement consistent governance across all AI applications: + +* **Workspace-Level Controls**: Enforce security policies by workspace +* **PII Detection**: Identify and redact sensitive information from AI interactions +* **Role-Based Access Control**: Granular permissions for admin, developer, and analyst roles +* **Request Traceability**: Track the origin, purpose, and cost of every AI request +* **Approval Workflows**: Implement change management for model deployments and configurations + +### 3. AWS GovCloud-Optimized Cost Management + +Achieve predictable budgeting and cost optimization: + +* **Department-Level Attribution**: Allocate AI costs to specific cost centers +* **Budget Enforcement**: Set hard or soft limits on model usage by team or project +* **Cost Optimization Engine**: Automatic routing to cost-effective models based on task complexity +* **Semantic Caching**: Reduce redundant API calls for common requests +* **Usage Analytics**: Comprehensive dashboards showing cost drivers and optimization opportunities + +### 4. Accelerated Development in Controlled Environments + +Enable your teams to build AI applications faster while maintaining security: + +* **Unified API**: One consistent interface for all AI models +* **Feature Parity**: Access to advanced capabilities across supported models +* **Managed Upgrades**: Seamless transitions when new model versions are released +* **Streamlined Testing**: A/B testing and canary deployments built into the platform +* **Developer Tooling**: SDKs, docs, and integrations that developers love + +## Enterprise Success Case Study + +> "With 30 million policies a month, managing over 25 GenAI use cases became a pain. Portkey helped with prompt management, tracking costs per use case, and ensuring our keys were used correctly. It gave us the visibility we needed into our AI operations." +> +> **— Prateek Jogani, CTO, Qoala** + +## Book a Consultation + +Our solutions architects will analyze your specific AWS GovCloud environment and AI requirements to design an optimal implementation path. + + + + +--- + +### Additional Resources + +* [AWS Cloud Deployment Guide](/product/enterprise-offering/private-cloud-deployments/aws) +* [Enterprise Security Docs](/product/enterprise-offering/security-portkey) diff --git a/integrations/cloud/aws.mdx b/integrations/cloud/aws.mdx new file mode 100644 index 00000000..c77e5e36 --- /dev/null +++ b/integrations/cloud/aws.mdx @@ -0,0 +1,208 @@ +--- +title: "AWS" +description: "Transform your AWS Bedrock AI infrastructure with Portkey's enterprise gateway, observability, and governance platform" +--- + +> Unlock enterprise-grade AI orchestration for your AWS Bedrock models with Portkey + +## What is Portkey for AWS? + +Portkey is an **AI Gateway and Observability Platform** that sits between your applications and AWS models, providing enterprise-grade controls, monitoring, and optimization. As a unified management layer, Portkey handles authentication, routing, caching, failover, governance, and comprehensive analytics for all your AWS AI operations. + +Our platform empowers organizations to transform their AWS infrastructure into a complete enterprise AI platform with full visibility and control. Companies building on AWS achieve **75% faster development cycles**, **significant cost reduction**, and **enhanced governance** while maintaining the security and compliance controls your enterprise demands. + +**Enterprise advantages with Portkey + AWS :** + +* Secure, air-gapped deployment options within your AWS environment +* Comprehensive cost attribution and optimization for your Bedrock spend +* Enterprise-grade controls: SSO, SCIM, and role-based access +* Unified API gateway with observability across all Bedrock models +* Seamless IAM role assumption for secure access management + +Schedule a 1:1 architecture review + +## Enterprise-Ready AWS Integration + + + + Deploy Portkey directly in your AWS account through AWS Marketplace for maximum security, compliance, and control. + + + Unified gateway for all AWS Bedrock models: Claude, Llama, Titan, Mistral, and more—with seamless version upgrades when new models arrive. + + + Bring your own encryption keys to Portkey AI to encrypt data at storage with AWS KMS support. + + + Secure access without API keys - Portkey temporarily assumes authorized IAM roles for Bedrock model access. + + + Implement granular budget controls, rate limits, and cost optimization strategies with project/department-level attribution. + + + + Maintain data sovereignty with air-gapped deployment options, audit logs, and advanced PII protection for regulated environments. + + + + Gain comprehensive visibility into your Bedrock usage patterns, request performance, and spend across teams and applications. + + + + Integrate seamlessly with your existing identity provider for enterprise-grade access control and automated user provisioning. + + + + Implement comprehensive organizational structures with role-based access control across multiple teams, projects, and environments. + + + + Portkey complies with SOC 2, GDPR, ISO27001, HIPAA to meet the most demanding industry standards. + + + Connect AWS with guardrails, agentic frameworks, popular LLMs, and developer libraries for an extensible AI infrastructure. + + + +## How Portkey Works with AWS + +Portkey acts as an intelligent middleware layer between your applications and AWS: + +1. **Unified Gateway**: Route all AWS Bedrock traffic through a single consistent interface +2. **Request Enrichment**: Automatically add metadata, tracking, and governance controls +3. **Intelligent Routing**: Direct requests to the optimal AWS Bedrock models based on cost, performance, and feature requirements +4. **Response Processing**: Apply caching, monitoring, and analytics to maximize performance and insights +5. **Comprehensive Observability**: Track usage, costs, and performance metrics across your entire organization +6. **Governance**: Implement role-based access control, workspace-level policies, and audit trails for all AI operations + +**Bedrock-Specific Features** + + + + Centralize and streamline your fine-tuning workflows for AWS models with version control and deployment management. + + + + Process large volumes of prompts efficiently with batch processing capabilities, optimizing throughput and resource utilization for Bedrock models. + + + + Optimize performance and reduce costs with intelligent caching strategies specifically tuned for AWS models. + + + + With Portkey, you can upload files in a unified OpenAI format and Portkey will handle transforming them into the format required by Bedrock on the fly. + + + Route to your AWS SageMaker models through Portkey for comprehensive management, logging, and access control. + + + + +**Enterprise Deployment Architecture** + +```mermaid +flowchart LR + A[Your AWS Services] --> B[Portkey AI Gateway] + B --> C[Unified Observability\n& Cost Control] + B --> D[Centralized Guardrails\n& Security] + B --> E[Enhanced Reliability\n& Performance] + B --> F[Accelerated Development\nCycles] + C --> G[Actionable Insights &\nROI on AWS Spend] + D --> G + E --> G + F --> G +``` + +Schedule a 1:1 architecture review + +**Enterprise Deployment Options** + +1. **AWS Marketplace SaaS Deployment** + * Deploy through AWS Marketplace with full AWS integration + * Maintain compliance with enterprise security standards + * Single-click deployment with immediate availability + +2. **Hybrid Deployment** + * Control plane in Portkey Cloud, data plane in your AWS environment + * Balance convenience with security requirements + * Flexible deployment to match your security posture + +3. **Fully Air-gapped Deployment** + * Air-gapped option for maximum security requirements + * All data and traffic contained within your AWS infrastructure + +## Implement in Minutes + + + + +## Why Enterprise CTOs Choose Portkey + AWS + +### 1. Zero Compromise on Security & Compliance + +Portkey's AWS integration preserves your security posture: + +* **AWS IAM Role Assumption**: No API keys exchanged - Portkey temporarily assumes authorized IAM roles +* **Data Sovereignty**: All data remains within your AWS environment with Private Cloud deployment +* **Air-Gapped Option**: Complete isolation with no outbound internet connectivity +* **End-to-End Encryption**: Data encrypted at rest and in transit with your KMS keys +* **Comprehensive Audit Logs**: Detailed audit trails for all administrative actions and requests + +### 2. Enterprise-Grade AI Governance + +Implement consistent governance across all AI applications: + +* **Workspace-Level Controls**: Enforce guardrails and security policies by workspace +* **Automatic PII Detection**: Identify and redact sensitive information from AI interactions +* **Role-Based Access Control**: Granular permissions for admin, developer, and analyst roles +* **Request Traceability**: Track the origin, purpose, and cost of every AI request +* **Approval Workflows**: Implement change management for model deployments and configurations + +### 3. AWS-Optimized Cost Management + +Achieve predictable budgeting and cost optimization: + +* **Department-Level Attribution**: Allocate Bedrock costs to specific cost centers +* **Budget Enforcement**: Set hard or soft limits on model usage by team or project +* **Cost Optimization Engine**: Automatic routing to cost-effective models based on task complexity +* **Semantic Caching**: Reduce redundant API calls for common requests +* **Usage Analytics**: Comprehensive dashboards showing cost drivers and optimization opportunities + +### 4. Accelerated Development Velocity + +Enable your teams to build AI applications faster: + +* **Unified API**: One consistent interface for all AWS models +* **Feature Parity**: Access to advanced capabilities across all supported models +* **Managed Upgrades**: Seamless transitions when new model versions are released +* **Streamlined Testing**: A/B testing and canary deployments built into the platform +* **Developer Tooling**: SDKs, docs, and integrations that developers love + +**Enterprise Success Case Study** + +> "With 30 million policies a month, managing over 25 GenAI use cases became a pain. Portkey helped with prompt management, tracking costs per use case, and ensuring our keys were used correctly. It gave us the visibility we needed into our AI operations." +> +> **— Prateek Jogani, CTO, Qoala** + +## Book an Enterprise Demo + + + + + + +--- + +### Enterprise Resources + +* [AWS Bedrock Model Integration Guide](/integrations/llms/bedrock/aws-bedrock) +* [AWS Private Cloud Deployment Whitepaper](/product/enterprise-offering/private-cloud-deployments/aws) +* [Enterprise Security & Compliance Docs](/product/enterprise-offering/security-portkey) diff --git a/product/guardrails.mdx b/product/guardrails.mdx index 64e22d2a..e3bb8c96 100644 --- a/product/guardrails.mdx +++ b/product/guardrails.mdx @@ -621,17 +621,6 @@ This example corresponds to a `config` like: - - - - - - - - - - - ## Defining Guardrails Directly in JSON On Portkey, you can also create the Guardrails in code and add them to your Configs. Read more about this here: @@ -640,8 +629,6 @@ On Portkey, you can also create the Guardrails in code and add them to your Conf --- - - ## Bring Your Own Guardrails If you already have a custom guardrail pipeline where you send your inputs/outputs for evaluation, you can integrate it with Portkey using a modular, custom webhook! Read more here: