Skip to content

Switch to @onkernel/cu-playwright package and template refactoring #39

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 15 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ create-kernel-app [app-name] [options]
- `browser-use`: Template with Browser Use SDK (Python only)
- `stagehand`: Template with Stagehand SDK (Typescript only)
- `advanced-sample`: Implements sample apps using advanced Kernel configs
- `computer-use`: Implements a prompt loop using Anthropic Computer Use
- `claude-cu`: Implements a prompt loop using Anthropic Computer Use
- `cua`: Implements a Computer Use Agent (OpenAI CUA) sample

### Examples
Expand All @@ -61,9 +61,9 @@ Create a Typescript application with Stagehand template:
npx @onkernel/create-kernel-app my-app --language typescript --template stagehand
```

Create a Typescript application with Computer Use template:
Create a Typescript application with Claude Computer Use template:
```bash
npx @onkernel/create-kernel-app my-app --language typescript --template computer-use
npx @onkernel/create-kernel-app my-app --language typescript --template claude-cu
```

Create a Python application with a sample app:
Expand All @@ -75,6 +75,10 @@ Create a Python application with Browser Use template:
```bash
npx @onkernel/create-kernel-app my-app --language python --template browser-use
```

Create a Python application with Claude Computer Use template:
```bash
npx @onkernel/create-kernel-app my-app --language python --template claude-cu
```

## Next Steps
Expand All @@ -98,10 +102,10 @@ export KERNEL_API_KEY=<YOUR_API_KEY>
4. Deploy your application:
```bash
# Typscript
kernel deploy index.ts # --env OPENAI_API_KEY=XXX if Stagehand; --env ANTHROPIC_API_KEY=XXX if Computer Use
kernel deploy index.ts # --env OPENAI_API_KEY=XXX if Stagehand; --env ANTHROPIC_API_KEY=XXX if Claude Computer Use

# Python
kernel deploy main.py # --env OPENAI_API_KEY=XXX if Browser Use
kernel deploy main.py # --env OPENAI_API_KEY=XXX if Browser Use; --env ANTHROPIC_API_KEY=XXX if Claude Computer Use
```

If deploying an app that requires environment variables, make sure to [set them](https://docs.onkernel.com/launch/deploy#environment-variables) when you `deploy`.
Expand All @@ -114,15 +118,18 @@ kernel invoke ts-basic get-page-title --payload '{"url": "https://www.google.com
# Typescript + Stagehand
kernel invoke ts-stagehand stagehand-task --payload '{"query": "Best wired earbuds"}'

# Typescript + Computer Use
kernel invoke ts-cu cu-task --payload '{"query": "Search for the top 3 restaurants in NYC according to Pete Wells"}'
# Typescript + Claude Computer Use
kernel invoke ts-claude-cu computer-use-query --payload '{"query": "Search for the top 3 restaurants in NYC according to Pete Wells"}'

# Python + Sample App
kernel invoke python-basic get-page-title --payload '{"url": "https://www.google.com"}'

# Python + Browser Use
kernel invoke python-bu bu-task --payload '{"task": "Compare the price of gpt-4o and DeepSeek-V3"}'

# Python + Claude Computer Use
kernel invoke python-claude-cu cu-task --payload '{"query": "Return the first url of a search result for NYC restaurant reviews Pete Wells"}'

# Typescript + CUA Sample
kernel invoke ts-cua cua-task --payload '{"task": "Go to https://news.ycombinator.com and get the top 5 articles"}'

Expand All @@ -140,7 +147,7 @@ These are the sample apps currently available when you run `npx @onkernel/create
| **browser-use** | Completes a specified task | Browser Use | `{ task }` |
| **stagehand** | Returns the first result of a specified Google search | Stagehand | `{ query }` |
| **advanced-sample** | Implements sample apps using advanced Kernel configs | n/a |
| **computer-use** | Implements a prompt loop | Anthropic Computer Use API | `{ query }` |
| **claude-cu** | Implements a prompt loop | Anthropic Computer Use API | `{ query }` |
| **cua** | Implements the OpenAI Computer Using Agent (CUA) | OpenAI CUA | `{ task }` |

## Documentation
Expand Down
25 changes: 13 additions & 12 deletions index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ type TemplateKey =
| "browser-use"
| "stagehand"
| "advanced-sample"
| "computer-use"
| "claude-cu"
| "cua";
type LanguageInfo = { name: string; shorthand: string };
type TemplateInfo = {
Expand All @@ -34,7 +34,8 @@ const TEMPLATE_SAMPLE_APP = "sample-app";
const TEMPLATE_BROWSER_USE = "browser-use";
const TEMPLATE_STAGEHAND = "stagehand";
const TEMPLATE_ADVANCED_SAMPLE = "advanced-sample";
const TEMPLATE_COMPUTER_USE = "computer-use";
const TEMPLATE_COMPUTER_USE = "claude-cu";
const TEMPLATE_ANTHROPIC_COMPUTER_USE = "claude-cu";
const TEMPLATE_CUA = "cua";
const LANGUAGE_SHORTHAND_TS = "ts";
const LANGUAGE_SHORTHAND_PY = "py";
Expand Down Expand Up @@ -70,8 +71,8 @@ const TEMPLATES: Record<TemplateKey, TemplateInfo> = {
"Implements sample actions with advanced Kernel configs",
languages: [LANGUAGE_TYPESCRIPT, LANGUAGE_PYTHON],
},
[TEMPLATE_COMPUTER_USE]: {
name: "Computer Use",
["claude-cu"]: {
name: "Claude Computer Use",
description: "Implements the Anthropic Computer Use SDK",
languages: [LANGUAGE_TYPESCRIPT, LANGUAGE_PYTHON],
},
Expand All @@ -93,8 +94,8 @@ const INVOKE_SAMPLES: Record<
'kernel invoke ts-stagehand stagehand-task --payload \'{"query": "Best wired earbuds"}\'',
[TEMPLATE_ADVANCED_SAMPLE]:
'kernel invoke ts-advanced test-captcha-solver',
[TEMPLATE_COMPUTER_USE]:
'kernel invoke ts-cu cu-task --payload \'{"query": "Return the first url of a search result for NYC restaurant reviews Pete Wells"}\'',
["claude-cu"]:
'kernel invoke ts-claude-cu computer-use-query --payload \'{"query": "Search for the top 3 restaurants in NYC according to Pete Wells"}\'',
[TEMPLATE_CUA]:
'kernel invoke ts-cua cua-task --payload \'{"query": "Go to https://news.ycombinator.com and get the top 5 articles"}\'',
},
Expand All @@ -105,8 +106,8 @@ const INVOKE_SAMPLES: Record<
'kernel invoke python-bu bu-task --payload \'{"task": "Compare the price of gpt-4o and DeepSeek-V3"}\'',
[TEMPLATE_ADVANCED_SAMPLE]:
'kernel invoke python-advanced test-captcha-solver',
[TEMPLATE_COMPUTER_USE]:
'kernel invoke python-cu cu-task --payload \'{"query": "Return the first url of a search result for NYC restaurant reviews Pete Wells"}\'',
["claude-cu"]:
'kernel invoke python-claude-cu cu-task --payload \'{"query": "Return the first url of a search result for NYC restaurant reviews Pete Wells"}\'',
[TEMPLATE_CUA]:
'kernel invoke python-cua cua-task --payload \'{"query": "Go to https://news.ycombinator.com and get the top 5 articles"}\'',
},
Expand All @@ -123,8 +124,8 @@ const REGISTERED_APP_NAMES: Record<
'ts-stagehand',
[TEMPLATE_ADVANCED_SAMPLE]:
'ts-advanced',
[TEMPLATE_COMPUTER_USE]:
'ts-cu',
["claude-cu"]:
'ts-claude-cu',
[TEMPLATE_CUA]:
'ts-cua',
},
Expand All @@ -135,8 +136,8 @@ const REGISTERED_APP_NAMES: Record<
'python-bu',
[TEMPLATE_ADVANCED_SAMPLE]:
'python-advanced',
[TEMPLATE_COMPUTER_USE]:
'python-cu',
["claude-cu"]:
'python-claude-cu',
[TEMPLATE_CUA]:
'python-cua',
},
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ class QueryOutput(TypedDict):
raise ValueError("ANTHROPIC_API_KEY is not set")

client = Kernel()
app = kernel.App("python-cu")
app = kernel.App("python-claude-cu")

@app.action("cu-task")
async def cu_task(
Expand Down
10 changes: 10 additions & 0 deletions templates/typescript/claude-cu/.prettierrc
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
{
"semi": true,
"trailingComma": "es5",
"singleQuote": true,
"printWidth": 80,
"tabWidth": 2,
"useTabs": false,
"bracketSpacing": true,
"arrowParens": "avoid"
}
3 changes: 3 additions & 0 deletions templates/typescript/claude-cu/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# Anthropic Computer Use Sample

This sample app demonstrates how to use the `@onkernel/cu-playwright` package to perform a simple search query.
17 changes: 17 additions & 0 deletions templates/typescript/claude-cu/_gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# Node
node_modules
dist
.DS_Store
npm-debug.log*
yarn-debug.log*
yarn-error.log*
pnpm-debug.log*

# Editor
.vscode
.idea
*.suo
*.ntvs*
*.njsproj
*.sln
*.sw?
38 changes: 38 additions & 0 deletions templates/typescript/claude-cu/eslint.config.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
import eslint from '@eslint/js';
import tseslint from '@typescript-eslint/eslint-plugin';
import tsparser from '@typescript-eslint/parser';
import prettierConfig from 'eslint-config-prettier';

export default [
eslint.configs.recommended,
{
files: ['**/*.ts', '**/*.tsx'],
languageOptions: {
parser: tsparser,
parserOptions: {
ecmaVersion: 'latest',
sourceType: 'module',
},
globals: {
process: 'readonly',
console: 'readonly',
Buffer: 'readonly',
__dirname: 'readonly',
__filename: 'readonly',
},
},
plugins: {
'@typescript-eslint': tseslint,
},
rules: {
...tseslint.configs.recommended.rules,
'@typescript-eslint/no-unused-vars': 'warn',
'@typescript-eslint/no-explicit-any': 'off',
'no-console': 'off',
},
},
prettierConfig,
{
ignores: ['node_modules/**', 'dist/**'],
},
];
59 changes: 59 additions & 0 deletions templates/typescript/claude-cu/index.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
import 'dotenv/config';
import { Kernel, type KernelContext } from '@onkernel/sdk';
import { chromium } from 'playwright';
import { ComputerUseAgent } from '@onkernel/cu-playwright';

const kernel = new Kernel();

const app = kernel.app('ts-claude-cu');

interface QueryInput {
query: string;
}

interface QueryOutput {
result: string;
}

app.action<QueryInput, QueryOutput>(
'computer-use-query',
async (ctx: KernelContext, payload?: QueryInput): Promise<QueryOutput> => {
const ANTHROPIC_API_KEY = process.env.ANTHROPIC_API_KEY;

if (!ANTHROPIC_API_KEY) {
throw new Error('ANTHROPIC_API_KEY is not set');
}

if (!payload?.query) {
throw new Error('Query is required');
}

const kernelBrowser = await kernel.browsers.create({
invocation_id: ctx.invocation_id,
});

const browser = await chromium.connectOverCDP(kernelBrowser.cdp_ws_url);

try {
const context = browser.contexts()[0];
if (!context) {
throw new Error('No browser context found.');
}
const page = context.pages()[0];
if (!page) {
throw new Error('No page found in browser context.');
}

const agent = new ComputerUseAgent({
apiKey: ANTHROPIC_API_KEY,
page,
});

const result = await agent.execute(payload.query);

return { result };
} finally {
await browser.close();
}
}
);
30 changes: 30 additions & 0 deletions templates/typescript/claude-cu/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
{
"name": "ts-claude-cu",
"module": "index.ts",
"type": "module",
"private": true,
"scripts": {
"format": "prettier --write .",
"lint": "eslint .",
"lint:fix": "eslint . --fix"
},
"peerDependencies": {
"typescript": "^5"
},
"dependencies": {
"@onkernel/cu-playwright": "^0.1.0",
"@onkernel/sdk": "^0.6.0",
"dotenv": "^16.6.0",
"playwright": "^1.52.0",
"zod": "^3.25.0"
},
"devDependencies": {
"@eslint/js": "^9.27.0",
"@typescript-eslint/eslint-plugin": "^8.32.1",
"@typescript-eslint/parser": "^8.32.1",
"eslint": "^9.27.0",
"eslint-config-prettier": "^10.1.5",
"prettier": "^3.0.0",
"typescript": "^5.0.0"
}
}
Loading