Skip to content

Commit

Permalink
Remove Async Validation in process async (#933)
Browse files Browse the repository at this point in the history 
Remove the async validators from instructors, targetting a re-release in subsequent versions.
  • Loading branch information
@ivanleomk
ivanleomk committed Aug 22, 2024
1 parent 09abebc commit b338582
Show file tree
Hide file tree
Showing 6 changed files with 4 additions and 888 deletions.
249 changes: 0 additions & 249 deletions docs/concepts/reask_validation.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -193,255 +193,6 @@ except ValidationError as e:
)
```

## Async Validation

Pydantic doesn't support async validators out of the box at the moment. We've implemented two validators that you can use out of the box to run async calls.

This is especially useful if you are making multiple api calls in parallel. This implementation is heavily inspired by [pydantic-async-validation](https://github.com/team23/pydantic-async-validation/tree/64e4190494141c881ade30963390f91c3286dafc).

!!! warning "Async Client Required"

The async validation features (`async_field_validator` and `async_model_validator`) only work with an asynchronous client (Eg. `AsyncOpenAI()` vs `OpenAI()`).

### Field Validators

To use the `async_field_validator`, all you'll need to do is to define your standard Pydantic `BaseModel` and import in the `async_field_validator` from instructor.

```python hl_lines="11"
from pydantic import BaseModel
import asyncio
from instructor import from_openai, async_field_validator
from openai import AsyncOpenAI


class User(BaseModel):
label: str
name: str

@async_field_validator("label", "name") # (1)!
async def validate_uppercase(self, v: str) -> str:
if not v.isupper():
raise ValueError(f"{v} must be an upper cased value")


client = from_openai(AsyncOpenAI())

resp = asyncio.run(
client.chat.completions.create(
model="gpt-4o-mini",
messages=[
{
"role": "user",
"content": "Extract out a user from this sentence - Ivan is a 27 year old software engineer",
}
],
response_model=User,
)
)
print(resp)
#> label='SOFTWARE ENGINEER' name='IVAN'
```

1. You can set the fields to run the same validator on. We don't support wildcard characters (eg. '\*' ) at this moment but might in the future.

### Model Validators

We can also use a model validator to validate a combination of fields. In this case, we want to make sure that every field is in caps. This helps us to prevent errors.

```python hl_lines="11"
from pydantic import BaseModel
import asyncio
from instructor import from_openai, async_model_validator
from openai import AsyncOpenAI


class User(BaseModel):
name: str
occupation: str

@async_model_validator() # (1)!
async def validate_uppercase(self):
if not self.name.isupper() or not self.occupation.isupper():
raise ValueError("Name and Occupation must be capitalized (Eg. TOM, COOK)")


client = from_openai(AsyncOpenAI())

resp = asyncio.run(
client.chat.completions.create(
model="gpt-4o-mini",
messages=[
{
"role": "user",
"content": "Extract out a user from this sentence - Ivan is a 27 year old software engineer",
}
],
response_model=User,
)
)

assert resp.name == "IVAN"
assert resp.occupation == "SOFTWARE ENGINEER"
```

1. This will run the validator on the entire model. You can access any of the fields in this manner and it will be propagated up on a model level

### Nested Pydantic Objects

We also support nested validation as long as they inherit from Pydantic's `BaseModel` class

```python hl_lines="9-15"
from instructor import async_model_validator
from pydantic import BaseModel
import asyncio
from instructor import from_openai
from openai import AsyncOpenAI
import time


class User(BaseModel): # (1)!
name: str
occupation: str

@async_model_validator()
async def validate_uppercase(self):
await asyncio.sleep(2)


class Users(BaseModel):
users: list[User]


client = from_openai(AsyncOpenAI())


start = time.time()
resp = asyncio.run(
client.chat.completions.create(
model="gpt-4o-mini",
messages=[
{
"role": "user",
"content": "Extract out all the users from this sentence - Thomas is a 24 year old software engineer and has a roomate called Jonathan that is around the same age and works as a Barista",
}
],
response_model=Users,
)
)
end = time.time()
print(f"Elapsed Time: {end-start}")
#> Elapsed Time: 3.375563859939575
print(resp.model_dump_json())
# {
# "users": [
# {"name": "Thomas", "occupation": "software engineer"},
# {"name": "Jonathan", "occupation": "Barista"},
# ]
# }
```

1. As long as the class inherits from the `BaseModel` class, we'll be able to run the validation you define on a property

### Validation Context

Sometimes, you might want your validators to be able to access some context you provide at runtime.

We allow you to access an arbitrary dictionary by defining an `info` parameter of type `ValidationInfo` in both the `async_model_validator` and `async_field_validator` objects.

```python hl_lines="12-16 36"
from pydantic import ValidationInfo, BaseModel
from instructor import async_model_validator
import asyncio
from instructor import from_openai
from openai import AsyncOpenAI


class User(BaseModel):
name: str
occupation: str

@async_model_validator()
async def validate_uppercase(self, info: ValidationInfo): # (1)!
context = info.context
if context and self.name in context.get("forbidden_names", []):
raise ValueError(f"Do not include the name {self.name}")


class Users(BaseModel):
users: list[User]


client = from_openai(AsyncOpenAI())


resp = asyncio.run(
client.chat.completions.create(
model="gpt-4o-mini",
messages=[
{
"role": "user",
"content": "Extract out all the users from this sentence - Thomas is a 24 year old software engineer and has a roomate called Jonathan that is around the same age and works as a Barista",
}
],
response_model=Users,
validation_context={"forbidden_names": "Thomas"}, # (2)!
)
)
print(resp.model_dump_json())
# {"users": [{"name": "Jonathan", "occupation": "Barista"}]}
```

1. To access the Validation Context in a validator, just define a property called `info:ValidationInfo` from Pydantic
2. You can pass in any arbitrary dictionary here for use in the validation_context

We also support the same api with the `field_validator` decorator

```python hl_lines="12-16 36"
from pydantic import ValidationInfo, BaseModel
from instructor import async_field_validator
import asyncio
from instructor import from_openai
from openai import AsyncOpenAI


class User(BaseModel):
name: str
occupation: str

@async_field_validator("name") # (1)!
async def validate_uppercase(self, v: str, info: ValidationInfo):
context = info.context
if context and v in context.get("forbidden_names", []):
raise ValueError(f"Do not include the user {v}")


class Users(BaseModel):
users: list[User]


client = from_openai(AsyncOpenAI())


resp = asyncio.run(
client.chat.completions.create(
model="gpt-4o-mini",
messages=[
{
"role": "user",
"content": "Extract out all the users from this sentence - Thomas is a 24 year old software engineer and has a roomate called Jonathan that is around the same age and works as a Barista",
}
],
response_model=Users,
validation_context={"forbidden_names": "Thomas"}, # (2)!
)
)
print(resp.model_dump_json())
# {"users": [{"name": "Jonathan", "occupation": "Barista"}]}
```

1. To access the Validation Context in a validator, just define a property called `info:ValidationInfo` from Pydantic
2. You can pass in any arbitrary dictionary here for use in the validation_context

## Advanced Validation Techniques

The docs are currently incomplete, but we have a few advanced validation techniques that we're working on documenting better such as model level validation, and using a validation context. Check out our example on [verifying citations](../examples/exact_citations.md) which covers:
Expand Down
3 changes: 0 additions & 3 deletions instructor/__init__.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,6 @@
from_litellm,
Provider,
)
from .validators import async_field_validator, async_model_validator

__all__ = [
"Instructor",
Expand All @@ -44,8 +43,6 @@
"Instructions",
"handle_parallel_model",
"handle_response_model",
"async_field_validator",
"async_model_validator",
]


Expand Down
Loading

0 comments on commit b338582

Please sign in to comment.