Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Incorrect override amounts when using binary expressions #1484

Open
mattrossman opened this issue Aug 29, 2024 · 3 comments · May be fixed by #1485
Open

Incorrect override amounts when using binary expressions #1484

mattrossman opened this issue Aug 29, 2024 · 3 comments · May be fixed by #1485

Comments

@mattrossman
Copy link
Contributor

When an expression is set to binary with isBinary: true, and these expressions also have blended overrides (e.g. overrideLookAt: "blend"), their weight is blended continuously instead of discretely when the corresponding expression group (e.g. gaze expression) has nonzero weight. I expect binary expressions to maintain the sharp 0.5 threshold whether using "blend" overrides or not.

I believe this stems from the overrideBlinkAmount / overrideLookAtAmount / overrideMouthAmount calculations not taking into account the isBinary flag, e.g.:

public get overrideLookAtAmount(): number {
if (this.overrideLookAt === 'block') {
return 0.0 < this.weight ? 1.0 : 0.0;
} else if (this.overrideLookAt === 'blend') {
return this.weight;
} else {
return 0.0;
}
}

Changing the expression in the 'blend' case to this.isBinary ? (this.weight <= 0.5 ? 0.0 : 1.0) : this.weight as is done elsewhere in the applyWeight logic produces the expected discrete blending behavior.

I wonder if this also affects the 'block' override mode, as the binary expression will begin blocking as soon as it has nonzero weight, even though it doesn't visually apply that weight until after the 0.5 weight threshold. So perhaps the 'block' case should look more like:

const actualWeight = this.isBinary ? (this.weight <= 0.5 ? 0.0 : 1.0) : this.weight;
return 0.0 < actualWeight ? 1.0 : 0.0;
@mattrossman mattrossman linked a pull request Aug 29, 2024 that will close this issue
@0b5vr
Copy link
Contributor

0b5vr commented Aug 30, 2024

Woow, that is a factor I haven't considered in the spec layer; thank you.
Let me consider the interaction between them, I'm not sure your PR is working perfect or not.

@mattrossman
Copy link
Contributor Author

A comparison of binary expressions with texture transform binds and "blend" lookAt overrides:

Before After
CleanShot.2024-08-30.at.10.36.18.mp4
CleanShot.2024-08-30.at.10.38.13.mp4

Notice before how when lookLeft is applied, texture transforms on the blink expression apply smoothly instead of a sharp binary change, which breaks the use case of expression textures.

@0b5vr
Copy link
Contributor

0b5vr commented Sep 5, 2024

Thank you for the detailed explanation! We are currently discussing the behavior with the VRM spec team. I'm 90% confident that we can fix it in the desired way, but we must ensure that the behavior can be described as a spec without confusion before we apply this in our implementation.

0b5vr added a commit that referenced this issue Sep 6, 2024
It's about interaction between isBinary and override expressions
See the discussion: #1484

Also add a test of VRMExpression to confirm the desired behavior
0b5vr added a commit to 0b5vr/vrm-specification that referenced this issue Sep 6, 2024
…e interacts each other

- The original discussion: pixiv/three-vrm#1484
- The implementation example on three-vrm: pixiv/three-vrm#1489

TODO: english translation
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants