Skip to content

Commit

Permalink
Rewrite code to work with a fixed-size video element
Browse files Browse the repository at this point in the history
  • Loading branch information
Valkryst committed Dec 4, 2023
1 parent ba1e530 commit ac81b6f
Show file tree
Hide file tree
Showing 4 changed files with 9 additions and 29 deletions.
12 changes: 4 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,17 +33,13 @@ Additionally, the [Mesh](https://github.com/Valkryst/Jellron/blob/master/js/mesh

### Styling

When styling the `video` element VIA CSS, it is assumed that the displayed video must take up the entire allotted
width. Internal code relies on this assumption and the element's height is calculated based on it's width. This is done
to maintain the camera's (MediaStream's) aspect ratio.

_Do not specify anything for the element's height. Not even `height: auto`._
You _must_ specify a fixed size for the `video` element VIA CSS. There are a number of issues in implementing dynamic
sizing, in a way that works across many browsers, so it is currently unsupported.

```css
video {
width: 100%; /* Set this to your desired value. */

object-fit: fill;
height: 480px;
width: 640px;
}
```

Expand Down
5 changes: 2 additions & 3 deletions css/styles.css
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,8 @@ html {
}

& > video {
width: 50%;

object-fit: fill;
height: 480px;
width: 640px;
}
}
}
Expand Down
13 changes: 2 additions & 11 deletions js/camera.js
Original file line number Diff line number Diff line change
Expand Up @@ -125,17 +125,8 @@ export class Camera {
* Both FaceDetector and HandDetector use the video element's height and width properties to scale the input
* MediaStream before processing it. This is why we need to set them.
*/
this.videoElement.height = this.videoElement.clientWidth / (await this.getMediaStreamAspectRatio());
this.videoElement.width = this.videoElement.clientWidth;
}

/**
* Calculates the aspect ratio of the MediaStream associated with the video element of this Camera object.
*
* @returns {Promise<number>} A promise that resolves to the aspect ratio.
*/
async getMediaStreamAspectRatio() {
return (await this.getMediaStreamWidth()) / (await this.getMediaStreamHeight());
this.videoElement.height = this.videoElement.scrollHeight;
this.videoElement.width = this.videoElement.scrollWidth;
}

/**
Expand Down
8 changes: 1 addition & 7 deletions js/detector/body_detector.js
Original file line number Diff line number Diff line change
Expand Up @@ -40,20 +40,14 @@ export class BodyDetector extends Detector {
this.intervalId = setInterval(async () => {
const currentTime = performance.now();

let frame = null;
let rawBodies = [];
try {
frame = tf.browser.fromPixels(videoElement);
frame = tf.image.resizeBilinear(frame, [videoElement.height, videoElement.width]);

rawBodies = await this.detector.estimatePoses(frame);
rawBodies = await this.detector.estimatePoses(videoElement);
} catch (e) {
/*
* Depending on the state of the video element, this can throw a "Requested texture size [0x0] is
* invalid." error. It doesn't seem to cause any issues, so we ignore it.
*/
} finally {
frame?.dispose();
}

if (rawBodies.length === 0) {
Expand Down

0 comments on commit ac81b6f

Please sign in to comment.