-
Notifications
You must be signed in to change notification settings - Fork 1.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
chore(qwik-city/middleware): add TextEncoderStream polyfill and tests #6310
Conversation
Deploying qwik-docs with Cloudflare Pages
|
✅ Deploy Preview for qwik-insights ready!
To edit notification comments on pull requests, go to your Netlify site configuration. |
@wmertens when you get time can you review |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Saw this PR and decided to add a few bits after looking closer myself to TextEncoderStream
. This implementation is shorter, and I believe it's fine to keep it as it is, but I have a few thoughts from what I understand:
- The
TextEncoderStream
class is not an extension of theTransformStream
(you can see that from the documentation on MDN here), but it has same generic interface withreadable
andwritable
properties (see the spec here). I'm not sure if we are trying to achieve the most compatibility, but you can refer to Deno's and Node.js' implementations if that's the case. - There are properties that I don't see on this class either. I left more comments on the specifics :)
yeah the props are there only to match the previous polyfill. so everything is just-in-case |
Oh, sure. But anyway here's TypeScript flavored version of TextEncoderStream polyfill based on Node.js' implementation, just in case: /**
* TextEncoderStream polyfill based on Node.js' implementation https://github.com/nodejs/node/blob/3f3226c8e363a5f06c1e6a37abd59b6b8c1923f1/lib/internal/webstreams/encoding.js#L38-L119 (MIT License)
*/
export class TextEncoderStream {
#pendingHighSurrogate: string | null = null
#handle = new TextEncoder()
#transform = new TransformStream<string, Uint8Array>({
transform: (chunk, controller) => {
// https://encoding.spec.whatwg.org/#encode-and-enqueue-a-chunk
chunk = String(chunk)
let finalChunk = ""
for (const item of chunk) {
const codeUnit = item.charCodeAt(0)
if (this.#pendingHighSurrogate !== null) {
const highSurrogate = this.#pendingHighSurrogate
this.#pendingHighSurrogate = null
if (codeUnit >= 0xDC00 && codeUnit <= 0xDFFF) {
finalChunk += highSurrogate + item
continue
}
finalChunk += "\uFFFD"
}
if (codeUnit >= 0xD800 && codeUnit <= 0xDBFF) {
this.#pendingHighSurrogate = item
continue
}
if (codeUnit >= 0xDC00 && codeUnit <= 0xDFFF) {
finalChunk += "\uFFFD"
continue
}
finalChunk += item
}
if (finalChunk) {
controller.enqueue(this.#handle.encode(finalChunk))
}
},
flush: controller => {
// https://encoding.spec.whatwg.org/#encode-and-flush
if (this.#pendingHighSurrogate !== null) {
controller.enqueue(new Uint8Array([0xEF, 0xBF, 0xBD]))
}
}
})
get encoding() {
return this.#handle.encoding
}
get readable() {
return this.#transform.readable
}
get writable() {
return this.#transform.writable
}
get [Symbol.toStringTag]() {
return "TextEncoderStream"
}
} |
looks good lets try it |
@octet-stream thanks for finding this better polyfill |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Tests pass locally and make sense on my end.
My understanding, is that this polyfill is removing the previous one, and is a more robust version that should support the 5% or so where this is not supported.
Is that the case? Anything I should look in-depth into?
Otherwise LGTM! 🙌
add _TextEncoderStream_polyfill and tests