Questions about specifics of Concurrent scheduling

[alvarlagerlof]

I want to start of by saying thanks to the team for the awesome content so far, and for the opportunity to be here!

That said, one of the things that didn't quite match the mental model I had before is exactly how prioritization in scheduling works. I think in general mismatches like these are important for us not on the team to point out, since it is likely that other people in the community are going to have the same misconceptions.

As far as I know, React is not currently using workers or wasm, which means that all code execution happens in one thread. Before Concurrent, my understanding is that every task started are executed linearly, meaning that code can block input events and such.

With Concurrent, there is still one thread, but there is prioritization going on. I thought this meant sometimes pausing code in order to first run things like input events. This does not seem to be entirely true. From what I've read so far, tasks can have different priorities and execute at different speed depending on how critical to user responsiveness they are.

I'd like deeper information on this, so here are a few open questions:

• How is work divided?
• Is work ever paused (if so what happens to network)? Or is it a queue of small tasks that can be reordered?
• Does work run at different speed or is it different amounts of work?
• Does work happening in Suspense mean an actual "new branch" of work, or is it interweaved with the rest?
• What level of granularity does the prioritization have?
• How does priority work of Offscreen?
• Will React always run things at max speed even if all work is low priority?

[acdlite]

Thanks for the questions! This is a big topic and I won't be able include everything in a single response. So I'll give quick answers to each of your bullets, then you can follow up with more specific questions if you like.

I'll also post them as separate answers to keep the threads organized.

[acdlite]

Q: How is work divided?

As you mentioned, we don't use workers or WASM for parallelism. Ours is a cooperative multitasking model — a single rendering thread, but interruptible so that rendering can be interleaved with other main thread tasks, like user input, network events, timers, animations, browser layout/paint, and so on.

Because rendering is interruptible, rendering can also be interleaved with other React renders, which is important because in a typical React app, rendering is most of what happens on the main thread. A low priority update won't starve a high priority update. We can start rendering an update in the background without worrying about blocking an update in response to new input; when that happens, we switch to the more urgent rendering task, then pick it up the original task after that has finished.

In a cooperative scheduler, we can't forcibly interrupt rendering whenever we want in the way that, say, an OS scheduler can suspend a process. But we can yield in between React different components. Since most individual components don't take longer than a single frame to render, that's enough granularity that, in practice, rendering is effectively interruptible whenever we need it to be.

Our current heuristic is to yield execution back to the main thread every 5ms. 5ms isn't a magic number, the important part is that it's smaller than a single frame even on 120fps devices, so it won't block animations.

If nothing is else is queued in the browser's event loop, then we'll immediately re-renter the React work loop and pick up where we left off.

[acdlite]

Q: Does work run at different speed or is it different amounts of work?

The rendering algorithm is mostly the same for all types of updates.

We do have a slight optimization for synchronous updates. (Which do still exist even in an app that has fully adopted concurrent features. We use them to process updates that are dispatched by a discrete event, like text input.) Because synchronous updates are the highest priority, there's no reason to track whether we've reached the end of the 5ms time slice, since we're already working on the most urgent thing. So we skip that check until the render is done.

We've discussed the possibility that we may fork the synchronous rendering algorithm from the concurrent one even further in the future. For example, a synchronous render could use the regular JS stack instead of storing stack variables on fiber objects. This could be especially useful for a built-in animations API, where every millisecond of CPU execution really matters.

But for now it's pretty much the same regardless of the type of update, with the same overhead.

Regarding whether some updates are "different amounts of work", that really isn't any different from legacy React. The more components that re-render as a result of an update, the slower the update will be. You can continue to use the DevTools Profiler to optimize your components.

However, the size of the update does usually correlate to its priority class. Synchronous discrete updates should be small and focused so that they can present immediate feedback to the user. But updates that trigger a full page transition can afford to take longer, especially since they're likely to be IO bound, anyway.

[alvarlagerlof]

That makes sense. To be clear I think I really meant "size" when I said "amount of work"

[alvarlagerlof]

Also, I can't help but notice "built-in animations api". I've heard Dan mention this once before, but is this something that's planned for the future? I totally understand if you do not want to reveal anything in detail right now.

[acdlite]

Yeah it's planned for the future, @sebmarkbage has been thinking a lot about it but no concrete proposal yet

[acdlite]

Q: What level of granularity does the prioritization have?

This is a fascinating topic and I would say it's the part of the our model that has changed the most radically since the original implementation.

Usually I try to explain our design decisions without mentioning implementation details, but in this case I think it might be easier to understand if you know a bit about how it's implemented.

We have 31 levels of granularity. It's 31 because that's how many we can fit into a single bitmask. Each bit in a bitmask is called a Lane.

Every React render is associated with one or more lanes, represented by a bitmask. (In the source, we call this variable renderLanes.)

Every update in React is assigned exactly one lane. If two updates are assigned the same lane, that means they will always render in the same batch. If they have different lanes, that means they may be rendered in separate batches.

For priorities that are assumed to be CPU-bound — meaning if something suspends we will immediately show a fallback instead of waiting for promises to resolve — we use only a single lane per priority. For example, all synchronous updates are assigned the SyncLane. If there are multiple pending sync updates, they will always render in a single batch. This is almost always better for performance because it reduces overhead of multiple layout passes, multiple style recalculations, multiple paints, and so on.

But the benefits of batching don't hold if the updates aren't CPU-bound. We have a several priorities like this; the main kind are called transitions. These are updates that are wrapped in a startTransition call. When a transition suspends, sometimes we choose not to show a fallback immediately (exactly why and how we decide to do this is a subject for another thread) and instead wait for the promises to resolve. This means that these updates can sometimes be IO-bound instead of CPU-bound. If we were to assign the same lane to all transitions, then one transition could effectively block all other transitions, even ones that are unrelated. In this way, batching can actually hurt perceived performance instead of helping.

So transitions don't all receive the same lane. We assign different lanes to successive transitions. Sometimes two unrelated transitions may happen to be assigned the same lane, because we only have a finite number. But usually in practice they'll have different lanes, and therefore can be finished independently.

This only scratches the surface of the design but I'll leave it there for now and provide more details if prompted.

The tl;dr is that lanes give us the flexibility to choose whether to render multiple transitions in a single batch or render them independently. Perhaps even out of order.

[alvarlagerlof]

Not at all sure what to ask about to understand in more detail, but I'm wondering, why a bitmask?

[acdlite]

It's so we can quickly check if a particular tree has matching updates, and skip over it if it doesn't.

Every component tree has a bitmask field called subtreeLanes that represents the union of all the scheduled updates in that tree.

So if we're rendering the lanes 0b011, and we encounter a tree with lanes 0b001, we will continue rendering through that three.

But if we encounter another tree with lanes 0b100, we know there are no matching updates, and can quickly re-use the current tree without re-rendering.

This is why setState is super fast for local updates — React can completely skip over the parents and siblings of the component that was updated, since they are guaranteed not to have changed.

[acdlite]

Q: Does work happening in Suspense mean an actual "new branch" of work, or is it interweaved with the rest?

We use a cooperative multitasking algorithm with a single rendering thread. The branch metaphor still works, though, because while a transition is suspended, we may not be able to make any additional progress on it until a promise resolves. If there are other pending transitions, we will switch to working on those in the meantime. We may switch back and forth multiple times before either transition has fully resolved. So although the rendering isn't truly parallel, in the sense of running on separate hardware threads, they still have many of the properties of parallel processes.

[acdlite]

Q: How does priority work of Offscreen?

In terms of prioritization, Offscreen works about the same as any other priority level. It happens to be the lowest one, so literally any other type of work will take precedent, and interrupt it if it already started.

Offscreen renders do have other special properties, though. Like if we're rendering at a normal priority level, and we reach an offscreen tree, we can defer the tree until a later render without showing inconsistent UI, because the offscreen tree is hidden. This is similar to what we do when we replace a Suspense boundary with a fallback; indeed, the Suspense component shares much of its implementation with the Offscreen component.

[sophiebits]

Does Offscreen use requestIdleCallback to schedule work?

[alvarlagerlof]

I was under the impression that everything uses https://www.npmjs.com/package/scheduler ?

[rickhanlonii]

Offscreen would use the idle lanes, which would cause them to be scheduled with IdlePriority. We removed requestIdelCallback from the scheduler and replaced it with a user-space queue that yields every 5ms instead (for now).