Ray Queries #6291

Vecvec · 2024-09-18T04:40:11Z

Connections
Works towards #1040

Description
This PR provides BLASes (bottom level acceleration structures), TLASes (top level acceleration structures), TLAS instances (which contain BLASes and data, like transforms, about them), TLAS packages (which contain vectors of TLAS instances).

Testing
Running tests & examples included in PR

This a updated version of #3631 and is inteanded to replace it.

Checklist

Run cargo fmt.
Run cargo clippy. If applicable, add:
- n/a --target wasm32-unknown-unknown
- n/a --target wasm32-unknown-emscripten
Run cargo xtask test to run tests.
Add change to CHANGELOG.md. See simple instructions inside file.
More tests & examples.
More docs

Later (follow-up PR)

Acceleration structure compaction.
Procedural geometry.
as_hal methods

…ructures

…u into hal-acceleration-structures

…ructures

+ seperated RAY_QUERY

…ructures

(better allocation strategy required)

Co-authored-by: JMS55 <47158642+JMS55@users.noreply.github.com>

…ructures

# Conflicts: # tests/tests/root.rs # wgpu-core/src/command/compute.rs # wgpu-core/src/command/mod.rs # wgpu-core/src/command/ray_tracing.rs # wgpu-core/src/command/render.rs # wgpu-core/src/device/life.rs # wgpu-core/src/device/queue.rs # wgpu-core/src/device/ray_tracing.rs # wgpu-core/src/device/resource.rs # wgpu-core/src/hub.rs # wgpu-core/src/lock/rank.rs # wgpu-core/src/ray_tracing.rs # wgpu-core/src/resource.rs # wgpu-core/src/track/mod.rs # wgpu/src/backend/wgpu_core.rs # wgpu/src/context.rs # wgpu/src/ray_tracing.rs

…cing-updated

JMS55 · 2024-09-18T23:13:02Z

I'm super excited for this!

What makes this PR a draft? What do you think is missing?

Vecvec · 2024-09-18T23:29:11Z

Examples, there is no case for no index buffer (but I'm working on it, hopefully done in a day or so).

…cing-updated

nical · 2024-09-19T12:17:45Z

This looks very exciting. This is a pretty large PR and to my knowledge, none of the wgpu maintainers are ray tracing experts. So to facilitate the review process, while we try to ramp up our knowledge of the domain, could you summarize here the context (that is scattered in #1040 and other places)? In particular:

The general ideas behind the design of the API compared to DX12/vulkan/metal:
- What subset of the functionality does the design cover.
- Tradeoffs (if any) compared to the underlying ray tracing APIs.
The scope of this work (for example, only implementing a specific portion of the overall plan).
Validation:
- What are the important aspects of validation to be aware of for safety? For example preventing undefined behavior and out-of-bound memory accesses in shaders.
- Any missing safety validation in this PR? Not saying the initial implementation should be bullet-proof enough to be a candidate for the WebGPU specification, but we should be aware of the shortcuts we take if any.
Any other important things to know (For example tricky aspects of mapping to this or that API)?

Vecvec · 2024-09-19T20:58:31Z

To preface this, I came later to this (after most of the design was finalized), so I may not know all of the missing validation, tradeoffs, etc. I will be talking about the safe tlas building (blas building is always safe but there is a extra unsafe function to do unsafe tlas building which might (?) be faster)

To my knowledge the only missing pieces of validation are ~~the vertex formats when creating a BLAS, metal (not yet implemented, but a consideration) only allows f32x3~~ (edit : finished), and tlas packages (which are only in wgpu) being able to have a different max instances from the max instances a tlas was created with (though I can't remember if it gets checked when building), this one is hard to validate, and later it might be a good idea to unify the tlases and tlas packages (especially if build acceleration structures unsafe gets removed).

The main tradeoffs are not exposing the scratch buffers because they are harder to validate (possibly) and could cause confusion (especially since most of that is lower level than wgpu provides, and also could be accidently be read back, giving garbage info and leaving the user confused) , which is why in an attempt to mitigate the allocation time blas building and tlas building are unified.

This implementation lacks compaction, which was in the initial API design, but needs another HAL function. Procedural geometry (naga needs to implement this, and if most people want to partially support this they will mainly use triangles). I believe there is a full wgpu API (though without ray-tracing pipelines)

Extra things to note: this is just for wgpu and wgpu-core, naga support (for ray queries) was added earlier. There is a ray-tracing pipeline that would be nice to have (it could allow a ray-tracer - e.g path-tracer - to be extended).

Edit: wgpu-hal support was also added earlier, but only for vulkan (so some things here still have to consider metal and dx12, though dx12 also was considered)

The naming is slightly confusing, this might be called ray-tracing, but something that uses this API to create something that simulates light bouncing around a scene might also be said to be ray-tracing (this does not do the second thing).

I think that is everything. Tell me if there are other questions about the API.

Edit # 2: mostly api is from #1040 (comment)

…RAY_TRACING_ACCELERATION_STRUCTURE is enabled

Vecvec · 2024-09-20T05:52:03Z

I've realised my previous comment requies quite some preexisting raytracing knowledge, so here are the API concepts:

BLASes
- A BLAS contains geometry in a device readable format, you can't interact directly with this, instead you have to build the BLAS with the buffers containing triangles or AABB.
TLAS instances
- A TLAS instance contains
  - A BLAS (or in native APIs it's handle)
  - A transformation matrix for the BLAS
  - Culling flags (if shader cull mask & instance.mask != 0 from vulkan this can be hit)
  - A custom index (this is data provided into the shader, unused elsewhere)
TLASes
- A TLAS contains TLAS instances in a device readable form, you cant interact directly with these either, instead you have to build the TLAS with TLAS instances
- Note: most of the native APIs require the TLAS instances on a buffer. Wgpu's safe API however requires TLAS instances on the CPU, this is because native APIs have a handle to the BLAS in the TLAS instance, but this is almost impossible to validate. Having TLAS instances in a buffer is supported by build_acceleration_structures_unsafe_tlas and is why it is unsafe.
TLAS packages
- A TLAS package contains
  - A TLAS
  - TLAS Instances
- Note: because TLASes build with their data in a buffer this does not exist in native APIs and is only used in safe building in wgpu

All metal, vulkan, and DXR use roughly these concepts (and they are based on one another).

Vecvec · 2024-09-20T06:30:01Z

I've noticed that a validation fix for spv (in trunk now see #5760) was duplicated in this PR, so I've removed that. (I think this was why gfx-rs/naga was requested for review)

nical

I did a first pass, it looks like a solid contribution, thanks a lot! Since I'm still wrapping my head around the domain, so what is coming out of this first review is the usual round of stylistic changes. There is probably going to be more of those in the next pass and hopefully soon a more thorough review of the more interesting/technical aspects of the work.

nical · 2024-09-20T08:11:52Z

wgpu-core/src/command/ray_tracing.rs

+            .map(|x| {
+                let geometries = match x.geometries {
+                    BlasGeometries::TriangleGeometries(triangle_geometries) => {
+                        crate::ray_tracing::TraceBlasGeometries::TriangleGeometries(


nitpick: let's use symbols like crate::ray_tracing::TraceBlasGeometries at the top of the file to avoid the long names.
Let's also use crate::device::trace to shorten these as well (keeping the trace part visible in the code is valuable in this case).

nical · 2024-09-20T08:13:02Z

wgpu-core/src/command/ray_tracing.rs

+        )
+        .unwrap();
+
+        #[cfg(feature = "trace")]


I think that it would be easier to read if this sequence of conditionally compiled code was all in a

#[cfg(feature = "trace")] { // move the code in here }

block.

nical · 2024-09-20T08:22:47Z

wgpu-core/src/command/ray_tracing.rs

+    Ok(())
+}
+
+/// Iterates over the buffers generated [iter_blas] and convert the barriers into hal barriers, and the triangles into [hal::AccelerationStructureEntries] (and also some validation).


Suggested change

/// Iterates over the buffers generated [iter_blas] and convert the barriers into hal barriers, and the triangles into [hal::AccelerationStructureEntries] (and also some validation).

/// Iterates over the buffers generated in [iter_blas], convert the barriers into hal barriers, and the triangles into [hal::AccelerationStructureEntries] (and also some validation).

nical · 2024-09-20T08:35:43Z

wgpu-core/src/device/ray_tracing.rs

+                    Vec::<hal::AccelerationStructureTriangles<dyn hal::DynBuffer>>::with_capacity(
+                        desc.len(),
+                    );
+                for x in desc {


Let's give x a descriptive name.

Suggested change

for x in desc {

for desc in descriptors {

nical · 2024-09-20T08:40:42Z

wgpu-core/src/ray_tracing.rs

+/// - lazy instance buffer allocation
+/// - maybe share scratch and instance staging buffer allocation
+/// - partial instance buffer uploads (api surface already designed with this in mind)
+/// - ([non performance] extract function in build (rust function extraction with guards is a pain))


It's a bit off how this comment for the file sits between two use statements and seems to apply to the use std::{num::NonZeroU64, slice}; under it. Let's move it to the first line and also make it a regular // comment instead of a doc comment.

nical · 2024-09-20T08:47:41Z

wgpu-types/src/lib.rs

+pub enum BlasGeometrySizeDescriptors {
+    /// Triangle geometry version.
+    Triangles {
+        /// Descriptor for each triangle geometry.
+        desc: Vec<BlasTriangleGeometrySizeDescriptor>,
+    },
+}


This descriptor should take a slice instead of a vector.

Also, wgpu so far has avoided chopping off words in the public API. I think that it's a good thing, and the plural form is informative.

Suggested change

pub enum BlasGeometrySizeDescriptors {

/// Triangle geometry version.

Triangles {

/// Descriptor for each triangle geometry.

desc: Vec<BlasTriangleGeometrySizeDescriptor>,

},

}

pub enum BlasGeometrySizeDescriptors<'a> {

/// Triangle geometry version.

Triangles {

/// Descriptor for each triangle geometry.

descriptors: &'[BlasTriangleGeometrySizeDescriptor],

},

}

For consistency with the rest of the API, BlasGeometrySizeDescriptors should be passed by reference (for example in create_blas.

This is required to have no lifetimes because it is used in the wgpu-core BLAS here
https://github.com/Vecvec/wgpu/blob/0fcc749efda0f22ccbcd5dffed3442d68fd17850/wgpu-core/src/resource.rs#L1924-L1925
doing this would require a duplicate size descriptor for wgpu-core that was a vector.

Edit: This sounds like I'm disagreeing, which I am not, I just want to know whether I should still do this if it had a reason.

nical · 2024-09-20T08:52:48Z

api_spec.rs

@@ -0,0 +1,154 @@
+// Ray tracing api proposal for wgpu (underlining Vulkan, Metal and DX12 implementations)


This file should be removed from the PR before merging.

nical · 2024-09-20T08:59:12Z

.gitattributes

 *.mtl binary
 *.obj binary
 wgpu/src/backend/webgpu/webgpu_sys/** linguist-generated=true
+>>>>>>> trunk


Looks like this change is not intentional.

expenses and others added 30 commits September 18, 2022 15:05

Merge remote-tracking branch 'origin/master' into hal-acceleration-st…

e786743

…ructures

Fix example, hopefully have fixed metal code

6f2b07a

Merge branch 'hal-acceleration-structures' of github.com:expenses/wgp…

9aca7f2

…u into hal-acceleration-structures

Changed surface_format in example.

3828af7

Changed based on PR 3020#pullrequestreview-1139691697 feedback.

e2de5a3

Merge remote-tracking branch 'origin/master' into hal-acceleration-st…

112b549

…ructures

unsafe blocks in unsafe functions

074ebe5

formatted code

8ce204b

added utility functions to AccelerationStructureInstance (bit packing)

f52a70b

WIP api change build_acceleration_structures

f15e865

preperation for changes in get_acceleration_structure_build_sizes

a3c0ffb

WIP api change get_acceleration_structure_build_sizes

a59d888

added aabbs and geometry flag support

93a347d

fixed RAY_TRACING feature collision

16cc4eb

+ seperated RAY_QUERY

renamed to build_acceleration_structure

5b84238

fixed missing type parameter for metal

caa4ae7

Merge remote-tracking branch 'origin/master' into hal-acceleration-st…

ca3a54c

…ructures

Merge remote-tracking branch 'origin/master' into hal-acceleration-st…

ca03f34

…ructures

multiple AS builds at once

bee8a36

(better allocation strategy required)

switched to using smallvec

a420b87

small api change + documentation

e48f7bd

added to changelog

c1dc803

typo

9c67400

Co-authored-by: JMS55 <47158642+JMS55@users.noreply.github.com>

Merge remote-tracking branch 'origin/master' into hal-acceleration-st…

c91ce76

…ructures

create_blas create_tlas

c931edf

example skeleton

e3a7fc9

WIP

e932040

moved flags to wgt

f5e94e7

renamed feature RAY_TRACING to RAY_TRACING_ACCELERATION_STRUCTURE

0b7e728

Merge branch 'hal-acceleration-structures' into ray-tracing + small fix

c078ab0

Vecvec added 10 commits September 14, 2024 21:12

fmt & docs

d5532b0

fix wasm clippy

7400129

fix issues with tests

3b573c8

fix issues with tests

86f7823

Merge remote-tracking branch 'refs/remotes/gfx-rs/trunk' into ray-tra…

93fba52

…cing-updated

Merge branch 'refs/heads/ray-tracing-updated' into ray-tracing-tests

32f9666

fmt & clippy (wasm)

7f69da4

wip ray-traced-triangle (non hal)

e8067ff

unify camera location in ray_shadows

5461327

Vecvec added 4 commits September 19, 2024 16:22

fix tests

c13baf1

Merge remote-tracking branch 'refs/remotes/gfx-rs/trunk' into ray-tra…

592270d

…cing-updated

Merge branch 'refs/heads/ray-tracing-new' into ray-tracing-tests

c137a33

add readme to ray_traced_triangle

ba098f2

Vecvec added 4 commits September 20, 2024 09:03

note that vertex format must be f32x3 (not yet validated)

8d1c193

changelog

ff6988d

require VertexFormat::Float32x3

638f4c5

fix compile error

cb167a4

Vecvec marked this pull request as ready for review September 19, 2024 21:36

Vecvec requested review from a team as code owners September 19, 2024 21:36

when creating and building acceleration structures require Features::…

e2168e6

…RAY_TRACING_ACCELERATION_STRUCTURE is enabled

remove duplicate workaround

0fcc749

nical reviewed Sep 20, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Ray Queries #6291

Ray Queries #6291

Vecvec commented Sep 18, 2024 •

edited

Loading

JMS55 commented Sep 18, 2024

Vecvec commented Sep 18, 2024

nical commented Sep 19, 2024 •

edited

Loading

Vecvec commented Sep 19, 2024 •

edited

Loading

Vecvec commented Sep 20, 2024

Vecvec commented Sep 20, 2024

nical left a comment

nical Sep 20, 2024

nical Sep 20, 2024

nical Sep 20, 2024

nical Sep 20, 2024

nical Sep 20, 2024

nical Sep 20, 2024

Vecvec Sep 20, 2024 •

edited

Loading

nical Sep 20, 2024

nical Sep 20, 2024

	/// Iterates over the buffers generated [iter_blas] and convert the barriers into hal barriers, and the triangles into [hal::AccelerationStructureEntries] (and also some validation).
	/// Iterates over the buffers generated in [iter_blas], convert the barriers into hal barriers, and the triangles into [hal::AccelerationStructureEntries] (and also some validation).

		@@ -0,0 +1,154 @@
		// Ray tracing api proposal for wgpu (underlining Vulkan, Metal and DX12 implementations)

Ray Queries #6291

Are you sure you want to change the base?

Ray Queries #6291

Conversation

Vecvec commented Sep 18, 2024 • edited Loading

JMS55 commented Sep 18, 2024

Vecvec commented Sep 18, 2024

nical commented Sep 19, 2024 • edited Loading

Vecvec commented Sep 19, 2024 • edited Loading

Vecvec commented Sep 20, 2024

Vecvec commented Sep 20, 2024

nical left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Vecvec Sep 20, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Vecvec commented Sep 18, 2024 •

edited

Loading

nical commented Sep 19, 2024 •

edited

Loading

Vecvec commented Sep 19, 2024 •

edited

Loading

Vecvec Sep 20, 2024 •

edited

Loading