[Release 2.6] Triton/inductor related optimisations #2008
Conversation
…8437) Splitting pytorch#147315 into two PRs. This PR adds general support for kpack and waves_per_eu triton kernel args for AMD backend. More detail in the PR above. A follow up PR will update the configs used by ROCm but this requires pytorch#147452 to land first Pull Request resolved: pytorch#148437 Approved by: https://github.com/eellison, https://github.com/jansel (cherry picked from commit 8059ead)
jataylo
requested review from
jeffdaily,
jithunnair-amd and
pruthvistony
as code owners
|
I'd like internal CI to kick off before merging, hopefully its working. |
|
Jenkins build for 8bf46b07ecdd31aea1641a30998e0ac00699ef4a commit finished as FAILURE |
|
Jenkins build for 8bf46b07ecdd31aea1641a30998e0ac00699ef4a commit finished as FAILURE |
jataylo
changed the title
|
@jithunnair-amd @pruthvistony local testing is good for inductor subset can we merge this? Looks like pytorch_inductor failed for this job but its not obvious if this was already failing.. |
|
Jenkins build for 8bf46b07ecdd31aea1641a30998e0ac00699ef4a commit finished as FAILURE |
|
Jenkins build for 8bf46b07ecdd31aea1641a30998e0ac00699ef4a commit finished as FAILURE Detected error during Pytorch building: |
|
Jenkins build for 8bf46b07ecdd31aea1641a30998e0ac00699ef4a commit finished as FAILURE |
|
Jenkins build for 8bf46b07ecdd31aea1641a30998e0ac00699ef4a commit finished as FAILURE |
|
Jenkins build for 8bf46b07ecdd31aea1641a30998e0ac00699ef4a commit finished as FAILURE |
|
Jenkins build for 8bf46b07ecdd31aea1641a30998e0ac00699ef4a commit finished as FAILURE |
|
!ci-build |
|
Jenkins build for 8bf46b07ecdd31aea1641a30998e0ac00699ef4a commit finished as ABORTED |
|
!ci-build |
|
Jenkins build for 8bf46b07ecdd31aea1641a30998e0ac00699ef4a commit finished as FAILURE |
|
Jenkins build for 8bf46b07ecdd31aea1641a30998e0ac00699ef4a commit finished as FAILURE |
|
Jenkins build for 8bf46b07ecdd31aea1641a30998e0ac00699ef4a commit finished as FAILURE |
|
Jenkins build for 8bf46b07ecdd31aea1641a30998e0ac00699ef4a commit finished as FAILURE |
|
Jenkins build for 8bf46b07ecdd31aea1641a30998e0ac00699ef4a commit finished as FAILURE |
|
Jenkins build for 8bf46b07ecdd31aea1641a30998e0ac00699ef4a commit finished as FAILURE |
|
Jenkins build for 8bf46b07ecdd31aea1641a30998e0ac00699ef4a commit is in progress |
|
Jenkins build for 8bf46b07ecdd31aea1641a30998e0ac00699ef4a commit finished as FAILURE |
|
Jenkins build for 8bf46b07ecdd31aea1641a30998e0ac00699ef4a commit finished as FAILURE |
|
Jenkins build for 8bf46b07ecdd31aea1641a30998e0ac00699ef4a commit finished as FAILURE |
|
Jenkins build for 8bf46b07ecdd31aea1641a30998e0ac00699ef4a commit finished as FAILURE |
|
Jenkins build for 8bf46b07ecdd31aea1641a30998e0ac00699ef4a commit finished as FAILURE |
|
Jenkins build for 8bf46b07ecdd31aea1641a30998e0ac00699ef4a commit finished as FAILURE |
|
Jenkins build for 8bf46b07ecdd31aea1641a30998e0ac00699ef4a commit finished as SUCCESS |
|
Jenkins build for 8bf46b07ecdd31aea1641a30998e0ac00699ef4a commit finished as FAILURE |
|
Jenkins build for 8bf46b07ecdd31aea1641a30998e0ac00699ef4a commit finished as FAILURE |
|
Jenkins build for 8bf46b07ecdd31aea1641a30998e0ac00699ef4a commit finished as FAILURE |
|
Jenkins build for 8bf46b07ecdd31aea1641a30998e0ac00699ef4a commit finished as FAILURE |
|
Jenkins build for 8bf46b07ecdd31aea1641a30998e0ac00699ef4a commit finished as FAILURE |
|
Jenkins build for 8bf46b07ecdd31aea1641a30998e0ac00699ef4a commit finished as FAILURE |
|
Jenkins build for 8bf46b07ecdd31aea1641a30998e0ac00699ef4a commit finished as FAILURE |
|
Jenkins build for 8bf46b07ecdd31aea1641a30998e0ac00699ef4a commit finished as FAILURE |
|
Jenkins build for 8bf46b07ecdd31aea1641a30998e0ac00699ef4a commit finished as FAILURE |
|
Jenkins build for 8bf46b07ecdd31aea1641a30998e0ac00699ef4a commit finished as FAILURE |
|
@jithunnair-amd can we merge this one now :) |
|
Jenkins build for 8bf46b07ecdd31aea1641a30998e0ac00699ef4a commit finished as ABORTED |
|
Jenkins build for 8bf46b07ecdd31aea1641a30998e0ac00699ef4a commit finished as ABORTED |
|
Jenkins build for 8bf46b07ecdd31aea1641a30998e0ac00699ef4a commit is in progress |
|
Jenkins build for 8bf46b07ecdd31aea1641a30998e0ac00699ef4a commit finished as ABORTED |
|
Jenkins build for 7d6c8e09710c81e5376f27d29148cfd397ac542b commit finished as FAILURE |
| ) | ||
| if error is not None and len(configs) == 1: | ||
| raise error | ||
|
|
There was a problem hiding this comment.
@jataylo Consider avoiding repetition by doing something like:
def append_choice_with_options(options):
return flex_attention_template.maybe_append_choice(
choices=choices,
call_sizes=query.get_size(),
**options,
)
if os.getenv("TORCHINDUCTOR_EXHAUSTIVE_FLEX_ATTENTION_EXPERIMENTAL") == "1":
for mfma in [0, 16]:
for wpeu in [0, 1, 2, 4, 8]:
cur_kernel_options["waves_per_eu"] = wpeu
cur_kernel_options["matrix_instr_non_kdim"] = mfma
error = append_choice_with_options(cur_kernel_options)
else:
error = append_choice_with_options(cur_kernel_options)
| for s in num_stages_list | ||
| if BLOCK2 % BLOCK1 == 0 | ||
| ] | ||
| ) | ||
|
|
||
|
|
|
Jenkins build for 7d6c8e09710c81e5376f27d29148cfd397ac542b commit finished as FAILURE |
jithunnair-amd
force-pushed
the
release/2.6
branch
from
3179d93 to
ea75c6e
Compare
Triton update to help with pointwise/gemm perf. The experimental env var is for helping with perf analysis internally.