Revert "Pin PT version: Fix FPX Inductor error"

[msaroufim]

Reverts #790

The problems are all mostly cpu specific, in particular this feels like a subclass on cpu problem but not sure - cc @bdhirsh

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/ao/843

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit 47acb46 with merge base 1b317f9 ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[msaroufim]

so fpx test is no longer failing but i see some new failures with bitnet that should probably be skipped @andrewor14 before we make the release. Also save/load is now failing on nightlies @jerryzh168

[bdhirsh]

@msaroufim do you have a link to the failing tests? I can try to help root cause (and/or at least figure out if it's subclass related)

[msaroufim]

Thank you! <3

They're in test/integration/test_integration.py

test_int8_weight_only_quant_subclass_api
test_int8_weight_only_quant_with_freeze
test_save_load_int8woqtensors

And only fail for cpu

[bdhirsh]

A couple things:

(1) I confirmed that python test/integration/test_integration.py -k test_int8_weight_only_quant_subclass_api_1_cpu fails for me locally

(2) I tweaked the test to compile with a few different backends: compile(... backend="inductor") fails, but compile(..., backend="aot_eager_decomp_partition") runs fine

(3) i checked out @eellison's beautiful inductor bisecting tool from here and ran it. It bisected down to to the lowering for prims.convert_element_type as the "problematic" bit of inductor. This is technically useful, but doesn't really give us the full story: that lowering hasn't been changed in months, and avoiding lowering that op probably just prevents inductor from doing some other fusions that are causing the numeric difference (I bet if the bisector could bisect on inductor fusions it could tell us more).

(4) Also confirmed that this only repros with cpu inputs.

Idk, @eellison do you know any suspicious commits coming from the cpu-inductor side in the last week that could affect numerics, especially in relation to casting ops?

[leslie-fang-intel]

Hi @bdhirsh

(1) I confirmed that python test/integration/test_integration.py -k test_int8_weight_only_quant_subclass_api_1_cpu fails for me locally

is it the same failure reported in #890

[eellison]

@bdhirsh, I was on pto last week so i'm not sure. sounds like @leslie-fang-intel has an idea. Would be easier to bisect maybe.

[bdhirsh]

@leslie-fang-intel not sure, but seems unlikely (mainly because the issue you linked is a hard error while the subclass is running at compile time, while this is a runtime / bad numerics error)

[leslie-fang-intel]

Thanks @bdhirsh. It sounds like a different error then. Maybe after we resolve the first one, we will meet the numeric error you met. Do you have any idea about the hard error in #890? Why it didn't happen in your test environment.

----------------- Update ---------------

• PyTorch: 86335e913557962bf8d415c80dcb7e615468ba42
• AO: 8236a87

comment out these lines to enable the test:

• ao/test/integration/test_integration.py

  Lines 820 to 821 in 8236a87

  	if TORCH_VERSION_AT_LEAST_2_5 and device == "cpu":
  	self.skipTest("Regression introduced in PT nightlies")

• ao/test/integration/test_integration.py

  Lines 832 to 833 in 8236a87

  	if TORCH_VERSION_AT_LEAST_2_5 and device == "cpu":
  	self.skipTest("Regression introduced in PT nightlies")

It seems we saw 2 different errors when running 2 different UT

• python test/integration/test_integration.py -k test_int8_weight_only_quant_with_freeze throws the hard error reported in [torchao]NotImplementedError: AffineQuantizedTensor dispatch: attempting to run unimplemented operator/function: aten.permute.default #890
• python test/integration/test_integration.py -k test_int8_weight_only_quant_subclass_api throws the numerical failure as reported in [torchao]AssertionError: tensor(2.3359, dtype=torch.float16) not greater than 40 : _int8wo_api failed when compiled with dtype=torch.float16, (m, k, n)=(32, 64, 32) pytorch#135831

[leslie-fang-intel]

Hi @bdhirsh, I have create PRs to fix these 2 failures

• PR in Torch: [AO][Inductor] Enable WOQ fusion pattern with permute pytorch#135928 addresses the numerical failures, which were caused by missing the fusion of this woq int8 pattern and falling back to the reference implementation. By re-enabling the fusions in Inductor for this pattern, the unit tests now pass on my local system.
• PR in AO: Fix WOQ int8 failures #884, as for another UT failure python test/integration/test_integration.py -k test_int8_weight_only_quant_with_freeze, it seems we still need unwrap_tensor_subclass for Inductor freezing. cc @eellison here.

Please kindly help to review these 2 PRs.

[bdhirsh]

@jerryzh168 I vaguely remember it being a problem to re-enable unwrap_tensor_subclass() in the torch.compile path for torchao, since dynamo had a bad interaction with the parametrization code. Does that ring a bell?

Either way, we should fix the freezing interaction (@IvanKobzarev is taking a look)