Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: increase sdk.Dec maxApproxRootIterations #12229

Merged

Conversation

catShaark
Copy link
Contributor

@catShaark catShaark commented Jun 11, 2022

Description

ApproxRoot return incorrect value for large sdk.Dec because maxApproxRootIterations is not big enough, I increase it to 300


Author Checklist

All items are required. Please add a note to the item if the item is not applicable and
please add links to any relevant follow up issues.

I have...

  • included the correct type prefix in the PR title
  • added ! to the type prefix if API or client breaking change
  • targeted the correct branch (see PR Targeting)
  • provided a link to the relevant issue or specification
  • followed the guidelines for building modules
  • included the necessary unit and integration tests
  • added a changelog entry to CHANGELOG.md
  • included comments for documenting Go code
  • updated the relevant documentation or specification
  • reviewed "Files changed" and left comments if necessary
  • confirmed all CI checks have passed

Reviewers Checklist

All items are required. Please add a note if the item is not applicable and please add
your handle next to the items reviewed if you only reviewed selected items.

I have...

  • confirmed the correct type prefix in the PR title
  • confirmed ! in the type prefix if API or client breaking change
  • confirmed all author checklist items have been addressed
  • reviewed state machine logic
  • reviewed API design and naming
  • reviewed documentation is accurate
  • reviewed tests and test coverage
  • manually tested (if applicable)

@catShaark catShaark requested a review from a team as a code owner June 11, 2022 20:57
@catShaark catShaark changed the title increase maxApproxRootIterations to 300 fix: ApproxRoot return incorrect value for large sdk.Dec Jun 11, 2022
@catShaark catShaark changed the title fix: ApproxRoot return incorrect value for large sdk.Dec fix: ApproxRoot return incorrect value for very large sdk.Dec Jun 11, 2022
@julienrbrt
Copy link
Member

julienrbrt commented Jun 12, 2022

Can we add a benchmark on ApproxRoot and an extra test case?

Copy link
Contributor

@ValarDragon ValarDragon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, lets just add a test case that used to fail?

We merged this into Osmosis

Also the current AppproxRoot behavior is imo unsafe, it needs to be returning an error or a flag at minimum, if it can't converege within max iterations, so the caller doesn't have incorrect expectations.

This actually would have seriously affected Osmosis in edge-cases that (thankfully) were not hit on mainnet.

@alexanderbez
Copy link
Contributor

@catShaark could you possibly address @ValarDragon's concerns (test case + returning an error) prior to us merging this PR?

@aaronc
Copy link
Member

aaronc commented Jun 14, 2022

Can we just start migrating to APD soon?

@catShaark
Copy link
Contributor Author

catShaark commented Jun 14, 2022

@catShaark could you possibly address @ValarDragon's concerns (test case + returning an error) prior to us merging this PR?

Yes, and from playing around with the function, I have some concerns of mine as well :

  • Right now the function will panic "out of bound" for very large sdk.Dec even if it has smaller bit len than maxDecBitLen. I think this is because we're using Newton-Raphson method with the initial guess of 1, which in cases of large input, makes the second guess far far exceeds the actual root value, causing the "out of bound" . We can fix this by including some condition check for guesses in the function but idk if this is the best way.

  • There's inaccuracy due to not enough precision, for example :

    ApproxSqrt(2290630776557940645655329181699877738996280875636842527) =
    1513482995133391198305976331.737501099575432618 but
    1513482995133391198305976331.737501099575432618 ^ 2 =
    2290630776557940645655329181699877738996280876203567639

    If we support dec as big as 256bit, we might as well increase precision to ensure accuracy for operation like
    AppoxSqrt, or we switch to APD like @aaronc said

  • I propose we make separate functions for calculating square root and cube root like this one implementation here.

@catShaark
Copy link
Contributor Author

catShaark commented Jun 14, 2022

Also the current AppproxRoot behavior is imo unsafe, it needs to be returning an error or a flag at minimum, if it can't converege within max iterations, so the caller doesn't have incorrect expectations.

Yes, I'll handle this.

@ValarDragon
Copy link
Contributor

Agreed that sqrt and cubed root should generally be different, dedicated functions.

Also agreed that we should switch to apd, but I'm not sure that we get to avoid maintenance for decimal for quite some time.

Perhaps we compile the issues you point out as well @catShaark into a second issue, and the flag for could-not-converge?

IMO it suffices for this PR if we just get a test case that would return something way off before, and is now fixed by bumping the iteration count

@aaronc
Copy link
Member

aaronc commented Jun 14, 2022

See #11783. Regen has prior work with apd that we can leverage

@aaronc
Copy link
Member

aaronc commented Jun 14, 2022

Also not sure we can avoid maintaining the current dec or not. It would be state machine breaking but maybe that's okay in the next release? At least we need something that supports Dec's (incorrect) binary encoding

@catShaark catShaark changed the title fix: ApproxRoot return incorrect value for very large sdk.Dec increase sdk.Dec maxApproxRootIterations Jun 15, 2022
@alexanderbez alexanderbez changed the title increase sdk.Dec maxApproxRootIterations fix: increase sdk.Dec maxApproxRootIterations Jun 15, 2022
@alexanderbez
Copy link
Contributor

@catShaark approved! Please add a changelog entry and I'll merge this. Should we backport this to 0.45?

@alexanderbez alexanderbez added the backport/0.46.x PR scheduled for inclusion in the v0.46's next stable release label Jun 15, 2022
Copy link
Member

@julienrbrt julienrbrt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure that this is the right way to test this. Now we just test ApproxRoot with different maxApproxRootIterations values, without actually testing the actual implementation (although it tests with 300 it is not clear from the test only that it is the default value).

I think we should just add one test case in TestApproxRoot that fails with 100 but passes with 300.
I don't think we need that extra TestApproxRootLargeDec nor having MaxApproxRootIterations public.
Moreover, we should update the comments in the tests that still mention 100 iterations.

@ValarDragon
Copy link
Contributor

Agreed that the variable shouldn't be exposed / altered, a simple one-off test vector suffices

@catShaark
Copy link
Contributor Author

catShaark commented Jun 16, 2022

I think we should just add one test case in TestApproxRoot that fails with 100 but passes with 300. I don't think we need that extra TestApproxRootLargeDec nor having MaxApproxRootIterations public. Moreover, we should update the comments in the tests that still mention 100 iterations.

Got it

@catShaark catShaark force-pushed the increase-maxApproxRootIterations-ApproxRoot branch from ece4440 to b8040fc Compare June 16, 2022 03:35
@catShaark catShaark force-pushed the increase-maxApproxRootIterations-ApproxRoot branch from 1a35c3f to 7f7fc97 Compare June 16, 2022 03:42
Copy link
Member

@julienrbrt julienrbrt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@julienrbrt julienrbrt added the A:automerge Automatically merge PR once all prerequisites pass. label Jun 16, 2022
@mergify mergify bot merged commit 6a3d10b into cosmos:main Jun 16, 2022
mergify bot pushed a commit that referenced this pull request Jun 16, 2022
## Description

`ApproxRoot` return incorrect value for large `sdk.Dec` because maxApproxRootIterations is not big enough, I increase it to 300

---

### Author Checklist

*All items are required. Please add a note to the item if the item is not applicable and
please add links to any relevant follow up issues.*

I have...

- [ ] included the correct [type prefix](https://github.com/commitizen/conventional-commit-types/blob/v3.0.0/index.json) in the PR title
- [ ] added `!` to the type prefix if API or client breaking change
- [ ] targeted the correct branch (see [PR Targeting](https://github.com/cosmos/cosmos-sdk/blob/main/CONTRIBUTING.md#pr-targeting))
- [ ] provided a link to the relevant issue or specification
- [ ] followed the guidelines for [building modules](https://github.com/cosmos/cosmos-sdk/blob/main/docs/building-modules)
- [ ] included the necessary unit and integration [tests](https://github.com/cosmos/cosmos-sdk/blob/main/CONTRIBUTING.md#testing)
- [ ] added a changelog entry to `CHANGELOG.md`
- [ ] included comments for [documenting Go code](https://blog.golang.org/godoc)
- [ ] updated the relevant documentation or specification
- [ ] reviewed "Files changed" and left comments if necessary
- [ ] confirmed all CI checks have passed

### Reviewers Checklist

*All items are required. Please add a note if the item is not applicable and please add
your handle next to the items reviewed if you only reviewed selected items.*

I have...

- [x] confirmed the correct [type prefix](https://github.com/commitizen/conventional-commit-types/blob/v3.0.0/index.json) in the PR title
- [x] confirmed `!` in the type prefix if API or client breaking change
- [x] confirmed all author checklist items have been addressed
- [x] reviewed state machine logic
- [x] reviewed API design and naming
- [ ] reviewed documentation is accurate
- [ ] reviewed tests and test coverage
- [x] manually tested (if applicable)

(cherry picked from commit 6a3d10b)
tac0turtle pushed a commit that referenced this pull request Jun 16, 2022
## Description

`ApproxRoot` return incorrect value for large `sdk.Dec` because maxApproxRootIterations is not big enough, I increase it to 300

---

### Author Checklist

*All items are required. Please add a note to the item if the item is not applicable and
please add links to any relevant follow up issues.*

I have...

- [ ] included the correct [type prefix](https://github.com/commitizen/conventional-commit-types/blob/v3.0.0/index.json) in the PR title
- [ ] added `!` to the type prefix if API or client breaking change
- [ ] targeted the correct branch (see [PR Targeting](https://github.com/cosmos/cosmos-sdk/blob/main/CONTRIBUTING.md#pr-targeting))
- [ ] provided a link to the relevant issue or specification
- [ ] followed the guidelines for [building modules](https://github.com/cosmos/cosmos-sdk/blob/main/docs/building-modules)
- [ ] included the necessary unit and integration [tests](https://github.com/cosmos/cosmos-sdk/blob/main/CONTRIBUTING.md#testing)
- [ ] added a changelog entry to `CHANGELOG.md`
- [ ] included comments for [documenting Go code](https://blog.golang.org/godoc)
- [ ] updated the relevant documentation or specification
- [ ] reviewed "Files changed" and left comments if necessary
- [ ] confirmed all CI checks have passed

### Reviewers Checklist

*All items are required. Please add a note if the item is not applicable and please add
your handle next to the items reviewed if you only reviewed selected items.*

I have...

- [x] confirmed the correct [type prefix](https://github.com/commitizen/conventional-commit-types/blob/v3.0.0/index.json) in the PR title
- [x] confirmed `!` in the type prefix if API or client breaking change
- [x] confirmed all author checklist items have been addressed
- [x] reviewed state machine logic
- [x] reviewed API design and naming
- [ ] reviewed documentation is accurate
- [ ] reviewed tests and test coverage
- [x] manually tested (if applicable)

(cherry picked from commit 6a3d10b)

Co-authored-by: khanh <50263489+catShaark@users.noreply.github.com>
halibobo1205 added a commit to halibobo1205/java-tron that referenced this pull request Jun 16, 2022
@catShaark catShaark mentioned this pull request Jun 17, 2022
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A:automerge Automatically merge PR once all prerequisites pass. backport/0.46.x PR scheduled for inclusion in the v0.46's next stable release
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants