Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Correctly rollback in ForkRepository #17034

Merged
merged 2 commits into from
Sep 14, 2021

Conversation

zeripath
Copy link
Contributor

The rollback functionality in
services/repository/repository.go:ForkRepository is incorrect and could
lead to a deadlock as it uses DeleteRepository to delete the rolled-back
repository - a function which creates its own transaction.

This PR adjusts the rollback function to only use RemoveAll as any
database changes will be automatically rolled-back. It also handles
panics and adjusts the Close within WithTx to ensure that if there is a
panic the session will always be closed.

Signed-off-by: Andrew Thornton art27@cantab.net

The rollback functionality in
services/repository/repository.go:ForkRepository is incorrect and could
lead to a deadlock as it uses DeleteRepository to delete the rolled-back
repository - a function which creates its own transaction.

This PR adjusts the rollback function to only use RemoveAll as any
database changes will be automatically rolled-back. It also handles
panics and adjusts the Close within WithTx to ensure that if there is a
panic the session will always be closed.

Signed-off-by: Andrew Thornton <art27@cantab.net>
@zeripath zeripath force-pushed the rollback-in-fork-in-transaction branch from 3d07d47 to 6da7f16 Compare September 13, 2021 18:56
@codecov-commenter
Copy link

Codecov Report

Merging #17034 (966439c) into main (8af7a21) will increase coverage by 0.02%.
The diff coverage is 45.45%.

❗ Current head 966439c differs from pull request most recent head 6da7f16. Consider uploading reports for the commit 6da7f16 to get more accurate results
Impacted file tree graph

@@            Coverage Diff             @@
##             main   #17034      +/-   ##
==========================================
+ Coverage   45.22%   45.24%   +0.02%     
==========================================
  Files         766      766              
  Lines       86632    86637       +5     
==========================================
+ Hits        39178    39203      +25     
+ Misses      41107    41088      -19     
+ Partials     6347     6346       -1     
Impacted Files Coverage Δ
modules/repository/fork.go 54.28% <40.00%> (+5.89%) ⬆️
models/context.go 56.00% <100.00%> (-1.15%) ⬇️
modules/queue/queue_bytefifo.go 59.28% <0.00%> (-3.60%) ⬇️
modules/queue/queue_channel.go 95.00% <0.00%> (-1.67%) ⬇️
services/pull/pull.go 41.78% <0.00%> (-0.41%) ⬇️
modules/queue/workerpool.go 46.94% <0.00%> (-0.39%) ⬇️
models/issue_comment.go 52.05% <0.00%> (+0.88%) ⬆️
modules/log/event.go 62.50% <0.00%> (+1.85%) ⬆️
modules/git/utils.go 70.83% <0.00%> (+2.77%) ⬆️
modules/git/repo_base_nogogit.go 85.71% <0.00%> (+2.85%) ⬆️
... and 4 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8af7a21...6da7f16. Read the comment docs.

@GiteaBot GiteaBot added the lgtm/need 2 This PR needs two approvals by maintainers to be considered for merging. label Sep 13, 2021
Copy link
Member

@delvh delvh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While I can completely understand the implementation,
adding a test might be beneficial here so that this potential deadlock does not occur again sometime else.

@GiteaBot GiteaBot added lgtm/need 1 This PR needs approval from one additional maintainer to be merged. and removed lgtm/need 2 This PR needs two approvals by maintainers to be considered for merging. labels Sep 13, 2021
@zeripath
Copy link
Contributor Author

While I can completely understand the implementation,
adding a test might be beneficial here so that this potential deadlock does not occur again sometime else.

I don't think it's testable. The deadlock could only happen if there was an error whilst creating the fork, even then it would only occur extremely randomly. We can't force an error, we can't force the deadlock. Even if we had a fully mocked up xorm we'd have to code in what it means to deadlock.

There some very complex solutions that could help reduce the risk of these errors in future but they'd come with their own issues too.

If you can think of a test then I'm happy to discuss.

@delvh
Copy link
Member

delvh commented Sep 14, 2021

That is unfortunately correct.
The only thing we could test is whether it behaves correctly under normal circumstances.

@GiteaBot GiteaBot added lgtm/done This PR has enough approvals to get merged. There are no important open reservations anymore. and removed lgtm/need 1 This PR needs approval from one additional maintainer to be merged. labels Sep 14, 2021
@zeripath
Copy link
Contributor Author

make lgtm work

@6543 6543 merged commit 26ef180 into go-gitea:main Sep 14, 2021
@zeripath zeripath deleted the rollback-in-fork-in-transaction branch September 14, 2021 16:27
zeripath added a commit to zeripath/gitea that referenced this pull request Sep 14, 2021
Backport go-gitea#17034

The rollback functionality in
services/repository/repository.go:ForkRepository is incorrect and could
lead to a deadlock as it uses DeleteRepository to delete the rolled-back
repository - a function which creates its own transaction.

This PR adjusts the rollback function to only use RemoveAll as any
database changes will be automatically rolled-back. It also handles
panics and adjusts the Close within WithTx to ensure that if there is a
panic the session will always be closed.

Signed-off-by: Andrew Thornton <art27@cantab.net>
@zeripath zeripath added the backport/done All backports for this PR have been created label Sep 14, 2021
lafriks pushed a commit that referenced this pull request Sep 15, 2021
Backport #17034

The rollback functionality in
services/repository/repository.go:ForkRepository is incorrect and could
lead to a deadlock as it uses DeleteRepository to delete the rolled-back
repository - a function which creates its own transaction.

This PR adjusts the rollback function to only use RemoveAll as any
database changes will be automatically rolled-back. It also handles
panics and adjusts the Close within WithTx to ensure that if there is a
panic the session will always be closed.

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
zeripath added a commit to zeripath/gitea that referenced this pull request Sep 19, 2021
## [1.15.3](https://github.com/go-gitea/gitea/releases/tag/v1.15.3) - 2021-09-19

* ENHANCEMENTS
  * Add fluid to ui container class to remove margin (go-gitea#16396) (go-gitea#16976)
  * Add caller to cat-file batch calls (go-gitea#17082) (go-gitea#17089)
* BUGFIXES
  * Render full plain readme. (go-gitea#17083) (go-gitea#17090)
  * Upgrade xorm to v1.2.4 (go-gitea#17059)
  * Fix bug of migrate comments which only fetch one page (go-gitea#17055) (go-gitea#17058)
  * Do not show issue context popup on external issues (go-gitea#17050) (go-gitea#17054)
  * Decrement Fork Num when converting from Fork (go-gitea#17035) (go-gitea#17046)
  * Correctly rollback in ForkRepository (go-gitea#17034) (go-gitea#17045)
  * Fix missing close in WalkGitLog (go-gitea#17008) (go-gitea#17009)
  * Add prefix to SVG id/class attributes (go-gitea#16997) (go-gitea#17000)
  * Fix bug of migrated repository not index (go-gitea#16991) (go-gitea#16996)
  * Skip AllowedUserVisibilityModes validation on update user if it is an organisation (go-gitea#16988) (go-gitea#16990)
  * Fix storage Iterate bug and Add storage doctor to delete garbage attachments (go-gitea#16971) (go-gitea#16977)
  * Fix issue with issue default mail template (go-gitea#16956) (go-gitea#16975)
  * Ensure that rebase conflicts are handled in updates (go-gitea#16952) (go-gitea#16960)
  * Prevent panic on diff generation (go-gitea#16950) (go-gitea#16951)

Signed-off-by: Andrew Thornton <art27@cantab.net>
@zeripath zeripath mentioned this pull request Sep 19, 2021
lunny pushed a commit that referenced this pull request Sep 20, 2021
## [1.15.3](https://github.com/go-gitea/gitea/releases/tag/v1.15.3) - 2021-09-19

* ENHANCEMENTS
  * Add fluid to ui container class to remove margin (#16396) (#16976)
  * Add caller to cat-file batch calls (#17082) (#17089)
* BUGFIXES
  * Render full plain readme. (#17083) (#17090)
  * Upgrade xorm to v1.2.4 (#17059)
  * Fix bug of migrate comments which only fetch one page (#17055) (#17058)
  * Do not show issue context popup on external issues (#17050) (#17054)
  * Decrement Fork Num when converting from Fork (#17035) (#17046)
  * Correctly rollback in ForkRepository (#17034) (#17045)
  * Fix missing close in WalkGitLog (#17008) (#17009)
  * Add prefix to SVG id/class attributes (#16997) (#17000)
  * Fix bug of migrated repository not index (#16991) (#16996)
  * Skip AllowedUserVisibilityModes validation on update user if it is an organisation (#16988) (#16990)
  * Fix storage Iterate bug and Add storage doctor to delete garbage attachments (#16971) (#16977)
  * Fix issue with issue default mail template (#16956) (#16975)
  * Ensure that rebase conflicts are handled in updates (#16952) (#16960)
  * Prevent panic on diff generation (#16950) (#16951)

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
6543 pushed a commit to 6543-forks/gitea that referenced this pull request Sep 21, 2021
## [1.15.3](https://github.com/go-gitea/gitea/releases/tag/v1.15.3) - 2021-09-19

* ENHANCEMENTS
  * Add fluid to ui container class to remove margin (go-gitea#16396) (go-gitea#16976)
  * Add caller to cat-file batch calls (go-gitea#17082) (go-gitea#17089)
* BUGFIXES
  * Render full plain readme. (go-gitea#17083) (go-gitea#17090)
  * Upgrade xorm to v1.2.4 (go-gitea#17059)
  * Fix bug of migrate comments which only fetch one page (go-gitea#17055) (go-gitea#17058)
  * Do not show issue context popup on external issues (go-gitea#17050) (go-gitea#17054)
  * Decrement Fork Num when converting from Fork (go-gitea#17035) (go-gitea#17046)
  * Correctly rollback in ForkRepository (go-gitea#17034) (go-gitea#17045)
  * Fix missing close in WalkGitLog (go-gitea#17008) (go-gitea#17009)
  * Add prefix to SVG id/class attributes (go-gitea#16997) (go-gitea#17000)
  * Fix bug of migrated repository not index (go-gitea#16991) (go-gitea#16996)
  * Skip AllowedUserVisibilityModes validation on update user if it is an organisation (go-gitea#16988) (go-gitea#16990)
  * Fix storage Iterate bug and Add storage doctor to delete garbage attachments (go-gitea#16971) (go-gitea#16977)
  * Fix issue with issue default mail template (go-gitea#16956) (go-gitea#16975)
  * Ensure that rebase conflicts are handled in updates (go-gitea#16952) (go-gitea#16960)
  * Prevent panic on diff generation (go-gitea#16950) (go-gitea#16951)

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
lunny pushed a commit that referenced this pull request Sep 22, 2021
## [1.15.3](https://github.com/go-gitea/gitea/releases/tag/v1.15.3) - 2021-09-19

* ENHANCEMENTS
  * Add fluid to ui container class to remove margin (#16396) (#16976)
  * Add caller to cat-file batch calls (#17082) (#17089)
* BUGFIXES
  * Render full plain readme. (#17083) (#17090)
  * Upgrade xorm to v1.2.4 (#17059)
  * Fix bug of migrate comments which only fetch one page (#17055) (#17058)
  * Do not show issue context popup on external issues (#17050) (#17054)
  * Decrement Fork Num when converting from Fork (#17035) (#17046)
  * Correctly rollback in ForkRepository (#17034) (#17045)
  * Fix missing close in WalkGitLog (#17008) (#17009)
  * Add prefix to SVG id/class attributes (#16997) (#17000)
  * Fix bug of migrated repository not index (#16991) (#16996)
  * Skip AllowedUserVisibilityModes validation on update user if it is an organisation (#16988) (#16990)
  * Fix storage Iterate bug and Add storage doctor to delete garbage attachments (#16971) (#16977)
  * Fix issue with issue default mail template (#16956) (#16975)
  * Ensure that rebase conflicts are handled in updates (#16952) (#16960)
  * Prevent panic on diff generation (#16950) (#16951)

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: techknowlogick <techknowlogick@gitea.io>

Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
@go-gitea go-gitea locked and limited conversation to collaborators Oct 19, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
backport/done All backports for this PR have been created lgtm/done This PR has enough approvals to get merged. There are no important open reservations anymore. type/bug
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants