Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add k/v store for repo settings #19400

Closed
wants to merge 1,289 commits into from

Conversation

techknowlogick
Copy link
Member

@techknowlogick techknowlogick commented Apr 14, 2022

No description provided.

@techknowlogick techknowlogick added the type/enhancement An improvement of existing functionality label Apr 14, 2022
@techknowlogick techknowlogick added this to the 1.17.0 milestone Apr 14, 2022
@techknowlogick techknowlogick marked this pull request as ready for review April 14, 2022 04:02
models/repo/setting_keys.go Outdated Show resolved Hide resolved
models/repo/setting.go Outdated Show resolved Hide resolved
models/repo/setting.go Outdated Show resolved Hide resolved
models/repo/setting.go Outdated Show resolved Hide resolved
models/repo/setting.go Outdated Show resolved Hide resolved
models/repo/setting.go Outdated Show resolved Hide resolved
models/repo/setting.go Outdated Show resolved Hide resolved
Comment on lines 34 to 36
settingValue, err := GetRepoSetting(99, keyName)
assert.NoError(t, err)
assert.EqualValues(t, newSetting.SettingValue, settingValue)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the case with the default value could be added...

models/migrations/v213.go Outdated Show resolved Hide resolved
models/repo/setting.go Outdated Show resolved Hide resolved
@GiteaBot GiteaBot added the lgtm/need 2 This PR needs two approvals by maintainers to be considered for merging. label Apr 15, 2022
@wxiaoguang
Copy link
Contributor

@delvh Most code comes from

Please read it first and follow it.

models/repo/setting.go Outdated Show resolved Hide resolved
models/migrations/v213.go Outdated Show resolved Hide resolved
models/repo/setting.go Outdated Show resolved Hide resolved
@lunny
Copy link
Member

lunny commented Jun 4, 2022

To avoid a data override when two requests, a version column is necessary.

@delvh
Copy link
Member

delvh commented Jun 4, 2022

Could you please elaborate why this would be necessary?
I don't exactly see the benefit...
As far I understand these tables, they are explicitly meant to store settings per repo/ user, so overwriting them should already be intended once the user changes the corresponding setting?
Why would we need a version?

@lunny
Copy link
Member

lunny commented Jun 4, 2022

Assume the user opened two pages for the same configuration, and the value is 1 on the page. And submit first page from 1 to 2. The second page will still display 1, but it should display 2, we should make it fail because the value has changed in first page, he should refresh first.

@techknowlogick techknowlogick modified the milestones: 1.17.0, 1.18.0 Jun 6, 2022
@lunny lunny modified the milestones: 1.18.0, 1.19.0 Oct 20, 2022
silverwind and others added 16 commits November 13, 2022 15:10
…#21791)

There was a bug introduced in go-gitea#21352 due to a change of behaviour caused
by go-gitea#19280. This causes a panic on running the default doctor checks
because the panic introduced by go-gitea#19280 assumes that the only way
opts.StdOut and opts.Stderr can be set in RunOpts is deliberately.
Unfortunately, when running a git.Command the provided RunOpts can be
set, therefore if you share a common set of RunOpts these two values can
be set by the previous commands.

This PR stops using common RunOpts for the commands in that doctor check
but secondly stops RunCommand variants from changing the provided
RunOpts.

Signed-off-by: Andrew Thornton <art27@cantab.net>
…#21813)

To use an anchor tag as a button and have it be accessible I added
`role="button" tabindex="0"`,
[reference](https://stackoverflow.com/a/10510353/7414734).

* Closes go-gitea#19912
…ea#21785)

The doctor check `storages` currently only checks the attachment
storage. This PR adds some basic garbage collection functionality for
the other types of storage.

Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
* Closes go-gitea#19997

Adds an `updated_at` time field to the `PullReview` API object to
specify when the pull request review's state changed.

Signed-off-by: Yarden Shoham <hrsi88@gmail.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
…1824)

GitHub migration tests will be skipped if the secret for the GitHub API
token hasn't been set.

This change should make all tests pass (or skip in the case of this one)
for anyone running the pipeline on their own infrastructure without
further action on their part.

Resolves go-gitea#21739

Signed-off-by: Gary Moon <gary@garymoon.net>
Added space between avatar and username which is missing on verified
commit message and avatar is too close to username which is don't look
nice.

Current state

![image](https://user-images.githubusercontent.com/3164256/202007728-d7d6ecac-f754-454c-a67f-e422f4aac5a5.png)


This is how it looks after change

![image](https://user-images.githubusercontent.com/3164256/202007984-d4a38a1c-7c24-4278-aa0f-9aa51c10f772.png)

Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
A file in `ISSUE_TEMPLATE` with the name `config.yml` shouldn't be
treated as a YAML template, it's for [configuring the template
chooser](https://docs.github.com/en/communities/using-templates-to-encourage-useful-issues-and-pull-requests/configuring-issue-templates-for-your-repository#configuring-the-template-chooser).

The old code tried to ignore the file, but it didn't work, caused by
go-gitea#20987. That's why the warning is displayed:

<img width="415" alt="image"
src="https://user-images.githubusercontent.com/9418365/202094067-804c42fe-0e9e-4fc5-bf01-d95fa336f54f.png">

Note that this PR is not an implementation of `config.yml`, there will
be another one to do it.
- Render directly into DOM, skipping string conversion
- Add limiting options to prevent excessive size/macros
- Remove invalid `display` option previously passed

Ref: https://katex.org/docs/options.html

Co-authored-by: John Olheiser <john.olheiser@gmail.com>
The headers can't be modified after it was send to the client.
The Ghost user should not be allowed to have access to a limited visible
user/org.

Co-authored-by: Lauris BH <lauris@nix.lv>
Bumps [loader-utils](https://github.com/webpack/loader-utils) from 2.0.3
to 2.0.4.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/webpack/loader-utils/releases">loader-utils's
releases</a>.</em></p>
<blockquote>
<h2>v2.0.4</h2>
<h3><a
href="https://github.com/webpack/loader-utils/compare/v2.0.3...v2.0.4">2.0.4</a>
(2022-11-11)</h3>
<h3>Bug Fixes</h3>
<ul>
<li>ReDoS problem (<a
href="https://github-redirect.dependabot.com/webpack/loader-utils/issues/225">#225</a>)
(<a
href="https://github.com/webpack/loader-utils/commit/ac09944dfacd7c4497ef692894b09e63e09a5eeb">ac09944</a>)</li>
</ul>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/webpack/loader-utils/blob/v2.0.4/CHANGELOG.md">loader-utils's
changelog</a>.</em></p>
<blockquote>
<h3><a
href="https://github.com/webpack/loader-utils/compare/v2.0.3...v2.0.4">2.0.4</a>
(2022-11-11)</h3>
<h3>Bug Fixes</h3>
<ul>
<li>ReDoS problem (<a
href="https://github-redirect.dependabot.com/webpack/loader-utils/issues/225">#225</a>)
(<a
href="https://github.com/webpack/loader-utils/commit/ac09944dfacd7c4497ef692894b09e63e09a5eeb">ac09944</a>)</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/webpack/loader-utils/commit/6688b5028106f144ee9f543bebc8e6a87b57829f"><code>6688b50</code></a>
chore(release): 2.0.4</li>
<li><a
href="https://github.com/webpack/loader-utils/commit/ac09944dfacd7c4497ef692894b09e63e09a5eeb"><code>ac09944</code></a>
fix: ReDoS problem (<a
href="https://github-redirect.dependabot.com/webpack/loader-utils/issues/225">#225</a>)</li>
<li>See full diff in <a
href="https://github.com/webpack/loader-utils/compare/v2.0.3...v2.0.4">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=loader-utils&package-manager=npm_and_yarn&previous-version=2.0.3&new-version=2.0.4)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
- `@dependabot use these labels` will set the current labels as the
default for future PRs for this repo and language
- `@dependabot use these reviewers` will set the current reviewers as
the default for future PRs for this repo and language
- `@dependabot use these assignees` will set the current assignees as
the default for future PRs for this repo and language
- `@dependabot use this milestone` will set the current milestone as the
default for future PRs for this repo and language

You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/go-gitea/gitea/network/alerts).

</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
- It's possible that the `user_redirect` table contains a user id that
no longer exists.
- Delete a user redirect upon deleting the user.
- Add a check for these dangling user redirects to check-db-consistency.
add webpack export type check

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
KN4CK3R and others added 24 commits January 14, 2023 23:57
closes go-gitea#13585
fixes go-gitea#9067
fixes go-gitea#2386
ref go-gitea#6226
ref go-gitea#6219
fixes #745

This PR adds support to process incoming emails to perform actions.
Currently I added handling of replies and unsubscribing from
issues/pulls. In contrast to go-gitea#13585 the IMAP IDLE command is used
instead of polling which results (in my opinion 😉) in cleaner code.

Procedure:
- When sending an issue/pull reply email, a token is generated which is
present in the Reply-To and References header.
- IMAP IDLE waits until a new email arrives
- The token tells which action should be performed

A possible signature and/or reply gets stripped from the content.

I added a new service to the drone pipeline to test the receiving of
incoming mails. If we keep this in, we may test our outgoing emails too
in future.

Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
…itea#22457)

The API endpoints for "git" can panic if they are called on an empty
repo. We can simply allow empty repos for these endpoints without worry
as they should just work.

Fix go-gitea#22452

Signed-off-by: Andrew Thornton <art27@cantab.net>
This PR introduce glob match for protected branch name. The separator is
`/` and you can use `*` matching non-separator chars and use `**` across
separator.

It also supports input an exist or non-exist branch name as matching
condition and branch name condition has high priority than glob rule.

Should fix go-gitea#2529 and go-gitea#15705

screenshots

<img width="1160" alt="image"
src="https://user-images.githubusercontent.com/81045/205651179-ebb5492a-4ade-4bb4-a13c-965e8c927063.png">

Co-authored-by: zeripath <art27@cantab.net>
In go-gitea#22447 it was noticed that display environments were not working
correctly. This was due to the setting displayMode not being set.

Further it was noticed that the error was not being displayed correctly.

This PR fixes both of these issues by forcibly setting the displayMode
setting and corrects an error in displayError.

Fix go-gitea#22447

Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
…NT_REPOS (go-gitea#22454)

There is a mistake in the code for SearchRepositoryCondition where it
tests topics as a string. This is incorrect for postgres where topics is
cast and stored as json. topics needs to be cast to text for this to
work. (For some reason JSON_ARRAY_LENGTH does not work, so I have taken
the simplest solution of casting to text and doing a string comparison.)

Ref go-gitea#21962 (comment)

Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: delvh <dev.lh@web.de>
Fix go-gitea#22370 and more.

Before Go 1.19, the `netgo` tag for Windows does nothing.

But Go 1.19 rewrite the net package code for Windows DNS, and there is a
bug:

* golang/go#57757

This PR just removes the `netgo` tag for Windows build, then the Gitea
for Windows can have the old DNS behavior.
…2462)

As suggest by Go developers, use `filepath.WalkDir` instead of
`filepath.Walk` because [*Walk is less efficient than WalkDir,
introduced in Go 1.16, which avoids calling `os.Lstat` on every file or
directory visited](https://pkg.go.dev/path/filepath#Walk).

This proposition address that, in a similar way as
go-gitea#22392 did.


Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
This PR adds a task to the cron service to allow garbage collection of
LFS meta objects. As repositories may have a large number of
LFSMetaObjects, an updated column is added to this table and it is used
to perform a generational GC to attempt to reduce the amount of work.
(There may need to be a bit more work here but this is probably enough
for the moment.)

Fix go-gitea#7045

Signed-off-by: Andrew Thornton <art27@cantab.net>
Addition to go-gitea#22056

This PR adds a hint to mail text if replies are supported.
I can't tell if the text structure is supported in every language. Maybe
we need to put the whole line in the translation file and use
parameters.
Reposting pull request for go-gitea#22465 

> Migration from GitBucket does not work due to a access for "Reviews"
API on GitBucket that makes 404 response. This PR has following changes.
> 1. Made to stop access for Reviews API while migrating from GitBucket.
> 2. Added support for custom URL (e.g.
`http://example.com/gitbucket/owner/repository`)
> 3. Made to accept for git checkout URL
(`http://example.com/git/owner/repository.git`)

Co-authored-by: zeripath <art27@cantab.net>
go-gitea#22479)

When offline mode is true, we should set `disable_gravatar` to `true`
and `enable_federated_avatar` to `false` in system settings.
This PR introduces a few minor changes to the gitea-monitoring-mixin,
specifically linting issues raised by
[Mixtool](https://github.com/monitoring-mixins/mixtool):
- Query selectors using `job` and `instance` have been update to allow
multi-select
- Added missing attributes to `job` and `instance` template

As this change is very minor I haven't created an issue, but please let
me know if you'd like me to do so. According to the guidelines, it
seemed to only be for larger designs :)
Frontport go-gitea#22471

Signed-off-by: jolheiser <john.olheiser@gmail.com>
Forgot to do this with the frontport

Signed-off-by: jolheiser <john.olheiser@gmail.com>
This just adds a mention on how to get the `swagger.json` for an
instance.

Signed-off-by: jolheiser <john.olheiser@gmail.com>
This PR adds the support for scopes of access tokens, mimicking the
design of GitHub OAuth scopes.

The changes of the core logic are in `models/auth` that `AccessToken`
struct will have a `Scope` field. The normalized (no duplication of
scope), comma-separated scope string will be stored in `access_token`
table in the database.
In `services/auth`, the scope will be stored in context, which will be
used by `reqToken` middleware in API calls. Only OAuth2 tokens will have
granular token scopes, while others like BasicAuth will default to scope
`all`.
A large amount of work happens in `routers/api/v1/api.go` and the
corresponding `tests/integration` tests, that is adding necessary scopes
to each of the API calls as they fit.


- [x] Add `Scope` field to `AccessToken`
- [x] Add access control to all API endpoints
- [x] Update frontend & backend for when creating tokens
- [x] Add a database migration for `scope` column (enable 'all' access
to past tokens)

I'm aiming to complete it before Gitea 1.19 release.

Fixes go-gitea#4300
Support [asciicast
files](https://github.com/asciinema/asciinema/blob/develop/doc/asciicast-v2.md)
as a new markup via
[asciinema-player](https://github.com/asciinema/asciinema-player). For
more on asciinema, see the [introduction](https://asciinema.org/).

So users can use asciinema recorder to generate an asciicast file (or
you can download a sample file from
https://asciinema.org/a/335480.cast?dl=1), then upload it to Gitea and
play it on Gitea.

Snapshots:
<details>

## Upload asciicast files

<img width="1134" alt="image"
src="https://user-images.githubusercontent.com/9418365/212461061-cc2c7181-0e14-4534-af55-1ec60a639fd1.png">

## Open an asciicast file

<img width="1137" alt="image"
src="https://user-images.githubusercontent.com/9418365/212461090-a3b5141f-4894-430d-a2b4-ea257801a0ed.png">

## Play it

<img width="1144" alt="image"
src="https://user-images.githubusercontent.com/9418365/212461157-4e82db69-0e41-471d-928f-ac1fe0737105.png">

## Copy contents from the "video"

<img width="1145" alt="image"
src="https://user-images.githubusercontent.com/9418365/212461286-211612bc-15d6-427a-89a9-6abff5c6a0a5.png">


## View the source

<img width="1140" alt="image"
src="https://user-images.githubusercontent.com/9418365/212461187-05473b2d-ba3d-4072-84a6-4aa1e7d82182.png">


</details>

Known issue:

Don't support the [v1 version asciicast
files](https://github.com/asciinema/asciinema/blob/develop/doc/asciicast-v1.md),
it's a poorly designed version, it does not specify the file extension
and uses `*.json` usually, so it's impossible to recognize the files.

Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
…ea#22354)

In some cases, the loading failure of `index.js` is not related to the
ROOT_URL directly, ex: https://gitea.com/gitea/helm-chart/issues/392

If the user's reversed proxy is mis-configured:
`http://public-domain/gitea/xxx` -> `http://gitea:3000/gitea/xxx`, it
also causes the loading failure.

So this PR removes the ROOT_URL related tip from the error message.
@lunny
Copy link
Member

lunny commented Jan 18, 2023

Don't know why I break this PR. So I just opened a new one #22500

@lunny lunny closed this Jan 18, 2023
@lunny lunny removed this from the 1.19.0 milestone Jan 18, 2023
@techknowlogick techknowlogick deleted the repo-settings branch January 18, 2023 14:22
@go-gitea go-gitea locked and limited conversation to collaborators May 3, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
lgtm/need 2 This PR needs two approvals by maintainers to be considered for merging. type/enhancement An improvement of existing functionality
Projects
None yet
Development

Successfully merging this pull request may close these issues.