GitLab

Co-authored-by: zeripath <art27@cantab.net>

Backport #20371

Xorm 1.3.2-0.20220714055524 contains a fix for interpreting db column sizes. Prior to this fix xorm would assume that the size of a column was within the range of an `int`. This is correct on 64bit machines where `int` is typical equivalent to `int64` however, on 32bit machines `int` tends to be `int32`. 

Unfortunately the size of a LONGTEXT field is actually `max_uint32`, thus using `strconv.Atoi` on these fields will fail and thus #20161 occurs on 32 bit arm. Xorm 1.3.2-0.20220714055524 changes this field to use int64 instead.

Fix  #20161

Backport #20300

  - Update goldmark to v1.4.13 to fix a issue with quotes after a empty list item(See https://github.com/yuin/goldmark/issues/313) and downstream issue https://codeberg.org/Codeberg/Community/issues/645

fgprof is a sampling Go profiler that allows you to analyze On-CPU as
well as Off-CPU (e.g. I/O) time together.

Go's builtin sampling CPU profiler can only show On-CPU time, but it's
better than fgprof at that. Go also includes tracing profilers that can
analyze I/O, but they can't be combined with the CPU profiler.

fgprof is designed for analyzing applications with mixed I/O and CPU
workloads. This kind of profiling is also known as wall-clock profiling.

Whilst fgprof can cause significant STW latencies in applications with a
lot of goroutines (> 1-10k), these latencies only occur if the profile
is requested - it doesn't cause a delay by simply being available.

The fgprof profile is mounted on
`http://localhost:6060/debug/fgprof?seconds=3`

Signed-off-by: Andrew Thornton <art27@cantab.net>

Use Golang 1.18 (as minimal requirement) for Gitea 1.17 release, make sure the Golang version is still actively supported during Gitea 1.17 lifecycle.
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: 6543 <6543@obermui.de>

A pr.Reviewer may be nil when migrating from Gitea if this is a team
request review.

We do not migrate teams therefore we cannot map these requests, but we can
migrate user requests.
Signed-off-by: Andrew Thornton <art27@cantab.net>

Fixes #12338

This allows use to talk to the API with our ssh certificate (and/or ssh-agent) without needing to fetch an API key or tokens.
It will just automatically work when users have added their ssh principal in gitea.

This needs client code in tea
Update: also support normal pubkeys

ref: https://tools.ietf.org/html/draft-cavage-http-signatures

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: zeripath <art27@cantab.net>
Signed-off-by: Andrew Thornton <art27@cantab.net>

Upgrade builder to v0.3.11
Upgrade xorm to v1.3.1 and fixed some hidden bugs.

Replace #19821
Replace #19834
Included #19850
Co-authored-by: zeripath <art27@cantab.net>

* update gitea.com/go-chi/cache -> v0.2.0

* ajust to new interface

* refactor

* update gitea.com/go-chi/binding
* update github.com/42wim/sshsig
* update github.com/PuerkitoBio/goquery
* update github.com/blevesearch/bleve/v2
* update github.com/caddyserver/certmagic
* update github.com/duo-labs/webauthn
* update github.com/editorconfig/editorconfig-core-go/v2
* update github.com/emirpasic/gods
* update github.com/gliderlabs/ssh
* update github.com/go-chi/cors
* update github.com/go-enry/go-enry/v2
* update github.com/go-git/go-git/v5
* update github.com/go-ldap/ldap/v3
* update github.com/golang-jwt/jwt/v4
* update github.com/google/pprof
* update github.com/klauspost/compress
* update github.com/lib/pq
* update github.com/markbates/goth
* update github.com/minio/minio-go/v7
* update github.com/olivere/elastic/v7
* update github.com/unrolled/render
* update github.com/urfave/cli
* update github.com/xanzy/go-gitlab
* update github.com/yuin/goldmark
* adopt breaking changes of certmagic

Remove two unmaintained vendor packages `i18n` and `paginater`. Changes:
* Rewrite `i18n` package with a more clear fallback mechanism. Fix an unstable `Tr` behavior, add more tests.
* Refactor the legacy `Paginater` to `Paginator`, test cases are kept unchanged.

Trivial enhancement (no breaking for end users):
* Use the first locale in LANGS setting option as the default, add a log to prevent from surprising users.

xorm/builder v0.3.10 add support to EXISTS and NOT EXISTS.

Follows: #19284
* The `CopyDir` is only used inside test code
* Rewrite `ToSnakeCase` with more test cases
* The `RedisCacher` only put strings into cache, here we use internal `toStr` to replace the legacy `ToStr`
* The `UniqueQueue` can use string as ID directly, no need to call `ToStr`

Previously if you tried to read a HostName in a config file that
looked like this:

```
Host github
    HostName github.com        # This is the host for code review
```

DefaultUserSettings.Get("HostName") would return "github.com        ",
which I think is unintuitive and unexpected.

This behavior is fixed in v1.2 which would return "github.com" in the
above example.

Continues on from #19202.

Following the addition of pprof labels we can now more easily understand the relationship between a goroutine and the requests that spawn them. 

This PR takes advantage of the labels and adds a few others, then provides a mechanism for the monitoring page to query the pprof goroutine profile.

The binary profile that results from this profile is immediately piped in to the google library for parsing this and then stack traces are formed for the goroutines.

If the goroutine is within a context or has been created from a goroutine within a process context it will acquire the process description labels for that process. 

The goroutines are mapped with there associate pids and any that do not have an associated pid are placed in a group at the bottom as unbound.

In this way we should be able to more easily examine goroutines that have been stuck.

A manager command `gitea manager processes` is also provided that can export the processes (with or without stacktraces) to the command line.
Signed-off-by: Andrew Thornton <art27@cantab.net>

Gitea was not able to supply any authentication parameters to it. So this brings support to do that, along with some light extraction of a couple of bits into some separate functions for easier testing.

I looked at other libraries supporting similar RedisUri-style connection strings (e.g. Lettuce), but it looks like this type of configuration is beyond what would typically be done in a connection string. Since gitea doesn't have configuration options for manually specifying all this redis connection detail, I went ahead and just chose straightforward names for these new parameters.

* Bump goldmark to v1.4.11

* add testcase
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Andrew Thornton <art27@cantab.net>

* use go1.18 to build gitea& update min go version to 1.17

* bump in a few more places

* add a few simple tests for isipprivate

* update go.mod

* update URL to https://go.dev/dl/

* golangci-lint

* attempt golangci-lint workaround

* change version

* bump fumpt version

* skip strings.title test

* go mod tidy

* update tests as some aren't private??

* update tests

This ensures the tools only run in the versions we've tested and it also
does not polute PATH with those tools so they are truly isolated. This
syntax of `go run` requires go 1.17, so the minimum version is set
accordingly.

Fixes: https://github.com/go-gitea/gitea/issues/18867

Co-authored-by: techknowlogick <techknowlogick@gitea.io>

* Update golang.org/x/crypto

- Update dependency to include fix for CVE.
- See https://groups.google.com/g/golang-announce/c/-cp44ypCT5s/m/wmegxkLiAQAJ?utm_medium=email&utm_source=footer

* Fix deprecation notice

* Remove workaround

- Introduced in https://github.com/go-gitea/gitea/pull/17281
- Fixed in x/crypto:
- https://github.com/golang/crypto/commit/5d542ad81a58c89581d596f49d0ba5d435481bcf
- & https://github.com/golang/crypto/commit/3147a52a75dda54ac3a611ef8978640d85188a2a

* Update Kex Algorithms

- Use standardized name for curve22519-sha256. https://github.com/golang/crypto/commit/9b076918e3c7e908b2bdea932f272a9979f2488a
- Prefer SHA256 version over SHA1 version. https://github.com/golang/crypto/commit/e4b3678e5f38521e67eba223ddd1902ceb3a303c

Fix #18855

* update gitea.com/go-chi/binding

* update gitea.com/go-chi/cache

* update github.com/42wim/sshsig

* update github.com/PuerkitoBio/goquery

* update github.com/blevesearch/bleve/v2

* update github.com/caddyserver/certmagic

* update github.com/denisenkom/go-mssqldb

* update github.com/duo-labs/webauthn

* update github.com/dustin/go-humanize

* update github.com/editorconfig/editorconfig-core-go/v2

* update github.com/ethantkoenig/rupture

* update github.com/go-chi/chi/v5

* update github.com/go-git/go-git/v5

* update github.com/go-ldap/ldap/v3

* update github.com/go-redis/redis/v8

* update github.com/go-swagger/go-swagger

* update github.com/gogs/chardet

* update github.com/golang-jwt/jwt/v4

* update github.com/hashicorp/go-version

* update github.com/jaytaylor/html2text

* update github.com/klauspost/compress

* update github.com/lib/pq

* update github.com/markbates/goth

* update github.com/mattn/go-sqlite3

* update github.com/mholt/archiver/v3

* update github.com/microcosm-cc/bluemonday

* update github.com/minio/minio-go/v7

* update github.com/msteinert/pam

* update github.com/niklasfasching/go-org

* update github.com/olivere/elastic/v7

* update github.com/prometheus/client_golang

* update github.com/quasoft/websspi

* update github.com/stretchr/testify

* update github.com/unknwon/i18n

* update github.com/unrolled/render

* update github.com/xanzy/go-gitlab

* update github.com/yuin/goldmark

* update github.com/yuin/goldmark-highlighting

* update github.com/yuin/goldmark-meta

* post make vendor
Signed-off-by: Andrew Thornton <art27@cantab.net>

* add make vendor back into update_dependencies.sh
Signed-off-by: Andrew Thornton <art27@cantab.net>

* Update update_dependencies.sh

* Update contrib/update_dependencies.sh
Co-authored-by: zeripath <art27@cantab.net>

* update mvdan.cc/xurls/v2
Co-authored-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>

- It was reported in the Matrix channel that Chroma [has support](https://github.com/alecthomas/chroma/commit/d40210297631298bde98a8fcfd52c3d2555cc367)
for the Fennel language, go-enry only added detection support for this
[in v2.7.2](https://github.com/go-enry/go-enry/commit/0affa3ccca37cd781600760956581c5f94ef8a3c

).
Co-authored-by: techknowlogick <techknowlogick@gitea.io>

- v1.6.0 has a inconsistency with the proxy and github's tagged version,
updating to v1.6.1 should fix this issue.
- Resolves #18864

Fix #14074
Signed-off-by: Andrew Thornton <art27@cantab.net>

* Update gitea-vet

- Update gitea-vet to include latest 2 changes
https://gitea.com/gitea/gitea-vet/compare/7c98703580bef612b10f6a603883052f79acf9c0...master



* Tidy up go.sum
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: zeripath <art27@cantab.net>

- Update json-iterator/go to handle 1.18's new memory model with slices.
- Don't panic while running gitea with go 1.18

* Update Goldmark to Goldmark 1.4.4
* nolint the deprecation
Signed-off-by: Andrew Thornton <art27@cantab.net>

The endpoint /{username}/{reponame}/milestone/{id} is not currently restricted to
the repo. This PR restricts the milestones to those within the repo.
Signed-off-by: Andrew Thornton <art27@cantab.net>

JSON Schema validation for data used by Gitea during migrations

Discussion at https://forum.forgefriends.org/t/common-json-schema-for-repository-information/563

Co-authored-by: Loïc Dachary <loic@dachary.org>

Signed-off-by: Andrew Thornton <art27@cantab.net>

Migrate from U2F to Webauthn
Co-authored-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>