Commit graph

852 commits

Author SHA1 Message Date
Lunny Xiao
3ca5dc7e32
Move keys to models/asymkey (#17917)
* Move keys to models/keys

* Rename models/keys -> models/asymkey

* change the missed package name

* Fix package alias

* Fix test

* Fix docs

* Fix test

* Fix test

* merge
2021-12-10 16:14:24 +08:00
Lunny Xiao
719bddcd76
Move repository model into models/repo (#17933)
* Some refactors related repository model

* Move more methods out of repository

* Move repository into models/repo

* Fix test

* Fix test

* some improvements

* Remove unnecessary function
2021-12-10 09:27:50 +08:00
mscherer
f49d160447
Replace "unix" by "http+unix" for PROTOCOL (#17771) 2021-12-05 23:46:11 -05:00
wxiaoguang
042cac5fed
Improve install code to avoid low-level mistakes. (#17779)
* Improve install code to avoid low-level mistakes.

If a user tries to do a re-install in a Gitea database, they gets a warning and double check.
When Gitea runs, it never create empty app.ini automatically.

Also some small (related) refactoring:

* Refactor db.InitEngine related logic make it more clean (especially for the install code)
* Move some i18n strings out from setting.go to make the setting.go can be easily maintained.
* Show errors in CLI code if an incorrect app.ini is used.
* APP_DATA_PATH is created when installing, and checked when starting (no empty directory is created any more).
2021-12-01 15:50:01 +08:00
zeripath
01087e9eef
Make Requests Processes and create process hierarchy. Associate OpenRepository with context. (#17125)
This PR registers requests with the process manager and manages hierarchy within the processes.

Git repos are then associated with a context, (usually the request's context) - with sub commands using this context as their base context.

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-11-30 20:06:32 +00:00
Lunny Xiao
1fee11d69a
Move accessmode into models/perm (#17828) 2021-11-28 12:58:28 +01:00
Lunny Xiao
a666829a37
Move user related model into models/user (#17781)
* Move user related model into models/user

* Fix lint for windows

* Fix windows lint

* Fix windows lint

* Move some tests in models

* Merge
2021-11-24 17:49:20 +08:00
Lunny Xiao
baed01f247
Remove unnecessary attributes of User struct (#17745)
* Remove unnecessary functions of User struct

* Move more database methods out of user struct

* Move more database methods out of user struct

* Fix template failure

* Fix bug

* Remove finished FIXME

* remove unnecessary code
2021-11-22 23:21:55 +08:00
techknowlogick
499b05da22
Add user settings key/value DB table (#16834) 2021-11-22 04:47:23 -05:00
zeripath
c96be0cd98
Make SSL cipher suite configurable (#17440) 2021-11-20 01:12:43 -05:00
Lunny Xiao
fc3d082609
Move attachment into models/repo/ (#17650)
* Move attachment into models/repo/

* Fix test

* Fix bug
2021-11-19 21:39:57 +08:00
KN4CK3R
f34151bdb2
Move user/org deletion to services (#17673) 2021-11-19 01:42:27 +08:00
Gusted
c98dd7a3e0
Remove unnecessary variable assignments (#17695)
* Remove unnecessary variable assignments

As title

* enable ineffassign

Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2021-11-18 09:33:06 +08:00
Lunny Xiao
5233051e64
Move some functions into services/repository (#17677) 2021-11-17 23:17:31 +08:00
wxiaoguang
750a8465f5
A better go code formatter, and now make fmt can run in Windows (#17684)
* go build / format tools
* re-format imports
2021-11-17 20:34:35 +08:00
Lunny Xiao
7e1ae38097
Move migrations into services and base into modules/migration (#17663)
* Move migrtions into services and base into modules/migration

* Fix imports

* Fix lint
2021-11-16 23:25:33 +08:00
Gusted
d2163df6a0
Fix offBy1 errors (#17606)
* Fix offBy1 errors

- Partially resolves #17596
- Resolve errors from go-critic `offBy1: Index() can return -1; maybe
you wanted to do Index()+1`.

* Match golang spec

* Remove comments

* Update migrations.go

* Apply suggestions from code review

Co-authored-by: delvh <dev.lh@web.de>

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: delvh <dev.lh@web.de>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2021-11-14 10:11:49 +02:00
Lunny Xiao
69b61d4373
Fix bug on admin subcommand (#17533)
* Fix bug on admin subcommand

* Add signals for all initDB

Co-authored-by: Lauris BH <lauris@nix.lv>
2021-11-07 11:11:27 +08:00
zeripath
9d855bd6a1
Simplify Gothic to use our session store instead of creating a different store (#17507)
* Simplify Gothic to use our session store instead of creating a different store

We have been using xormstore to provide a separate session store for our OAuth2 logins
however, this relies on using gorilla context and some doubling of our session storing.
We can however, simplify and simply use our own chi-based session store. Thus removing
a cookie and some of the weirdness with missing contexts.

Signed-off-by: Andrew Thornton <art27@cantab.net>

* as per review

Signed-off-by: Andrew Thornton <art27@cantab.net>

* as per review

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Handle MaxTokenLength

Signed-off-by: Andrew Thornton <art27@cantab.net>

* oops

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: Lauris BH <lauris@nix.lv>
2021-11-03 08:33:54 +08:00
wxiaoguang
599ff1c054
Only allow webhook to send requests to allowed hosts (#17482) 2021-11-01 16:39:52 +08:00
Lunny Xiao
63c0dc89ef
Rename db Engines related functions (#17481)
* Rename db Engines related functions

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2021-10-30 22:32:11 +08:00
Lunny Xiao
f494776931
Use a variable but a function for IsProd because of a slight performance increment (#17368) 2021-10-20 16:37:19 +02:00
zeripath
123f0aea00
Allow LDAP Sources to provide Avatars (#16851)
* Allow LDAP Sources to provide Avatars

Add setting to LDAP source to allow it to provide an Avatar.

Currently this is required to point to the image bytes.

Fix #4144

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Rename as Avatar Attribute (drop JPEG)

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Always synchronize avatar if there is change

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Actually get the avatar from the ldap

Signed-off-by: Andrew Thornton <art27@cantab.net>

* clean-up

Signed-off-by: Andrew Thornton <art27@cantab.net>

* use len()>0 rather than != ""

Signed-off-by: Andrew Thornton <art27@cantab.net>

* slight shortcut in IsUploadAvatarChanged

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2021-09-26 22:39:36 -04:00
Lunny Xiao
5842a55b31
Move login related structs and functions to models/login (#17093)
* Move login related structs and functions to models/login

* Fix test

* Fix lint

* Fix lint

* Fix lint of windows

* Fix lint

* Fix test

* Fix test

* Only load necessary fixtures when preparing unit tests envs

* Fix lint

* Fix test

* Fix test

* Fix error log

* Fix error log

* Fix error log

* remove unnecessary change

* fix error log

* merge main branch
2021-09-24 19:32:56 +08:00
Eng Zer Jun
f2e7d5477f
refactor: move from io/ioutil to io and os package (#17109)
The io/ioutil package has been deprecated as of Go 1.16, see
https://golang.org/doc/go1.16#ioutil. This commit replaces the existing
io/ioutil functions with their new definitions in io and os packages.

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2021-09-22 13:38:34 +08:00
Lunny Xiao
a4bfef265d
Move db related basic functions to models/db (#17075)
* Move db related basic functions to models/db

* Fix lint

* Fix lint

* Fix test

* Fix lint

* Fix lint

* revert unnecessary change

* Fix test

* Fix wrong replace string

* Use *Context

* Correct committer spelling and fix wrong replaced words

Co-authored-by: zeripath <art27@cantab.net>
2021-09-19 19:49:59 +08:00
zeripath
27b351aba5
Make LDAP be able to skip local 2FA (#16954)
This PR extends #16594 to allow LDAP to be able to be set to skip local 2FA too. The technique used here would be extensible to PAM and SMTP sources.

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-09-17 12:43:47 +01:00
zeripath
04b233e940
Always emit the configuration path (#17036)
Often when handling problems it is not clear which configuration file Gitea is
using. This PR simply ensures that the configuration file is emitted.

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-09-13 21:24:57 -04:00
zeripath
9ca0e7905c
Add setting to OAuth handlers to skip local 2FA authentication (#16594)
This PR adds a setting to OAuth and OpenID login sources to allow the source to
skip local 2FA authentication.

Fix #13939

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-09-10 18:37:57 +02:00
Lunny Xiao
a807031a30
Fix storage Iterate bug and Add storage doctor to delete garbage attachments (#16971)
* Fix storage Iterate bug and Add storage doctor to delete garbage attachments

* Close object when used
2021-09-06 16:46:20 +02:00
John Olheiser
648464b504
Add bundle download for repository (#14538)
* Add bundle download

Signed-off-by: jolheiser <john.olheiser@gmail.com>

* Fix fmt

Signed-off-by: jolheiser <john.olheiser@gmail.com>

* Fix build tags

Signed-off-by: jolheiser <john.olheiser@gmail.com>

* Download specific commit

Signed-off-by: jolheiser <john.olheiser@gmail.com>
2021-08-24 11:47:09 -05:00
zeripath
73defbbd1c
Ensure that template compilation panics are sent to the logs (#16788)
Although panics within the rendering pipeline are caught and dealt with,
panics that occur before that starts are unprotected and will kill Gitea
without being sent to the logs.

This PR adds a basic recovery handler to catch panics that occur after
the logger is initialised and ensure that they're sent to the logger.

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-08-23 20:40:59 +01:00
a1012112796
3705168837
Add agit flow support in gitea (#14295)
* feature: add agit flow support

ref: https://git-repo.info/en/2020/03/agit-flow-and-git-repo/

example:

```Bash
git checkout -b test
echo "test" >> README.md
git commit -m "test"
git push origin HEAD:refs/for/master -o topic=test
```

Signed-off-by: a1012112796 <1012112796@qq.com>

* fix lint

* simplify code add fix some nits

* update merge help message

* Apply suggestions from code review. Thanks @jiangxin

* add forced-update message

* fix lint

* splite writePktLine

* add refs/for/<target-branch>/<topic-branch> support also

* Add test code add fix api

* fix lint

* fix test

* skip test if git version < 2.29

* try test with git 2.30.1

* fix permission check bug

* fix some nit

* logic implify and test code update

* fix bug

* apply suggestions from code review

* prepare for merge

Signed-off-by: Andrew Thornton <art27@cantab.net>

* fix permission check bug

- test code update
- apply suggestions from code review @zeripath

Signed-off-by: a1012112796 <1012112796@qq.com>

* fix bug when target branch isn't exist

* prevent some special push and fix some nits

* fix lint

* try splite

* Apply suggestions from code review

- fix permission check
- handle user rename

* fix version negotiation

* remane

* fix template

* handle empty repo

* ui: fix  branch link under the title

* fix nits

Co-authored-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2021-07-28 17:42:56 +08:00
Lunny Xiao
9f31f3aa8a
Add an abstract json layout to make it's easier to change json library (#16528)
* Add an abstract json layout to make it's easier to change json library

* Fix import

* Fix import sequence

* Fix blank lines

* Fix blank lines
2021-07-24 18:03:58 +02:00
techknowlogick
29a22ade82
switch to maintained lib (#16532)
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Andrew Thornton <art27@cantab.net>
2021-07-24 13:00:41 +02:00
zeripath
5d2e11eedb
Refactor: Move login out of models (#16199)
`models` does far too much. In particular it handles all `UserSignin`.

It shouldn't be responsible for calling LDAP, SMTP or PAM for signing in.

Therefore we should move this code out of `models`.

This code has to depend on `models` - therefore it belongs in `services`.

There is a package in `services` called `auth` and clearly this functionality belongs in there.

Plan:

- [x] Change `auth.Auth` to `auth.Method` - as they represent methods of authentication.
- [x] Move `models.UserSignIn` into `auth`
- [x] Move `models.ExternalUserLogin`
- [x] Move most of the `LoginVia*` methods to `auth` or subpackages
- [x] Move Resynchronize functionality to `auth`
  - Involved some restructuring of `models/ssh_key.go` to reduce the size of this massive file and simplify its files.
- [x] Move the rest of the LDAP functionality in to the ldap subpackage
- [x] Re-factor the login sources to express an interfaces `auth.Source`?
  - I've done this through some smaller interfaces Authenticator and Synchronizable - which would allow us to extend things in future
- [x] Now LDAP is out of models - need to think about modules/auth/ldap and I think all of that functionality might just be moveable
- [x] Similarly a lot Oauth2 functionality need not be in models too and should be moved to services/auth/source/oauth2
  - [x] modules/auth/oauth2/oauth2.go uses xorm... This is naughty - probably need to move this into models.
  - [x] models/oauth2.go - mostly should be in modules/auth/oauth2 or services/auth/source/oauth2 
- [x] More simplifications of login_source.go may need to be done
- Allow wiring in of notify registration -  *this can now easily be done - but I think we should do it in another PR*  - see #16178 
- More refactors...?
  - OpenID should probably become an auth Method but I think that can be left for another PR
  - Methods should also probably be cleaned up  - again another PR I think.
  - SSPI still needs more refactors.* Rename auth.Auth auth.Method
* Restructure ssh_key.go

- move functions from models/user.go that relate to ssh_key to ssh_key
- split ssh_key.go to try create clearer function domains for allow for
future refactors here.

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-07-24 11:16:34 +01:00
zeripath
33a8eec33e
Retry rename on lock induced failures (#16435)
* Retry rename on lock induced failures

Due to external locking on Windows it is possible for an
os.Rename to fail if the files or directories are being
used elsewhere.

This PR simply suggests retrying the rename again similar
to how we handle the os.Remove problems.

Fix #16427

Signed-off-by: Andrew Thornton <art27@cantab.net>

* resolve CI fail

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2021-07-15 11:46:07 -04:00
zeripath
3dcb3e9073
Second attempt at preventing zombies (#16326)
* Second attempt at preventing zombies

* Ensure that the pipes are closed in ssh.go
* Ensure that a cancellable context is passed up in cmd/* http requests
* Make cmd.fail return properly so defers are obeyed
* Ensure that something is sent to stdout in case of blocks here

Signed-off-by: Andrew Thornton <art27@cantab.net>

* placate lint

Signed-off-by: Andrew Thornton <art27@cantab.net>

* placate lint 2

Signed-off-by: Andrew Thornton <art27@cantab.net>

* placate lint 3

Signed-off-by: Andrew Thornton <art27@cantab.net>

* fixup

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Apply suggestions from code review

Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Lauris BH <lauris@nix.lv>
2021-07-14 10:43:13 -04:00
Stanley Hu
3dba75fb97
Support HTTP/2 in Let's Encrypt (#16371)
Modify the tlsConfig.NextProtos for Let's Encrypt and built-in HTTPS server in order to support HTTP/2.

Co-authored-by: 6543 <6543@obermui.de>
2021-07-13 18:17:46 +01:00
luzpaz
e0296b6a6d
Fix various documentation, user-facing, and source comment typos (#16367)
* Fix various doc, user-facing, and source comment typos

Found via `codespell -q 3 -S ./options/locale,./vendor -L ba,pullrequest,pullrequests,readby`
2021-07-08 13:38:13 +02:00
zeripath
302e8b6d02
Prevent zombie processes (#16314)
Unfortunately go doesn't always ensure that execd processes are completely
waited for. On linux this means that zombie processes can occur.

This PR ensures that these are waited for by using signal notifier in serv and
passing a context elsewhere.

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-06-30 22:07:23 +02:00
zeripath
35f37a3625
Add --quiet and --verbose to gitea web to control initial logging (#16260)
One of the repeatedly reported issues has been that gitea produces too much console
logging during set up even if the console logger is turned off.

Fundamentally this is due to some otherwise very helpful logging that has to occur
before logging is set up. This has come to a head with the merging of #16243 where
otherwise potentially helpful Trace logging in the git module now appears on the
console.

This PR proposes three things:

1. Change the initial default logger to Info not Trace.
2. Change the logging for the AppPath things to Info in recompense.
3. Add two new command line options to gitea web: --quiet and --verbose

`gitea web -q` or `gitea web --quiet` will only log Fatal level initially.
`gitea web -verbose` will log at Trace.

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2021-06-26 20:56:58 -04:00
KN4CK3R
44b8b07631
Add tag protection (#15629)
* Added tag protection in hook.

* Prevent UI tag creation if protected.

* Added settings page.

* Added tests.

* Added suggestions.

* Moved tests.

* Use individual errors.

* Removed unneeded methods.

* Switched delete selector.

* Changed method names.

* No reason to be unique.

* Allow editing of protected tags.

* Removed unique key from migration.

* Added docs page.

* Changed date.

* Respond with 404 to not found tags.

* Replaced glob with regex pattern.

* Added support for glob and regex pattern.

* Updated documentation.

* Changed white* to allow*.

* Fixed edit button link.

* Added cancel button.

Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2021-06-25 16:28:55 +02:00
KN4CK3R
383ffcfa34
Small refactoring of modules/private (#15947)
* Use correct variable name.

* doer is never nil here.

* Use status code constants.

* Replaced generic map with concrete struct.

* Fixed windows lint.

* Removed unused method.

* Changed error codes.

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2021-06-23 15:38:19 -04:00
KN4CK3R
29695cd6d5
Add asymmetric JWT signing (#16010)
* Added asymmetric token signing.

* Load signing key from settings.

* Added optional kid parameter.

* Updated documentation.

* Add "kid" to token header.
2021-06-17 23:56:46 +02:00
Lunny Xiao
1bfb0a24d8
Refactor routers directory (#15800)
* refactor routers directory

* move func used for web and api to common

* make corsHandler a function to prohibit side efects

* rm unused func

Co-authored-by: 6543 <6543@obermui.de>
2021-06-09 01:33:54 +02:00
KN4CK3R
efe77eec85
Unified custom config creation (#16012)
* Unified custom config creation.

* Fixed log message.

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2021-05-29 19:44:14 +01:00
a1012112796
fec8324026
add a new internal hook to save ssh log (#15787)
* add a new internal hook to save ssh log

as title, when a ssh error ocure like #15785.
only when switch ``RUN_MODE`` to dev can we
found which error is ocure. But this way is
not a good idea for production envirment.

this changes try save ssh error mesage to the
log file like other log by a new internal hook.
I think it's usefull for find error message
in production envirment. Thanks.

Signed-off-by: a1012112796 <1012112796@qq.com>

* rename and fix nit

* Update modules/private/hook.go

Co-authored-by: silverwind <me@silverwind.io>

Co-authored-by: techknowlogick <matti@mdranta.net>
Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2021-05-21 17:37:16 -04:00
Naohisa Murakami
9f19c2b8cc
Fix error message when saving generated LOCAL_ROOT_URL config (#15880) 2021-05-15 11:23:21 +01:00
Blake Miner
e2f39c2b64
Fix bound address/port for caddy's certmagic library (see #15848) (#15859) 2021-05-14 15:39:10 +01:00
Blake Miner
6a8e5f69cf
Fix bound address/port for caddy's certmagic library (#15758)
* Fix bound address/port for caddy's certmagic library

* Fix bug

Co-authored-by: zeripath <art27@cantab.net>
2021-05-12 20:58:55 +01:00
Lunny Xiao
e5723d6556
Move restore repo to internal router and invoke from command to avoid open the same db file or queues files (#15790)
* Move restore repo to internal router and invoke from command to avoid open the same db file or queues files

* Follow @zeripath's review

* set no timeout for resotre repo private request

* make restore repo cancelable
2021-05-10 15:57:45 +08:00
6543
8ea1d32bea
[Vendor] update certmagic (#15590)
* update github.com/caddyserver/certmagic v0.12.0 -> v0.13.0

* migrate
2021-04-22 22:42:33 +02:00
Lunny Xiao
e7fc078891
Fix missing storage init (#15589) 2021-04-22 18:48:47 +02:00
Johan Van de Wauw
6d2866f20c
dump: Add option to skip LFS/attachment files (#15407)
* Add option to skip dumping LFS/attachment files

* Fix fmt issues

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2021-04-12 17:33:32 +08:00
zeripath
9d2c251214
Move FCGI req.URL.Path fix-up to the FCGI listener (#15292)
Simplify the web.go FCGI path by moving the req.URL.Path fix-up to listener

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: 6543 <6543@obermui.de>
2021-04-09 21:17:57 +08:00
6543
9c4601bdf8
Code Formats, Nits & Unused Func/Var deletions (#15286)
* _ to unused func options

* rm useless brakets

* rm trifial non used models functions

* rm dead code

* rm dead global vars

* fix routers/api/v1/repo/issue.go

* dont overload import module
2021-04-09 09:40:34 +02:00
KN4CK3R
c03e488e14
Add LFS Migration and Mirror (#14726)
* Implemented LFS client.

* Implemented scanning for pointer files.

* Implemented downloading of lfs files.

* Moved model-dependent code into services.

* Removed models dependency. Added TryReadPointerFromBuffer.

* Migrated code from service to module.

* Centralised storage creation.

* Removed dependency from models.

* Moved ContentStore into modules.

* Share structs between server and client.

* Moved method to services.

* Implemented lfs download on clone.

* Implemented LFS sync on clone and mirror update.

* Added form fields.

* Updated templates.

* Fixed condition.

* Use alternate endpoint.

* Added missing methods.

* Fixed typo and make linter happy.

* Detached pointer parser from gogit dependency.

* Fixed TestGetLFSRange test.

* Added context to support cancellation.

* Use ReadFull to probably read more data.

* Removed duplicated code from models.

* Moved scan implementation into pointer_scanner_nogogit.

* Changed method name.

* Added comments.

* Added more/specific log/error messages.

* Embedded lfs.Pointer into models.LFSMetaObject.

* Moved code from models to module.

* Moved code from models to module.

* Moved code from models to module.

* Reduced pointer usage.

* Embedded type.

* Use promoted fields.

* Fixed unexpected eof.

* Added unit tests.

* Implemented migration of local file paths.

* Show an error on invalid LFS endpoints.

* Hide settings if not used.

* Added LFS info to mirror struct.

* Fixed comment.

* Check LFS endpoint.

* Manage LFS settings from mirror page.

* Fixed selector.

* Adjusted selector.

* Added more tests.

* Added local filesystem migration test.

* Fixed typo.

* Reset settings.

* Added special windows path handling.

* Added unit test for HTTPClient.

* Added unit test for BasicTransferAdapter.

* Moved into util package.

* Test if LFS endpoint is allowed.

* Added support for git://

* Just use a static placeholder as the displayed url may be invalid.

* Reverted to original code.

* Added "Advanced Settings".

* Updated wording.

* Added discovery info link.

* Implemented suggestion.

* Fixed missing format parameter.

* Added Pointer.IsValid().

* Always remove model on error.

* Added suggestions.

* Use channel instead of array.

* Update routers/repo/migrate.go

* fmt

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: zeripath <art27@cantab.net>
2021-04-08 18:25:57 -04:00
zeripath
c03f530212
Make internal SSH server host key path configurable (#14918)
* Make SSH server host key path configurable

* make it possible to have multiple keys

* Make gitea.rsa the default key

* Add some more logging

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-03-08 03:43:59 +01:00
zeripath
f0e15250b9
Migrate to use jsoniter instead of encoding/json (#14841)
* Migrate to use jsoniter

* fix tests

* update gitea.com/go-chi/binding

Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
2021-03-01 22:08:10 +01:00
zeripath
3477e616ab
Exclude the current dump file from the dump (#14606)
* Exclude the current dump file from the dump

Always prevent the current file from being added to the dump.

Fix #13618

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Add skip custom directory option

Signed-off-by: Andrew Thornton <art27@cantab.net>

* placate lint

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: 6543 <6543@obermui.de>
2021-02-08 09:00:12 +08:00
Lunny Xiao
6433ba0ec3
Move macaron to chi (#14293)
Use [chi](https://github.com/go-chi/chi) instead of the forked [macaron](https://gitea.com/macaron/macaron). Since macaron and chi have conflicts with session share, this big PR becomes a have-to thing. According my previous idea, we can replace macaron step by step but I'm wrong. :( Below is a list of big changes on this PR.

- [x] Define `context.ResponseWriter` interface with an implementation `context.Response`.
- [x] Use chi instead of macaron, and also a customize `Route` to wrap chi so that the router usage is similar as before.
- [x] Create different routers for `web`, `api`, `internal` and `install` so that the codes will be more clear and no magic .
- [x] Use https://github.com/unrolled/render instead of macaron's internal render
- [x] Use https://github.com/NYTimes/gziphandler instead of https://gitea.com/macaron/gzip
- [x] Use https://gitea.com/go-chi/session which is a modified version of https://gitea.com/macaron/session and removed `nodb` support since it will not be maintained. **BREAK**
- [x] Use https://gitea.com/go-chi/captcha which is a modified version of https://gitea.com/macaron/captcha
- [x] Use https://gitea.com/go-chi/cache which is a modified version of https://gitea.com/macaron/cache
- [x] Use https://gitea.com/go-chi/binding which is a modified version of https://gitea.com/macaron/binding
- [x] Use https://github.com/go-chi/cors instead of https://gitea.com/macaron/cors
- [x] Dropped https://gitea.com/macaron/i18n and make a new one in `code.gitea.io/gitea/modules/translation`
- [x] Move validation form structs from `code.gitea.io/gitea/modules/auth` to `code.gitea.io/gitea/modules/forms` to avoid dependency cycle.
- [x] Removed macaron log service because it's not need any more. **BREAK**
- [x] All form structs have to be get by `web.GetForm(ctx)` in the route function but not as a function parameter on routes definition.
- [x] Move Git HTTP protocol implementation to use routers directly.
- [x] Fix the problem that chi routes don't support trailing slash but macaron did.
- [x] `/api/v1/swagger` now will be redirect to `/api/swagger` but not render directly so that `APIContext` will not create a html render. 

Notices:
- Chi router don't support request with trailing slash
- Integration test `TestUserHeatmap` maybe mysql version related. It's failed on my macOS(mysql 5.7.29 installed via brew) but succeed on CI.

Co-authored-by: 6543 <6543@obermui.de>
2021-01-26 16:36:53 +01:00
techknowlogick
d2ea21d0d8
Use caddy's certmagic library for extensible/robust ACME handling (#14177)
* use certmagic for more extensible/robust ACME cert handling

* accept TOS based on config option

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: Lauris BH <lauris@nix.lv>
2021-01-25 01:37:35 +02:00
Lunny Xiao
84b147c7f0
Use IsProd instead of testing if it's equal. (#14336)
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: zeripath <art27@cantab.net>
2021-01-14 23:17:03 +02:00
6543
74a0481586
[Refactor] Passwort Hash/Set (#14282)
* move SaltGeneration into HashPasswort and rename it to what it does

* Migration: Where Password is Valid with Empty String delete it

* prohibit empty password hash

* let SetPassword("") unset pwd stuff
2021-01-10 20:05:18 +02:00
Daniil Pankratov
3175d08626
CLI support for OAuth sources custom icons (#14166)
Co-authored-by: Daniil Pankratov <daniil.pankratov@t-systems.com>
2020-12-28 09:39:12 +01:00
Lunny Xiao
dd08853b10
Dump github/gitlab/gitea repository data to a local directory and restore to gitea (#12244)
* Dump github/gitlab repository data to a local directory

* Fix lint

* Adjust directory structure

* Allow migration special units

* Allow migration ignore release assets

* Fix lint

* Add restore repository

* stage the changes

* Merge

* Fix lint

* Update the interface

* Add some restore methods

* Finish restore

* Add comments

* Fix restore

* Add a token flag

* Fix bug

* Fix test

* Fix test

* Fix bug

* Fix bug

* Fix lint

* Fix restore

* refactor downloader

* fmt

* Fix bug isEnd detection on getIssues

* Refactor maxPerPage

* Remove unused codes

* Remove unused codes

* Fix bug

* Fix restore

* Fix dump

* Uploader should not depend downloader

* use release attachment name but not id

* Fix restore bug

* Fix lint

* Fix restore bug

* Add a method of DownloadFunc for base.Release to make uploader not depend on downloader

* fix Release yml marshal

* Fix trace information

* Fix bug when dump & restore

* Save relative path on yml file

* Fix bug

* Use relative path

* Update docs

* Use git service string but not int

* Recognize clone addr to service type
2020-12-27 11:34:19 +08:00
6543
a19447aed1
migrate from com.* to alternatives (#14103)
* remove github.com/unknwon/com from models

* dont use "com.ToStr()"

* replace "com.ToStr" with "fmt.Sprint" where its easy to do

* more refactor

* fix test

* just "proxy" Copy func for now

* as per @lunny
2020-12-25 11:59:32 +02:00
zeripath
5e09de174c
fix label of --id in admin delete user (#14005)
* fix label of --id in admin delete user

This pr fixes the label descriptor of `gitea admin delete user`
but also adds a `--username` option.

Fix #13995

Signed-off-by: Andrew Thornton <art27@cantab.net>

* fix-spacing

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Add delete email support

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2020-12-17 02:57:32 -05:00
zeripath
287b594803
Whenever the password is updated ensure that the hash algorithm is too (#13966)
`user.HashPassword` may potentially - and in fact now likely does - change
the `passwd_hash_algo` therefore whenever the `passwd` is updated, this
also needs to be updated.

Fix #13832

Thanks @fblaese for the hint

Signed-off-by: Andrew Thornton <art27@cantab.net>
2020-12-12 19:59:49 +01:00
zeripath
4569339a4b
Refactor doctor (#12264)
* Refactor Logger

Refactor Logger to make a logger interface and make it possible to
wrap loggers for specific purposes.

* Refactor Doctor

    Move the gitea doctor functions into its own module.
    Use a logger for its messages instead of returning a results string[]

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Update modules/doctor/misc.go

Co-authored-by: 6543 <6543@obermui.de>

* Update modules/doctor/misc.go

Co-authored-by: 6543 <6543@obermui.de>

Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2020-12-02 06:56:04 +02:00
zeripath
742e21aeba
Handle and propagate errors when checking if paths are Dirs, Files or Exist (#13186)
* Ensure errors from IsDir propagate

* Handle errors when checking IsFile

* Handle and propagate errors from IsExist

* Update modules/templates/static.go

* Update modules/templates/static.go

* Return after ctx.ServerError

* Apply suggestions from code review

* Fix tests

The previous merge managed to break repo_form.go

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: Lauris BH <lauris@nix.lv>
2020-11-27 21:42:08 -05:00
Lunny Xiao
586bfb9f32
Use mount but not register for chi routes (#13555)
* Use mount but not register for chi routes

* Fix test

* Fix test

* Fix test

* Fix comment

* turn back unnecessary change

* Remove the timout middleware since some operations may spend much time.
2020-11-16 15:33:41 +08:00
Lunny Xiao
c296f4fed6
Introduce go chi web framework as frontend of macaron, so that we can move routes from macaron to chi step by step (#7420)
* When route cannot be found on chi, go to macaron

* Stick chi version to 1.5.0

* Follow router log setting
2020-11-13 14:51:07 +02:00
John Olheiser
172eca1cd3
Add docs command (#13429)
Signed-off-by: jolheiser <john.olheiser@gmail.com>
2020-11-07 08:32:57 +08:00
zeripath
dd12384f22
Fix --port setting (#13288)
* Fix --port setting

Unfortunately there was an error in #13195 which set the --port
option before the settings were read. This PR fixes this by
moving applying this option to after the the settings are read

However, on looking further into this code I believe that the setPort
code was slightly odd.

Firstly, it may make sense to run the install page on a different
temporary port to the full system and this should be possible with
a --install-port option.

Secondy, if the --port option is provided we should apply it to both
otherwise there will be unusual behaviour on graceful restart

Thirdly, the documentation for --port says that the setting is
temporary - it should therefore not save its result to the configuration

(This however, does mean that authorized_keys and internal links may
not be correct. - I think we need to discuss this option further.)

Fix #13277

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Update cmd/web.go

* Apply suggestions from code review

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2020-10-30 15:26:03 -04:00
Lunny Xiao
38d11eea58
Fix send mail (#13312)
* Fix send mail

* Fix send mail

* Update modules/private/mail.go

Co-authored-by: techknowlogick <matti@mdranta.net>
2020-10-26 12:42:27 -04:00
Maxim Zhiburt
a1952afc38
Sendmail command (#13079)
* Add SendSync method

Usefull to have when you need to be confident that message was sent.

* Add sendmail command

* add checks that if either title or content is empty then error out

* Add a confirmation step

* Add --force option to bypass confirm step

* Move implementation of runSendMail to a different file

* Add copyrighting comment

* Make content optional

Print waring if it's empty or haven't been set up.
The warning will be skiped if there's a `--force` flag.

* Fix import style

Co-authored-by: 6543 <6543@obermui.de>

* Use batch when getting all users

IterateUsers uses batching by default.

Signed-off-by: Maxim Zhiburt <zhiburt@gmail.com>

* Send emails one by one instead of as one chunck

Signed-off-by: Maxim Zhiburt <zhiburt@gmail.com>

* Send messages concurantly

Signed-off-by: Maxim Zhiburt <zhiburt@gmail.com>

* Use SendAsync+Flush instead of SendSync

Signed-off-by: Maxim Zhiburt <zhiburt@gmail.com>

* Add timeout parameter to sendemail command

Signed-off-by: Maxim Zhiburt <zhiburt@gmail.com>

* Fix spelling mistake

Signed-off-by: Maxim Zhiburt <zhiburt@gmail.com>

* Update cmd/admin.go

Co-authored-by: 6543 <6543@obermui.de>

* Connect to a running Gitea instance

* Fix mispelling

* Add copyright comment

Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2020-10-24 16:38:14 -04:00
zeripath
2f1353a2f3
Move install pages out of main macaron routes (#13195)
* Move install pages out of main macaron loop

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Update templates/post-install.tmpl

Co-authored-by: Lauris BH <lauris@nix.lv>

* remove prefetch

Signed-off-by: Andrew Thornton <art27@cantab.net>
2020-10-19 17:03:08 -04:00
ydelafollye
cb171dbd56
Improve users management through the CLI (#6001) (#10492)
* Fix images in wiki edit preview (#11546)

Make sure wiki editor sets wiki to true so gitea renders it as a wiki page.

Also change the context data attr for edit form. This looks wrong but everywhere else in our code assumes the urlPrefix to be just the repo url when rendering and manually adds /wiki to the rendered url regardless.

Fixes #11540
2020-10-15 22:48:38 -04:00
Lunny Xiao
80a6b0f5bc
Avatars and Repo avatars support storing in minio (#12516)
* Avatar support minio

* Support repo avatar minio storage

* Add missing migration

* Fix bug

* Fix test

* Add test for minio store type on avatars and repo avatars; Add documents

* Fix bug

* Fix bug

* Add back missed avatar link method

* refactor codes

* Simplify the codes

* Code improvements

* Fix lint

* Fix test mysql

* Fix test mysql

* Fix test mysql

* Fix settings

* Fix test

* fix test

* Fix bug
2020-10-14 21:07:51 +08:00
zeripath
2d75d6f664
Move update-server-info to hooks (#12826)
Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2020-10-14 00:24:06 +08:00
zeripath
6b1266b6b3
Provide self-registering storage system (#12978)
* Provide self-registering storage system

Signed-off-by: Andrew Thornton <art27@cantab.net>

* More simplification

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Remove old strings from setting

Signed-off-by: Andrew Thornton <art27@cantab.net>

* oops attachments not attachment

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2020-10-12 23:58:34 -04:00
Wim
9066d09c57
Add ssh certificate support (#12281)
* Add ssh certificate support

* Add ssh certificate support to builtin ssh

* Write trusted-user-ca-keys.pem based on configuration

* Update app.example.ini

* Update templates/user/settings/keys_principal.tmpl

Co-authored-by: silverwind <me@silverwind.io>

* Remove unused locale string

* Update options/locale/locale_en-US.ini

Co-authored-by: silverwind <me@silverwind.io>

* Update options/locale/locale_en-US.ini

Co-authored-by: silverwind <me@silverwind.io>

* Update models/ssh_key.go

Co-authored-by: silverwind <me@silverwind.io>

* Add missing creation of SSH.Rootpath

* Update cheatsheet, example and locale strings

* Update models/ssh_key.go

Co-authored-by: zeripath <art27@cantab.net>

* Update models/ssh_key.go

Co-authored-by: zeripath <art27@cantab.net>

* Update models/ssh_key.go

Co-authored-by: zeripath <art27@cantab.net>

* Update models/ssh_key.go

Co-authored-by: zeripath <art27@cantab.net>

* Update models/ssh_key.go

* Optimizations based on feedback

* Validate CA keys for external sshd

* Add filename option and change default filename

Add a SSH_TRUSTED_USER_CA_KEYS_FILENAME option which default is
RUN_USER/.ssh/gitea-trusted-user-ca-keys.pem

Do not write a file when SSH_TRUSTED_USER_CA_KEYS is empty.

Add some more documentation.

* Remove unneeded principalkey functions

* Add blank line

* Apply suggestions from code review

Co-authored-by: zeripath <art27@cantab.net>

* Add SSH_AUTHORIZED_PRINCIPALS_ALLOW option

This adds a SSH_AUTHORIZED_PRINCIPALS_ALLOW which is default
email,username this means that users only can add the principals
that match their email or username.

To allow anything the admin need to set the option anything.

This allows for a safe default in gitea which protects against malicious
users using other user's prinicipals. (before that user could set it).

This commit also has some small other fixes from the last code review.

* Rewrite principal keys file on user deletion

* Use correct rewrite method

* Set correct AuthorizedPrincipalsBackup default setting

* Rewrite principalsfile when adding principals

* Add update authorized_principals option to admin dashboard

* Handle non-primary emails

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Add the command actually to the dashboard template

* Update models/ssh_key.go

Co-authored-by: silverwind <me@silverwind.io>

* By default do not show principal options unless there are CA keys set or they are explicitly set

Signed-off-by: Andrew Thornton <art27@cantab.net>

* allow settings when enabled

* Fix typos in TrustedUserCAKeys path

* Allow every CASignatureAlgorithms algorithm

As this depends on the content of TrustedUserCAKeys we should allow all
signature algorithms as admins can choose the specific algorithm on their
signing CA

* Update models/ssh_key.go

Co-authored-by: Lauris BH <lauris@nix.lv>

* Fix linting issue

Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: techknowlogick <matti@mdranta.net>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2020-10-10 20:38:09 -04:00
zeripath
1bf40ca0df
Gitea serv, hooks, manager and the like should always display Fatals (#13032)
The gitea cmd/serv.go setup function deletes the console logger to
prevent stdout interference with commands. However, the problem is that
it then calls setting.NewContext which may cause an exit with log.Fatal
- which will then not be shown.

Now, log.Fatal is so disastrous that the interference that logging
causes is wanted. However, we can avoid stdout interference by emitting
on stderr.

This PR forces the creation of a console logger on stderr listening on
Fatal for these commands. Most commands have a `--debug` option now that
will print more reasons too - however, defaulting to give the log for
Fatal seems reasonable.

Signed-off-by: Andrew Thornton <art27@cantab.net>
2020-10-07 16:44:16 -04:00
6543
818d921bbb
Refactor use TrimSuffix instead of TrimRight (#12993)
* Refactor use TrimSuffix instead of TrimRight

* TrimRight right

* has #12990
2020-10-01 06:54:34 +01:00
Lunny Xiao
3878e985b6
Add default storage configurations (#12813)
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: zeripath <art27@cantab.net>
2020-09-29 12:05:13 +03:00
John Olheiser
c6e4bc53aa
Check passwords against HaveIBeenPwned (#12716)
* Implement pwn

Signed-off-by: jolheiser <john.olheiser@gmail.com>

* Update module

Signed-off-by: jolheiser <john.olheiser@gmail.com>

* Apply suggestions mrsdizzie

Co-authored-by: mrsdizzie <info@mrsdizzie.com>

* Add link to HIBP

Signed-off-by: jolheiser <john.olheiser@gmail.com>

* Add more details to admin command

Signed-off-by: jolheiser <john.olheiser@gmail.com>

* Add context to pwn

Signed-off-by: jolheiser <john.olheiser@gmail.com>

* Consistency and making some noise ;)

Signed-off-by: jolheiser <john.olheiser@gmail.com>

Co-authored-by: mrsdizzie <info@mrsdizzie.com>
Co-authored-by: zeripath <art27@cantab.net>
2020-09-08 17:06:39 -05:00
Lunny Xiao
7a5465fc56
LFS support to be stored on minio (#12518)
* LFS support to be stored on minio

* Fix test

* Fix lint

* Fix lint

* Fix check

* Fix test

* Update documents and add migration for LFS

* Fix some bugs
2020-09-08 23:45:10 +08:00
6543
cfd8e8a489
gitea dump: include version & Check InstallLock (#12760)
* gitea dump: include version

* Check InstallLock (close #12759)

* fix test

* fix lint
2020-09-07 23:27:17 +01:00
zeripath
a722dd72db
Set setting.AppURL as GITEA_ROOT_URL environment variable during pushes (#12752)
* Set setting.AppURL as GITEA_ROOT_URL environment variable during pushes

Fix #11738

Signed-off-by: Andrew Thornton <art27@cantab.net>
2020-09-06 23:53:42 -04:00
zeripath
1b9d5074a7
Add command to recreate tables (#12407)
Provides new command: `gitea doctor recreate-table` which will recreate
db tables and copy the old data in to the new table.

This function can be used to remove the old warning of struct defaults being
out of date.

Fix #8868
Fix #3265
Fix #8894

Signed-off-by: Andrew Thornton <art27@cantab.net>
2020-09-06 22:52:01 +01:00
zeripath
82b71881d8
Ensure repoNames are lowercased in serv (#12668)
#12624 missed lowering the provided repoPath.

(Additionally make a few fixes to the way the debug flag works.)

Fix #12659
Fix #12667

Signed-off-by: Andrew Thornton <art27@cantab.net>
2020-08-31 18:12:49 +01:00
zeripath
d257485bc0
Rename models.ProtectedBranchRepoID to models.EnvRepoID and ensure EnvPusherEmail is set (#12646)
Signed-off-by: Andrew Thornton <art27@cantab.net>
2020-08-30 08:24:39 +01:00
zeripath
7ba6fea0b7
Use shellquote to unpack arguments to gitea serv (#12624)
Fix #12471

Signed-off-by: Andrew Thornton <art27@cantab.net>
2020-08-28 20:55:25 +01:00
John Olheiser
43a397ce9a
Initial support for push options (#12169)
* Initial support for push options

Signed-off-by: jolheiser <john.olheiser@gmail.com>

* Fix misspelling 🤦

Signed-off-by: jolheiser <john.olheiser@gmail.com>

* Fix formatting after conflict resolution

* defer close git repo

* According the GitLab documentation, git >= 2.10

Signed-off-by: jolheiser <john.olheiser@gmail.com>

* Words are hard. Thanks @mrsdizzie 😅

Co-authored-by: mrsdizzie <info@mrsdizzie.com>

* Only update if there are push options

Signed-off-by: jolheiser <john.olheiser@gmail.com>

Co-authored-by: mrsdizzie <info@mrsdizzie.com>
2020-08-23 17:02:35 +01:00
Lunny Xiao
62e6c9bc6c
Add a storage layer for attachments (#11387)
* Add a storage layer for attachments

* Fix some bug

* fix test

* Fix copyright head and lint

* Fix bug

* Add setting for minio and flags for migrate-storage

* Add documents

* fix lint

* Add test for minio store type on attachments

* fix test

* fix test

* Apply suggestions from code review

Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>

* Add warning when storage migrated successfully

* Fix drone

* fix test

* rebase

* Fix test

* display the error on console

* Move minio test to amd64 since minio docker don't support arm64

* refactor the codes

* add trace

* Fix test

* remove log on xorm

* Fi download bug

* Add a storage layer for attachments

* Add setting for minio and flags for migrate-storage

* fix lint

* Add test for minio store type on attachments

* Apply suggestions from code review

Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>

* Fix drone

* fix test

* Fix test

* display the error on console

* Move minio test to amd64 since minio docker don't support arm64

* refactor the codes

* add trace

* Fix test

* Add URL function to serve attachments directly from S3/Minio

* Add ability to enable/disable redirection in attachment configuration

* Fix typo

* Add a storage layer for attachments

* Add setting for minio and flags for migrate-storage

* fix lint

* Add test for minio store type on attachments

* Apply suggestions from code review

Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>

* Fix drone

* fix test

* Fix test

* display the error on console

* Move minio test to amd64 since minio docker don't support arm64

* don't change unrelated files

* Fix lint

* Fix build

* update go.mod and go.sum

* Use github.com/minio/minio-go/v6

* Remove unused function

* Upgrade minio to v7 and some other improvements

* fix lint

* Fix go mod

Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>
Co-authored-by: Tyler <tystuyfzand@gmail.com>
2020-08-18 12:23:45 +08:00
zeripath
ac3cfad23d
Make the default PID file compile-time settable (#12485)
#12391 offered to change the default PID file from /var/run/gitea.pid however in discussion it was decided that this could break users of older systems. An alternative was offered that we could make the PID file compile/link time settable.

This PR does this, and changes the name of the setting from CustomPID to simply PIDFile. It also updates the from-source docs to show how to change the compiler settings to do this.

Closes #12391

Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Florian Klink <flokli@flokli.de>
2020-08-15 21:15:27 +01:00
zeripath
74bd9691c6
Re-attempt to delete temporary upload if the file is locked by another process (#12447)
Replace all calls to os.Remove/os.RemoveAll by retrying util.Remove/util.RemoveAll and remove circular dependencies from util.

Fix #12339

Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: silverwind <me@silverwind.io>
2020-08-11 21:05:34 +01:00
Wim
4609eba2e7
Fix ipv6 parsing (#12321)
* Fix ipv6 parsing

* Update modules/setting/setting.go

Co-authored-by: 6543 <6543@obermui.de>

Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2020-07-26 16:31:28 -04:00
赵智超
12f9dd8fa9
Decrease the num_stars when deleting a repo (#11954)
* Decrease the  num_stars when deleting a repo

fix #11949

Signed-off-by: a1012112796 <1012112796@qq.com>

* Add migration

* use batch

* Apply suggestions from code review

Co-authored-by: Lauris BH <lauris@nix.lv>

* fix lint

* fix lint

* fix ci

* fix ci2

* add doctor

* duplicate code

* fix migration

* fix some nits

* add start

Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2020-07-07 15:16:34 -04:00
zeripath
c5b08f6d5a
Pause, Resume, Release&Reopen, Add and Remove Logging from command line (#11777)
* Make LogDescriptions race safe

* Add manager commands for pausing, resuming, adding and removing loggers

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Placate lint

* Ensure that file logger is run!

* Add support for smtp and conn

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Add release-and-reopen

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: Lauris BH <lauris@nix.lv>
2020-07-05 20:07:07 -04:00
zeripath
b9e281265e
Add migration to set IsArchived false if it is null (#11853)
* Add migration to set IsArchived false if it is null

Fix #11824

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Add doctor

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2020-06-11 16:18:11 -04:00
PhilippHomann
684b7a999f
Dump: add output format tar and output to stdout (#10376)
* Dump: Use mholt/archive/v3 to support tar including many compressions

Signed-off-by: Philipp Homann <homann.philipp@googlemail.com>

* Dump: Allow dump output to stdout

Signed-off-by: Philipp Homann <homann.philipp@googlemail.com>

* Dump: Fixed bug present since #6677 where SessionConfig.Provider is never "file"

Signed-off-by: Philipp Homann <homann.philipp@googlemail.com>

* Dump: never pack RepoRootPath, LFS.ContentPath and LogRootPath when they are below AppDataPath

Signed-off-by: Philipp Homann <homann.philipp@googlemail.com>

* Dump: also dump LFS (fixes #10058)

Signed-off-by: Philipp Homann <homann.philipp@googlemail.com>

* Dump: never dump CustomPath if CustomPath is a subdir of or equal to AppDataPath (fixes #10365)

Signed-off-by: Philipp Homann <homann.philipp@googlemail.com>

* Use log.Info instead of fmt.Fprintf

Signed-off-by: Philipp Homann <homann.philipp@googlemail.com>

* import ordering

* make fmt

Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: Matti R <matti@mdranta.net>
2020-06-05 16:47:39 -04:00
6543
726e1e5279
Doctor check & fix db consistency (#11111)
needed to fix issue as described in #10280

* rename check-db to check-db-version
* add check-db-consistency:
* find issues without existing repository
* find pulls without existing issues
* find tracked times without existing issues/pulls
* find labels without repository or org reference

Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2020-05-29 14:24:15 +01:00
zeripath
8730b091e3
Add debug option to hooks (#11624)
Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2020-05-28 23:04:44 -04:00
zeripath
c81ab17de9
TrimSpace when reading InternalToken from a file (#11502)
InternalTokens are fixed as alphanum strings therefore TrimSpace from these.
Also use isatty to not add a terminal newline when redirecting generate.

Fix #11498

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: Lauris BH <lauris@nix.lv>
2020-05-20 19:16:06 +03:00
zeripath
c58bc4bf80
Prevent timer leaks in Workerpool and others (#11333)
There is a potential memory leak in `Workerpool` due to the intricacies of
`time.Timer` stopping.

Whenever a `time.Timer` is `Stop`ped its channel must be cleared using a
`select` if the result of the `Stop()` is `false`.

Unfortunately in `Workerpool` these were checked the wrong way round.

However, there were a few other places that were not being checked.

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2020-05-08 16:46:05 +01:00
Kyle Evans
a5018d099d
cmd: dump: check value of skip-repository flag (#11254)
This is a boolean flag; simply checking if it's set isn't enough, we must check the value as well.

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2020-05-03 00:57:45 -03:00
Kyle Evans
1bdffefc05
cmd: dump: add an -L/--skip-log option (#11253)
Not all dumps need to include the logs, in a similar vain to not all dumps
needing to include repositories; these may be subject to different backup
mechanisms/constraints. Add a simple option to let them be excluded from the
dump to simplify workflows that need to exclude them or not collect in the
first place.

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: zeripath <art27@cantab.net>
2020-04-30 21:30:31 -04:00
zeripath
089b34858c
Return error for authorized_keys and mergebase problems (#10990)
Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2020-04-08 02:04:19 -03:00
zeripath
d26885e2bf
Mulitple Gitea Doctor improvements (#10943)
* Add `gitea doctor --list` flag to list the checks that will be run, including those by default
* Add `gitea doctor --run` to run specific checks
* Add `gitea doctor --all` to run all checks
* Add db version checker
* Add non-default recalculate merge bases check/fixer to doctor
* Add hook checker (Fix #9878) and ensure hooks are executable (Fix #6319)
* Fix authorized_keys checker - slight change of functionality here because parsing the command is fragile and we should just check if the authorized_keys file is essentially the same as what gitea would produce. (This is still not perfect as order matters - we should probably just md5sum the two files.)
* Add SCRIPT_TYPE check (Fix #10977)
* Add `gitea doctor --fix` to attempt to fix what is possible to easily fix
* Add `gitea doctor --log-file` to set the log-file, be it a file, stdout or to switch off completely. (Fixes previously undetected bug with certain xorm logging configurations - see @6543 comment.)

Signed-off-by: Andrew Thornton <art27@cantab.net>
2020-04-06 11:44:47 +01:00
zeripath
1737fca220
make gitea admin auth list formatting configurable (#10844)
* make admin auth list formatting configurable

Signed-off-by: Andrew Thornton <art27@cantab.net>

* As per @guillep2k

Signed-off-by: Andrew Thornton <art27@cantab.net>
2020-03-27 22:26:43 +01:00
zeripath
9269b7f627
Multiple LFS improvements (#10667)
* Add more logging in the LFS server

Adds more logging in the LFS server and stops sending internal server
error information to the client

* Add LFS Lock cursor implementation

* Simplify Claims in LFS and remove the float64 casts

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: Lauris BH <lauris@nix.lv>
2020-03-09 19:56:18 +00:00
Lauris BH
37c3db7be6
Add restricted user filter to LDAP authentication (#10600)
* Add restricted user filter to LDAP authentification

* Fix unit test cases
2020-03-05 08:30:33 +02:00
John Olheiser
95013fde60
Fix push-create SSH bugs (#10145)
* Attempt to fix push-create SSH bugs

Signed-off-by: jolheiser <john.olheiser@gmail.com>

* Fix binding

Signed-off-by: jolheiser <john.olheiser@gmail.com>

* Invalid ctx

Signed-off-by: jolheiser <john.olheiser@gmail.com>
2020-02-05 10:40:35 +01:00
guillep2k
bcb52aef09
Implement "embedded" command to extract static resources (#9982)
* draft

* Implement extract command

* Fix nits and force args on extract

* Add !bindata stub, support Windows, fmt

* fix vendored flag

* Remove leading slash for matching

* Add docs

* Fix typos

* Add embedded view command

Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2020-02-02 10:17:44 +08:00
guillep2k
04cbdf5c08
Implement basic app.ini and path checks to doctor cmd (#10064)
* Add doctor check of app.ini paths

* Make /custom dir not mandatory

* Fix message and improve interface

* Update cmd/doctor.go

Co-Authored-By: John Olheiser <42128690+jolheiser@users.noreply.github.com>

* Apaise lint

* Isn't the linter a sweet? (1)

* Isn't the linter a sweet? (2)

* Isn't the linter a sweet?? (3)

* Restart CI

Co-authored-by: John Olheiser <42128690+jolheiser@users.noreply.github.com>
Co-authored-by: zeripath <art27@cantab.net>
2020-01-30 10:00:27 +08:00
zeripath
c01221e70f
Queue: Make WorkerPools and Queues flushable (#10001)
* Make WorkerPools and Queues flushable

Adds Flush methods to Queues and the WorkerPool
Further abstracts the WorkerPool
Adds a final step to Flush the queues in the defer from PrintCurrentTest
Fixes an issue with Settings inheritance in queues

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Change to for loop

* Add IsEmpty and begin just making the queues composed WorkerPools

* subsume workerpool into the queues and create a flushable interface

* Add manager command

* Move flushall to queue.Manager and add to testlogger

* As per @guillep2k

* as per @guillep2k

* Just make queues all implement flushable and clean up the wrapped queue flushes

* cope with no timeout

Co-authored-by: Lauris BH <lauris@nix.lv>
2020-01-28 20:01:06 -05:00
SpaWn2KiLl
1f01f53c53 API add/generalize pagination (#9452)
* paginate results

* fixed deadlock

* prevented breaking change

* updated swagger

* go fmt

* fixed find topic

* go mod tidy

* go mod vendor with go1.13.5

* fixed repo find topics

* fixed unit test

* added Limit method to Engine struct; use engine variable when provided; fixed gitignore

* use ItemsPerPage for default pagesize; fix GetWatchers, getOrgUsersByOrgID and GetStargazers; fix GetAllCommits headers; reverted some changed behaviors

* set Page value on Home route

* improved memory allocations

* fixed response headers

* removed logfiles

* fixed import order

* import order

* improved swagger

* added function to get models.ListOptions from context

* removed pagesize diff on unit test

* fixed imports

* removed unnecessary struct field

* fixed go fmt

* scoped PR

* code improvements

* code improvements

* go mod tidy

* fixed import order

* fixed commit statuses session

* fixed files headers

* fixed headers; added pagination for notifications

* go mod tidy

* go fmt

* removed Private from user search options; added setting.UI.IssuePagingNum as default valeu on repo's issues list

* Apply suggestions from code review

Co-Authored-By: 6543 <6543@obermui.de>
Co-Authored-By: zeripath <art27@cantab.net>

* fixed build error

* CI.restart()

* fixed merge conflicts resolve

* fixed conflicts resolve

* improved FindTrackedTimesOptions.ToOptions() method

* added backwards compatibility on ListReleases request; fixed issue tracked time ToSession

* fixed build error; fixed swagger template

* fixed swagger template

* fixed ListReleases backwards compatibility

* added page to user search route

Co-authored-by: techknowlogick <matti@mdranta.net>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: zeripath <art27@cantab.net>
2020-01-24 14:00:29 -05:00
Lunny Xiao
d92781bf94 Refactor repository check and sync functions (#9854)
Move more general repository functions out of models/repo.go
2020-01-20 20:01:19 +00:00
zeripath
c5014a7f2c Add option to prevent LDAP from deactivating everything on empty search (#9879)
* Add option to prevent LDAP from deactivating everything on empty search

* Update options/locale/locale_en-US.ini

Co-Authored-By: guillep2k <18600385+guillep2k@users.noreply.github.com>

Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2020-01-20 11:47:39 +08:00
Lunny Xiao
9f40bb020e fix dump non-exist log directory (#9818) 2020-01-16 21:56:51 -05:00
zeripath
65baacf227
Make hook status printing configurable with delay (#9641)
* Delay printing hook statuses until after 1 second

* Move to a 5s delay, wrapped writer structure and add config

* Update cmd/hook.go

* Apply suggestions from code review

* Update cmd/hook.go

Co-authored-by: Antoine GIRARD <sapk@users.noreply.github.com>
2020-01-12 08:46:03 +00:00
Lunny Xiao
f69f5a9f10 Add a new command doctor to check if some wrong configurations on gitea instance (#9095)
* add doctor

* Add a new command doctor to check if some wrong configurations on gitea instance

* fix import

* use regex match authorized_keys on doctor

* Add documentation
2020-01-11 15:24:57 +01:00
Shashvat Kedia
1e9b3d4743 Fix #9530: admin auth list make readable and admin auth delete bug fix (#9628) 2020-01-07 23:41:16 +00:00
zeripath
4acca9d2e8
Properly enforce gitea environment for pushes (#9501)
#8982 attempted to enforce the gitea environment for pushes - unfortunately it tested the settings before they were actually read in - and therefore does not do that!
2019-12-27 21:15:04 +00:00
zeripath
154424623a
Remove unnecessary loading of settings in update hook (#9496)
This PR simply makes update an empty command rather than needlessly load the settings for each reference.
2019-12-27 15:21:33 +00:00
zeripath
7bfb83e064 Batch hook pre- and post-receive calls (#8602)
* make notifyWatchers work on multiple actions

* more efficient multiple notifyWatchers

* Make CommitRepoAction take advantage of multiple actions

* Batch post and pre-receive results

* Set batch to 30

* Auto adjust timeout & add logging

* adjust processing message

* Add some messages to pre-receive

* Make any non-200 status code from pre-receive an error

* Add missing hookPrintResults

* Remove shortcut for single action

* mistaken merge fix

* oops

* Move master branch to the front

* If repo was empty and the master branch is pushed ensure that that is set as the default branch

* fixup

* fixup

* Missed HookOptions in setdefaultbranch

* Batch PushUpdateAddTag and PushUpdateDelTag

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2019-12-26 13:29:45 +02:00
zeripath
3e166bd055 Add debug option to serv to help debug problems (#9492)
* Add debug option to serv to help debug problems

* fixup! Add debug option to serv to help debug problems
2019-12-25 23:44:57 +08:00
Rychu
2b0e4e0cb8 Fixed errors logging in dump.go (#9218)
* Fixed errors logging in dump.go

* Replaced overlooked log.Fatal() with fatal()

* Run make fmt on dump.go

* Added missing new line to the fatal log

* Run make fmt on dump.go
2019-12-17 18:12:10 +02:00
zeripath
d1a49977b0 AuthorizedKeysCommand should not query db directly (#9371)
* AuthorizedKeysCommand should not query db directly

* Update routers/private/internal.go

* Fix import order
2019-12-16 20:49:07 -05:00
zeripath
e3c3b33ea7
Graceful: Xorm, RepoIndexer, Cron and Others (#9282)
* Change graceful to use a singleton obtained through GetManager instead of a global.
* Graceful: Make TestPullRequests shutdownable
* Graceful: Make the cron tasks graceful
* Graceful: AddTestPullRequest run in graceful ctx
* Graceful: SyncMirrors shutdown
* Graceful: SetDefaultContext for Xorm to be HammerContext
* Avoid starting graceful for migrate commands and checkout
* Graceful: DeliverHooks now can be shutdown
* Fix multiple syncing errors in modules/sync/UniqueQueue &  Make UniqueQueue closable
* Begin the process of making the repo indexer shutdown gracefully
2019-12-15 09:51:28 +00:00
Lunny Xiao
7b2ff82d20 Move some repository methods from models to modules/repository (#9353)
* Move some repository methods from models to modules/repository

* fix test
2019-12-14 18:30:01 +01:00
zeripath
2c83dac5d4 FCGI: Allow FCGI over unix sockets (#9298)
* FCGI: Allow FCGI over unix sockets

* fixup! FCGI: Allow FCGI over unix sockets
2019-12-10 13:23:26 +01:00
zeripath
60c5339042 Graceful: Cancel Process on monitor pages & HammerTime (#9213)
* Graceful: Create callbacks to with contexts

* Graceful: Say when Gitea is completely finished

* Graceful: Git and Process within HammerTime

Force all git commands to terminate at HammerTime
Force all process commands to terminate at HammerTime
Move almost all git processes to run as git Commands

* Graceful: Always Hammer after Shutdown

* ProcessManager: Add cancel functionality

* Fix tests

* Make sure that process.Manager.Kill() cancels

* Make threadsafe access to Processes and remove own unused Kill

* Remove cmd from the process manager as it is no longer used

* the default context is the correct context

* get rid of double till
2019-11-30 16:40:22 +02:00
zeripath
7523314ef8
Graceful: Allow graceful restart for fcgi (#9112)
* Graceful: Allow graceful restart for fcgi

My previous interpretation was incorrect - we do not handle sockets
being passed in over stdin

* Update web.go
2019-11-24 10:39:50 +00:00
zeripath
e3f22ad2cc Graceful: Allow graceful restart for unix sockets (#9113)
Previously we could not handle graceful restarts for http over unix
sockets. These can now be handled.
2019-11-24 03:11:24 +01:00
zeripath
cbaa1de9ec Add Graceful shutdown for Windows and hooks for shutdown of goroutines (#8964)
* Graceful Shutdown for windows and others

Restructures modules/graceful, adding shutdown for windows, removing and
replacing the old minwinsvc code.

Creates a new waitGroup - terminate which allows for goroutines to
finish up after the shutdown of the servers.

Shutdown and terminate hooks are added for goroutines.

* Remove unused functions - these can be added in a different PR

* Add startup timeout functionality

* Document STARTUP_TIMEOUT
2019-11-21 13:32:02 -05:00
zeripath
dd1beee2ef
Enforce Gitea environment for pushes (#8982)
* Enforce Gitea environment for pushes

* Update custom/conf/app.ini.sample

Co-Authored-By: Antoine GIRARD <sapk@users.noreply.github.com>
2019-11-14 22:39:48 +00:00
zeripath
722a7c902d
Add Close() method to gogitRepository (#8901)
In investigating #7947 it has become clear that the storage component of go-git repositories needs closing.

This PR adds this Close function and adds the Close functions as necessary.

In TransferOwnership the ctx.Repo.GitRepo is closed if it is open to help prevent the risk of multiple open files.

Fixes #7947
2019-11-13 07:01:19 +00:00
zeripath
b48cad6a2a
Adjust the must-change-password help (#8755) 2019-10-30 18:28:14 +00:00
zeripath
f067e12859
Graceful fixes (#8645)
* Only attempt to kill parent once

* Apply suggestions from code review

Co-Authored-By: guillep2k <18600385+guillep2k@users.noreply.github.com>

* Add waitgroup for running servers
2019-10-23 16:32:19 +01:00
zeripath
0bfe5eb10b
Allow Protected Branches to Whitelist Deploy Keys (#8483)
Add an option to protected branches to add writing deploy keys to the whitelist for pushing.

Please note this is technically a breaking change: previously if the owner of a repository was on the whitelist then any writing deploy key was effectively on the whitelist. This option will now need to be set if that is desired.

Closes #8472 

Details:
* Allow Protected Branches to Whitelist Deploy Keys
* Add migration
* Ensure that IsDeployKey is set to false on the http pushes
* add not null default false
2019-10-21 09:21:45 +01:00
zeripath
167e8f18da
Restore Graceful Restarting & Socket Activation (#7274)
* Prevent deadlock in indexer initialisation during graceful restart

* Move from gracehttp to our own service to add graceful ssh

* Add timeout for start of indexers and make hammer time configurable

* Fix issue with re-initialization in indexer during tests

* move the code to detect use of closed to graceful

* Handle logs gracefully - add a pid suffix just before restart

* Move to using a cond and a holder for indexers

* use time.Since

* Add some comments and attribution

* update modules.txt

* Use zero to disable timeout

* Move RestartProcess to its own file

* Add cleanup routine
2019-10-15 14:39:51 +01:00
Maxim Tkachenko
db657192d0 Password Complexity Checks (#6230)
Add password complexity checks. The default settings require a lowercase, uppercase, number and a special character within passwords.

Co-Authored-By: T-M-A <maxim.tkachenko@gmail.com>
Co-Authored-By: Lanre Adelowo <adelowomailbox@gmail.com>
Co-Authored-By: guillep2k <18600385+guillep2k@users.noreply.github.com>
Co-Authored-By: Lauris BH <lauris@nix.lv>
2019-10-14 16:24:26 +01:00
Lunny Xiao
f83db078f0 Move database settings from models to setting (#7806)
* move database settings from models to setting

* update docs

* fix checkout pr

* fix tests

* fix lint

* remove unsupported tidb options

* correct wrong variable name

* remove tidb totally
2019-08-24 11:24:45 +02:00
Tamal Saha
171b359877 Use gitea forked macaron (#7933)
Signed-off-by: Tamal Saha <tamal@appscode.com>
2019-08-23 12:40:29 -04:00
zeripath
def84840db Attempt to fix hook problem (#7854) 2019-08-14 12:25:05 +03:00
clavinet
8101d803e6 Update serv.go (#7822)
small semantics fix
"Hi there, user!" looks better than "Hi there: user!"
2019-08-11 20:15:58 +08:00