gitea/integrations
zeripath bb0ff77e46
Share HTML template renderers and create a watcher framework (#20218)
The recovery, API, Web and package frameworks all create their own HTML
Renderers. This increases the memory requirements of Gitea
unnecessarily with duplicate templates being kept in memory.

Further the reloading framework in dev mode for these involves locking
and recompiling all of the templates on each load. This will potentially
hide concurrency issues and it is inefficient.

This PR stores the templates renderer in the context and stores this
context in the NormalRoutes, it then creates a fsnotify.Watcher
framework to watch files.

The watching framework is then extended to the mailer templates which
were previously not being reloaded in dev.

Then the locales are simplified to a similar structure.

Fix #20210 
Fix #20211
Fix #20217

Signed-off-by: Andrew Thornton <art27@cantab.net>
2022-08-28 10:43:25 +01:00
..
gitea-repositories-meta rm .sample hooks which aren't used (#19101) 2022-03-16 10:33:07 +00:00
migration-test Refactor legacy git init (#20376) 2022-08-09 11:22:24 +08:00
admin_user_test.go Refactor AssertExistsAndLoadBean to use generics (#20797) 2022-08-16 10:22:25 +08:00
api_activitypub_person_test.go Share HTML template renderers and create a watcher framework (#20218) 2022-08-28 10:43:25 +01:00
api_admin_org_test.go format with gofumpt (#18184) 2022-01-20 18:46:10 +01:00
api_admin_test.go Refactor AssertExistsAndLoadBean to use generics (#20797) 2022-08-16 10:22:25 +08:00
api_branch_test.go Update HTTP status codes to modern codes (#18063) 2022-03-23 12:54:07 +08:00
api_comment_test.go Refactor AssertExistsAndLoadBean to use generics (#20797) 2022-08-16 10:22:25 +08:00
api_fork_test.go Fix "data race" in testlogger (#9159) 2019-11-26 01:21:37 +02:00
api_gpg_keys_test.go format with gofumpt (#18184) 2022-01-20 18:46:10 +01:00
api_helper_for_declarative_test.go Auto merge pull requests when all checks succeeded via API (#9307) 2022-05-08 01:05:52 +08:00
api_httpsig_test.go Implement http signatures support for the API (#17565) 2022-06-05 08:16:14 +01:00
api_issue_label_test.go Refactor AssertExistsAndLoadBean to use generics (#20797) 2022-08-16 10:22:25 +08:00
api_issue_milestone_test.go Refactor AssertExistsAndLoadBean to use generics (#20797) 2022-08-16 10:22:25 +08:00
api_issue_reaction_test.go Refactor AssertExistsAndLoadBean to use generics (#20797) 2022-08-16 10:22:25 +08:00
api_issue_stopwatch_test.go Refactor AssertExistsAndLoadBean to use generics (#20797) 2022-08-16 10:22:25 +08:00
api_issue_subscription_test.go Refactor AssertExistsAndLoadBean to use generics (#20797) 2022-08-16 10:22:25 +08:00
api_issue_test.go Refactor AssertExistsAndLoadBean to use generics (#20797) 2022-08-16 10:22:25 +08:00
api_issue_tracked_time_test.go Refactor AssertExistsAndLoadBean to use generics (#20797) 2022-08-16 10:22:25 +08:00
api_keys_test.go Refactor AssertExistsAndLoadBean to use generics (#20797) 2022-08-16 10:22:25 +08:00
api_nodeinfo_test.go Share HTML template renderers and create a watcher framework (#20218) 2022-08-28 10:43:25 +01:00
api_notification_test.go Move some files into models' sub packages (#20262) 2022-08-25 10:31:57 +08:00
api_oauth2_apps_test.go Refactor AssertExistsAndLoadBean to use generics (#20797) 2022-08-16 10:22:25 +08:00
api_org_test.go Remove dependent on session auth for api/v1 routers (#19321) 2022-04-08 06:22:10 +02:00
api_packages_composer_test.go Refactor AssertExistsAndLoadBean to use generics (#20797) 2022-08-16 10:22:25 +08:00
api_packages_conan_test.go Refactor AssertExistsAndLoadBean to use generics (#20797) 2022-08-16 10:22:25 +08:00
api_packages_container_test.go Refactor AssertExistsAndLoadBean to use generics (#20797) 2022-08-16 10:22:25 +08:00
api_packages_generic_test.go Refactor AssertExistsAndLoadBean to use generics (#20797) 2022-08-16 10:22:25 +08:00
api_packages_helm_test.go Refactor AssertExistsAndLoadBean to use generics (#20797) 2022-08-16 10:22:25 +08:00
api_packages_maven_test.go Refactor AssertExistsAndLoadBean to use generics (#20797) 2022-08-16 10:22:25 +08:00
api_packages_npm_test.go Refactor AssertExistsAndLoadBean to use generics (#20797) 2022-08-16 10:22:25 +08:00
api_packages_nuget_test.go Refactor AssertExistsAndLoadBean to use generics (#20797) 2022-08-16 10:22:25 +08:00
api_packages_pub_test.go Refactor AssertExistsAndLoadBean to use generics (#20797) 2022-08-16 10:22:25 +08:00
api_packages_pypi_test.go Refactor AssertExistsAndLoadBean to use generics (#20797) 2022-08-16 10:22:25 +08:00
api_packages_rubygems_test.go Refactor AssertExistsAndLoadBean to use generics (#20797) 2022-08-16 10:22:25 +08:00
api_packages_test.go Refactor AssertExistsAndLoadBean to use generics (#20797) 2022-08-16 10:22:25 +08:00
api_private_serv_test.go Fix the bug: deploy key with write access can not push (#19010) 2022-03-22 17:29:07 +08:00
api_pull_commits_test.go Refactor AssertExistsAndLoadBean to use generics (#20797) 2022-08-16 10:22:25 +08:00
api_pull_review_test.go Refactor AssertExistsAndLoadBean to use generics (#20797) 2022-08-16 10:22:25 +08:00
api_pull_test.go Refactor AssertExistsAndLoadBean to use generics (#20797) 2022-08-16 10:22:25 +08:00
api_releases_test.go Move some files into models' sub packages (#20262) 2022-08-25 10:31:57 +08:00
api_repo_archive_test.go Refactor AssertExistsAndLoadBean to use generics (#20797) 2022-08-16 10:22:25 +08:00
api_repo_collaborator_test.go Refactor AssertExistsAndLoadBean to use generics (#20797) 2022-08-16 10:22:25 +08:00
api_repo_edit_test.go Refactor AssertExistsAndLoadBean to use generics (#20797) 2022-08-16 10:22:25 +08:00
api_repo_file_create_test.go Refactor AssertExistsAndLoadBean to use generics (#20797) 2022-08-16 10:22:25 +08:00
api_repo_file_delete_test.go Refactor AssertExistsAndLoadBean to use generics (#20797) 2022-08-16 10:22:25 +08:00
api_repo_file_get_test.go Add API to serve blob or LFS file content (#19689) 2022-06-04 21:17:53 +08:00
api_repo_file_helpers.go Propagate context and ensure git commands run in request context (#17868) 2022-01-19 23:26:57 +00:00
api_repo_file_update_test.go Refactor AssertExistsAndLoadBean to use generics (#20797) 2022-08-16 10:22:25 +08:00
api_repo_get_contents_list_test.go Refactor AssertExistsAndLoadBean to use generics (#20797) 2022-08-16 10:22:25 +08:00
api_repo_get_contents_test.go Refactor AssertExistsAndLoadBean to use generics (#20797) 2022-08-16 10:22:25 +08:00
api_repo_git_blobs_test.go Refactor AssertExistsAndLoadBean to use generics (#20797) 2022-08-16 10:22:25 +08:00
api_repo_git_commits_test.go Refactor AssertExistsAndLoadBean to use generics (#20797) 2022-08-16 10:22:25 +08:00
api_repo_git_hook_test.go Refactor AssertExistsAndLoadBean to use generics (#20797) 2022-08-16 10:22:25 +08:00
api_repo_git_notes_test.go Refactor AssertExistsAndLoadBean to use generics (#20797) 2022-08-16 10:22:25 +08:00
api_repo_git_ref_test.go Refactor AssertExistsAndLoadBean to use generics (#20797) 2022-08-16 10:22:25 +08:00
api_repo_git_tags_test.go Refactor AssertExistsAndLoadBean to use generics (#20797) 2022-08-16 10:22:25 +08:00
api_repo_git_trees_test.go Refactor AssertExistsAndLoadBean to use generics (#20797) 2022-08-16 10:22:25 +08:00
api_repo_languages_test.go Update HTTP status codes to modern codes (#18063) 2022-03-23 12:54:07 +08:00
api_repo_lfs_locks_test.go Refactor AssertExistsAndLoadBean to use generics (#20797) 2022-08-16 10:22:25 +08:00
api_repo_lfs_migrate_test.go Refactor AssertExistsAndLoadBean to use generics (#20797) 2022-08-16 10:22:25 +08:00
api_repo_lfs_test.go Refactor AssertExistsAndLoadBean to use generics (#20797) 2022-08-16 10:22:25 +08:00
api_repo_raw_test.go Refactor AssertExistsAndLoadBean to use generics (#20797) 2022-08-16 10:22:25 +08:00
api_repo_tags_test.go Refactor AssertExistsAndLoadBean to use generics (#20797) 2022-08-16 10:22:25 +08:00
api_repo_teams_test.go Refactor AssertExistsAndLoadBean to use generics (#20797) 2022-08-16 10:22:25 +08:00
api_repo_test.go Refactor AssertExistsAndLoadBean to use generics (#20797) 2022-08-16 10:22:25 +08:00
api_repo_topic_test.go Refactor AssertExistsAndLoadBean to use generics (#20797) 2022-08-16 10:22:25 +08:00
api_settings_test.go Make mirror feature more configurable (#16957) 2021-09-07 17:49:36 +02:00
api_team_test.go Fix SQL Query for SearchTeam (#20844) 2022-08-21 17:24:05 +01:00
api_team_user_test.go Refactor AssertExistsAndLoadBean to use generics (#20797) 2022-08-16 10:22:25 +08:00
api_token_test.go Move some files into models' sub packages (#20262) 2022-08-25 10:31:57 +08:00
api_user_email_test.go Don't panic on ErrEmailInvalid (#19441) 2022-04-20 23:39:30 +02:00
api_user_heatmap_test.go Move some files into models' sub packages (#20262) 2022-08-25 10:31:57 +08:00
api_user_org_perm_test.go Add ContextUser to http request context (#18798) 2022-03-26 17:04:22 +08:00
api_user_orgs_test.go Fix SQL Query for SearchTeam (#20844) 2022-08-21 17:24:05 +01:00
api_user_search_test.go Refactor AssertExistsAndLoadBean to use generics (#20797) 2022-08-16 10:22:25 +08:00
api_wiki_test.go Add API to get/edit wiki (#17278) 2021-10-25 11:43:40 +08:00
attachment_test.go Update HTTP status codes to modern codes (#18063) 2022-03-23 12:54:07 +08:00
auth_ldap_test.go Refactor AssertExistsAndLoadBean to use generics (#20797) 2022-08-16 10:22:25 +08:00
benchmarks_test.go Refactor AssertExistsAndLoadBean to use generics (#20797) 2022-08-16 10:22:25 +08:00
branches_test.go Make better use of i18n (#20096) 2022-06-26 22:19:22 +08:00
change_default_branch_test.go Refactor AssertExistsAndLoadBean to use generics (#20797) 2022-08-16 10:22:25 +08:00
cmd_keys_test.go format with gofumpt (#18184) 2022-01-20 18:46:10 +01:00
compare_test.go Allow default branch to be inferred on compare page (#17908) 2021-12-06 11:04:07 -06:00
cors_test.go Fix "data race" in testlogger (#9159) 2019-11-26 01:21:37 +02:00
create_no_session_test.go Share HTML template renderers and create a watcher framework (#20218) 2022-08-28 10:43:25 +01:00
csrf_test.go Refactor AssertExistsAndLoadBean to use generics (#20797) 2022-08-16 10:22:25 +08:00
delete_user_test.go Move issues related files into models/issues (#19931) 2022-06-13 17:37:59 +08:00
download_test.go A better go code formatter, and now make fmt can run in Windows (#17684) 2021-11-17 20:34:35 +08:00
dump_restore_test.go Refactor AssertExistsAndLoadBean to use generics (#20797) 2022-08-16 10:22:25 +08:00
editor_test.go Add more linters to improve code readability (#19989) 2022-06-20 12:02:49 +02:00
empty_repo_test.go Refactor AssertExistsAndLoadBean to use generics (#20797) 2022-08-16 10:22:25 +08:00
eventsource_test.go Move some files into models' sub packages (#20262) 2022-08-25 10:31:57 +08:00
explore_repos_test.go Fix "data race" in testlogger (#9159) 2019-11-26 01:21:37 +02:00
git_clone_wiki_test.go Unexport git.GlobalCommandArgs (#18376) 2022-01-25 18:15:58 +00:00
git_helper_for_declarative_test.go Remove git.Command.Run and git.Command.RunInDir* (#19280) 2022-04-01 10:55:30 +08:00
git_smart_http_test.go Update HTTP status codes to modern codes (#18063) 2022-03-23 12:54:07 +08:00
git_test.go Refactor AssertExistsAndLoadBean to use generics (#20797) 2022-08-16 10:22:25 +08:00
goget_test.go Refactor legacy unknwon/com package, improve golangci lint (#19284) 2022-04-01 16:47:50 +08:00
gpg_git_test.go Refactor AssertExistsAndLoadBean to use generics (#20797) 2022-08-16 10:22:25 +08:00
html_helper.go Frontend refactor, PascalCase to camelCase, remove unused code (#17365) 2021-10-21 15:37:43 +08:00
integration_test.go Share HTML template renderers and create a watcher framework (#20218) 2022-08-28 10:43:25 +01:00
issue_test.go Refactor AssertExistsAndLoadBean to use generics (#20797) 2022-08-16 10:22:25 +08:00
lfs_getobject_test.go Refactor legacy git init (#20376) 2022-08-09 11:22:24 +08:00
lfs_local_endpoint_test.go format with gofumpt (#18184) 2022-01-20 18:46:10 +01:00
links_test.go Remove tab/TabName usage where it's not needed (#19973) 2022-06-15 23:05:32 +08:00
migrate_test.go Refactor AssertExistsAndLoadBean to use generics (#20797) 2022-08-16 10:22:25 +08:00
mirror_pull_test.go Move some files into models' sub packages (#20262) 2022-08-25 10:31:57 +08:00
mirror_push_test.go Move some files into models' sub packages (#20262) 2022-08-25 10:31:57 +08:00
mssql.ini.tmpl Use new config options (#20125) 2022-06-24 16:38:19 -04:00
mysql8.ini.tmpl Use new config options (#20125) 2022-06-24 16:38:19 -04:00
mysql.ini.tmpl Use new config options (#20125) 2022-06-24 16:38:19 -04:00
nonascii_branches_test.go Add more linters to improve code readability (#19989) 2022-06-20 12:02:49 +02:00
oauth_test.go Add more linters to improve code readability (#19989) 2022-06-20 12:02:49 +02:00
org_count_test.go Refactor AssertExistsAndLoadBean to use generics (#20797) 2022-08-16 10:22:25 +08:00
org_test.go Fix SQL Query for SearchTeam (#20844) 2022-08-21 17:24:05 +01:00
pgsql.ini.tmpl Use new config options (#20125) 2022-06-24 16:38:19 -04:00
private-testing.key Use non-expiring key. (#17984) 2021-12-14 15:02:20 -05:00
privateactivity_test.go Move some files into models' sub packages (#20262) 2022-08-25 10:31:57 +08:00
pull_compare_test.go Fix "data race" in testlogger (#9159) 2019-11-26 01:21:37 +02:00
pull_create_test.go Update HTTP status codes to modern codes (#18063) 2022-03-23 12:54:07 +08:00
pull_merge_test.go Move some files into models' sub packages (#20262) 2022-08-25 10:31:57 +08:00
pull_review_test.go Fix "data race" in testlogger (#9159) 2019-11-26 01:21:37 +02:00
pull_status_test.go Change commit status icons to SVG (#20736) 2022-08-09 23:55:29 +02:00
pull_update_test.go Move some files into models' sub packages (#20262) 2022-08-25 10:31:57 +08:00
README_ZH.md upgrade postgres refrence to 14 (#19416) 2022-04-16 11:59:56 -04:00
README.md Fix various typos of software names (#18083) 2021-12-24 11:56:57 +08:00
release_test.go Refactor AssertExistsAndLoadBean to use generics (#20797) 2022-08-16 10:22:25 +08:00
rename_branch_test.go Refactor AssertExistsAndLoadBean to use generics (#20797) 2022-08-16 10:22:25 +08:00
repo_activity_test.go format with gofumpt (#18184) 2022-01-20 18:46:10 +01:00
repo_branch_test.go Make better use of i18n (#20096) 2022-06-26 22:19:22 +08:00
repo_commits_search_test.go Make TestCreateBranch and TestRepoCommitsSearch less noisy (#13471) 2020-11-08 16:24:54 -05:00
repo_commits_test.go Change commit status icons to SVG (#20736) 2022-08-09 23:55:29 +02:00
repo_fork_test.go Refactor AssertExistsAndLoadBean to use generics (#20797) 2022-08-16 10:22:25 +08:00
repo_generate_test.go Refactor AssertExistsAndLoadBean to use generics (#20797) 2022-08-16 10:22:25 +08:00
repo_migrate_test.go Update HTTP status codes to modern codes (#18063) 2022-03-23 12:54:07 +08:00
repo_search_test.go Automatically pause queue if index service is unavailable (#15066) 2022-01-27 10:30:51 +02:00
repo_tag_test.go Move some files into models' sub packages (#20262) 2022-08-25 10:31:57 +08:00
repo_test.go Remove useless JS operation for relative time tooltips (#20756) 2022-08-12 17:03:41 -04:00
repo_topic_test.go Refactor CSRF protection modules, make sure CSRF tokens can be up-to-date. (#19337) 2022-04-08 13:21:05 +08:00
repo_watch_test.go Some repository refactors (#17950) 2021-12-12 23:48:20 +08:00
repofiles_delete_test.go Renamed ctx.User to ctx.Doer. (#19161) 2022-03-22 15:03:22 +08:00
repofiles_update_test.go Add latest commit's SHA to content response (#20398) 2022-07-30 16:09:04 +08:00
setting_test.go Update HTTP status codes to modern codes (#18063) 2022-03-23 12:54:07 +08:00
signin_test.go Refactor AssertExistsAndLoadBean to use generics (#20797) 2022-08-16 10:22:25 +08:00
signout_test.go Update HTTP status codes to modern codes (#18063) 2022-03-23 12:54:07 +08:00
signup_test.go Refactor AssertExistsAndLoadBean to use generics (#20797) 2022-08-16 10:22:25 +08:00
sqlite.ini.tmpl Use new config options (#20125) 2022-06-24 16:38:19 -04:00
ssh_key_test.go format with gofumpt (#18184) 2022-01-20 18:46:10 +01:00
testlogger.go Fix typo (#19889) 2022-06-04 17:43:25 -04:00
timetracking_test.go format with gofumpt (#18184) 2022-01-20 18:46:10 +01:00
user_avatar_test.go Refactor AssertExistsAndLoadBean to use generics (#20797) 2022-08-16 10:22:25 +08:00
user_test.go Refactor AssertExistsAndLoadBean to use generics (#20797) 2022-08-16 10:22:25 +08:00
version_test.go Fix "data race" in testlogger (#9159) 2019-11-26 01:21:37 +02:00
view_test.go Display SVG files as images instead of text (#14101) 2021-01-12 22:45:19 -05:00
webfinger_test.go Refactor AssertExistsAndLoadBean to use generics (#20797) 2022-08-16 10:22:25 +08:00
xss_test.go Refactor AssertExistsAndLoadBean to use generics (#20797) 2022-08-16 10:22:25 +08:00

Integrations tests

Integration tests can be run with make commands for the appropriate backends, namely:

make test-mysql
make test-pgsql
make test-sqlite

Make sure to perform a clean build before running tests:

make clean build

Run all tests via local drone

drone exec --local --build-event "pull_request"

Run sqlite integrations tests

Start tests

make test-sqlite

Run MySQL integrations tests

Setup a MySQL database inside docker

docker run -e "MYSQL_DATABASE=test" -e "MYSQL_ALLOW_EMPTY_PASSWORD=yes" -p 3306:3306 --rm --name mysql mysql:latest #(just ctrl-c to stop db and clean the container)
docker run -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" --rm --name elasticsearch elasticsearch:7.6.0 #(in a second terminal, just ctrl-c to stop db and clean the container)

Start tests based on the database container

TEST_MYSQL_HOST=localhost:3306 TEST_MYSQL_DBNAME=test TEST_MYSQL_USERNAME=root TEST_MYSQL_PASSWORD='' make test-mysql

Run pgsql integrations tests

Setup a pgsql database inside docker

docker run -e "POSTGRES_DB=test" -p 5432:5432 --rm --name pgsql postgres:latest #(just ctrl-c to stop db and clean the container)

Start tests based on the database container

TEST_PGSQL_HOST=localhost:5432 TEST_PGSQL_DBNAME=test TEST_PGSQL_USERNAME=postgres TEST_PGSQL_PASSWORD=postgres make test-pgsql

Run mssql integrations tests

Setup a mssql database inside docker

docker run -e "ACCEPT_EULA=Y" -e "MSSQL_PID=Standard" -e "SA_PASSWORD=MwantsaSecurePassword1" -p 1433:1433 --rm --name mssql microsoft/mssql-server-linux:latest #(just ctrl-c to stop db and clean the container)

Start tests based on the database container

TEST_MSSQL_HOST=localhost:1433 TEST_MSSQL_DBNAME=gitea_test TEST_MSSQL_USERNAME=sa TEST_MSSQL_PASSWORD=MwantsaSecurePassword1 make test-mssql

Running individual tests

Example command to run GPG test:

For SQLite:

make test-sqlite#GPG

For other databases(replace mssql to mysql, mysql8 or pgsql):

TEST_MSSQL_HOST=localhost:1433 TEST_MSSQL_DBNAME=test TEST_MSSQL_USERNAME=sa TEST_MSSQL_PASSWORD=MwantsaSecurePassword1 make test-mssql#GPG

Setting timeouts for declaring long-tests and long-flushes

We appreciate that some testing machines may not be very powerful and the default timeouts for declaring a slow test or a slow clean-up flush may not be appropriate.

You can either:

  • Within the test ini file set the following section:
[integration-tests]
SLOW_TEST = 10s ; 10s is the default value
SLOW_FLUSH = 5S ; 5s is the default value
  • Set the following environment variables:
GITEA_SLOW_TEST_TIME="10s" GITEA_SLOW_FLUSH_TIME="5s" make test-sqlite