Commit graph

69 commits

Author SHA1 Message Date
Abhishek kumar
76517d101c
feat: add --skip-fixtures option to bench migrate command (#33754) 2025-08-22 16:51:17 +05:30
Ankush Menat
54c12b594d
feat: Show progress of long migration queries (#33660)
Often large databases take several minutes to alter a table, this gives
zero feedback right now and makes it hard to guess where the problem
is if things eventually fail. Was the query slow? Was it making adequate
progress? after how much time it failed?

All of these questions will now be easier to answer. This PR spawns a
new thread everytime we migrate database and...
1. Every 10 seconds check if any command has been running for >10
   seconds from current DB connection.
2. Print details about ongoing command like state, progress, time take
   and query.

The progress detail is often misleading but it's better than nothing.
2025-08-17 07:56:29 +00:00
Sagar Vora
627516711d fix: remove explicit call to add migration_hash column 2025-06-26 14:30:40 +05:30
Ankush Menat
7110e79926
fix: Avoid killing any system threads (#31442) 2025-02-26 16:07:46 +00:00
Ankush Menat
e3ce05b57e
perf: use client cache for notifications (#31094) 2025-02-04 09:24:30 +00:00
Ankush Menat
4805f1d7c4 fix: force kill all other DB connections before migrating 2025-01-10 14:05:29 +05:30
Ankush Menat
e67bbf8bb8 fix: Avoid indefinitely waiting for metadata locks 2025-01-09 15:09:33 +05:30
David
d87b786687
chore: fix all invocation of init for singledispatch -> min 1 positional arg 2024-09-13 11:53:52 +02:00
Rushabh Mehta
774d86f642 feat: Workspace Settings - allow enabling and disabling of workspaces during setup 2024-08-04 13:41:08 +05:30
barredterra
e041603aca feat: increase verbosity of post_schema_updates 2024-05-24 15:48:00 +02:00
Ankush Menat
4350f94616
refactor: add_column (#24867)
- Skip cache which can be invalid
- Do check at db level
2024-02-12 07:45:18 +00:00
Hussain Nagaria
8d2137c265 docs: consistent doc strings 2023-12-18 18:27:39 +05:30
Ankush Menat
c354b31b1f
fix(migrate): raise exception even if db is not available (#22922)
* fix(migrate): raise exception even if db is not available

* fix: wrap correctly
2023-10-30 12:51:22 +00:00
Ankush Menat
c189b7e622 revert: syncing of report and pages 2023-09-15 16:22:06 +05:30
Ankush Menat
b9ee6827b4 refactor: stale object removal
- Do it after migration, post migration might use some old doctype.
- stale -> orpahn
- dont commit changes individually
- Remove table doctype too
2023-09-15 16:18:25 +05:30
14987
b9449a4f13 fix: Delete stale doctypes, reports and pages 2023-09-15 16:18:25 +05:30
Ankush Menat
8107781f1c
fix: move filelock imports to functions (#19300) 2022-12-15 15:21:31 +05:30
Ankush Menat
49437f53b4
feat: operation level locking for CLI commands (#19162)
This prevents mistakenly issuing same commands twice which can be
dangerous.

added global lock(s):
- [x] bench build

added site level lock(s):
- [x] bench new-site sitename
- [x] bench --site sitename migrate
- [x] bench install-app appname
- [x] bench build
- [x] bench restore (the code is just meh, needs some cleanup)

closes https://github.com/frappe/frappe/issues/13215
2022-12-08 13:24:43 +05:30
Shadrak Gurupnor
9c0e111383
fix: handle exceptions thrown in post_schema_updates (#18648)
* fix: handle exceptions thrown in post_schema_updates

* chore: removed nested try-finally block
2022-10-28 22:37:58 +05:30
Ankush Menat
98b57f6a1a fix: ensure deferred insert are flushed during update 2022-09-09 17:34:46 +05:30
Ankush Menat
fbee80f734
perf: rebuild website search index in background (#17974)
* perf: rebuild website search index in background

* refactor: allow enqueueing jobs during migrate

This was added a long time ago to handle missing redis during migrate.
It is not the case right now as redis HAS to be availabe during
migration.

ref: https://github.com/frappe/frappe/pull/2988

* ci: pass correct build type

* chore: warn about redis unavailability
2022-08-26 17:41:07 +05:30
Gavin D'souza
91e0cac029 fix: Check for required services running after frappe init
Each site on a single app server can run on separate DBMS' on separate
servers. This site specific config resides in each site's
site_config.json file. Thereby, we need to load site's config before
checking for service's availability.
2022-04-20 11:25:38 +05:30
Suraj Shetty
c0c5b2ebdd
style: format all python files using black (#16453)
Co-authored-by: Frappe Bot <developers@frappe.io>
2022-04-12 10:59:25 +05:30
Gavin D'souza
891d1fbd86 refactor: Site Migration 2022-02-02 17:44:49 +05:30
Ankush Menat
fd227d38f4
feat: post model-sync patches (#15351)
Ability to run a few patches after the doctype model schema is synced.

Read module-level docstring of patch_handler.py for more info.
2022-01-26 11:21:30 +05:30
abhishek
69d8eddf5a refactor: remove unused args
refactor: remove unused args
2021-10-11 17:23:17 +05:30
abhishek
278883edb0 fix: missing migration hash column 2021-10-11 17:23:17 +05:30
Gavin D'souza
db6fc6a5fb feat: Hash based comparison migration 2021-10-11 17:23:17 +05:30
Gavin D'souza
3446026555 chore: Update header: license.txt => LICENSE
The license.txt file has been replaced with LICENSE for quite a while
now. INAL but it didn't seem accurate to say "hey, checkout license.txt
although there's no such file". Apart from this, there were
inconsistencies in the headers altogether...this change brings
consistency.
2021-09-03 12:02:59 +05:30
Suraj Shetty
2d8c6c1710 Merge branch 'develop' of https://github.com/frappe/frappe into refactor-website 2021-06-03 11:43:28 +05:30
Gavin D'souza
e407b78506 chore: Drop dead and deprecated code
* Remove six for PY2 compatability since our dependencies are not, PY2
  is legacy.
* Removed usages of utils from future/past libraries since they are
  deprecated. This includes 'from __future__ ...' and 'from past...'
  statements.
* Removed compatibility imports for PY2, switched from six imports to
  standard library imports.
* Removed utils code blocks that handle operations depending on PY2/3
  versions.
* Removed 'from __future__ ...' lines from templates/code generators
* Used PY3 syntaxes in place of PY2 compatible blocks. eg: metaclass
2021-05-26 15:31:29 +05:30
Suraj Shetty
e6d701b038 refactor: Use clear_website_cache instead of clear_cache 2021-05-25 15:02:18 +05:30
Suraj Shetty
86897c1808 refactor: Remove render.py and move all utility functions to utils.py
- Replace or remove all render imports
2021-05-25 10:53:35 +05:30
Gavin D'souza
ab34d1d9e7 fix: Remove dead option "rebuild-website" in migrate 2020-09-29 21:13:41 +05:30
Gavin D'souza
2d96a2c4cc fix: Docstring latest=>current 2020-09-09 13:42:30 +05:30
Shivam Mishra
68e4509792 chore: update comment 2020-08-06 19:47:46 +05:30
Shivam Mishra
6f0c4a4bf1 fix: import for build_index_for_all_routes 2020-08-06 19:47:46 +05:30
Shivam Mishra
20f06f0d24 chore: revert "feat: make search indexing optional in migrate"
This reverts commit b5fe9d4f7be8dae3e3e946b4327bf44b9a35448c.
2020-08-06 19:47:46 +05:30
Shivam Mishra
8b7ae16cdd feat: make search indexing optional in migrate 2020-08-06 19:47:46 +05:30
Shivam Mishra
b2100279d6 feat: update code for new search 2020-08-06 19:47:46 +05:30
Shivam Mishra
5145c58b81 feat: [breaking] remove global search from migrate
* Run search last as it changes the session user
2020-08-06 19:47:46 +05:30
Shivam Mishra
19f42d1877 feat: added index all routes function 2020-08-06 19:47:46 +05:30
Shivam Mishra
51b8e467ad feat: add flag to disable search index rebuild on migrate 2020-08-06 19:47:46 +05:30
Shivam Mishra
d2d37a1e04 feat: move whoosh full_text_search build to migrate from hooks 2020-08-06 19:47:46 +05:30
mergify[bot]
b4e5df62d3
Merge pull request #10293 from gavindsouza/installed-apps
feat: single doctype to store installed apps data
2020-05-14 23:55:16 +00:00
Gavin D'souza
2473b1ccd7 fix: explicitly called method instead of updating implicitly 2020-05-12 10:20:10 +05:30
Gavin D'souza
ed74f52e20 feat: single doctype to store installed apps data
updated on migrate
versions enabled
2020-05-11 20:17:40 +05:30
Chinmay Pai
9cdcf56288
Merge branch 'develop' into bench-ux-fix 2020-05-10 15:43:56 +05:30
Shivam Mishra
db152b28de feat: sync dashboards after setup and migrate 2020-05-06 14:45:07 +05:30
Chinmay D. Pai
a25088a606
chore: show message on migrate when services are not running
shows message before migrate if the database or redis service is not running
instead of trying to migrate and then failing.

Signed-off-by: Chinmay D. Pai <chinmaydpai@gmail.com>
2020-04-13 19:10:19 +05:30