vassili/seitime-frappe
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1
49231 commits 1 branch 0 tags 587 MiB
Commit graph

126 commits

Author SHA1 Message Date
Ankush Menat
5a5ad3032c
fix: procline crash (#31845) 
* fix: Skip rollback if no DB

* fix: drop proctitle from dependency

- While it improves our own processes, it makes gunicorn process titles worse
- It also crashes on mac because of some unknown reason related to fork-safety.

Best to not use this.
 2025-03-21 12:30:19 +05:30
Ankush Menat
967d3e828c fix: granular status in system health report 
differentiate between dead process, manually disabled or dormant.
 2025-02-25 10:38:25 +05:30
Ankush Menat
6db7cb096d fix: Better proxy for active site and disable on selfhosted 2025-02-25 10:37:29 +05:30
Ankush Menat
0d85bec58f fix: allow disabling dormancy behaviour 
Set it to 0 to never trigger this again.
 2025-02-25 10:37:29 +05:30
Corentin Forler
83f572d3d6 feat(scheduler)!: Make dormant sites opt-in 2025-02-25 10:37:29 +05:30
Ankush Menat
d0c3a8ee56
fix: check scheduler process status in health report (#31284) 
Currently it's just checking if the scheduler is enabled or not. This PR
also adds a check to see if the process is running or not.
 2025-02-17 05:04:07 +00:00
Gavin D'souza
c6580b5880
refactor: Replace pytz to std lib zoneinfo & datetime 
Signed-off-by: Gavin D'souza <gavin.dsouza@switchup.de>
 2024-12-06 15:43:33 +05:30
David
d87b786687
chore: fix all invocation of init for singledispatch -> min 1 positional arg 2024-09-13 11:53:52 +02:00
Akhil Narang
3efd8c6d71
chore(scheduler): never mark sites as dormant if running in developer mode 
Signed-off-by: Akhil Narang <me@akhilnarang.dev>
 2024-08-12 13:27:14 +05:30
Ankush Menat
82e6a9a081
fix: Slightly efficient scheduling (#26657) 
- Scheduler tick is 1 minute
- Fastest jobs are at least 4 minutes apart.

3/4 times we are wasting cpu cycle for each site.

Fix:
- Check every 4 minutes.
- Align the sleep around real clock to fix problem of jobs executing
  randomly in any part of the minute.

Cons:
Anything not aligned with scheduler interval will be delayed.
 2024-06-04 05:21:47 +00:00
Ankush Menat
59bbbd7b56 perf: optimize scheduling 
- Randomize scheduling order
- Cache dormant status for 1 hour (this gets checked every second, too much work)
 2024-05-27 12:21:11 +05:30
Ankush Menat
433e7281f7 feat: make running scheduler with worker optional 2024-05-27 12:14:15 +05:30
Ankush Menat
f7ff829ea7 perf: Merge worker and scheduler 2024-05-26 12:44:00 +05:30
Ankush Menat
b2ef2cd506 feat: show oldest unscheduled job in health report 2024-05-02 13:39:51 +05:30
Ankush Menat
b0aaeb5096 fix: Dont let one invalid cron fail scheduler 
Scenario:
- One bad cron job exists
- When it fails nothing after that job is enqueued.

After this fix, that failure is skipped and rest of the jobs are enqueued.
 2024-05-02 13:36:14 +05:30
Ankush Menat
071e269548 fix!: Switch to creation as default sort order 2024-03-27 11:18:28 +05:30
Ankush Menat
8bbb69a883
fix: Prevent multiple schedulers from running on same bench (#25340) 2024-03-11 14:12:31 +00:00
Akhil Narang
bf12be6928
fix: 755 -> 644 
Signed-off-by: Akhil Narang <me@akhilnarang.dev>
 2024-02-23 16:54:10 +05:30
Ankush Menat
de9ac89748 style: re-format with ruff 2024-02-05 18:53:33 +05:30
Ankush Menat
585200988b feat: simple procline for scheduler 
Makes it easy to tell if scheduler is for some reason "stuck".
 2023-12-29 14:39:40 +05:30
Hussain Nagaria
ecdd7b3e37 docs: consistency 2023-12-20 14:02:32 +05:30
Hussain Nagaria
8d2137c265 docs: consistent doc strings 2023-12-18 18:27:39 +05:30
Akhil Narang
659cbcb88f
refactor(scheduler): drop unused parameter (#23174) 
Signed-off-by: Akhil Narang <me@akhilnarang.dev>
 2023-11-11 11:29:12 +05:30
Ankush Menat
19d33a9a88
fix: fair-er scheduling for multitenant deploys (#22022) 2023-08-12 10:55:35 +05:30
Ankush Menat
ef51dde052
perf: set lower priority for background processes (#21841) 2023-07-28 13:01:46 +05:30
Sagar Vora
363a40bfb1 perf: get all Scheduled Job Types in one query 2023-07-23 15:02:50 +05:30
Ankush Menat
8a37d6d278
perf: reduce memory usage of background processes (#21467) 
* perf: defer translation.py imports

This indirectly imports babel which isn't really required most of the
time.

* perf: defer gzip import

* perf: move validate_and_sanitize_search_inputs

This causes all sorts of indirect imports and increases memory usage

* perf: defer requests module imports

* perf: defer system settings import

* perf: defer LOG_DOCTYPES import

Causes many indirect imports

* perf: defer update_site_config

* perf: defer notifications import

* perf: remove unused import

* perf: defer safe exec import

* test: memory usage overhead
 2023-06-23 12:51:45 +05:30
Ankush Menat
7ac6199213
feat: re-enable scheduler from desk (#20434) 
[skip ci]
 2023-03-23 11:54:24 +05:30
Gavin D'souza
4738a1422d fix: Add format, verbose options to scheduler 2023-02-10 14:39:16 +05:30
Gavin D'souza
6b84c9ccf5 feat: Check scheduler status via CLI 
new: `bench --site scheduler status`
 2023-02-10 14:38:50 +05:30
Ankush Menat
e00023deb1 refactor: replace bad usage of db.set_value 2023-01-04 11:13:53 +05:30
Ankush Menat
d13ab320b7 refactor: Duplicate enqueue checks 
scheduled_job_type.enqueue already does check for duplicate job and is
more "recent"
 2022-12-14 15:46:46 +05:30
Ankush Menat
2b050f9fc3 test: test without adding hacky flags 2022-12-14 15:46:46 +05:30
Ankush Menat
82d699a801 refactor: misleading "log and raise" and types 2022-12-14 15:46:46 +05:30
Ankush Menat
c3b547f897
refactor!: scheduler without external libraries (#18331) 2022-10-10 13:09:34 +05:30
Ankush Menat
0642708f01
perf: use modified instead of creation in scheduler (#18234) 
creation doesn't have index and does full table scan. On large site with
~3M activity logs it takes ~1.5 seconds just to begin scheduler.


[skip ci]
 2022-09-27 19:21:02 +05:30
Ankush Menat
46df97c3f2 feat: show scheduler status on RQ job page 2022-09-12 11:49:25 +05:30
Ankush Menat
81b37cb7d2
refactor: clean up code to py310 supported features (#17367) 
refactor: clean up code to py39+ supported syntax

- f-strings instead of format
- latest typing support instead of pre 3.9 TitleCase
- remove UTF-8 declarations.
- many more changes

Powered by https://github.com/asottile/pyupgrade/ + manual cleanups
 2022-07-01 11:51:05 +05:30
Ankush Menat
d35d7ffbe2 fix: remove bare exception catching 
A bare except catches lots of things (like generator iteration end) and should never be used.
 2022-06-28 18:05:00 +05:30
Gavin D'souza
d0efa8e3ff fix: Show site name along with debugging messages 
Without the site name, the output doesn't really make sense :')
 2022-05-10 13:33:53 +05:30
Gavin D'souza
66655eb8ed Merge branch 'develop' of github.com:frappe/frappe into bg-rename_doc 2022-04-14 13:41:56 +05:30
Suraj Shetty
c0c5b2ebdd
style: format all python files using black (#16453) 
Co-authored-by: Frappe Bot <developers@frappe.io>
 2022-04-12 10:59:25 +05:30
Gavin D'souza
311a42ae52 fix: Print to STDOUT to help debug via is_scheduler_inactive 2022-02-28 19:10:50 +05:30
Rushabh Mehta
853287f49d fix: load user_info on-demand 2022-01-13 16:39:24 +05:30
Gavin D'souza
3446026555 chore: Update header: license.txt => LICENSE 
The license.txt file has been replaced with LICENSE for quite a while
now. INAL but it didn't seem accurate to say "hey, checkout license.txt
although there's no such file". Apart from this, there were
inconsistencies in the headers altogether...this change brings
consistency.
 2021-09-03 12:02:59 +05:30
Gavin D'souza
e407b78506 chore: Drop dead and deprecated code 
* Remove six for PY2 compatability since our dependencies are not, PY2
  is legacy.
* Removed usages of utils from future/past libraries since they are
  deprecated. This includes 'from __future__ ...' and 'from past...'
  statements.
* Removed compatibility imports for PY2, switched from six imports to
  standard library imports.
* Removed utils code blocks that handle operations depending on PY2/3
  versions.
* Removed 'from __future__ ...' lines from templates/code generators
* Used PY3 syntaxes in place of PY2 compatible blocks. eg: metaclass
 2021-05-26 15:31:29 +05:30
Gavin D'souza
6b88c5979e chore: drop unused code 2020-05-27 19:53:51 +05:30
Gavin D'souza
c39a52d764 feat: seperate loggers into frappe, web, scheduler categories and not by 
module
 2020-05-27 18:33:03 +05:30
Suraj Shetty
e26064436b chore: Do not print traceback in console 
- Instead print class of error and the site name
 2020-04-12 15:05:28 +05:30
Aditya Hase
692055499a feat(scheduler): Frequent job scheduling 
Scheduler tick interval (default 60 seconds) can now be configured with scheduler_tick_interval key in common_site_config.json
 2020-03-05 15:45:19 +05:30