Commit graph

40 commits

Author SHA1 Message Date
Ankush Menat
8baeb5151d
fix: check for running jobs before migrating (#31438) 2025-02-26 09:30:38 +00:00
David
202cce8a0d
refactor: type hint cli context 2024-09-13 17:25:43 +02:00
David
d87b786687
chore: fix all invocation of init for singledispatch -> min 1 positional arg 2024-09-13 11:53:52 +02:00
Ankush Menat
d13178755b refactor: keep scheduler+worker behaviour to workerpool
- Single worker adds some extra costs to each worker (importing scheduler)
- Workerpool can import it all once and share
- This way we need not have many different
2024-05-27 12:29:41 +05:30
Ankush Menat
433e7281f7 feat: make running scheduler with worker optional 2024-05-27 12:14:15 +05:30
Ankush Menat
de9ac89748 style: re-format with ruff 2024-02-05 18:53:33 +05:30
Akhil Narang
d67a6d168e
fix(commands/scheduler): typo (#23441)
* fix(commands/scheduler): typo

Signed-off-by: Akhil Narang <me@akhilnarang.dev>

* chore: Update frappe/commands/scheduler.py

[skip ci]

---------

Signed-off-by: Akhil Narang <me@akhilnarang.dev>
Co-authored-by: Ankush Menat <ankushmenat@gmail.com>
2023-11-27 11:30:33 +05:30
Ankush Menat
1092eef7bd perf: faster pending jobs check 2023-06-29 17:31:06 +05:30
Ankush Menat
73bca16d77 feat: RQ WorkerPool
RQ now has experimental support for workerpools.

When to use this?

Roughly when you have more than 2 workers a workerpool might make
sense, below 2 it's overhead as master "pool" process will need to run
to manager workerpool itself.

Why is it any better?

Currently we just let supervisor duplicate the worker process N number
of times. This is inefficient from shared memory POV. Forking the
original process to create workers enables sharing of more memory thus
leading upwards of 60-70% reduction in memory usage with pool size of 8
workers.
2023-06-26 18:10:00 +05:30
Gavin D'souza
32cf13cb29 chore: Cleanup imports 2023-02-10 14:45:39 +05:30
Gavin D'souza
4738a1422d fix: Add format, verbose options to scheduler 2023-02-10 14:39:16 +05:30
Gavin D'souza
6b84c9ccf5 feat: Check scheduler status via CLI
new: `bench --site scheduler status`
2023-02-10 14:38:50 +05:30
Ankush Menat
5d1444c21c
docs: add missing help text for commands (#19039)
closes https://github.com/frappe/frappe/issues/19036

[skip ci]
2022-11-29 16:01:25 +05:30
Ankush Menat
d00b98f460 chore: typo 2022-11-25 13:10:43 +05:30
Ankush Menat
ed28626021 refactor: simpler deque strat selection
Co-authored-by: Ritwik Puri <ritwikpuri5678@gmail.com>
2022-11-25 12:14:37 +05:30
Ankush Menat
a8bf86ef75 feat: support dequeuing strategies for worker 2022-11-25 12:14:37 +05:30
Ankush Menat
aece93fbc5 feat: burst mode in workers
https://python-rq.org/docs/workers/#burst-mode
2022-11-25 12:14:33 +05:30
Ankush Menat
40b2929c0d feat(workers): allow consuming multiple queues 2022-11-25 12:03:34 +05:30
Himanshu Shivhare
c7461a05c6
chore: Typo correction (#18997)
[skip ci]
2022-11-24 16:25:11 +05:30
Gavin D'souza
71b5c77e6e Merge branch 'develop' of github.com:frappe/frappe into mariadb-client-refactor 2022-07-05 14:55:02 +05:30
Ankush Menat
81b37cb7d2
refactor: clean up code to py310 supported features (#17367)
refactor: clean up code to py39+ supported syntax

- f-strings instead of format
- latest typing support instead of pre 3.9 TitleCase
- remove UTF-8 declarations.
- many more changes

Powered by https://github.com/asottile/pyupgrade/ + manual cleanups
2022-07-01 11:51:05 +05:30
Gavin D'souza
cf699fe40b Merge branch 'develop' of github.com:frappe/frappe into mariadb-client-refactor 2022-06-29 13:49:18 +05:30
Ankush Menat
d35d7ffbe2 fix: remove bare exception catching
A bare except catches lots of things (like generator iteration end) and should never be used.
2022-06-28 18:05:00 +05:30
gavin
73c3db12ee refactor(minor): Move internal util to the module where it's used 2022-05-24 19:19:53 +05:30
Suraj Shetty
c0c5b2ebdd
style: format all python files using black (#16453)
Co-authored-by: Frappe Bot <developers@frappe.io>
2022-04-12 10:59:25 +05:30
leela
db09a85183 refactor: Add authentication for Redis Queue 2021-07-26 21:09:34 +05:30
Gavin D'souza
e407b78506 chore: Drop dead and deprecated code
* Remove six for PY2 compatability since our dependencies are not, PY2
  is legacy.
* Removed usages of utils from future/past libraries since they are
  deprecated. This includes 'from __future__ ...' and 'from past...'
  statements.
* Removed compatibility imports for PY2, switched from six imports to
  standard library imports.
* Removed utils code blocks that handle operations depending on PY2/3
  versions.
* Removed 'from __future__ ...' lines from templates/code generators
* Used PY3 syntaxes in place of PY2 compatible blocks. eg: metaclass
2021-05-26 15:31:29 +05:30
gavin
edf92d4450
fix(cli): Trigger Scheduler Event (#12955)
* Triggers events via Scheduled Job Type's execute method
* Exits with code 1 if no event with that name found or process
termination
* Added feedback if event not found
2021-04-22 12:41:31 +05:30
Gavin D'souza
cfa7afb08c fix(doctor): update broken command behaviour 2020-06-10 11:34:20 +05:30
Gavin D'souza
9d11f2c44f fix: redo improper merge commit 2020-06-01 19:31:41 +05:30
Gavin D'souza
07c15d0fe1 fix: replace terrible code with bad one 2020-06-01 18:44:47 +05:30
Aditya Hase
4164c62efc fix(doctor): Show status of scheduler in bench doctor 2019-07-31 20:14:58 +05:30
Aditya Hase
6b72faf44f style: Linting fixes (#6809)
* style: Remove unused imports

* style: Remove unused local variables
2019-01-23 16:39:55 +05:30
Achilles Rasquinha
255e88d297 Ensure worker is quiet during development 2018-02-08 23:58:55 +05:30
tundebabzy
7b41d7e5eb Python 3 compatible print statements (#3199)
* changes print statements in file to python 3 compatible style using `__future__`

* adds python 3 style for print statements

* adds python 3 style for print statements

* adds python 3 style for print statements

* adds python 3 style for print statements

* adds python 3 style for print statements

* adds python 3 style for print statements

* adds python 3 style for print statements

* adds python 3 style for print statements

* adds python 3 style for print statements

* adds python 3 style for print statements

* adds python 3 style for print statements

* adds python 3 style for print statements

* adds python 3 style for print statements

* adds python 3 style for print statements

* adds python 3 style for print statements

* adds python 3 style for print statements

* adds python 3 style for print statements

* adds python 3 style for print statements

* adds python 3 style for print statements

* adds python 3 style for print statements

* adds python 3 style for print statements

* adds python 3 style for print statements

* adds python 3 style for print statements

* adds python 3 style for print statements

* adds python 3 style for print statements

* adds python 3 style for print statements

* adds python 3 style for print statements

* adds python 3 style for print statements

* adds python 3 style for print statements

* adds python 3 style for print statements

* adds python 3 style for print statements

* adds python 3 style for print statements

* changes deprecated md5 module to hashlib

* adds python 3 style for print statements

* adds python 3 style for print statements

* adds python 3 style for print statements

* adds python 3 style for print statements

* adds python 3 style for print statements

* adds python 3 style for print statements

* adds python 3 style for print statements

* adds python 3 style for print statements

* adds python 3 style for print statements

* adds python 3 style for print statements

* adds python 3 style for print statements

* adds python 3 style for print statements

* adds python 3 style for print statements

* adds python 3 style for print statements

* adds python 3 style for print statements
2017-05-01 12:34:29 +05:30
Nabin Hait
4cf123bd8f Fixed merge conflict 2016-11-28 14:25:11 +05:30
Shreyas Patil
6f59a23f47 [Minor] Removed not available scheduler commands 'dump-queue-status' (#2385) 2016-11-28 12:38:20 +05:30
Rushabh Mehta
2fe0e2acad [fix] set Message-Id in email based on reference and tag replies to the same object 2016-11-17 09:45:44 +05:30
Valmik Jangla
f4fb315082 Updated docs for new features 2016-08-12 04:01:29 -07:00
Anand Doshi
49de395407 [fix] split commands into multiple files 2016-05-18 17:14:06 +05:30