vassili/seitime-frappe
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 4
42283 commits 1 branch 0 tags 587 MiB
Commit graph

105 commits

Author SHA1 Message Date
Ankush Menat
eb17c12dda fix: reuse redis connection if not rq auth 2023-06-11 11:40:22 +05:30
Ankush Menat
07e1d34568 refactor: RQ enqueue after commit and tests 2023-06-03 18:44:19 +05:30
Ankush Menat
6519651002 refactor: change implementation of enqueue_after_commit 
if enqueue_after_commit then pass partial function after commit instead
of storing it in flags. SLIGHTLY less efficient, but uses consistent API.
 2023-06-03 18:44:19 +05:30
Ankush Menat
ef6f4b5956
refactor!: Use job_id for bg job deduplication (#20951) 2023-05-10 12:45:05 +05:30
Ankush Menat
ecce4ba5a7 test: deduplication with unique job id 2023-05-09 14:23:07 +05:30
Ankush Menat
bc3871b657 refactor: standardize assigning job_id 2023-05-09 14:23:04 +05:30
Ankush Menat
4bd02a4ed1 perf: Faster scheduled job deduplication 
O(n) to O(1) dedup by specifying unique IDs on scheduled RQ jobs.
 2023-05-08 18:28:09 +05:30
Sagar Vora
8418d3c7e2 fix: only call RQ synchronously as fallback during migration 2023-05-02 13:46:50 +05:30
Bernhard Sirlinger
c5cfe8f5aa
feat(minor): log datetime in worker log (#20414) 2023-03-21 12:55:03 +05:30
Ankush Menat
da086a4410
feat: allow configuring ttl for RQ job retention (#20331) 
Some might want to keep them around for long, others might wanna get rid
of it to free up memory. Hardcoded defaults dont work for everyone hence
make it configurable.

`no-docs`


[skip ci]
 2023-03-14 12:43:41 +05:30
Gavin D'souza
fe26c542b7 refactor: Move before/after tasks as hooks 
Moved before/after tasks in Requests as hooks for:
- monitor
- rate_limiter
- recorder

Moved before/after tasks in Jobs as hooks for:
- monitor
- releasing document locks
 2023-02-15 15:30:02 +05:30
Gavin D'souza
2ada98fbdb fix: Pass result of job to after_job hooks 2023-02-10 13:31:46 +05:30
Gavin D'souza
83f3cf1991 fix(background_jobs): Pass retval in execute_job 2023-02-10 13:23:24 +05:30
Gavin D'souza
34731d1e9e feat: Befor/After Job Hooks 2023-02-10 11:37:36 +05:30
Ankush Menat
290a3df68d chore!: drop backup.py direct calls + pragma nocov 2022-12-13 14:18:19 +05:30
Ankush Menat
d00b98f460 chore: typo 2022-11-25 13:10:43 +05:30
Ankush Menat
ed28626021 refactor: simpler deque strat selection 
Co-authored-by: Ritwik Puri <ritwikpuri5678@gmail.com>
 2022-11-25 12:14:37 +05:30
Ankush Menat
a8bf86ef75 feat: support dequeuing strategies for worker 2022-11-25 12:14:37 +05:30
Ankush Menat
aece93fbc5 feat: burst mode in workers 
https://python-rq.org/docs/workers/#burst-mode
 2022-11-25 12:14:33 +05:30
Ankush Menat
0ebd3945ff refactor: consider multi-queue consumption 2022-11-25 12:03:34 +05:30
Ankush Menat
40b2929c0d feat(workers): allow consuming multiple queues 2022-11-25 12:03:34 +05:30
Ankush Menat
b7cef3ae71 fix: prioritize short queue when using all queues 2022-11-25 12:03:34 +05:30
Aradhya Tripathi
07bd958dfd
Merge branch 'develop' into bg-submissions 2022-11-12 08:37:55 +05:30
Ankush Menat
ce360b6fce feat: Set default SQL statement timeouts 2022-11-08 14:51:43 +05:30
Aradhya
f587140f6f Merge branch 'develop' of https://github.com/Aradhya-Tripathi/frappe into bg-submissions 2022-11-04 18:34:41 +05:30
Ankush Menat
0ed7600604
build: py3.11 support (#18550) 
* chore: bump RestrictedPython

* ci: use py3.11 on development version

* fix: py311 compat changes

* chore: bump to final versions
 2022-11-03 20:49:05 +05:30
Aradhya
3852eaea74 feat: Added on_success and on_failure to enqueue 2022-10-15 14:41:08 +05:30
Aradhya
9cc826541f fix: fixed functionality 2022-10-04 02:41:14 +05:30
Aradhya
eae73ae5d4 feat: log reports for queued submits 2022-10-03 17:36:33 +05:30
Ankush Menat
1a60049af0
refactor: simpler job queued checking (#18091) 2022-09-12 12:59:00 +05:30
Ankush Menat
11936a76df feat: kill a running background job -f 
Also refactor permission checks
 2022-09-12 11:49:25 +05:30
Ankush Menat
5152f13b10 fix: RQ reduce failed job TTL from 1 year to 7 days 2022-09-10 23:54:43 +05:30
Ankush Menat
fbee80f734
perf: rebuild website search index in background (#17974) 
* perf: rebuild website search index in background

* refactor: allow enqueueing jobs during migrate

This was added a long time ago to handle missing redis during migrate.
It is not the case right now as redis HAS to be availabe during
migration.

ref: https://github.com/frappe/frappe/pull/2988

* ci: pass correct build type

* chore: warn about redis unavailability
 2022-08-26 17:41:07 +05:30
Gavin D'souza
750618ca7c fix: Re-raise original exception from tenacity's retry 2022-08-03 12:01:12 +05:30
Ankush Menat
81b37cb7d2
refactor: clean up code to py310 supported features (#17367) 
refactor: clean up code to py39+ supported syntax

- f-strings instead of format
- latest typing support instead of pre 3.9 TitleCase
- remove UTF-8 declarations.
- many more changes

Powered by https://github.com/asottile/pyupgrade/ + manual cleanups
 2022-07-01 11:51:05 +05:30
Ankush Menat
d35d7ffbe2 fix: remove bare exception catching 
A bare except catches lots of things (like generator iteration end) and should never be used.
 2022-06-28 18:05:00 +05:30
Gavin D'souza
52359a0ad9 test: Scheduler tests cleanup 2022-06-14 12:22:12 +05:30
chillaranand
1d763a6659 refactor: Fix flake8 issues 2022-05-19 15:34:35 +05:30
Gavin D'souza
66655eb8ed Merge branch 'develop' of github.com:frappe/frappe into bg-rename_doc 2022-04-14 13:41:56 +05:30
Suraj Shetty
c0c5b2ebdd
style: format all python files using black (#16453) 
Co-authored-by: Frappe Bot <developers@frappe.io>
 2022-04-12 10:59:25 +05:30
Sagar Vora
830a925b82 feat: allow enqueue at front of Redis Queue 2022-04-03 11:50:02 +05:30
Suraj Shetty
bb98e47e11
Merge pull request #16470 from netchampfaris/refactor-background-jobs-page 2022-04-01 12:06:13 +05:30
Faris Ansari
c7dbb61e55 fix: Refactor Background Jobs page 
- Jobs/Worker view
- Filter by Queue Timeout and Job Status
- Toggle Auto Refresh
- Consistent theme
 2022-03-31 16:35:54 +05:30
Gavin D'souza
448fb8a207 Merge branch 'develop' of github.com:frappe/frappe into bg-rename_doc 2022-03-22 12:29:43 +05:30
Ankush Menat
23b70df784 fix!: deprecate is_async=False usage outside of tests 2022-03-14 14:27:59 +05:30
Gavin D'souza
9e36005c70 fix: Remove taken file locks at end of background job 2022-03-03 20:05:05 +05:30
mergify[bot]
a7cd6003ce
fix: Wrap timeout getting for custom queues in function (backport #15933) (#16054) 
Co-authored-by: Lev Vereshchagin <mail@vrslev.com>
Co-authored-by: Gavin D'souza <gavin18d@gmail.com>
 2022-02-22 10:26:42 +05:30
mergify[bot]
a053e8b33c
fix: allow custom queues (backport #15653) (#15667) 
* fix: allow custom queues

(cherry picked from commit ab7077dd31c854f5a739f1a1a653e0fbffb2c53c)

* fix: remove site config and reformat the code

(cherry picked from commit b3e11c01ee08533e8ce151873e974730b16f26f7)

* fix: remove redundant code

(cherry picked from commit 51ae9a57fde9276877ef47a22d3a468a3fcfb479)

Co-authored-by: Lucas Vazquez <lucas5zvazquez@gmail.com>
 2022-01-20 09:34:32 +00:00
leela
cb696d2966 fix: rq utiliy module conflicts with redis queue's rq package 2021-09-23 12:42:25 +05:30
Gavin D'souza
783165c01e fix: Retry get_redis_conn until "sure" 
If ConnectionError or BusyLoadingError occurs, try every second for
up-to 10 times.

Why: `bench start` exits just as i run it at times. This happens when
the worker's processes each try to fetch a redis conn but redis isnt up
yet. The 3 workeer processes exit with 1 and our procman gives up too.
 2021-08-30 12:00:31 +05:30