Commit graph

336 commits

Author SHA1 Message Date
Ankush Menat
adf30693a9 ci: update pyupgrade 2023-07-14 14:24:08 +05:30
David Arnold
6b2bb9a2ab
fix: add env overrides for service orchestration (#21577)
This avoids having to manipulate config files in brittle bash
entrypoints that need to react to dynamic service discovery.

This significantly improves the operability of various bench sites.
2023-07-14 11:39:01 +05:30
Ankush Menat
8a37d6d278
perf: reduce memory usage of background processes (#21467)
* perf: defer translation.py imports

This indirectly imports babel which isn't really required most of the
time.

* perf: defer gzip import

* perf: move validate_and_sanitize_search_inputs

This causes all sorts of indirect imports and increases memory usage

* perf: defer requests module imports

* perf: defer system settings import

* perf: defer LOG_DOCTYPES import

Causes many indirect imports

* perf: defer update_site_config

* perf: defer notifications import

* perf: remove unused import

* perf: defer safe exec import

* test: memory usage overhead
2023-06-23 12:51:45 +05:30
Ankush Menat
3f3ee12338 Revert: keep supporting set_value for singles
w/ explicit check for singles

This shouldn't have any performance impact as last function call only
happens if we THINK it's single doctype. use set_single_value to avoid
that extra function call.
2023-06-14 17:30:32 +05:30
Ankush Menat
3005e66e45 refactor!: Drop previously deprecated code 2023-06-13 16:00:43 +05:30
Ankush Menat
fa6dc03cc8
refactor: frappe.cache() usage to frappe.cache (#21282) 2023-06-08 11:47:17 +05:30
Ankush Menat
98b4693dcf perf: finer cache eviction on db.set_value
Instead of nuking everything, just clear matching prefix
2023-06-03 22:53:10 +05:30
Ankush Menat
07e1d34568 refactor: RQ enqueue after commit and tests 2023-06-03 18:44:19 +05:30
Ankush Menat
54ae0c4a21 refactor: move flush_realtime_log to realtime.py
This doesn't have anything to do with databases
2023-06-03 18:44:19 +05:30
Ankush Menat
6ce7444669 refactor: generic callback manager 2023-06-03 18:44:19 +05:30
Ankush Menat
3f1c66de10 refactor: move flush_local_link_count to hook 2023-06-03 18:44:19 +05:30
Ankush Menat
ccc107b41f test: use db.before_commit 2023-06-03 18:44:19 +05:30
Ankush Menat
6519651002 refactor: change implementation of enqueue_after_commit
if enqueue_after_commit then pass partial function after commit instead
of storing it in flags. SLIGHTLY less efficient, but uses consistent API.
2023-06-03 18:44:19 +05:30
Ankush Menat
b3d370a0b1 refactor!: remove rollback_observers
use `frappe.db.after_rollback.add` instead
2023-06-03 18:44:19 +05:30
Ankush Menat
7e9ef00bea refactor!: Remove frappe.db.add_before_commit
Not used anywhere, use `frappe.db.before_commit.add()` instead.
2023-06-03 18:44:19 +05:30
Ankush Menat
235e4855c2 feat: DBHooks to run things before/after commit/rollback
This is a common pattern which is implemented in inconsistent and undocumented ways using these:

- `frappe.local.rollback_observers`
- `frappe.flags.enqueue_after_commit`
- `frappe.local.realtime_log`
- `frappe.local.before_commit`
- `flush_local_link_count`

Instead new simple api:

- Simple function call `frappe.db.after_commit.run(function)`
- If you need args just pass partial function `frappe.db.after_commit.run(lambda: frappe.clear_cache(doctype, name)`
2023-06-03 18:44:19 +05:30
Ankush Menat
691cbd6da7 chore: remove select from ddl prefix 2023-06-02 16:39:58 +05:30
Ankush Menat
a09e29cfa0
Merge pull request #21173 from resilient-tech/safe-filters
fix!: improved filter validation in `Engine.get_query`
2023-06-02 11:45:06 +05:30
Ankush Menat
4104e7d733
feat: show db table utilization on doctype (#21193)
* feat: show db table utilization on doctype

* fix: nicer error message with docs
2023-05-31 16:28:44 +05:30
Ankush Menat
950277a88d chore: increase logging level 2023-05-31 16:04:56 +05:30
Ankush Menat
48539dc0a3
feat: log all DDL queries (#21107) 2023-05-31 16:00:59 +05:30
Sagar Vora
9f5a994f70 fix!: improved filter validation in Engine.get_query 2023-05-31 14:16:52 +05:30
Sagar Vora
d6d5d488a2 fix: ensure correct return value for get_values_from_single 2023-05-29 10:06:49 +05:30
Ritwik Puri
d099c9376b
feat(minor): db.get_column_type for postgres (#21125) 2023-05-26 16:03:28 +05:30
Ankush Menat
4bd02a4ed1 perf: Faster scheduled job deduplication
O(n) to O(1) dedup by specifying unique IDs on scheduled RQ jobs.
2023-05-08 18:28:09 +05:30
Ankush Menat
cf53b1a58e chore: set_value docstring
[skip ci]
2023-05-08 16:23:29 +05:30
Ikko Eltociear Ashimine
3efaee9597
docs: fix typo in database.py (#20894) 2023-05-04 17:07:08 +05:30
Ankush Menat
da086a4410
feat: allow configuring ttl for RQ job retention (#20331)
Some might want to keep them around for long, others might wanna get rid
of it to free up memory. Hardcoded defaults dont work for everyone hence
make it configurable.

`no-docs`


[skip ci]
2023-03-14 12:43:41 +05:30
Ankush Menat
4ec874ac6e style: format 2023-03-06 11:05:29 +05:30
Saif Ur Rehman
e9dfa80cf0
fix(Database): clear background jobs and realtime logs on rollback (#20236) 2023-03-06 10:47:11 +05:30
Sagar Vora
70ede18942
fix: improved implementation of between filter (#20190) 2023-03-02 09:51:02 +05:30
Ankush Menat
9c28368213
chore: simpler "*" check condition (#20201) 2023-03-01 15:48:02 +05:30
Ankush Menat
d8ae5cd187
Revert "chore(Database): simplify condition" (#20200) 2023-03-01 15:26:24 +05:30
Sagar Vora
ad90ea888a chore(Database): simplify condition 2023-03-01 12:11:10 +05:30
Ankush Menat
9321a2e86a fix(UX): Better message for read only mode
- The default message keeps confusing user because it doesn't contain the word "site update" which is usually the reason for putting site in read only mode.
- Also fixes annoying popup while opening notifications in read only mode.

[skip ci]
2023-02-27 16:16:22 +05:30
Ankush Menat
30f0b4ea2d
perf!: frappe.db.exists without order_by (#20093) 2023-02-20 17:28:03 +05:30
Gavin D'souza
3cd781e316 Merge branch 'develop' into get_all-virtual-dts 2023-02-02 13:55:24 +05:30
Gavin D'souza
5d3453eeb9 refactor: Re-use DefaultOrderBy value as global constant 2023-02-02 13:43:31 +05:30
Gavin D'souza
7ce0c4c8b3 Merge branch 'develop' into permlevel-apis 2023-01-31 17:47:59 +05:30
Ankush Menat
b8deb72411
fix: prepared report patch (#19807)
- the patch is failing if system setting is updated and column is gone
- the message is misleading "invalid column", it should be "missing field".
2023-01-27 14:26:54 +05:30
gavin
4be74bc013
Merge branch 'develop' into permlevel-apis 2023-01-19 11:00:15 +05:30
Ankush Menat
fa55793a52
Merge pull request #19405 from netchampfaris/refactor-qb-engine
refactor: qb.engine
2023-01-18 21:17:17 +05:30
gavin
d1b7a69141
Merge branch 'develop' into permlevel-apis 2023-01-16 14:54:36 +05:30
Pruthvi Patel
2e87b405fe
chore(docstring): fix docstring to match the value of MAX_WRITES_PER_TRANSACTION (#19572) 2023-01-12 17:47:59 +05:30
Faris Ansari
a0f6a5ff46 fix: move pluck to run 2023-01-10 18:21:50 +05:30
Gavin D'souza
058c49f439 fix: Pass parenttype in meta calls, handle count(*) type queries 2023-01-09 17:09:36 +05:30
Faris Ansari
35c2654f00 chore: indentation fix 2023-01-09 15:34:50 +05:30
Faris Ansari
be654eaa60
Merge branch 'develop' into refactor-qb-engine 2023-01-09 15:23:20 +05:30
Ankush Menat
28375e8e6e
refactor: deprecate db.set_value on singles completely (#19481) 2023-01-05 13:04:08 +05:30
Ankush Menat
3e7a572aa2 refactor: Deprecate db.set_value with None as docname 2023-01-04 11:05:09 +05:30