Commit graph

728 commits

Author SHA1 Message Date
David Arnold
6b2bb9a2ab
fix: add env overrides for service orchestration (#21577)
This avoids having to manipulate config files in brittle bash
entrypoints that need to react to dynamic service discovery.

This significantly improves the operability of various bench sites.
2023-07-14 11:39:01 +05:30
Ankush Menat
8a37d6d278
perf: reduce memory usage of background processes (#21467)
* perf: defer translation.py imports

This indirectly imports babel which isn't really required most of the
time.

* perf: defer gzip import

* perf: move validate_and_sanitize_search_inputs

This causes all sorts of indirect imports and increases memory usage

* perf: defer requests module imports

* perf: defer system settings import

* perf: defer LOG_DOCTYPES import

Causes many indirect imports

* perf: defer update_site_config

* perf: defer notifications import

* perf: remove unused import

* perf: defer safe exec import

* test: memory usage overhead
2023-06-23 12:51:45 +05:30
Ankush Menat
3f3ee12338 Revert: keep supporting set_value for singles
w/ explicit check for singles

This shouldn't have any performance impact as last function call only
happens if we THINK it's single doctype. use set_single_value to avoid
that extra function call.
2023-06-14 17:30:32 +05:30
Ankush Menat
3005e66e45 refactor!: Drop previously deprecated code 2023-06-13 16:00:43 +05:30
Ankush Menat
fa6dc03cc8
refactor: frappe.cache() usage to frappe.cache (#21282) 2023-06-08 11:47:17 +05:30
Ankush Menat
98b4693dcf perf: finer cache eviction on db.set_value
Instead of nuking everything, just clear matching prefix
2023-06-03 22:53:10 +05:30
Ankush Menat
07e1d34568 refactor: RQ enqueue after commit and tests 2023-06-03 18:44:19 +05:30
Ankush Menat
54ae0c4a21 refactor: move flush_realtime_log to realtime.py
This doesn't have anything to do with databases
2023-06-03 18:44:19 +05:30
Ankush Menat
6ce7444669 refactor: generic callback manager 2023-06-03 18:44:19 +05:30
Ankush Menat
3f1c66de10 refactor: move flush_local_link_count to hook 2023-06-03 18:44:19 +05:30
Ankush Menat
ccc107b41f test: use db.before_commit 2023-06-03 18:44:19 +05:30
Ankush Menat
6519651002 refactor: change implementation of enqueue_after_commit
if enqueue_after_commit then pass partial function after commit instead
of storing it in flags. SLIGHTLY less efficient, but uses consistent API.
2023-06-03 18:44:19 +05:30
Ankush Menat
b3d370a0b1 refactor!: remove rollback_observers
use `frappe.db.after_rollback.add` instead
2023-06-03 18:44:19 +05:30
Ankush Menat
7e9ef00bea refactor!: Remove frappe.db.add_before_commit
Not used anywhere, use `frappe.db.before_commit.add()` instead.
2023-06-03 18:44:19 +05:30
Ankush Menat
235e4855c2 feat: DBHooks to run things before/after commit/rollback
This is a common pattern which is implemented in inconsistent and undocumented ways using these:

- `frappe.local.rollback_observers`
- `frappe.flags.enqueue_after_commit`
- `frappe.local.realtime_log`
- `frappe.local.before_commit`
- `flush_local_link_count`

Instead new simple api:

- Simple function call `frappe.db.after_commit.run(function)`
- If you need args just pass partial function `frappe.db.after_commit.run(lambda: frappe.clear_cache(doctype, name)`
2023-06-03 18:44:19 +05:30
Ankush Menat
691cbd6da7 chore: remove select from ddl prefix 2023-06-02 16:39:58 +05:30
Ankush Menat
a09e29cfa0
Merge pull request #21173 from resilient-tech/safe-filters
fix!: improved filter validation in `Engine.get_query`
2023-06-02 11:45:06 +05:30
Ankush Menat
4104e7d733
feat: show db table utilization on doctype (#21193)
* feat: show db table utilization on doctype

* fix: nicer error message with docs
2023-05-31 16:28:44 +05:30
Ankush Menat
950277a88d chore: increase logging level 2023-05-31 16:04:56 +05:30
Ankush Menat
48539dc0a3
feat: log all DDL queries (#21107) 2023-05-31 16:00:59 +05:30
Sagar Vora
1b2d1dd567 chore: move statement to set validate_filters property 2023-05-31 14:20:26 +05:30
Sagar Vora
9f5a994f70 fix!: improved filter validation in Engine.get_query 2023-05-31 14:16:52 +05:30
Ankush Menat
a1c40d9158 feat: support tree-link-fields filtering in QB 2023-05-29 17:00:37 +05:30
Ankush Menat
842195ce29 fix: simplify plucking
no idea why itertools are used for this dumb operation
2023-05-29 17:00:37 +05:30
Ankush Menat
3df13ca392 feat: new operator - descendants of (inclusive)
Co-Authored-By: Faris Ansari <netchamp.faris@gmail.com>
2023-05-29 16:59:57 +05:30
Sagar Vora
38c1207abe
fix: multiple fixes to Engine.get_query (#21135) 2023-05-29 10:48:26 +05:30
Sagar Vora
d6d5d488a2 fix: ensure correct return value for get_values_from_single 2023-05-29 10:06:49 +05:30
Ritwik Puri
d099c9376b
feat(minor): db.get_column_type for postgres (#21125) 2023-05-26 16:03:28 +05:30
Ritwik Puri
e277b94714
fix: allow default for text and long text when altering table (#21109) 2023-05-26 11:09:28 +05:30
Ritwik Puri
c5e62cac26
fix: allow setting default in longtext and text columns (#21089) 2023-05-25 12:29:20 +05:30
Faris Ansari
2df7fdd79e
Merge pull request #20908 from netchampfaris/qb-getquery-child-fields-syntax-sugar 2023-05-11 12:02:30 +05:30
Faris Ansari
c4bb732eaa fix: use dict syntax instead of string 2023-05-08 18:55:11 +05:30
Faris Ansari
41d7563aff feat: child_field[] syntax sugar for qb
- fetch child table rows in qb.get_query
- runs one query each for each child field
2023-05-08 18:55:11 +05:30
Ankush Menat
4bd02a4ed1 perf: Faster scheduled job deduplication
O(n) to O(1) dedup by specifying unique IDs on scheduled RQ jobs.
2023-05-08 18:28:09 +05:30
Ankush Menat
cf53b1a58e chore: set_value docstring
[skip ci]
2023-05-08 16:23:29 +05:30
Ikko Eltociear Ashimine
3efaee9597
docs: fix typo in database.py (#20894) 2023-05-04 17:07:08 +05:30
Ankush Menat
024faff025
build: bump pymysql (#20478)
Actual fix for this bandaid fix: https://github.com/frappe/frappe/pull/20475

Keeping pymysql hard pinned until we have better way to get last full
query.
2023-03-28 13:04:27 +05:30
Ankush Menat
da086a4410
feat: allow configuring ttl for RQ job retention (#20331)
Some might want to keep them around for long, others might wanna get rid
of it to free up memory. Hardcoded defaults dont work for everyone hence
make it configurable.

`no-docs`


[skip ci]
2023-03-14 12:43:41 +05:30
Ankush Menat
4ec874ac6e style: format 2023-03-06 11:05:29 +05:30
Saif Ur Rehman
e9dfa80cf0
fix(Database): clear background jobs and realtime logs on rollback (#20236) 2023-03-06 10:47:11 +05:30
Sagar Vora
70ede18942
fix: improved implementation of between filter (#20190) 2023-03-02 09:51:02 +05:30
Ankush Menat
9c28368213
chore: simpler "*" check condition (#20201) 2023-03-01 15:48:02 +05:30
Ankush Menat
d8ae5cd187
Revert "chore(Database): simplify condition" (#20200) 2023-03-01 15:26:24 +05:30
Sagar Vora
ad90ea888a chore(Database): simplify condition 2023-03-01 12:11:10 +05:30
Ankush Menat
9321a2e86a fix(UX): Better message for read only mode
- The default message keeps confusing user because it doesn't contain the word "site update" which is usually the reason for putting site in read only mode.
- Also fixes annoying popup while opening notifications in read only mode.

[skip ci]
2023-02-27 16:16:22 +05:30
Ankush Menat
30f0b4ea2d
perf!: frappe.db.exists without order_by (#20093) 2023-02-20 17:28:03 +05:30
Gavin D'souza
3cd781e316 Merge branch 'develop' into get_all-virtual-dts 2023-02-02 13:55:24 +05:30
Gavin D'souza
5d3453eeb9 refactor: Re-use DefaultOrderBy value as global constant 2023-02-02 13:43:31 +05:30
Ritwik Puri
73078e89ba
Merge pull request #19888 from ankush/drop_tables
fix: drop table if exists for action and links
2023-02-01 22:55:40 +05:30
Ankush Menat
dad0e5cbba fix: drop table if exists for action and links
closes https://github.com/frappe/frappe/issues/19712
2023-02-01 22:37:43 +05:30