Commit graph

740 commits

Author SHA1 Message Date
Sagar Vora
7e9b08a040 fix: convert value column to longtext for Singles 2023-09-06 16:23:01 +05:30
Ankush Menat
961585f1d9
fix: misc dx improvemnts (#22188)
* fix(dx): simplify adding callbacks

frappe.db.after_commit(func) == frappe.db.after_commit.add(func)

* fix: trace id missing

DB gets initted before request
2023-08-28 07:01:25 +00:00
Ankush Menat
730e906dfd
refactor!: remove implicit primary key from logs (#22209) 2023-08-26 16:01:47 +05:30
Ankush Menat
59e49d89fe feat: include Trace ID in all SQL queries 2023-08-23 13:59:24 +05:30
Ankush Menat
79392260d4
fix: autoincr caching and clear site cache after restore (#22079)
* fix: remove hazardous cache for autoincr

* fix: move cache to redis

* fix: clear all redis cache after restoring a site
2023-08-17 11:11:27 +05:30
mergify[bot]
fd10ab25cc
Merge pull request #22003 from gavindsouza/refactor-doc-bits
refactor: Use single query to delete child rows on doc.save
2023-08-11 06:45:07 +00:00
Gavin D'souza
28e657b603
fix: Check if fieldtype in set rather than str comparison 2023-08-11 11:45:42 +05:30
barredterra
88c8baa9ee refactor: for append to extend, merge list extend
Replace a for append loop with list extend.
Create the list with values instead of creating
an empty list and extending it with another list.
2023-08-09 13:25:39 +02:00
barredterra
b553ed98d8 refactor: inline immediately returned variable
Inline a variable to a return in the case when the
variable being declared is immediately returned
2023-08-08 18:09:49 +02:00
Ankush Menat
22a3a56d44 chore: remove conflict-marker look-alike
This is annoying when trying to find git conflicts

[skip ci]
2023-07-28 11:45:56 +05:30
David Arnold
a926e64ec9
fix: procure db config from single authority (#21578)
* fix: procure db config from single authority

ensures that configuration is uniformely procured from local.conf
instead of making use of hard to audit multilevel fallback logic

Implementation Note:
- `get_db(host, port, user, password)` was stripped of any optional
  argument and therefrom all errors where fixed.
- All occurances of `grep 'frappe.db.db_'` where changed to
  `frappe.conf.db_`

* fix: revert unnecessary breaking changes
2023-07-24 10:48:29 +05:30
Ankush Menat
adf30693a9 ci: update pyupgrade 2023-07-14 14:24:08 +05:30
David Arnold
6b2bb9a2ab
fix: add env overrides for service orchestration (#21577)
This avoids having to manipulate config files in brittle bash
entrypoints that need to react to dynamic service discovery.

This significantly improves the operability of various bench sites.
2023-07-14 11:39:01 +05:30
Ankush Menat
8a37d6d278
perf: reduce memory usage of background processes (#21467)
* perf: defer translation.py imports

This indirectly imports babel which isn't really required most of the
time.

* perf: defer gzip import

* perf: move validate_and_sanitize_search_inputs

This causes all sorts of indirect imports and increases memory usage

* perf: defer requests module imports

* perf: defer system settings import

* perf: defer LOG_DOCTYPES import

Causes many indirect imports

* perf: defer update_site_config

* perf: defer notifications import

* perf: remove unused import

* perf: defer safe exec import

* test: memory usage overhead
2023-06-23 12:51:45 +05:30
Ankush Menat
3f3ee12338 Revert: keep supporting set_value for singles
w/ explicit check for singles

This shouldn't have any performance impact as last function call only
happens if we THINK it's single doctype. use set_single_value to avoid
that extra function call.
2023-06-14 17:30:32 +05:30
Ankush Menat
3005e66e45 refactor!: Drop previously deprecated code 2023-06-13 16:00:43 +05:30
Ankush Menat
fa6dc03cc8
refactor: frappe.cache() usage to frappe.cache (#21282) 2023-06-08 11:47:17 +05:30
Ankush Menat
98b4693dcf perf: finer cache eviction on db.set_value
Instead of nuking everything, just clear matching prefix
2023-06-03 22:53:10 +05:30
Ankush Menat
07e1d34568 refactor: RQ enqueue after commit and tests 2023-06-03 18:44:19 +05:30
Ankush Menat
54ae0c4a21 refactor: move flush_realtime_log to realtime.py
This doesn't have anything to do with databases
2023-06-03 18:44:19 +05:30
Ankush Menat
6ce7444669 refactor: generic callback manager 2023-06-03 18:44:19 +05:30
Ankush Menat
3f1c66de10 refactor: move flush_local_link_count to hook 2023-06-03 18:44:19 +05:30
Ankush Menat
ccc107b41f test: use db.before_commit 2023-06-03 18:44:19 +05:30
Ankush Menat
6519651002 refactor: change implementation of enqueue_after_commit
if enqueue_after_commit then pass partial function after commit instead
of storing it in flags. SLIGHTLY less efficient, but uses consistent API.
2023-06-03 18:44:19 +05:30
Ankush Menat
b3d370a0b1 refactor!: remove rollback_observers
use `frappe.db.after_rollback.add` instead
2023-06-03 18:44:19 +05:30
Ankush Menat
7e9ef00bea refactor!: Remove frappe.db.add_before_commit
Not used anywhere, use `frappe.db.before_commit.add()` instead.
2023-06-03 18:44:19 +05:30
Ankush Menat
235e4855c2 feat: DBHooks to run things before/after commit/rollback
This is a common pattern which is implemented in inconsistent and undocumented ways using these:

- `frappe.local.rollback_observers`
- `frappe.flags.enqueue_after_commit`
- `frappe.local.realtime_log`
- `frappe.local.before_commit`
- `flush_local_link_count`

Instead new simple api:

- Simple function call `frappe.db.after_commit.run(function)`
- If you need args just pass partial function `frappe.db.after_commit.run(lambda: frappe.clear_cache(doctype, name)`
2023-06-03 18:44:19 +05:30
Ankush Menat
691cbd6da7 chore: remove select from ddl prefix 2023-06-02 16:39:58 +05:30
Ankush Menat
a09e29cfa0
Merge pull request #21173 from resilient-tech/safe-filters
fix!: improved filter validation in `Engine.get_query`
2023-06-02 11:45:06 +05:30
Ankush Menat
4104e7d733
feat: show db table utilization on doctype (#21193)
* feat: show db table utilization on doctype

* fix: nicer error message with docs
2023-05-31 16:28:44 +05:30
Ankush Menat
950277a88d chore: increase logging level 2023-05-31 16:04:56 +05:30
Ankush Menat
48539dc0a3
feat: log all DDL queries (#21107) 2023-05-31 16:00:59 +05:30
Sagar Vora
1b2d1dd567 chore: move statement to set validate_filters property 2023-05-31 14:20:26 +05:30
Sagar Vora
9f5a994f70 fix!: improved filter validation in Engine.get_query 2023-05-31 14:16:52 +05:30
Ankush Menat
a1c40d9158 feat: support tree-link-fields filtering in QB 2023-05-29 17:00:37 +05:30
Ankush Menat
842195ce29 fix: simplify plucking
no idea why itertools are used for this dumb operation
2023-05-29 17:00:37 +05:30
Ankush Menat
3df13ca392 feat: new operator - descendants of (inclusive)
Co-Authored-By: Faris Ansari <netchamp.faris@gmail.com>
2023-05-29 16:59:57 +05:30
Sagar Vora
38c1207abe
fix: multiple fixes to Engine.get_query (#21135) 2023-05-29 10:48:26 +05:30
Sagar Vora
d6d5d488a2 fix: ensure correct return value for get_values_from_single 2023-05-29 10:06:49 +05:30
Ritwik Puri
d099c9376b
feat(minor): db.get_column_type for postgres (#21125) 2023-05-26 16:03:28 +05:30
Ritwik Puri
e277b94714
fix: allow default for text and long text when altering table (#21109) 2023-05-26 11:09:28 +05:30
Ritwik Puri
c5e62cac26
fix: allow setting default in longtext and text columns (#21089) 2023-05-25 12:29:20 +05:30
Faris Ansari
2df7fdd79e
Merge pull request #20908 from netchampfaris/qb-getquery-child-fields-syntax-sugar 2023-05-11 12:02:30 +05:30
Faris Ansari
c4bb732eaa fix: use dict syntax instead of string 2023-05-08 18:55:11 +05:30
Faris Ansari
41d7563aff feat: child_field[] syntax sugar for qb
- fetch child table rows in qb.get_query
- runs one query each for each child field
2023-05-08 18:55:11 +05:30
Ankush Menat
4bd02a4ed1 perf: Faster scheduled job deduplication
O(n) to O(1) dedup by specifying unique IDs on scheduled RQ jobs.
2023-05-08 18:28:09 +05:30
Ankush Menat
cf53b1a58e chore: set_value docstring
[skip ci]
2023-05-08 16:23:29 +05:30
Ikko Eltociear Ashimine
3efaee9597
docs: fix typo in database.py (#20894) 2023-05-04 17:07:08 +05:30
Ankush Menat
024faff025
build: bump pymysql (#20478)
Actual fix for this bandaid fix: https://github.com/frappe/frappe/pull/20475

Keeping pymysql hard pinned until we have better way to get last full
query.
2023-03-28 13:04:27 +05:30
Ankush Menat
da086a4410
feat: allow configuring ttl for RQ job retention (#20331)
Some might want to keep them around for long, others might wanna get rid
of it to free up memory. Hardcoded defaults dont work for everyone hence
make it configurable.

`no-docs`


[skip ci]
2023-03-14 12:43:41 +05:30