Commit graph

129 commits

Author SHA1 Message Date
Ankush Menat
de48dc2c04
fix: db explain (#25724)
This broke from some recent changes.

It's just a wrapper, so probably no one uses it.
Anyway, fixed and added test to prevent it.
2024-03-29 10:23:12 +00:00
Ankush Menat
23204c0c05
Merge pull request #25256 from ankush/no_commit_doc_events
fix: disable transaction commits during doc events
2024-03-28 12:28:12 +05:30
Ankush Menat
e5a64fd50c test: transaction control 2024-03-28 11:58:38 +05:30
Ankush Menat
b7bc5d3236
fix(round 3): switch to creation sorting (#25669) 2024-03-27 08:59:04 +00:00
Ankush Menat
0fff2755d6
test: separate out risky tests (#25179) 2024-02-29 16:10:08 +00:00
Ankush Menat
0c9cc2e6ce test: NOWAIT functionality 2024-02-29 17:46:30 +05:30
Ankush Menat
5116768a54 test: utils for simulating two connections 2024-02-29 17:31:46 +05:30
Akhil Narang
3f1e19de85
refactor(treewide): enable RUF rules
Signed-off-by: Akhil Narang <me@akhilnarang.dev>
2024-02-21 16:20:28 +05:30
Raffael Meyer
68eb2d978d
fix: keep order in get_values_from_single (#24907)
* fix: keep order in `get_values_from_single`

* fix: add test for destructuring

---------

Co-authored-by: Ankush Menat <ankush@frappe.io>
2024-02-19 05:35:35 +00:00
Akhil Narang
26ae0f3460
fix: ruff fixes
Signed-off-by: Akhil Narang <me@akhilnarang.dev>
2024-02-07 17:04:31 +05:30
Ankush Menat
de9ac89748 style: re-format with ruff 2024-02-05 18:53:33 +05:30
Ankush Menat
a2525e545a
perf: Unbuffered cursors for large result sets (#24365)
If you're reading 1000s of rows from MySQL, the default behaviour is to
read all of them in memory at once.

One of the use case for reading large rows is reporting where a lot of
data is read and then processed in Python. The read row is hoever not
used again but still consumes memory until entire function exits.

SSCursor (Server Side Cursor) allows fetching one row at a time.

Note: This is slower than fetching everything at once AND has risk of
connection loss. So, don't use this as a crutch. If possible rewrite
code so processing is done in SQL.
2024-01-16 11:00:12 +05:30
Ankush Menat
588157df74 feat: frappe.db.sql results as iterator
- Also avoid self.last_result that holds on to large result set reference.
2024-01-15 12:32:02 +05:30
Ankush Menat
e45e313bfb
feat: Skip locked rows while selecting (#24298) 2024-01-13 09:49:27 +05:30
Ankush Menat
acd1b9d64e
fix: Don't run query if dn is None (#22475) 2023-09-19 15:11:15 +00:00
0xsaif
aa7ee1c6b9
refactor: force ipv4 localhost (#22394)
* reafctor: force ipv4 localhost

Replacing "localhost" with "127.0.0.1" in the codebase; sometimes the name localhost force-resolves to ipv6

* revert: leave localhost usage in oauth tests

Change not required.

---------

Co-authored-by: Ankush Menat <ankush@frappe.io>
2023-09-13 16:23:23 +05:30
Ankush Menat
961585f1d9
fix: misc dx improvemnts (#22188)
* fix(dx): simplify adding callbacks

frappe.db.after_commit(func) == frappe.db.after_commit.add(func)

* fix: trace id missing

DB gets initted before request
2023-08-28 07:01:25 +00:00
Ankush Menat
fefd9ac2e2
fix: connect_replica and read_only should be idempotent (#21674)
* fix: `connect_replica` should be idempotent

Calling `connect_replica` twice ends up forgetting orginal writable
replica completely.

* test: replicas

closes https://github.com/frappe/frappe/issues/21619
2023-07-14 13:56:57 +05:30
Ankush Menat
c73d9fb783 test: add perf test helper for counting rows read 2023-06-28 19:51:18 +05:30
Ankush Menat
3f3ee12338 Revert: keep supporting set_value for singles
w/ explicit check for singles

This shouldn't have any performance impact as last function call only
happens if we THINK it's single doctype. use set_single_value to avoid
that extra function call.
2023-06-14 17:30:32 +05:30
Ankush Menat
3d0888a5d5 refactor: set_value usage 2023-06-13 16:00:43 +05:30
Ankush Menat
26722b1a1c fix: ignore ConnectionError in frappe.cache().exists() 2023-06-03 22:53:10 +05:30
Ankush Menat
0d056a3a2b test: fix broken tests
Fixture test:
This is broken cause it's trying to find doctype after it has been
deleted (wut?)
It was working so far because cache wasn't cleared correctly so you'd
still find it from cache.

db.set_value test:
converted to use last query instead of patching SQL
2023-06-03 22:53:10 +05:30
Ankush Menat
0b9dee4791 test: db callbacks 2023-06-03 18:44:19 +05:30
Faris Ansari
60febc9799 fix: list filter
filters as list must always be list of list
2023-01-09 19:44:11 +05:30
Ankush Menat
a7377d23fc
refactor!: Drop deprecated functionality (#18815) 2022-11-11 19:54:36 +05:30
Ankush Menat
b71d93ef9f test: db timeout computation 2022-11-09 15:55:31 +05:30
Ankush Menat
ce360b6fce feat: Set default SQL statement timeouts 2022-11-08 14:51:43 +05:30
Ankush Menat
64a0e19329 Revert "fix!: Dont update modified by default in db.set_value (#18301)"
This reverts commit 6aaefd6633.
2022-10-06 22:02:31 +05:30
Ankush Menat
bfa6a5fbdf fix: better cache validation
- Only delete a single doc if we know which doc changed
- Drop all docs other wise (kinda bad, but this isn't used frequently,
  will fix when visiting entire caching system again)
2022-10-06 19:46:58 +05:30
Ankush Menat
6aaefd6633
fix!: Dont update modified by default in db.set_value (#18301) 2022-10-06 16:01:20 +05:30
Ankush Menat
f96505fae0 feat: wrap read only mode SQL errors 2022-09-09 17:34:44 +05:30
Ankush Menat
3e2d2a703a test: Use FrappeTestCase everywhere 2022-08-17 16:39:42 +05:30
Gavin D'souza
d1fbab1c45 test(db): Add tests for untested db methods 2022-07-22 13:30:47 +05:30
Gavin D'souza
2bef29bb46 chore: Deprecate backup script + minor refactors 2022-07-22 13:21:29 +05:30
Gavin D'souza
bcfa8c276e fix: Set default port attribute for Database classes
db.default_port wil be available as a class attribute to hold defaults
for DB types.

Usage: frappe.conf.db_port or frappe.db.default_port
Why: I couldn't run the mariadb command because the defaults aren't set
for my system. server is remote / containerized. Setting port in
equivalent mysql command fixes this.
2022-07-22 13:16:26 +05:30
Gavin D'souza
63e618c709 test: More resilient tests for sequences 2022-07-22 12:11:01 +05:30
Gavin D'souza
71b5c77e6e Merge branch 'develop' of github.com:frappe/frappe into mariadb-client-refactor 2022-07-05 14:55:02 +05:30
Ankush Menat
81b37cb7d2
refactor: clean up code to py310 supported features (#17367)
refactor: clean up code to py39+ supported syntax

- f-strings instead of format
- latest typing support instead of pre 3.9 TitleCase
- remove UTF-8 declarations.
- many more changes

Powered by https://github.com/asottile/pyupgrade/ + manual cleanups
2022-07-01 11:51:05 +05:30
gavin
574f160d3c
Merge branch 'develop' into mariadb-client-refactor 2022-06-23 16:13:03 +05:30
phot0n
5e1007940c refactor(minor): move test_modify_query and test_modify_values to TestDB class 2022-06-18 10:55:40 +05:30
phot0n
215507a742 test: add list value in test_modify_values 2022-06-18 10:45:40 +05:30
Gavin D'souza
71ed8417d3 test: frappe.db.describe returns List[Tuple] through mariadb 2022-06-16 16:23:27 +05:30
Ankush Menat
359c7768f5 fix: multiple postgres transaction abort issues
- wrap setup fixtures in savepoint
- wrap scheduled_job_type in savepoint
- ignore duplicates where it's ignored by exc
- dont attempt to delete from deleted table
- delete custom field and commit - stale meta causes future inserts to
  insert unknown field.
2022-06-05 00:26:15 +05:30
Ankush Menat
8861752675 fix: restart transaction after commit 2022-06-05 00:26:12 +05:30
Aradhya Tripathi
7a0dc40693
feat!: Added support for IS in frappe qb (#16877)
Fixes #16869
Using 53148c3f90
2022-05-18 09:09:22 +00:00
Ritwik Puri
968343b302
Merge branch 'develop' into autoinc-customize-form-fix 2022-05-13 00:41:58 +05:30
phot0n
b1d61906d1 test: update and rename test_change_autoname
* chore: remove unnecessary decorators for skiprun
2022-05-13 00:03:19 +05:30
saxenabhishek
0450a25822 test: get_count 2022-04-28 17:10:31 +05:30
phot0n
6405b0510a chore: fix linter 2022-04-20 17:56:48 +05:30