Commit graph

628 commits

Author SHA1 Message Date
Ankush Menat
bc63c17287
perf: don't load doc before save for child tables (#32770)
Consider this:

```python
for row in doc.get_children():
    row.db_set("amount", 0)
```

This sounds like it will do one write query for each row but it does 2
because of this unnecessary locking of child tables.
2025-06-04 07:52:37 +00:00
Ankush Menat
78f7fa1429
perf: hoist constant computations for child table column order (#32677)
These don't/shouldn't change during execution of the function.
2025-05-26 19:39:29 +05:30
Ankush Menat
e2d619504f
perf: batch bulk_insert (#32675)
* fix: reduce bulk insert batch size

Back when this feature was added it used to lazily evaluate the input.
Now the iterator is consumed upfront so large batch sizes == huge memory usage.

* perf: bring back iterator for bulk_insert

Bulk insert used to support iterator for consuming arbitrarily large
amount of data and inserting it. Since child table support was added, it
can't do it anymore because that requires collecting values.

This change now brings back iterators by batching input iterator (by
default 1000) documents.

This is almost as good as original change from design POV. Performance
is still meh for flat documents.
2025-05-26 13:36:53 +00:00
Ankush Menat
9920c8d033 fix: duplicate follow
This exists in `doc.insert()` not need to add it here too
2025-05-26 14:43:06 +05:30
Ankush Menat
c2d2fda212 perf: avoid clearing cache for newly imported documents 2025-05-26 14:42:54 +05:30
Ankush Menat
d3c88556bc perf: Skip version checking on new docs 2025-05-26 14:38:26 +05:30
Ankush Menat
cc1e843260 perf: Skip realtime notifications while importing 2025-05-26 14:31:46 +05:30
Akhil Narang
33dbf97db2
fix(document): don't try to overwrite name if a string is already set
This allows us to pass `name` in via API when creating documents.

Signed-off-by: Akhil Narang <me@akhilnarang.dev>
2025-05-05 12:24:58 +05:30
Sagar Vora
f93b87ddd8 fix(Document): improve __onload implementation 2025-05-03 20:35:52 +05:30
Ankush Menat
266ec9f76e
perf: Fix child table queries for int-PK parent (#32293)
Lessen but not eliminate impact of https://github.com/frappe/frappe/issues/32287
2025-04-25 05:33:07 +00:00
Ankush Menat
d998d520fb
perf: use raw query for reading int-pk docs too (#32286)
Currently only string PK is supported but that wasn't the intention,
isinstance check is only there to avoid dicts/filters.
2025-04-24 13:11:07 +00:00
Akhil Narang
9e6a73f7c2
Merge pull request #31156 from akhilnarang/sqlite
feat: sqlite support
2025-04-16 08:56:52 +05:30
Ankush Menat
56e2a9ad2d
Revert "fix(HTML Editor): extraction of images" (#32129) 2025-04-15 22:21:40 +05:30
Akhil Narang
ad32216040
fix: support sqlite
Signed-off-by: Akhil Narang <me@akhilnarang.dev>
2025-04-15 13:59:16 +05:30
Ankush Menat
eb79ecaca9
chore: Revert document read only mode (#32102)
We can trust mappers to do the right thing, restricting few methods
doesn't prevent anything real anyway, unnecessary overhead and breakages.
2025-04-12 18:27:16 +05:30
Sagar Vora
bd844579d2 refactor: trim frappe.__init__ 2025-04-06 10:30:23 +05:30
gavin
554c76aa13
Merge pull request #31706 from gavindsouza/fix-add_viewed
refactor: doc.add_viewed
2025-03-17 09:07:47 +01:00
Sagar Vora
09d0abcb54 fix: simplify load_from_db logic 2025-03-15 16:14:55 +05:30
Gavin D'souza
dc807832fe
refactor: doc.add_viewed
* Don't check for unique_views if not needed - one less query/table scan
* Remove unnecessary indents
* Fix docstring
2025-03-13 11:11:47 +01:00
Sagar Vora
146d50e5d4 chore: fix typo in new DocStatus constant 2025-03-05 08:29:47 +05:30
Sagar Vora
99574a74e2
Merge pull request #31400 from sagarvora/better-table-fieldnames 2025-02-25 10:43:29 +05:30
Ankush Menat
efee329adf
fix: Unlock old locks automatically (#31411) 2025-02-24 13:49:08 +00:00
Sagar Vora
26b5d8de15 perf: better _table_fieldnames cache 2025-02-24 17:41:00 +05:30
Sagar Vora
b381cc4874
Merge branch 'develop' into ref-docstatus 2025-02-24 15:22:06 +05:30
Sagar Vora
be5e856dd9
Merge pull request #31391 from sagarvora/perf-nosingledispatchmethod 2025-02-24 15:21:05 +05:30
Sagar Vora
bb1080dfac perf: dont use singledispatchmethod 2025-02-24 15:11:14 +05:30
Sagar Vora
fcf657ba80 perf: improved DocStatus API and other minor improvements 2025-02-24 12:51:10 +05:30
Ankush Menat
37da1d43bd
fix: remove duplicate clear_cache calls (#31374)
This is internally handled in clear_document_cache

Fix was only required for websitegenerator specific things.
2025-02-21 08:54:05 +00:00
Ankush Menat
4997e5d223
fix: clear cache after db transaction ends (#31315)
This reduces probability of stale cache, but it's still very easily
possible because of repeatable read (!)
2025-02-19 11:44:37 +00:00
Sagar Vora
f4062b4d7a fix: ensure consistent error in response 2025-02-19 12:10:59 +05:30
Ankush Menat
e3ce05b57e
perf: use client cache for notifications (#31094) 2025-02-04 09:24:30 +00:00
Ankush Menat
fdba41c682
perf: misc client cache improvements (#29070)
* perf: Reduce penalty for lack of redis connection

If redis isn't running than this client cache is slower than default
implementation because of the extra locking overhead.

* test: update perf redis counts

* perf: cache table columns in client-cache

* fix: race condition on cache-client_cache init

Rare but apparant in synthetic benchmarks.

Cache is set but client cache is still being initialized then request
will fail.

* perf: Don't run notifications when loading document

WHAT?

* fix: use cached doc to repopulate

* perf: reduce get_meta calls
2025-01-07 16:14:43 +05:30
krantheman
bb38c62642 fix(HTML Editor docfield): extraction of images 2024-12-23 14:55:05 +05:30
Ankush Menat
17cc356915
perf: speed up flt by 1.06x and get_system_settings by 1.32x (#28841)
* perf: resolve rounding method once

When rounding method is explcitly specified it's 1.4x faster.

* perf: reorder checks

Bankers rounding is default and most common now

* perf: speedup get_system_settings
2024-12-19 14:38:45 +00:00
Ankush Menat
5e4f23720a
perf: Use raw SQL for fetching documents (#28789)
`get_doc` is single most used commands, directly and indirectly.
Currently there's QB overhead of building queries.

These queries are SO SIMPLE, I don't believe we need QB to keep these
maintainable or compatible with databases.

Microbenchmarks (first is simple flat document, second has many child tables):

```
bench_orm_bench_get_doc: Mean +- std dev: [before] 10.7 ms +- 0.2 ms -> [after] 7.24 ms +- 0.15 ms: 1.48x faster
bench_orm_bench_get_user: Mean +- std dev: [before] 16.3 ms +- 0.2 ms -> [after] 9.50 ms +- 0.14 ms: 1.72x faster
```
2024-12-16 13:27:07 +00:00
barredterra
beab110ce9 fix: clarify error message for child tables 2024-12-10 17:06:55 +01:00
Akhil Narang
84ef6ec677
refactor: fixup with ruff 0.8.1
Signed-off-by: Akhil Narang <me@akhilnarang.dev>
2024-12-04 13:18:04 +05:30
l0gesh29
35d8ff40ec feat: validate amended from record docstatus 2024-11-28 17:04:44 +05:30
David Arnold
6f35a554a5
fix: make read only mode thread safe (#28359)
* fix: Apply read_only_method decorator to Document methods

* fix: update tests for read-only document context manager

* refactor: place mappers into read-only mode

Reapply "refactor: place mappers into read-only mode"

This reverts commit a8208d57069c63f0982bf2881bcad28a4b349f3c.

read-only mode is now thread safe
2024-11-18 15:00:50 +01:00
Raffael Meyer
69a8a362f6
fix: improve permission error message (#28292) 2024-11-06 12:28:09 +01:00
David Arnold
d5fd8d7c20
chore(docref): fix circular imports (#28282) 2024-10-24 22:31:12 +02:00
David Arnold
90e44d950e
fix: hashable is enough to serve as (composite) cache key (#28189) 2024-10-19 23:19:48 +02:00
David Arnold
2abba7b51b
fix: don't force values into the string type (#28185) 2024-10-19 19:00:25 +00:00
David Arnold
7348572af8
feat: docref identifier / proxy (#27973)
* feat: add DocRef

* feat: Add comprehensive test cases for DocRef functionality

* chore(db): add field type hints

* fix: ensure document stringer fulfills the DocRef contract
2024-10-19 09:40:26 +05:30
Gavin D'souza
b1d96dd532
refactor: Move children load db into separate method
Signed-off-by: Gavin D'souza <gavin.dsouza@switchup.de>
2024-10-17 19:18:59 +02:00
David Arnold
0204db6547
chore: move to use new test record api (#28105) 2024-10-12 23:13:41 +00:00
David Arnold
5d3697500e
refactor: improve maintainability with a simple dispatcher (#27975)
* refactor: improve maintainability with a simple dispatcher

* refactor: improve maintainability with a init dispatcher on Document

* refactor: improve maintainability with an init dispatcher on meta
2024-10-06 16:56:40 +00:00
Mohamud Amin Ali
e6223b04fc
refactor: Explicit feedback to timestamp mismatch errror
Not necessarily a needed improvement but it adds explicitness to the feedback.

fix: text order

fix: type
2024-10-04 11:46:39 +02:00
David
1c4a0fe54f
feat: add read only document mode 2024-10-02 12:02:52 +02:00
barredterra
4f3b096f64 fix: translatable strings in doc.validate_value 2024-07-24 18:38:22 +02:00