Commit graph

1692 commits

Author SHA1 Message Date
Ankush Menat
fdba41c682
perf: misc client cache improvements (#29070)
* perf: Reduce penalty for lack of redis connection

If redis isn't running than this client cache is slower than default
implementation because of the extra locking overhead.

* test: update perf redis counts

* perf: cache table columns in client-cache

* fix: race condition on cache-client_cache init

Rare but apparant in synthetic benchmarks.

Cache is set but client cache is still being initialized then request
will fail.

* perf: Don't run notifications when loading document

WHAT?

* fix: use cached doc to repopulate

* perf: reduce get_meta calls
2025-01-07 16:14:43 +05:30
Ankush Menat
73dc99275d refactor: move get_system_settings
- This should not be in init.py
- set_single_value is not a good alternate for properly cleaning up cache.
2025-01-06 20:31:38 +05:30
Ankush Menat
b7ddd1df92 perf: cache hooks in client-side cache 2025-01-06 20:24:27 +05:30
Ankush Menat
b60a1292bd perf: store app-module mappings in client cache 2025-01-06 19:46:42 +05:30
Ankush Menat
7ea0421624 perf: cache system settings in client side cache 2025-01-06 19:40:07 +05:30
Ankush Menat
15f5adc25a refactor: use namedtuple for readability 2025-01-06 19:28:33 +05:30
Ankush Menat
0a85992331 fix: avoid clearing client cache for single doctype
Similar to recent _SITE_CACHE change, it's not required.

Properly cleared keys will get invalidated anyway.
2025-01-06 18:57:57 +05:30
Ankush Menat
e7139a1395 perf: store meta in client cache 2025-01-06 18:57:57 +05:30
Ankush Menat
b96b8c815d feat: Initate client-side caching 2025-01-06 18:57:57 +05:30
Ankush Menat
46dbb2f0cd
perf: Evict site cache only on frappe.clear_cache() (#29033)
For doctype/user specific cache eviction, no need to remove site_cache.

Rationale:
- Site cache is worker specific, so this eviction doesn't help much.
- Anything that might need to be evicted from site cache should be
  manually cleared or use a TTL.

Maybe we can just replace all of site_cache usage with
https://github.com/frappe/frappe/pull/28992 once it's stable.
2025-01-03 11:38:53 +05:30
Ankush Menat
c75febb5d4
fix: register fault hander after gunicorn registers signals (#28976)
Ugh, currently gunicorn is overriding our signal handler and we don't
care about gunicorn's SIGUSR1 handler.
2024-12-31 14:01:19 +00:00
Ankush Menat
766cb64d55
perf!: Cache site configs in memory for 60 seconds (#28869)
This is middle ground between caching it completely and requiring a
restart/signal to reload vs always reloading it.

I don't know any use cases that can break from this, nowhere in code
configs should be expected to reload instantly.

This change is only applied to requests for now
2024-12-27 16:21:14 +00:00
Ankush Menat
c4b8560247
fix: Ensure only one Redis connection is created (#28930)
There is no gaurantee that setup_cache is only called once. This PR adds
a mutex lock to ensure only one thread gets to create the connection. If
both arrive at same time then one of them will be blocked until
connection is setup.

So far this hasn't been an issue because the "orphan" connection would
just get garbage collected but if you setup any kind of listener on it
or refer to it then it will keep running forever hurting performance.

This just has small performance impact on first request that sets up the
connection, in absence of contention the lock should have almost no
overhead. I make up for it by eliminating one function call :pinch:
2024-12-27 12:55:32 +05:30
Ankush Menat
11a6dfb6a0
revert: bencher/bench intrface (#28925) 2024-12-26 14:41:36 +00:00
Ankush Menat
4f628ca091
fix: Never query flag_print_sql in developer_mode=0 (#28884)
Unnecessary overhead and need to disable this everytime I want to get
realistic performance numbers out.

All the performance affecting toggles should be directly controlled by
just `developer_mode` alone.
2024-12-23 13:57:01 +00:00
Ankush Menat
fe63af5449
refactor: make optimizations.py private entirely (#28872)
Avoids having to prefix everything with `_`.
2024-12-23 09:56:56 +00:00
Ankush Menat
197a49cf27
perf: speedup frappe.call by ~8x (#28866)
Before: 8.81us
After: 1.1us

Benchmarks in caffeine repo
2024-12-23 06:41:20 +00:00
Ankush Menat
17cc356915
perf: speed up flt by 1.06x and get_system_settings by 1.32x (#28841)
* perf: resolve rounding method once

When rounding method is explcitly specified it's 1.4x faster.

* perf: reorder checks

Bankers rounding is default and most common now

* perf: speedup get_system_settings
2024-12-19 14:38:45 +00:00
Ankush Menat
9e8ab92371
refactor: move all optimizations and pre/post fork hooks to separate file (#28832)
Now they will truly execute before/after fork = :pinch: few bytes saved!
2024-12-19 16:46:26 +05:30
Ankush Menat
6040145109 fix: Set some expiry for cached documents
IMO 1 cache miss per document is fine. This at least ensure that a
missed-invalidation won't cause a perpetual problem.
2024-12-18 17:26:50 +05:30
Ankush Menat
30ec033747
perf: Speedup get_doc by another ~1.5x (#28807)
* perf: Reduce impact of forced cache replacement on every doc access

* fix: clear cache before processing users

Note: This is just an artifact of testing model, this won't have any real
effect on execution in real system.

Basically `enqueue_on_commit` is not respected in tests and it can't be
practically supported either.
2024-12-17 16:48:43 +05:30
Ankush Menat
c1c4a7dd48
perf: speed up @request_cache by ~2x (#28803)
- Eagerly initialize request_cache, all requests use it, so what is the
  point of doing it lazily?
- Reduce accesses to `frappe.local` namespace, get cache once and reuse
  it in rest of the execution.

Before: 1250ns +/- 1%
After: 645ns +/- 1%

Source: Trust me bro.
(no really, for now just trust me or look at the diff)
2024-12-17 07:35:38 +00:00
Ankush Menat
5bf50b6bc2 perf: Avoid patching QB in every request 2024-12-17 10:58:31 +05:30
David Arnold
d17136cd04
fix: redirect cssutils logger to file (#28692) 2024-12-08 13:42:18 +05:30
David Arnold
fef569e284
refactor: simplify bencher (#28694)
* chore: flatten bencher class layout

* chore: rename bencher to clarify

* docs: add file level docstrings
2024-12-07 14:10:03 +00:00
David Arnold
3d71f594d8
refactor: bench class inauguration (#28158)
* feat: Add bench layout classes and configuration handler

Bench layout: (`frappe.bench`)

- Layout by env variable, e.g. FRAPPE_BENCH_PATH, FRAPPE_SITES_PATH, etc
- Detecting modules and apps by the presence of a sentinel .frappe file
- Site is scoped by frappe.local.site_name (thread safe)

Config handler: (`frappe.bench.sites{,.site}.config`)

- Optional config registry for better discovery; warning if not specced
- Env variable overload with `FRAPPE_` prefix

* chore: type frappe.config

* chore: type frappe.bencher

* chore: py310 compat
2024-12-06 19:07:34 +01:00
David Arnold
75377aaaf5
refactor(typing): type filters (#28218)
* chore(typing): type filters

* chore(typing): type filters for get_list et al

* fix: dashboard chart filter expression

* test: fix case with new-style right hand object to equality check

* chore: place new typed filter under typing verification

* chore: remove debug print statment

* chore: inverse logic of type guard

* fix: add float to filter value types

* chore: clarify value naming
2024-12-04 23:18:53 +00:00
Akhil Narang
84ef6ec677
refactor: fixup with ruff 0.8.1
Signed-off-by: Akhil Narang <me@akhilnarang.dev>
2024-12-04 13:18:04 +05:30
Faris Ansari
f7c005dc09
Merge pull request #28629 from netchampfaris/autoincrement-renaming
fix: allow renaming autoincrement documents
2024-12-02 14:25:05 +05:30
Faris Ansari
3b014a2548 test: rename autoincrement document 2024-11-30 16:49:04 +05:30
David Arnold
e2a8c7fed3
test: fix universal type checker based on downstream use and more testing (#28619)
* test: fix universal type checker based on downstream use and more testing

* fix: type validation error reporting

* fix: types; various

* chore: switch off test-time type checking

still too many errors
2024-11-28 23:35:32 +01:00
David Arnold
c7eb914679
chore: fixup type hint oversight (#28467) 2024-11-14 13:27:42 +00:00
David Arnold
268c33d3ed
fix(typing): hack-hint to type checkers for ide use (#28257) 2024-10-24 21:16:05 +02:00
David Arnold
8d6f8bce01
chore(typing): add some more typing to frappe.__init__ (#28215) 2024-10-21 11:30:11 +00:00
David Arnold
f7a7cef54a
chore: split custom types into types module (#28204) 2024-10-20 23:01:36 +00:00
Akhil Narang
ff0fe1228e
Merge pull request #28118 from Priyansh121096/fix
refactor: Replace hard-coded pg default port with PostgresDatabase.default_port
2024-10-14 16:00:20 +05:30
Priyansh Agrawal
92a0ffb191 refactor: Replace hard-coded pg default port with PostgresDatabase.default_port 2024-10-13 18:00:48 +00:00
Ankush Menat
a25df17efb
Revert "refactor: Replace hard-coded pg default port with PostgresDatabase.de…" (#28117)
This reverts commit cfb04b93e1.
2024-10-13 13:05:50 +00:00
David Arnold
bd07ae994d
feat: may copy MappingProxyType (#28083) 2024-10-11 04:01:23 +00:00
David Arnold
443c38daa9
refactor: toml test records for readability (#28065)
* refactor: toml test records for readability

* fix: maintain backwards compatibility

* refactor: transform in-tree records

* chore: don't use deprecated functions (treewide)

* chore: revert migration of tests which depend on old test records list

* feat: add cls.globalTestRecords on IntegrationTestCase
2024-10-10 13:24:02 +02:00
David Arnold
8cfeb156df
devx: add deprecation dumpster (#27887)
* feat: Add deprecation_dumpster.py file

* docs: add jovial and jocose docstring for frappe/deprecation_dumpster.py

* refactor: fill the dumpster with its own kind

* refactor: move to the deprecation dumpster

* chore: color coding class

* fix: only check import error when import errors
2024-10-08 18:56:10 +02:00
David Arnold
c2c9d9062a
Testing Improvements 3 (#27995)
* feat: set doctype on test classes

* refactor: Transform `make_test_records` into a generator

* feat: lazy create doctype records on first use

* perf: improve file walker

* fix: submission queue test

* refactor: improve logging a bit

* fix: global records install for app (semifix)
2024-10-06 15:04:47 +00:00
Priyansh Agrawal
cfb04b93e1
refactor: Replace hard-coded pg default port with PostgresDatabase.default_port (#27989) 2024-10-05 17:59:26 +00:00
Akhil Narang
c044680c3f
Merge pull request #27505 from dj12djdjs/fix-link-title-href
fix: show link title in generated form hrefs
2024-09-13 18:17:15 +05:30
Rushabh Mehta
ddee23f5ad fix(ui): sortable sidebar, fixes to breadcrumbs etc 2024-09-06 14:12:36 +05:30
Devin Slauenwhite
a2f4013e1e fix: show link title in generated form hrefs 2024-08-22 20:35:25 -04:00
Rushabh Mehta
1d678146a9 fix(styles): cleaner sidebars for list and form 2024-08-14 16:13:57 +05:30
Philipp Gruener
7df4185f87 fix: removed TODO marks 2024-07-08 15:16:51 +02:00
Philipp Gruener
1b56b4f3f7 fix: Reset to orig env vars in specs 2024-07-05 15:44:04 +02:00
Philipp Gruener
abba28be3b feat: Added env db options for db, password and pg_schema 2024-07-05 12:05:20 +02:00