vassili/seitime-frappe
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1
54583 commits 1 branch 0 tags 587 MiB
Commit graph

134 commits

Author SHA1 Message Date
Saqib Ansari
53e7e34948 refactor: make token initialization simple 2026-04-22 16:20:54 +05:30
Saqib Ansari
850cc58664 fix: clear_cache for shared cache 2026-04-22 16:20:54 +05:30
Saqib Ansari
0064eb80b4 fix: support shared RedisSemaphores for concurrency limits 2026-04-22 16:20:54 +05:30
Saqib Ansari
e8c7eb946b refactor: rewrite concurrent_limit to use LIST + BLPOP semaphore 
Replace the INCRBY-based polling loop with a proper token pool backed by
a Redis LIST. BLPOP blocks until a token is available instead of sleeping
and retrying, which is more efficient and avoids the check-then-act race
of the old counter approach.

Other fixes bundled in:
- Add `blpop` and `setnx` wrappers to `RedisWrapper` so all key prefixing
  goes through `make_key` consistently
- Cache `_default_limit()` result with `@redis_cache(shared=True)` to
  avoid importing `multiprocessing` on every request
- Fix `limit=0` edge case: use `is not None` guard instead of falsy check
- Guard `_release()` against pushing the `"fallback"` token back into the
  pool when Redis was unavailable during acquire
 2026-04-18 14:21:33 +05:30
Ankush Menat
6091040e97
fix: Skip redis cache signal if redis is down (#32967) 
https://github.com/frappe/frappe/pull/32888#issuecomment-2975345660
 2025-06-17 13:59:31 +00:00
Ankush Menat
3167e03133 fix: invalidate persistent caches 2025-06-11 00:28:38 +05:30
Ankush Menat
12ab20f705 fix: Change local variable name 
Shouldn't be same as function name
 2025-06-10 23:52:27 +05:30
Ankush Menat
856513423f
fix: recorder auto-disable (#32296) 
This broke with client cache implementation... oopsie.
 2025-04-25 11:32:55 +00:00
Sagar Vora
f70c32c23e
fix: dont recheck healthy state after value is retrieved from client cache (#32099) 2025-04-12 18:16:02 +05:30
Ankush Menat
fe5be4b3d1
fix: Add a long timeout instead of indefinite blocking (#32090) 
Redis handles non-blocking calls differently and auto-recovers from
timeouts
 2025-04-11 13:05:25 +00:00
Sagar Vora
322312a154 perf: make redis key faster 2025-03-15 13:12:57 +05:30
Ankush Menat
6ea200a48f fix: handle typical client tracking errors 2025-01-25 10:15:50 +05:30
Ankush Menat
b6a0e849a0 fix: Avoid mixing client cache and redis_wrapper 2025-01-14 15:20:47 +05:30
Ankush Menat
b49d512404 feat: cache documents in client cache 
No need to query 1 key at a time.
 2025-01-14 14:33:10 +05:30
Ankush Menat
4d1ef02e25 feat: generator support for client cache 
Similar to redis, for ergonomics
 2025-01-14 13:55:28 +05:30
Ankush Menat
fab464effb feat: support shared keys in client cache 2025-01-14 13:52:29 +05:30
Ankush Menat
d46e2c20af
fix: Forward shared flag to cache generator (#29146) 
Right now assets.json is read on every requests because of this bug.
 2025-01-14 08:15:29 +00:00
Ankush Menat
fdba41c682
perf: misc client cache improvements (#29070) 
* perf: Reduce penalty for lack of redis connection

If redis isn't running than this client cache is slower than default
implementation because of the extra locking overhead.

* test: update perf redis counts

* perf: cache table columns in client-cache

* fix: race condition on cache-client_cache init

Rare but apparant in synthetic benchmarks.

Cache is set but client cache is still being initialized then request
will fail.

* perf: Don't run notifications when loading document

WHAT?

* fix: use cached doc to repopulate

* perf: reduce get_meta calls
 2025-01-07 16:14:43 +05:30
Ankush Menat
b9c0662de6 fix: Make client cache work without redis too 2025-01-07 13:46:47 +05:30
Ankush Menat
06816f2817 perf: client-cache for defaults 2025-01-07 12:51:10 +05:30
Ankush Menat
33bd1bc09b fix: Flushall shouldn't attempt reading keys 2025-01-07 11:56:22 +05:30
Ankush Menat
b5e2569c0c feat: statistics for cache performance 
These can be used for logging or debugging the behaviour of the cache.
 2025-01-07 11:47:52 +05:30
Ankush Menat
8dd4ad53e2 fix: data race between GET and INVALIDATE 
Fix for the documented example given here: https://redis.io/docs/latest/develop/reference/client-side-caching/#avoiding-race-conditions
 2025-01-07 11:09:42 +05:30
Ankush Menat
c14d416d19 docs: add usage / notes in code 2025-01-06 19:28:37 +05:30
Ankush Menat
15f5adc25a refactor: use namedtuple for readability 2025-01-06 19:28:33 +05:30
Ankush Menat
b51cfc1705 perf: minimize penalty for broken client cache 2025-01-06 18:57:57 +05:30
Ankush Menat
119af71ae3 refactor: variable names, force RESP2 2025-01-06 18:57:57 +05:30
Ankush Menat
98b1df7dac fix: guard all writes with an RLock 2025-01-06 18:57:57 +05:30
Ankush Menat
53f085e0f4 test: ttl and maxsize 2025-01-06 18:57:57 +05:30
Ankush Menat
55ae5615d0 fix: clear all meta cache only when doctype is not specified 2025-01-06 18:57:57 +05:30
Ankush Menat
f332852415 fix: start tracking keys set by us immediately 2025-01-06 18:57:57 +05:30
Ankush Menat
f74ada155e fix: Don't serve local cache values when it becomes unhealthy 2025-01-06 18:57:57 +05:30
Ankush Menat
45d414fe29 refactor: per-key TTL 2025-01-06 18:57:57 +05:30
Ankush Menat
21f0cda732 refactor: Reuse original cache connection 
Drops redis connection per worker from 3 to 2!

Nothing wrong with doing this.
 2025-01-06 18:57:57 +05:30
Ankush Menat
01cfa647a2 fix: set/del locally too 
This is required if we do NOLOOP
 2025-01-06 18:57:57 +05:30
Ankush Menat
e7139a1395 perf: store meta in client cache 2025-01-06 18:57:57 +05:30
Ankush Menat
cd47bee65e fix: Avoid data races by accessing request specific cache 
Imagine:
| client 1 | client 2 |
| ---      | ---      |
| rd(x)    |          |
|          |  wr(x)   |
| inv(x)   |          |
| rd(x)    |          |  <- This will end up using request specific cache!
 2025-01-06 18:57:57 +05:30
Ankush Menat
764c3134cb fix: expire client cache every 10 minutes 
We can be more aggressive later, this is a good starting parting.
 2025-01-06 18:57:57 +05:30
Ankush Menat
796e51df62 fix: limit # of keys by maxsize 
Picking 1024 keys assuming 4kb avg size. In practice most things are
smaller so 1024 should be good enough!

This will likely only affect multitenant deploy with many sites.
 2025-01-06 18:57:57 +05:30
Ankush Menat
590e7ff185 fix: Ensure that RESP3 is not in use 
One can set it in conf and it won't work as expected because pubsub
format is different
 2025-01-06 18:57:57 +05:30
Ankush Menat
67848aa920 fix: handle temporary disconnects 2025-01-06 18:57:57 +05:30
Ankush Menat
e21a7112e9 fix: flushall handler 
Also no need to check channel names?
 2025-01-06 18:57:57 +05:30
Ankush Menat
b96b8c815d feat: Initate client-side caching 2025-01-06 18:57:57 +05:30
Ankush Menat
416919e880
Revert "feat: Use RESP3 for Redis cache connections (#28929)" (#28938) 
This reverts commit 6f7b4c412e.
 2024-12-27 13:19:35 +00:00
Ankush Menat
6f7b4c412e
feat: Use RESP3 for Redis cache connections (#28929) 
RESP3 has PUSH support which is useful for implementing client side
caching. Enabling this before I work on that to test if anything breaks
with this.

No need to do this for background jobs instance just yet, infrequent
accesses and performance doesn't matter as much.
 2024-12-27 06:15:59 +00:00
Ankush Menat
7dd15e3613
perf: speedup pickling of document objects (#28823) 
* perf: Use latest pickle protocol

* perf: pop flags from cached documents

This is also the right thing to do, things like `doc.flags.for_update`
shouldn't be "cached".
 2024-12-18 10:18:04 +00:00
Ankush Menat
3297aff294 fix: Always set value in local even if it expires 2024-12-17 18:34:45 +05:30
Ankush Menat
d0c314090c
perf: speed up recurring redis cache accesses (#28805) 2024-12-17 14:06:01 +05:30
Gavin D'souza
aae0cd896e
refactor: frappe.cache.set_value 
redis.set accepts ex kwarg, etc
 2024-07-12 19:55:38 +02:00
Ankush Menat
724d886f88
perf: Reduce 1 redis call while dumping monitor logs (#26337) 2024-05-06 12:53:36 +00:00