This was always broken apparantly. Cache key only contains the path so
querystring is being ignored. If request has query params then we
shouldn't cache it.
WIP - better solution for caching based on cache headers. Anyway this
caching isn't THAT helpful since #29170
This feels overengineered and it kinda is, but other efforts to
inroduce sequential naming/UUID naming haven't been that fruitful
either.
10 character random "hash" i now changed to.
1. first character - last character in UUID4 ID of request/job
2. three characters - derived from current timestamp.
4. 6 characters - random data.
This satisfies all three requirements:
1. Readers - temporal locality should result in spatial locality on disk. (fewer pages accessed)
2. Single writer - temporal locality should result in spatial locality. (fewer dirty pages)
3. Multiple writers - temporal locality should NOT result in spatial locality. (less lock contention)
Mostly concludes https://github.com/frappe/frappe/pull/25309 and https://github.com/frappe/frappe/pull/28349
Rough probabiliy numbers
Assumptions:
- Unique per worker prefix - 16 (uuid's base16 version)
- Rough time spent generating names - 10% of request (very very conservative estimate)
Probability(collision) = P(at least one prefix collision) * P(time collision)
Probability(collision) = (1 - p(all different)) * 10%
Probability(collision) = (1 - (16! / 16-N! )/ 16^N ) * 10%
| N (concurrency) | Probability(collision) |
| 1 | 0.0% |
| 2 | 0.6% |
| 3 | 1.8% |
| 4 | 3.3% |
| 5 | 5.0% |
| 6 | 6.6% |
| 7 | 7.9% |
| 8 | 8.8% |
We have limited count to 1000 by default but this can still easily read
entire table if:
- There are filters and filters don't use index and there aren't 1000
matching rows.
This unknown tail latency by default is not acceptable, so by default
any count query that fails to return results in 2 seconds will be
terminated and user will have to explicitly ask for the count.
* fix: update instead of extend
None of these are supposed to be extended over defaults.
* feat: custom response header support
* refactor: use response_headers to set cache control headers
* feat: Decorator to cache API response using cache-control headers
* perf: cache notifications for 1 minute
+ SWR for 5 minutes
* perf: cache `get_events` on desk load
* perf: slow down auto-refresh even more
once every 5 seconds instead of 2 seconds.
* perf: Cache plain link validation for 30 minutes
Very often you're picking same documents again and again, there's no
need to validate them.
Also, document is JUST selected using search_link, so it's 99%
guaranteed to be valid.
The real purpose of this function is to provide "fetch from" feature,
not link validation like the name suggests.
It will get validated server side anyway.
* fix: Never use HTTP cache in developer mode
* perf: cache "is_document_amended"
A document that is amended from something stays amended.
* perf: proxy-cache website_script.js
* fix: only cache if called directly
Very often you're picking same documents again and again, there's no
need to validate them.
Also, document is JUST selected using search_link, so it's 99%
guaranteed to be valid.
The real purpose of this function is to provide "fetch from" feature,
not link validation like the name suggests.
It will get validated server side anyway.
* fix: remove whitespace in restrict ip in validate
* fix: added check for request_ip
* fix: return if no restrict ip
* fix: set to localhost if none, refactor validate_ip_addr
* fix: validate ip_address cleanup and removed uncessary comments
* fix: validate ip_addr cleanup
* fix: remove unecessary check
