This feels overengineered and it kinda is, but other efforts to
inroduce sequential naming/UUID naming haven't been that fruitful
either.
10 character random "hash" i now changed to.
1. first character - last character in UUID4 ID of request/job
2. three characters - derived from current timestamp.
4. 6 characters - random data.
This satisfies all three requirements:
1. Readers - temporal locality should result in spatial locality on disk. (fewer pages accessed)
2. Single writer - temporal locality should result in spatial locality. (fewer dirty pages)
3. Multiple writers - temporal locality should NOT result in spatial locality. (less lock contention)
Mostly concludes https://github.com/frappe/frappe/pull/25309 and https://github.com/frappe/frappe/pull/28349
Rough probabiliy numbers
Assumptions:
- Unique per worker prefix - 16 (uuid's base16 version)
- Rough time spent generating names - 10% of request (very very conservative estimate)
Probability(collision) = P(at least one prefix collision) * P(time collision)
Probability(collision) = (1 - p(all different)) * 10%
Probability(collision) = (1 - (16! / 16-N! )/ 16^N ) * 10%
| N (concurrency) | Probability(collision) |
| 1 | 0.0% |
| 2 | 0.6% |
| 3 | 1.8% |
| 4 | 3.3% |
| 5 | 5.0% |
| 6 | 6.6% |
| 7 | 7.9% |
| 8 | 8.8% |
We have limited count to 1000 by default but this can still easily read
entire table if:
- There are filters and filters don't use index and there aren't 1000
matching rows.
This unknown tail latency by default is not acceptable, so by default
any count query that fails to return results in 2 seconds will be
terminated and user will have to explicitly ask for the count.
* fix: update instead of extend
None of these are supposed to be extended over defaults.
* feat: custom response header support
* refactor: use response_headers to set cache control headers
* feat: Decorator to cache API response using cache-control headers
* perf: cache notifications for 1 minute
+ SWR for 5 minutes
* perf: cache `get_events` on desk load
* perf: slow down auto-refresh even more
once every 5 seconds instead of 2 seconds.
* perf: Cache plain link validation for 30 minutes
Very often you're picking same documents again and again, there's no
need to validate them.
Also, document is JUST selected using search_link, so it's 99%
guaranteed to be valid.
The real purpose of this function is to provide "fetch from" feature,
not link validation like the name suggests.
It will get validated server side anyway.
* fix: Never use HTTP cache in developer mode
* perf: cache "is_document_amended"
A document that is amended from something stays amended.
* perf: proxy-cache website_script.js
* fix: only cache if called directly
Very often you're picking same documents again and again, there's no
need to validate them.
Also, document is JUST selected using search_link, so it's 99%
guaranteed to be valid.
The real purpose of this function is to provide "fetch from" feature,
not link validation like the name suggests.
It will get validated server side anyway.
* fix: remove whitespace in restrict ip in validate
* fix: added check for request_ip
* fix: return if no restrict ip
* fix: set to localhost if none, refactor validate_ip_addr
* fix: validate ip_address cleanup and removed uncessary comments
* fix: validate ip_addr cleanup
* fix: remove unecessary check
This broke for some cases because people had doc permissions, but `has_website_permission()` returns False
Signed-off-by: Akhil Narang <me@akhilnarang.dev>
This allows for caching the count which are frequently requested but
rarely change and even if they changed, UI doesn't show accurate value
to begin with.
After this change any filters that result in "1,000+" response will be
just cached by client and won't be requested again and again (for at
least 30 minutes, it's SWR after that so UI should still be snappy)
* refactor: Trim `__init__.py`
We strayed from the goal here: https://github.com/frappe/frappe/pull/29181
So making up for it.
* chore!: Delete unused get_version
Not used anywhere
* refactor: move few more functions to relevant place
* asd
* perf: faster `LocalProxy`
* refactor: use callable style local
* test: add some tests for local proxy override
---------
Co-authored-by: Ankush Menat <ankush@frappe.io>
