vassili/seitime-frappe
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1
seitime-frappe/frappe
History
Ankush Menat 9b79dfeb7b
perf: "random" naming to improve concurrency and locality (#30053) 
This feels overengineered and it kinda is, but other efforts to
inroduce sequential naming/UUID naming haven't been that fruitful
either.

10 character random "hash" i now changed to.

1. first character - last character in UUID4 ID of request/job
2. three characters - derived from current timestamp.
4. 6 characters - random data.

This satisfies all three requirements:

1. Readers - temporal locality should result in spatial locality on disk. (fewer pages accessed)
2. Single writer - temporal locality should result in spatial locality. (fewer dirty pages)
3. Multiple writers - temporal locality should NOT result in spatial locality. (less lock contention)

Mostly concludes https://github.com/frappe/frappe/pull/25309 and https://github.com/frappe/frappe/pull/28349

Rough probabiliy numbers

Assumptions:
- Unique per worker prefix - 16 (uuid's base16 version)
- Rough time spent generating names - 10% of request (very very conservative estimate)

Probability(collision) = P(at least one prefix collision) * P(time collision)
Probability(collision) = (1 - p(all different)) * 10%
Probability(collision) = (1 - (16! / 16-N! )/ 16^N ) * 10%

| N (concurrency) | Probability(collision) |
| 1  |    0.0% |
| 2  |    0.6% |
| 3  |    1.8% |
| 4  |    3.3% |
| 5  |    5.0% |
| 6  |    6.6% |
| 7  |    7.9% |
| 8  |    8.8% |
2025-01-25 07:04:09 +00:00
..
api Revert "perf: Avoid checking server script map (#29057)" 2025-01-06 19:43:53 +05:30
automation perf: lower reminders frequency (#28931) 2024-12-27 07:37:25 +00:00
change_log
commands chore: added ipv6 to hosts file (#29199) 2025-01-16 19:23:32 +05:30
contacts fix(Contact): export company name and department in vCard (#29190) 2025-01-15 14:16:03 +00:00
core fix: remove whitespace from restrict ip and always check request_ip (#29867) 2025-01-24 13:12:58 +00:00
custom perf: skip perm checking on property setters (#29031) 2025-01-02 16:39:21 +00:00
data
database perf: misc client cache improvements (#29070) 2025-01-07 16:14:43 +05:30
desk perf: Set timeout in same query / avoid side effects 2025-01-25 10:22:08 +05:30
email fix: set login_id for sengrid 2025-01-24 12:20:55 +05:30
geo perf: Dont send useless translations 2025-01-14 15:46:21 +05:30
gettext refactor: fixup with ruff 0.8.1 2024-12-04 13:18:04 +05:30
integrations perf: speed up oauth bearer token query (#29184) 2025-01-15 13:24:46 +00:00
locale chore: update POT file 2025-01-19 09:33:11 +00:00
model perf: "random" naming to improve concurrency and locality (#30053) 2025-01-25 07:04:09 +00:00
modules fix(export_file): check paths before writing 2025-01-09 14:56:10 +05:30
patches fix(NamingPatch): convert numerics to str for regex match 2025-01-18 14:49:33 +05:30
printing fix: alignment Issues with Serial Numbers and Descriptions (#29093) 2025-01-13 11:50:01 +05:30
public fix(UX): change cursor for clickable list count 2025-01-25 11:07:30 +05:30
query_builder fix: datetime comparison in QB (#26364) 2024-05-09 10:19:27 +00:00
search refactor: change filter from tuple to dict 2024-11-26 11:38:15 +05:30
social revert: changes to energy point log 2024-11-05 12:25:33 +05:30
templates fix: alignment Issues with Serial Numbers and Descriptions (#29093) 2025-01-13 11:50:01 +05:30
testing chore: set default max_module_depth to 0 2024-12-04 17:19:02 +05:30
tests perf: "random" naming to improve concurrency and locality (#30053) 2025-01-25 07:04:09 +00:00
types perf: Make frappe._dict great again (#28824) 2024-12-18 16:36:31 +05:30
utils fix: handle typical client tracking errors 2025-01-25 10:15:50 +05:30
website refactor: use response_headers to set cache control headers 2025-01-24 18:35:26 +05:30
workflow fix: use get_email_template to support html template in workflow email 2024-10-24 14:43:39 +05:30
www fix: only cache if called directly 2025-01-24 19:08:04 +05:30
__init__.py feat: custom response header support 2025-01-24 18:35:26 +05:30
_optimizations.py fix(DX): add faulthandler for SEGV and other bad signals (#30046) 2025-01-24 14:24:54 +00:00
app.py feat: HTTP caching on API endpoints (#29997) 2025-01-24 20:04:57 +05:30
apps.py fix: consider default path for Website User if applicable 2024-08-30 15:42:58 +05:30
auth.py fix: remove whitespace from restrict ip and always check request_ip (#29867) 2025-01-24 13:12:58 +00:00
boot.py perf: cache unseen notes 2025-01-14 19:43:22 +05:30
build.py fix: Clear all cache after rebuilding assets (#29876) 2025-01-21 18:57:10 +05:30
cache_manager.py refactor: trim init.py (#29866) 2025-01-20 13:05:02 +00:00
client.py perf: cache "is_document_amended" 2025-01-24 18:58:39 +05:30
config.py perf!: Cache site configs in memory for 60 seconds (#28869) 2024-12-27 16:21:14 +00:00
coverage.py ci: make gh actions functions (#28299) 2024-10-27 02:09:02 +00:00
defaults.py perf: client-cache for defaults 2025-01-07 12:51:10 +05:30
deferred_insert.py fix: ruff fixes 2024-02-07 17:04:31 +05:30
deprecation_dumpster.py test: undo breaking change to test-fixture regeneration (#29034) 2025-01-03 11:44:12 +05:30
exceptions.py revert: bencher/bench intrface (#28925) 2024-12-26 14:41:36 +00:00
frappeclient.py refactor: fixup with ruff 0.8.1 2024-12-04 13:18:04 +05:30
handler.py Revert "perf: Avoid checking server script map (#29057)" 2025-01-06 19:43:53 +05:30
hooks.py Merge pull request #28767 from frappe/remove-billing-pages 2025-01-15 13:21:32 +05:30
installer.py perf!: Cache site configs in memory for 60 seconds (#28869) 2024-12-27 16:21:14 +00:00
locale.py feat: cache documents in client cache 2025-01-14 14:33:10 +05:30
middlewares.py
migrate.py fix: force kill all other DB connections before migrating 2025-01-10 14:05:29 +05:30
modules.txt
monitor.py perf: "random" naming to improve concurrency and locality (#30053) 2025-01-25 07:04:09 +00:00
oauth.py perf: Skip link checking on internal deletes 2025-01-06 11:48:19 +05:30
parallel_test_runner.py test: avoid flaky behaviour on test ordering 2025-01-04 11:54:37 +05:30
patches.txt chore: added patch to create missing keys and update counter 2024-12-09 11:32:50 +05:30
permissions.py perf: Skip link checking on internal deletes 2025-01-06 11:48:19 +05:30
push_notification.py chore: Add Push Notification Settings to Integrations workspace 2024-02-28 11:07:22 +05:30
rate_limiter.py perf: No need to set expiry for rate limiter key everytime (#28956) 2024-12-30 07:31:47 +00:00
realtime.py refactor: better permission checking on sockets 2024-04-06 21:14:48 +05:30
recorder.py refactor: minor cleanup / avoid repeat calls 2025-01-06 18:57:57 +05:30
sessions.py refactor: trim init.py (#29866) 2025-01-20 13:05:02 +00:00
share.py style: re-format with ruff 2024-02-05 18:53:33 +05:30
test_runner.py refactor: organize test contextmanagers (#28041) 2024-10-09 02:09:19 +02:00
translate.py chore: remove unused method (#29163) 2025-01-14 21:39:09 +00:00
twofactor.py chore(twofactor): specify key to display during decryption failures 2024-09-16 13:00:38 +05:30