Comparing varchar field with ints makes indexes unusable in MariaDB.
This PR is just one small fix for DB query, similar fixes won't be made
for DB APIs which do not assume anything about database schema.
Applying something basic like `order id like xyz` filter can result in
full table scan on years of data. This isn't necessary most of the time.
Interactive queries are usually done on recent data, so add this filter
to ensure only recent data is checked first.
Users can remove this filter if they want to.
This feels overengineered and it kinda is, but other efforts to
inroduce sequential naming/UUID naming haven't been that fruitful
either.
10 character random "hash" i now changed to.
1. first character - last character in UUID4 ID of request/job
2. three characters - derived from current timestamp.
4. 6 characters - random data.
This satisfies all three requirements:
1. Readers - temporal locality should result in spatial locality on disk. (fewer pages accessed)
2. Single writer - temporal locality should result in spatial locality. (fewer dirty pages)
3. Multiple writers - temporal locality should NOT result in spatial locality. (less lock contention)
Mostly concludes https://github.com/frappe/frappe/pull/25309 and https://github.com/frappe/frappe/pull/28349
Rough probabiliy numbers
Assumptions:
- Unique per worker prefix - 16 (uuid's base16 version)
- Rough time spent generating names - 10% of request (very very conservative estimate)
Probability(collision) = P(at least one prefix collision) * P(time collision)
Probability(collision) = (1 - p(all different)) * 10%
Probability(collision) = (1 - (16! / 16-N! )/ 16^N ) * 10%
| N (concurrency) | Probability(collision) |
| 1 | 0.0% |
| 2 | 0.6% |
| 3 | 1.8% |
| 4 | 3.3% |
| 5 | 5.0% |
| 6 | 6.6% |
| 7 | 7.9% |
| 8 | 8.8% |
Steps to reproduce:
- enable developer mode (doesn't happen in prod)
- Save a document with set only once fields
- Reload the page (requests meta again which is now polluted)
This is new category of bug surfaced because meta objects now live
longer than request and all kinds of weird `self._cached_property`
starts getting serialized.
Co-Authored-By: ruthra kumar <ruthra@erpnext.com>
* perf: Reduce penalty for lack of redis connection
If redis isn't running than this client cache is slower than default
implementation because of the extra locking overhead.
* test: update perf redis counts
* perf: cache table columns in client-cache
* fix: race condition on cache-client_cache init
Rare but apparant in synthetic benchmarks.
Cache is set but client cache is still being initialized then request
will fail.
* perf: Don't run notifications when loading document
WHAT?
* fix: use cached doc to repopulate
* perf: reduce get_meta calls
These are deletes that aren't user triggered and these documents are
typically never "linked" somewhere else. So skip all expensive link /
dynamic link checks.
Note about correctness: Once site has seen enough usage this map will rarely change. So the
problem of "cache inconsistency" is very rare, still care is taken to
avoid possible cache inconsistencies.
* perf: resolve rounding method once
When rounding method is explcitly specified it's 1.4x faster.
* perf: reorder checks
Bankers rounding is default and most common now
* perf: speedup get_system_settings
* perf: Use latest pickle protocol
* perf: pop flags from cached documents
This is also the right thing to do, things like `doc.flags.for_update`
shouldn't be "cached".
`get_doc` is single most used commands, directly and indirectly.
Currently there's QB overhead of building queries.
These queries are SO SIMPLE, I don't believe we need QB to keep these
maintainable or compatible with databases.
Microbenchmarks (first is simple flat document, second has many child tables):
```
bench_orm_bench_get_doc: Mean +- std dev: [before] 10.7 ms +- 0.2 ms -> [after] 7.24 ms +- 0.15 ms: 1.48x faster
bench_orm_bench_get_user: Mean +- std dev: [before] 16.3 ms +- 0.2 ms -> [after] 9.50 ms +- 0.14 ms: 1.72x faster
```
* chore(typing): type filters
* chore(typing): type filters for get_list et al
* fix: dashboard chart filter expression
* test: fix case with new-style right hand object to equality check
* chore: place new typed filter under typing verification
* chore: remove debug print statment
* chore: inverse logic of type guard
* fix: add float to filter value types
* chore: clarify value naming
