Commit graph

86 commits

Author SHA1 Message Date
Gavin D'souza
aae0cd896e
refactor: frappe.cache.set_value
redis.set accepts ex kwarg, etc
2024-07-12 19:55:38 +02:00
Ankush Menat
724d886f88
perf: Reduce 1 redis call while dumping monitor logs (#26337) 2024-05-06 12:53:36 +00:00
Ankush Menat
b7a5884344 perf: import sentinal locally 2024-03-15 11:31:04 +05:30
Revant Nandgaonkar
77618cde1f
feat: connect to redis sentinel for redis cache (#25398) 2024-03-14 13:54:46 +05:30
Akhil Narang
648828c3b5
fix(redis_wrapper): handle redis not being connected during cache clearing
Ignore the redis connection errors, we can still proceed here

Signed-off-by: Akhil Narang <me@akhilnarang.dev>
2024-02-21 11:26:57 +05:30
Akhil Narang
5343bcb756
fix(redis_wrapper/hdel): don't pipeline if we can avoid it
For a single deletion it doesn't make much sense, so just directly delete and return

Signed-off-by: Akhil Narang <me@akhilnarang.dev>
2024-02-19 14:28:35 +05:30
Akhil Narang
7c82115f5b
refactor(redis): DEL -> UNLINK
This is beneficial since unlink is non blocking, it will run in a different thread if the data passed is large
https://redis.io/commands/unlink/

Signed-off-by: Akhil Narang <me@akhilnarang.dev>
2024-02-19 14:28:35 +05:30
Akhil Narang
aa38d9e2f4
refactor: pipeline wherever possible, optimize calls
Signed-off-by: Akhil Narang <me@akhilnarang.dev>
2024-02-19 14:28:35 +05:30
Akhil Narang
26ae0f3460
fix: ruff fixes
Signed-off-by: Akhil Narang <me@akhilnarang.dev>
2024-02-07 17:04:31 +05:30
Hussain Nagaria
8d2137c265 docs: consistent doc strings 2023-12-18 18:27:39 +05:30
Ankush Menat
fa6dc03cc8
refactor: frappe.cache() usage to frappe.cache (#21282) 2023-06-08 11:47:17 +05:30
Ankush Menat
40b3cd82bc
feat!: frappe.cache() -> frappe.cache (#21279)
This is more intuitive and consistent with other things like `frappe.db`.

PS: This is quite likely to break some weird usage which I can't guess right now. Normal usage inside request/job cycles will continue to work as it used to.
2023-06-07 22:47:45 +05:30
Ankush Menat
26722b1a1c fix: ignore ConnectionError in frappe.cache().exists() 2023-06-03 22:53:10 +05:30
Ankush Menat
7d50ef19d3 perf: Delete multiple keys in O(1) redis calls
Currently we call redis for each key, redis already supports deleting
multiple keys in one go.
2023-06-03 22:53:10 +05:30
Saqib Ansari
32dbbb47bf
feat: redis cache decorator (#20452)
* feat: redis cache decorator

* fix: review changes

* fix: remove unintentional changes

* fix: remove unintentional changes

* refactor: cleanup and simplify code for redis

AIs suck

* fix: bug

* test: redis cache

* fix: remove unused import

* feat: make redis cache user specific

redis cache utils already support this, extending so everyone can use it

* feat: support @redis_cache without params

* test: flake in request site cache test

---------

Co-authored-by: Ankush Menat <ankush@frappe.io>
2023-03-27 17:03:20 +05:30
Sagar Vora
05c03a9345 chore!: remove cache_locally parameter 2023-01-21 17:32:24 +05:30
Daizy Modi
fce9ccedaa
fix: only execute generator if value is not found in redis cache (#18472)
* fix: use of generator in

* fix: improve docstring

* fix: improve docstring

* fix: directly assign value to flags

Co-authored-by: Daizy <DaizyModi>
2022-10-20 16:48:03 +05:30
Ankush Menat
1f089f44f6 feat: RedisearchWrapper
Like rediswrapper but for redisearch to allow multitenancy.
2022-08-31 17:04:31 +05:30
Ankush Menat
81b37cb7d2
refactor: clean up code to py310 supported features (#17367)
refactor: clean up code to py39+ supported syntax

- f-strings instead of format
- latest typing support instead of pre 3.9 TitleCase
- remove UTF-8 declarations.
- many more changes

Powered by https://github.com/asottile/pyupgrade/ + manual cleanups
2022-07-01 11:51:05 +05:30
Sagar Vora
b7514a05ca fix(redis): pass shared param when setting value based on generator 2022-06-28 00:37:31 +05:30
Sagar Vora
99388652c9 chore(redis): remove unnecessary exception handling 2022-06-27 01:56:58 +05:30
Sagar Vora
df28f32b8b
perf: ~90% faster frappe.get_cached_doc (#17107) 2022-06-08 15:49:11 +05:30
Ankush Menat
111df3a8d7 perf: improve document caching
As per current implementation whenever `get_doc` is called, document is
cached. However, this cache is only ever used by `get_cached_doc`. Going
through the codebase of both FF/ERPNext you'll find that `get_doc` is
used a lot more than `get_cached_doc`. So in many places, all this
caching overhead is unnecessary.

This change removes implicit caching from get_doc and replaces it with
cache-replacement instead. i.e. cache is only updated if it exists but
not created from get_doc.

Pros:
- faster `get_doc`
- lower memory usage on Redis
- Reduces chances of OOM by blowing up worker's memory as old docs can't
  be GCed until
- Correctness i.e. caching only what gets used from cache.

Con:
- After this change. First call to `get_cached_doc` will always be a cache miss. DUH.
2022-06-06 16:18:23 +05:30
Ankush Menat
d3b366f147 feat: cache_locally to limit caching in local.cache 2022-06-06 16:18:23 +05:30
Suraj Shetty
c0c5b2ebdd
style: format all python files using black (#16453)
Co-authored-by: Frappe Bot <developers@frappe.io>
2022-04-12 10:59:25 +05:30
Suraj Shetty
42643c3faa fix: Follow FIFO while inserting global search record
the latest enqueued value should override the value of the existing document
2022-04-01 18:47:05 +05:30
barredterra
745297a49d refactor: a not in b
Search: if not ([\w\d]*?) in ([\w\d]*?)
Replace: if $1 not in $2
2022-02-21 19:54:34 +01:00
barredterra
dac9349aef refactor: use is bool instead of == bool 2022-01-17 15:40:33 +01:00
Gavin D'souza
3446026555 chore: Update header: license.txt => LICENSE
The license.txt file has been replaced with LICENSE for quite a while
now. INAL but it didn't seem accurate to say "hey, checkout license.txt
although there's no such file". Apart from this, there were
inconsistencies in the headers altogether...this change brings
consistency.
2021-09-03 12:02:59 +05:30
Gavin D'souza
793b5638d3 style: Fix E201, E202, E401, E701
reported by sider: https://sider.review/gh/repos/1864194/pulls/13344?statuses#issues
2021-05-27 12:48:01 +05:30
Gavin D'souza
e407b78506 chore: Drop dead and deprecated code
* Remove six for PY2 compatability since our dependencies are not, PY2
  is legacy.
* Removed usages of utils from future/past libraries since they are
  deprecated. This includes 'from __future__ ...' and 'from past...'
  statements.
* Removed compatibility imports for PY2, switched from six imports to
  standard library imports.
* Removed utils code blocks that handle operations depending on PY2/3
  versions.
* Removed 'from __future__ ...' lines from templates/code generators
* Used PY3 syntaxes in place of PY2 compatible blocks. eg: metaclass
2021-05-26 15:31:29 +05:30
Faris Ansari
f8ca990a83 Merge remote-tracking branch 'upstream/develop' into esbuild 2021-05-16 11:26:43 +05:30
Gavin D'souza
b3af9f0c72 fix: Use raw strings for strings with \
Avoid DeprecationWarning which will turn into SyntaxError in later
Python versions
2021-05-07 17:59:20 +05:30
Faris Ansari
dd69f1ab43 fix: Hash based file naming
- For better HTTP caching and cache busting
- assets.json is created under [app]/dist folder which contains the map
of input file and output file name, this is used to get the correct path for
bundled assets
2021-04-29 13:30:07 +05:30
gregor-horvath
603a60e0a4
fix: py3 AttributeError decode
str has no attribute decode in python3, use cstr to convert to str
2020-10-08 21:12:09 +02:00
marty
256428572f resolve error redis_wrapper 2020-03-31 12:54:30 +00:00
Aditya Hase
af3c4feb64 feat: Monitor
Collect HTTP Request and Background Job logs
2020-03-05 15:06:15 +05:30
Rushabh Mehta
74e2e05162 fix(test): two_factor 2019-11-07 12:14:09 +05:30
Rushabh Mehta
463d1c5ec5 fix(major): upgrade redis-py and rq and try and fix scheduler tests 2019-11-07 12:14:09 +05:30
Rushabh Mehta
0704928fd8 fix(global search): fallback if redis is not yet started 2019-01-30 11:26:07 +05:30
Ameya Shenoy
d1d0edc636
Merge branch 'master' into staging-fixes 2018-11-23 08:07:36 +00:00
Aditya Hase
fc085a2b39 fix(redis-wrapper): Do not assume existence of superclass of redis.Redis
RedisWrapper inherits from Redis which inherits from StrictRedis

In redis-py 3.0.0 StrictRedis was renamed to Redis.

This seems like a harmless change, but, instead of using
`self(RedisWrapper ...`, all methods use `self(Redis ...`
which assumes previous hierarchy (i.e. RedisWrapper <- Redis <- StrictRedis)
2018-11-15 20:06:03 +05:30
Saurabh
3e7b387a7a [resolved] merge-conflicts 2018-09-12 14:52:46 +05:30
Ameya Shenoy
21bbbd2dbb
[feature] show popup for version update
- added a new weekly hook to check if new version update is available
- added function to desk.js to show popup in case a new version is
available
- added redis wrappers for set related commands, namely sadd, srem,
sismember, spop, srandmember, smembers
2018-08-15 20:34:48 +00:00
Aditya Hase
dca702d898 Allow multiple values to be passed as arguments to rpush and lpush (#5802) 2018-07-10 15:34:29 +05:30
Achilles Rasquinha
0fc518ace4 python 3 fixes 2018-03-31 18:15:43 +05:30
Achilles Rasquinha
0bf3411391 removed dill, revert to cPickle 2018-03-19 19:23:14 +05:30
Achilles Rasquinha
473d3eb6be Fixes for Python 3 (#4940)
* Fixed dict_keys and lists

* [FIX] config.get keys must be list

* [FIX] pickle all objects

* [FIX] get versions in unicode

* [FIX] get log versions

* debugging

* Fixed commit ID reference

* Fixed branch reference

* Fixed doc keys to list

* [LOG] test log

* [LOG] test log

* Convert iterators to list

* removed logs

* [FIX] Trial to load templates

* Fixed codacy
2018-01-31 10:51:13 +05:30
Aditya Hase
00c655b2f4 Decode string before passsing to regex.match 2017-08-22 17:20:35 +05:30
Aditya Hase
aac8822d05 Replaced cPickle import with six.moves.cPickle (#3858) 2017-08-04 10:53:21 +05:30