Commit graph

1565 commits

Author SHA1 Message Date
Akhil Narang
eb45da3913
feat: Allow usage of print() within safe_exec() (#23084)
* feat(safe_exec): allow usage of `print()`

Signed-off-by: Akhil Narang <me@akhilnarang.dev>

* refactor(system_console): update description to mention `print()` instead of `log()`

Signed-off-by: Akhil Narang <me@akhilnarang.dev>

* feat: unconditionally add debug logs to response if present

Signed-off-by: Akhil Narang <me@akhilnarang.dev>

* chore(safe_exec): add in a test for running `print()` within safe_exec

Signed-off-by: Akhil Narang <me@akhilnarang.dev>

* fix(safe_exec): ignore warning

RestrictedPython warns us if we call `print()` don't use their `printed` variable

Signed-off-by: Akhil Narang <me@akhilnarang.dev>

* feat: store debug logs from scheduled jobs

Signed-off-by: Akhil Narang <me@akhilnarang.dev>

* fix: avoid ignoring warnings, disabled in prod anyway

* chore: remove unnecessary logging

This can be moved to level 2 when required

---------

Signed-off-by: Akhil Narang <me@akhilnarang.dev>
Co-authored-by: Ankush Menat <ankush@frappe.io>
2023-11-20 12:45:41 +05:30
Ankush Menat
089985e53b feat: frappe.job local for reading job info 2023-11-04 14:06:31 +05:30
Raffael Meyer
f0155674a5
refactor!: remove redundant sending of translations (#22962)
* chore!: remove redundant translation

* refactor!: remove "get_translated_dict" hook

Instead, include the results from
`frappe.geo.country_info.get_translated_dict` in `get_all_translations`.

* test: money_in_words uses translated currency

... and now we have translations.

* Revert "test: money_in_words uses translated currency"

This reverts commit e07d594ac1ce4b120c0fceb5d23377cb981fe820.

* refactor!: stop translating timezone and currency

* refactor: rename `get_translated_dict`

... to `get_translated_countries`
2023-11-02 20:04:14 +05:30
Ankush Menat
08b92858a3
fix: ignore duplicate perm check on assign hooks (#22832)
* fix: Ignore permissions while assigning if flag set

* fix: Avoid double permission checks on assignment rule

When it's triggered via doc events either:
- Permission check is done or
- Permission checks are not applicable
2023-10-20 06:18:19 +00:00
Corentin Flr
1a589d9354
fix: Fix get_message_log extra json.loads (#22827) 2023-10-19 23:32:09 +05:30
Sagar Vora
93a8c37977 fix: type hint for keyword-only parameter for_update
[skip ci]
2023-10-18 17:36:45 +05:30
Ankush Menat
e2714c3e1c feat: v2 error and debug log structure 2023-10-16 18:13:50 +05:30
Ankush Menat
018ed845bd refactor: defer unnecessary json-dumping of messages
Also avoid accessing locals where interface is present like for popping last message.
2023-10-16 18:13:50 +05:30
Ankush Menat
0a244b10fc test: basic tests v2 API 2023-10-16 18:12:53 +05:30
Ankush Menat
9983bf82e6 fix: backward compatible get_site_path
Before: `./sitename`
After:  `sitename`

This used to return `.` before, even though these are paths identical
they aren't REALLY. Tar command strips 2 subpaths so this ended up
breaking file restores.
2023-10-11 17:30:13 +05:30
Ankush Menat
a81fb2a11a
Merge pull request #22509 from netchampfaris/file-backup-path
fix: use relative path in file backup
2023-09-24 10:06:47 +05:30
Faris Ansari
fa88f1a628 fix: resolve path instead of absolute path
#22142 made the change to absolute path but it affected file backups.
Resolve path should probably be enough in most cases.
2023-09-23 11:05:49 +05:30
Ankush Menat
8cb13f922a chore(DX): Annotate all local proxies 2023-09-18 15:36:58 +05:30
Ankush Menat
970a740164
revert: #22308 (#22389)
* Revert "chore: move function to correct file"

This reverts commit ebfdfa283b.

* Revert "refactor!: merge get_site_url into get_url (#22308)"

This reverts commit 2001bc278f.
2023-09-13 14:34:52 +05:30
Ankush Menat
ebfdfa283b chore: move function to correct file 2023-09-11 19:02:34 +05:30
David Arnold
2001bc278f
refactor!: merge get_site_url into get_url (#22308)
* chore: merge get_site_url into get_url

* fix: keep alias with deprecation notice
2023-09-11 15:09:05 +05:30
niraj gautam
db61deef72
fix: Avoid doc caching if for_update is set (#22203)
* refactor: Avoid doc caching if for_update is set

* chore: simplify condition

---------

Co-authored-by: Ankush Menat <ankushmenat@gmail.com>
2023-08-25 19:35:32 +05:30
Ankush Menat
6e0b522ae3 refactor!: Disable server scripts by default
- Move the config to bench level and not site level because, server
  script "threat model" requires consent from a bench owner and not
  individual site.
- While this is a breaking change which people may not like, we believe
  it's essential to improve security model of Frappe.
2023-08-23 14:49:05 +05:30
mergify[bot]
fe820ae8c8
Merge pull request #22142 from barredterra/abs-pymodule-path
feat(DX): return absolute path
2023-08-23 05:40:38 +00:00
Ankush Menat
04a13ca97c
fix: frappe.clear_cache should drop all keys (#22156)
We are dropping some keys which are hardcoded in function.

- this over time might be out of sync with rest of the code.
- other apps cache stuff too.

Best fix: just clear everything.
2023-08-23 11:02:45 +05:30
Sagar Vora
29e5ad4fca fix: set correct document label in case doc is of type str 2023-08-22 15:24:26 +05:30
barredterra
158a6f8140 refactor: use get_app_path in get_module_list 2023-08-21 17:53:47 +02:00
barredterra
8971709655 feat: return absolute site path 2023-08-21 17:53:10 +02:00
barredterra
8cfee4dfe2 feat: return absolute pymodule path 2023-08-21 17:13:35 +02:00
Raffael Meyer
8f7a4f6697
Merge pull request #21873 from blaggacao/dont-exceed-python-module-boundary 2023-08-21 17:10:27 +02:00
Ankush Menat
3531f58b54 refactor: unify safe_exec and safe_eval code
A lot of duplication was present for this code.
2023-08-21 11:59:53 +05:30
barredterra
5944d387be Merge remote-tracking branch 'upstream/develop' into refactor-attach-print 2023-08-20 15:38:05 +02:00
David Arnold
8dabd2bae6
refactor: make the app source accessor explicit
The parent folder of a python module (inside `./site-packages`) can be
`./lib`. Separate code paths that depend on the app source into its own
getter so that patching that function becomes easier in those cases
where the python module is not served from the source checkout.
2023-08-20 01:11:05 +02:00
Ankush Menat
4d5a945861
refactor: reduce duplication in unsafe attr checks (#22033) 2023-08-13 18:23:41 +05:30
barredterra
88c8baa9ee refactor: for append to extend, merge list extend
Replace a for append loop with list extend.
Create the list with values instead of creating
an empty list and extending it with another list.
2023-08-09 13:25:39 +02:00
barredterra
8d6480df1c Merge remote-tracking branch 'upstream/develop' into refactor-attach-print 2023-08-04 00:25:09 +02:00
barredterra
b1a2c31fe8 feat: letterhead param for attach_print 2023-07-27 18:29:22 +02:00
barredterra
44a3d9d69e refactor: attach_print
- Use print_language context manager
- Move filename and ignore_print_permissions closer to usage
2023-07-27 18:07:02 +02:00
Ankush Menat
6e94cd2eb9
fix: Guess most likely exception source (#21827) 2023-07-27 17:30:04 +05:30
gavin
6cb51a536e
fix: re module cache settings - purge & cache size (#21808)
* fix: Remove re cache internals manipulation

* fix: Purge re cache after module loads

Empty cache would work better as we already got our pre-compiled
patterns at the top level of every module. This leaves the cache open
for dynamically generated patterns which are in better need of it. Over
time, workers would converge to this anyway. This change only reduces
the cache hit and eviction effort.

I'd improve this by executing `re.purge` on every module import but
complexity tradeoff lol. I'd prefer if re didn't cache patterns
generated by `re.compile` but I dont see this behaviour or any escape
hatches so this will have to do for now.
2023-07-27 10:40:50 +05:30
David Arnold
a926e64ec9
fix: procure db config from single authority (#21578)
* fix: procure db config from single authority

ensures that configuration is uniformely procured from local.conf
instead of making use of hard to audit multilevel fallback logic

Implementation Note:
- `get_db(host, port, user, password)` was stripped of any optional
  argument and therefrom all errors where fixed.
- All occurances of `grep 'frappe.db.db_'` where changed to
  `frappe.conf.db_`

* fix: revert unnecessary breaking changes
2023-07-24 10:48:29 +05:30
Ankush Menat
adf30693a9 ci: update pyupgrade 2023-07-14 14:24:08 +05:30
Ankush Menat
fefd9ac2e2
fix: connect_replica and read_only should be idempotent (#21674)
* fix: `connect_replica` should be idempotent

Calling `connect_replica` twice ends up forgetting orginal writable
replica completely.

* test: replicas

closes https://github.com/frappe/frappe/issues/21619
2023-07-14 13:56:57 +05:30
David Arnold
6b2bb9a2ab
fix: add env overrides for service orchestration (#21577)
This avoids having to manipulate config files in brittle bash
entrypoints that need to react to dynamic service discovery.

This significantly improves the operability of various bench sites.
2023-07-14 11:39:01 +05:30
Ankush Menat
a63398778e perf: tune gc by default
This is running on several prod site without any noticable problems, so
making it default behaviour.

Opt out by setting env variable `FRAPPE_TUNE_GC=False`
2023-07-02 16:29:19 +05:30
Ankush Menat
b9f000e1f9 refactor!: Log 5xx error to error log instead of error snapshot
Also move log_error function to right location
2023-06-28 10:59:19 +05:30
Ankush Menat
fbe3174914 perf: Bump alloc count to 7,000 for generation 0
This has overall 1-2% CPU usage reduction for little to no costs.
Benefits increase when doing bulk processing with lots of objects.
2023-06-24 17:02:45 +05:30
Ankush Menat
8a37d6d278
perf: reduce memory usage of background processes (#21467)
* perf: defer translation.py imports

This indirectly imports babel which isn't really required most of the
time.

* perf: defer gzip import

* perf: move validate_and_sanitize_search_inputs

This causes all sorts of indirect imports and increases memory usage

* perf: defer requests module imports

* perf: defer system settings import

* perf: defer LOG_DOCTYPES import

Causes many indirect imports

* perf: defer update_site_config

* perf: defer notifications import

* perf: remove unused import

* perf: defer safe exec import

* test: memory usage overhead
2023-06-23 12:51:45 +05:30
Ankush Menat
3005e66e45 refactor!: Drop previously deprecated code 2023-06-13 16:00:43 +05:30
Ankush Menat
fa6dc03cc8
refactor: frappe.cache() usage to frappe.cache (#21282) 2023-06-08 11:47:17 +05:30
Sagar Vora
b6669bb56e
perf: remove localproxy for frappe.cache (#21281)
* perf: remove localproxy for `frappe.cache`

* chore: fix type hints
2023-06-08 10:21:45 +05:30
Ankush Menat
40b3cd82bc
feat!: frappe.cache() -> frappe.cache (#21279)
This is more intuitive and consistent with other things like `frappe.db`.

PS: This is quite likely to break some weird usage which I can't guess right now. Normal usage inside request/job cycles will continue to work as it used to.
2023-06-07 22:47:45 +05:30
Ankush Menat
392a506a76 perf: Cache published web forms 2023-06-04 15:13:39 +05:30
Ankush Menat
7c7c11b454 perf: Cache web view routes
Each call to evaluate if route is web view makes N queries where N = #
of web view doctypes. This entire computation can be definitely cached
for short duration.

- Added cache bursting in WebsiteGenerator doctype updates.
- Added 60 minutes TTL in case cache invalidation wasn't done reliably.
2023-06-04 15:13:39 +05:30
Suraj Shetty
f0df450c6f
Merge pull request #21224 from surajshetty3416/fix-safe-eval 2023-06-04 09:24:10 +05:30