Commit graph

2725 commits

Author SHA1 Message Date
Suraj Shetty
d4166dbe20 refactor: Replace html2text with markdownify 2022-07-06 08:23:14 +05:30
Ankush Menat
261fbfcd11
Revert "fix(doc)!: Always cast datetime, date and time fields"
Revert "fix(doc)!: Always cast datetime, date and time fields (#15891)"

This reverts commit d7789ab6ff.
2022-07-05 13:43:32 +05:30
Himanshu
d7789ab6ff
fix(doc)!: Always cast datetime, date and time fields (#15891)
### BREAKING CHANGE
#### Datetime, Date and Time fields will always be cast to respective objects in `setattr`, this will ensure uniformity while accessing the values, no more `getdate`, `get_datetime`, `to_timedelta` wrapper.
- While importing data, the framework does check for `set_only_once`.
- In normal case scenarios, this will work flawlessly since most date fields might not be set_only_once.
- But in Subscription, the date field is set to `set_only_once` and in `after_insert`, `document.save` is called, and while doing so, `set_only_once` is checked [here](1944a547f9/frappe/model/document.py (L566)).
-This works fine if the data imported is in the correct format.
- If the date's data is not in the correct format, the framework throws an error.
- for eg `06-02-2022 00:00:00 != 06-02-2022`
- fixes [Issue/#15370](https://github.com/frappe/frappe/issues/15370)

> no-docs
2022-07-05 07:07:16 +00:00
Ankush Menat
9a03c53572
Merge pull request #17405 from ankush/feat/currency_symbol_loc
feat: configurable location of currency symbol
2022-07-05 11:22:02 +05:30
Ankush Menat
054ad9c30c feat: Allow showing currency symbol to the right 2022-07-05 11:21:23 +05:30
gavin
93ef06f233
Merge pull request #17388 from gavindsouza/ci-mariadb-10_6
ci: Bump mariadb from 10.3 to 10.6
2022-07-05 09:51:02 +05:30
Ankush Menat
4e6ea5b554
fix: handle None as amount in fmt_money (#17395) 2022-07-04 16:27:57 +05:30
Ankush Menat
60c4819373
refactor: safer binding of outer scope in closures (#17389) 2022-07-04 13:39:56 +05:30
Gavin D'souza
65c6914068 ci: Bump mariadb from 10.3 to 10.6 2022-07-04 12:33:29 +05:30
Ankush Menat
81b37cb7d2
refactor: clean up code to py310 supported features (#17367)
refactor: clean up code to py39+ supported syntax

- f-strings instead of format
- latest typing support instead of pre 3.9 TitleCase
- remove UTF-8 declarations.
- many more changes

Powered by https://github.com/asottile/pyupgrade/ + manual cleanups
2022-07-01 11:51:05 +05:30
Ankush Menat
aad7ebc938
fix: extra notification triggered on value change (#17364)
Since value in DB and value on object can be in different type the
notification gets triggered unnecessarily

e.g. data(2012, 10, 10) != "2012-10-10"
2022-06-30 20:25:29 +05:30
Gavin D'souza
404d5a3cd7 build!: Bump minimum Python version to 3.10
Given how widespread PY310's usage has become, and how we're just a
few months away from PY311 major release. This is a slightly late
bumping but necessary to ensure smoother updates & maintenance for
Frappe, ERPNext & other apps in the coming years. Almost all people
who participated in the pool from the community as well as Frappe team
voted (via active telegram groups) PY310 as their preferred minimum
requirement for v14.
2022-06-30 13:11:26 +05:30
Ankush Menat
813dcc1848
Revert "feat: Adding support to Query engine" 2022-06-29 23:30:23 +05:30
Ankush Menat
181833848f
Merge pull request #17337 from ankush/lint_cleanup
fix: remove bare exception catching
2022-06-29 11:39:05 +05:30
gavin
d89c1e1666
Merge pull request #17346 from gavindsouza/ci-min-PY38
ci: Match CI with minimum Python version requirement
2022-06-29 11:33:15 +05:30
Gavin D'souza
bd3d7bf3ed ci: Run test builds on min requirement PY38
Since version pinned in pyproject.toml is 38 & we've been running on 39,
we've pushed a few incompatible changes since :crie:
2022-06-29 10:44:55 +05:30
gavin
91e76ff874
Merge branch 'develop' into get-all-mod 2022-06-28 19:47:20 +05:30
Aradhya
4af2e1e886 refactor: replaced frappe.db.query with frappe.qb.engine 2022-06-28 19:43:50 +05:30
Ankush Menat
d35d7ffbe2 fix: remove bare exception catching
A bare except catches lots of things (like generator iteration end) and should never be used.
2022-06-28 18:05:00 +05:30
Sagar Vora
b7514a05ca fix(redis): pass shared param when setting value based on generator 2022-06-28 00:37:31 +05:30
Sagar Vora
99388652c9 chore(redis): remove unnecessary exception handling 2022-06-27 01:56:58 +05:30
Suraj Shetty
b0d53d93ad
Merge pull request #17264 from netchampfaris/page-builder-section-ids 2022-06-22 12:48:36 +05:30
Faris Ansari
d49f9bfb8f fix: handle case where web_block is invoked
from places other than web page
2022-06-22 12:04:57 +05:30
Faris Ansari
8814041b33 fix: set web-template attr in script and style tags from web templates
Also, deduplication of styles and scripts when web_blocks jinja method is used
2022-06-21 20:42:14 +05:30
Gavin D'souza
59ab6531c9 ci: Invalidate GHA pip cache on pyproject or setup changes
- Cache invalidated if any of the files change: *requirements.txt,
pyproject.toml, setup.py or setup.cfg
- Updated boilerplate for new apps too
2022-06-15 15:05:37 +05:30
Gavin D'souza
52359a0ad9 test: Scheduler tests cleanup 2022-06-14 12:22:12 +05:30
Gavin D'souza
fc422f4a97 chore(deps): Bump Pillow from 9.0.1 to 9.1.1
Includes security fixes, API & constants deprecations

Changelog: https://pillow.readthedocs.io/en/stable/releasenotes/index.html
2022-06-10 13:12:35 +05:30
Gavin D'souza
ef0a92c849 chore(deps): Bump PyPDF2 from 1.26.0 to 2.1.0
* Updated changes in API usages
* changelog: https://github.com/py-pdf/PyPDF2/blob/main/CHANGELOG
2022-06-10 12:08:10 +05:30
Gavin D'souza
3aef47200b chore(deps): Bumped pdfkit from 0.6.1 to 1.0.0
* Updated from_string API to match latest signature
* changelog: https://github.com/JazzCore/python-pdfkit/blob/master/HISTORY.rst
2022-06-10 11:53:50 +05:30
Gavin D'souza
7e346933c7 fix: Cleaner error message on invalid encryption_key 2022-06-09 16:45:23 +05:30
gavin
9ce74fb6c4
Merge branch 'develop' into request-cache 2022-06-09 12:32:03 +05:30
Gavin D'souza
2e5784629b fix: Avoid possible iteration failures passing default in pop 2022-06-09 12:31:05 +05:30
Gavin D'souza
13b18a9118 fix: Regex compilation in boilerplate utils
Flagged via tests in frappe.tests.test_boilerplate
2022-06-09 12:05:07 +05:30
Suraj Shetty
8e2555f510
fix: Erroneous regex usage
Fixes: https://github.com/frappe/frappe/pull/17111#issuecomment-1150118499

Issue introduce via: https://github.com/frappe/frappe/pull/17111
2022-06-09 07:12:04 +05:30
Sagar Vora
df28f32b8b
perf: ~90% faster frappe.get_cached_doc (#17107) 2022-06-08 15:49:11 +05:30
Gavin D'souza
b696fa6da5 perf: Pre-compile and re-use regexp pattern
Converted all possible usages of re.* that weren't compiling the regex
separately and re-using it. Separated out the compiled patterns as
global variables. Repetitive patterns could be made DRY-er.

Would be nicer to have all regexes in a single module so that we could
re-use better, keep track of outdated, and keep checks for possible
reDos' etc
2022-06-08 14:07:38 +05:30
gavin
44dba28159
Merge branch 'develop' into refactor-file 2022-06-08 12:43:35 +05:30
Ankush Menat
59a9be4caf refactor: simplify timegrain conditions
All special handling for timegrains is same as general handling, no idea why it's done
like this.
2022-06-08 11:53:14 +05:30
mmdanny89
4850c0d4fe
infinite loop graphic bar
When Period is Yearly and Date Range is Last Year in Graphic Bar
2022-06-07 09:12:23 -04:00
Gavin D'souza
f03a7162ed feat(site_cache): Add maxsize to site_cache, defaults to None 2022-06-07 18:35:46 +05:30
Gavin D'souza
b04e1908f1 perf: Speed up request_cache
Sped up request_cache access times multi-fold with the help of a
benchmarking script. Access times for this generic cache is comparable
to specific caches written (eg: get_meta's local cache) by an additional
overhead of 15% as compared to implementing it in each function separately

* Got rid of logging
* Optimized bits with the help of benchmarking script against
  frappe.get_meta's performance
* Use hash instead of json.dumps
2022-06-07 15:56:26 +05:30
Gavin D'souza
061703ab17 feat(site_cache): Allow 'time to live' param for cache
Want to automatically expire your cache every 60s? site_cache has got you.
2022-06-07 15:56:26 +05:30
gavin
3be3b83c4f feat: frappe.utils.caching.site_cache
Decorator to cache method calls across requests. The cache is stored in
frappe.utils.caching._SITE_CACHE. The cache persists on the parent process.
It offers a light-weight cache for the current process without the additional
overhead of serializing / deserializing Python objects.

Note: This cache isn't shared among workers. If you need to share data across
workers, use redis (frappe.cache API) instead.

Usage:
    from frappe.utils.caching import site_cache

    @site_cache
    def calculate_pi():
        import math, time
        precision = get_precision("Math Constant", "Pi") # depends on
        site data
	return round(math.pi, precision)

    calculate_pi(10) # will calculate value
    calculate_pi(10) # will return value from cache
    calculate_pi.clear_cache() # clear this function's cache for all sites
    calculate_pi(10) # will calculate value
2022-06-07 15:56:26 +05:30
gavin
db9b0d3d4f feat: frappe.utils.caching.request_cache
Utility method for boundless cache. This method maintains it's cache
using Werkzeug local. Therefore, it's invalidated at the end ofthe
lifecycle of a request in our WSGI app.

Cache keys generated are a function of func.__name__, func.__module__,
passed args & kwargs. Key generation will be successful only if args and
kwargs can be safely `json.dumps`'ed.

Usage:

    from frappe.utils.caching import request_cache

    @request_cache
    def calculate_pi(num_terms=0):
        import math, time
        print(f"{num_terms = }")
        time.sleep(10)
        return math.pi

    calculate_pi(10) # will calculate value
    calculate_pi(10) # will return value from cache
2022-06-07 15:56:26 +05:30
gavin
88592eac5d fix!: Set default logging level ERROR when DEV_SERVER envvar is unset 2022-06-07 15:56:26 +05:30
Ankush Menat
111df3a8d7 perf: improve document caching
As per current implementation whenever `get_doc` is called, document is
cached. However, this cache is only ever used by `get_cached_doc`. Going
through the codebase of both FF/ERPNext you'll find that `get_doc` is
used a lot more than `get_cached_doc`. So in many places, all this
caching overhead is unnecessary.

This change removes implicit caching from get_doc and replaces it with
cache-replacement instead. i.e. cache is only updated if it exists but
not created from get_doc.

Pros:
- faster `get_doc`
- lower memory usage on Redis
- Reduces chances of OOM by blowing up worker's memory as old docs can't
  be GCed until
- Correctness i.e. caching only what gets used from cache.

Con:
- After this change. First call to `get_cached_doc` will always be a cache miss. DUH.
2022-06-06 16:18:23 +05:30
Ankush Menat
d3b366f147 feat: cache_locally to limit caching in local.cache 2022-06-06 16:18:23 +05:30
Suraj Shetty
0cd41fbf0c
Merge branch 'develop' into fix-document-signature-2 2022-06-01 18:01:12 +05:30
Suraj Shetty
af287a04a9 refactor: Remove unnecessary exception handling 2022-06-01 17:50:07 +05:30
Gavin D'souza
5ec2690160 Merge branch 'develop' into refactor-file 2022-06-01 15:47:55 +05:30