Commit graph

1710 commits

Author SHA1 Message Date
Sagar Vora
b395a84d0b
fix!: remove version API (#21786) 2023-07-24 13:01:45 +05:30
David Arnold
a926e64ec9
fix: procure db config from single authority (#21578)
* fix: procure db config from single authority

ensures that configuration is uniformely procured from local.conf
instead of making use of hard to audit multilevel fallback logic

Implementation Note:
- `get_db(host, port, user, password)` was stripped of any optional
  argument and therefrom all errors where fixed.
- All occurances of `grep 'frappe.db.db_'` where changed to
  `frappe.conf.db_`

* fix: revert unnecessary breaking changes
2023-07-24 10:48:29 +05:30
Ankush Menat
2c99583247 perf: Lazily fetch shared documents
We eagerly fetch shared documents for ANY `get_list` query, even when
user has full read acess doctype, where it's moot to consider adding
shared document as separately.

This eliminates one entire db call from get_list and in most cases
get_list will translate to single DB call, hence probably worth the
additional complexity.
2023-07-22 18:04:35 +05:30
Ankush Menat
42aff950ce
feat(DX): normalize queries in recorder (#21735)
Attempt to normalize query by removing "variables"
This gives a different view of similar duplicate queries.

These two are distinct queries:
```sql
select * from user where name = 'x'
select * from user where name = 'z'
```

But their "normalized" form would be same:
```sql
select * from user where name = ?
```

This helps highlight queries ran in loop which might not register as
duplicate but are possibly "duplicate".
2023-07-19 16:45:30 +05:30
Ankush Menat
8436f775a5
feat(DX): PEP517 compatible build in new apps (#21704)
* chore: remove setup.py requirements.txt boilerplate

* feat(DX): PEP517 compatible builds on new apps

closes https://github.com/frappe/frappe/issues/21612
2023-07-17 12:56:41 +05:30
Deepesh Garg
e38078a69f
test: Patch hooks for testing (#21702)
* test: Patch hooks for testing

* test: hooks patcher

---------

Co-authored-by: Ankush Menat <ankush@frappe.io>
2023-07-17 12:41:27 +05:30
Shariq Ansari
d423dedcd3
Merge pull request #21377 from shariquerik/move-form-builder-in-doctype-form 2023-07-16 14:38:08 +05:30
Deepesh Garg
e32e74f2f1
feat: Custom naming series parser via hooks (#21690)
* feat: Custom naming series parser via hooks

* chore: use assignment operatot

Co-authored-by: Ankush Menat <ankushmenat@gmail.com>

* test: Unit test for custom parser

* test: Unit test for custom parser

---------

Co-authored-by: Ankush Menat <ankushmenat@gmail.com>
2023-07-16 12:57:40 +05:30
Shariq Ansari
3b2f2ef418 test: fix sidebar UI Test 2023-07-16 12:06:48 +05:30
Ankush Menat
79532ea0f2 fix: incorrect cache clearing of assets
- Build version wasn't correctly computed since v14 update of build
  system. This makes client side cache useless.
- We clear cache assuming rapid reloads,but opening new tab also does
  that. This makes the cache effectively useless for most users.
2023-07-15 21:59:04 +05:30
Ankush Menat
adf30693a9 ci: update pyupgrade 2023-07-14 14:24:08 +05:30
Ankush Menat
fefd9ac2e2
fix: connect_replica and read_only should be idempotent (#21674)
* fix: `connect_replica` should be idempotent

Calling `connect_replica` twice ends up forgetting orginal writable
replica completely.

* test: replicas

closes https://github.com/frappe/frappe/issues/21619
2023-07-14 13:56:57 +05:30
David Arnold
6b2bb9a2ab
fix: add env overrides for service orchestration (#21577)
This avoids having to manipulate config files in brittle bash
entrypoints that need to react to dynamic service discovery.

This significantly improves the operability of various bench sites.
2023-07-14 11:39:01 +05:30
Ankush Menat
154a6dd9ed
perf: Reduce recorder overhead (#21656)
By processing `EXPLAIN` of queries in background after recording is
completed, the overhead reduces significantly and hence can be used
"more" in production environment too.
2023-07-12 19:34:23 +05:30
Ankush Menat
8730de41f9
fix: Only clear doctype cache if specified (#21611) 2023-07-09 10:44:43 +05:30
Ankush Menat
d7990368a8
perf: preload more modules (#21557)
* perf: preload more modules

- bleach is used frequently for sanitization
- File gets imported anytime a private file is viewed. Indirect import
  of PIL is costly in each worker.

* test: warm up perf test
2023-07-02 21:47:21 +05:30
mergify[bot]
721035b313
Merge pull request #21276 from gavindsouza/bump-pydantic-v2
build(deps): Bump Pydantic from v1 to v2
2023-07-02 13:38:18 +00:00
Ankush Menat
4ece3da47d test: publish_progress ui test 2023-07-01 18:06:34 +05:30
Ankush Menat
ab44a9cca8 test: socketio tests
- conflict check (moved from form)
- list view updates
- custom event subscribe and recieve
2023-07-01 12:43:53 +05:30
gavin
03fbdde007
Merge branch 'develop' into bump-pydantic-v2 2023-06-30 22:19:09 +05:30
Corentin Flr
4ccfd1e8fa
Merge branch 'frappe:develop' into fix-test_never_render 2023-06-30 11:52:33 +02:00
Ankush Menat
a52485cc53 feat: RQ jobs can show count 2023-06-29 18:06:52 +05:30
Ankush Menat
1092eef7bd perf: faster pending jobs check 2023-06-29 17:31:06 +05:30
Ankush Menat
c73d9fb783 test: add perf test helper for counting rows read 2023-06-28 19:51:18 +05:30
Ankush Menat
0e1236b6be refactor: Simplify expiry queries.
Dont rely on mysql dateutils, simply compare dates with a cutoff.
2023-06-28 18:05:12 +05:30
Ankush Menat
38f08bb695
Merge branch 'develop' into fix-test_never_render 2023-06-28 11:41:51 +05:30
gavin
faab26ce4f
Merge branch 'develop' into bump-pydantic-v2 2023-06-26 13:12:24 +05:30
Ankush Menat
793f4ebba3 perf: defer loading JWT 2023-06-24 19:50:08 +05:30
Ankush Menat
b3840596fc
test: mock github API calls (#21450)
[skip ci]
2023-06-21 16:33:09 +05:30
Smit Vora
db6a06d204
fix: make sure number is not zero for bankers_rounding (#21431)
* fix: make sure num is not zero for bankers_rounding

* test: rounding near zero

---------

Co-authored-by: Ankush Menat <ankush@frappe.io>
2023-06-20 12:00:22 +05:30
Corentin Flr
9afedfae25
fix(test): Remove frappe.local.request between requests
`frappe.local.request` was not cleared between tests, which would not be a problem if all tests did set it to another Request object. But, some tests directly fetch the response content using get_response_content without first setting the frappe.local.request object (using set_request).
2023-06-16 13:27:49 +02:00
Corentin Flr
bcdc483a13
fix(test): Fix test_never_render to get path as string, exclude PYC files from static downloads
This test code never actually tested the behaviour for two reasons:
- first, the page had an error which meant that a 500 Error page was returned (because `path` is not a string)
- second, every page contains the string "400" because it's contained in some of the icons.svg icons!

I also found a minor related bug in static_page.py, allowing people to download PYC files (pycache)
2023-06-15 18:36:30 +02:00
Ankush Menat
3f3ee12338 Revert: keep supporting set_value for singles
w/ explicit check for singles

This shouldn't have any performance impact as last function call only
happens if we THINK it's single doctype. use set_single_value to avoid
that extra function call.
2023-06-14 17:30:32 +05:30
Ankush Menat
039be73af4
refactor: Consider singles for dynamic set_value usage (#21367)
Found all usage using this semgrep rule:

```yaml
    - pattern: frappe.db.set_value($DOCTYPE, ...)
    - pattern-not: frappe.db.set_value("$STR", ...)
```
2023-06-14 10:46:25 +05:30
Ankush Menat
3d0888a5d5 refactor: set_value usage 2023-06-13 16:00:43 +05:30
Ankush Menat
3005e66e45 refactor!: Drop previously deprecated code 2023-06-13 16:00:43 +05:30
Ritwik Puri
fbc25d206e
Merge pull request #21304 from phot0n/fix-expiry-for-email-queue
feat: bulk retry for email queue & remove infinite* retry for certain smtp exceptions
2023-06-12 15:08:44 +05:30
Ankush Menat
18e791a353 build(deps): PyPDF2 -> pypdf
closes https://github.com/frappe/frappe/issues/19861
2023-06-10 20:47:27 +05:30
Ankush Menat
8485ac5d62 build(deps): bump many dependencies
test: change code to adapt to new werkzeug client

fix: avoid setting charset

utf8 is default and assumed now by werkzeug, setting this manually is
deprecated.

fix: use string instead of bytes for setting headers

DeprecationWarning: Passing bytes as a header value is deprecated and will not be supported in Werkzeug 3.0.
12:23:34 web.1         |   response.headers["X-Page-Name"] = path.encode("ascii", errors="xmlcharrefreplace")
2023-06-10 20:47:27 +05:30
phot0n
1f8598891e fix: remove infinite sending retries for certain exceptions for email queue
* chore: remove expired status & set_expiry job from email queue
2023-06-09 14:38:55 +05:30
Ankush Menat
fa6dc03cc8
refactor: frappe.cache() usage to frappe.cache (#21282) 2023-06-08 11:47:17 +05:30
gavin
86b1382e32
Merge branch 'develop' into bump-pydantic-v2 2023-06-08 11:32:03 +05:30
Ankush Menat
40b3cd82bc
feat!: frappe.cache() -> frappe.cache (#21279)
This is more intuitive and consistent with other things like `frappe.db`.

PS: This is quite likely to break some weird usage which I can't guess right now. Normal usage inside request/job cycles will continue to work as it used to.
2023-06-07 22:47:45 +05:30
Gavin D'souza
ae3a61b3fc
test: Add tests for Document arg validations 2023-06-07 19:26:32 +05:30
Gavin D'souza
5c2fe7c292
test: Add tests for validate_argument_types decorator 2023-06-07 17:17:13 +05:30
Ankush Menat
26722b1a1c fix: ignore ConnectionError in frappe.cache().exists() 2023-06-03 22:53:10 +05:30
Ankush Menat
0d056a3a2b test: fix broken tests
Fixture test:
This is broken cause it's trying to find doctype after it has been
deleted (wut?)
It was working so far because cache wasn't cleared correctly so you'd
still find it from cache.

db.set_value test:
converted to use last query instead of patching SQL
2023-06-03 22:53:10 +05:30
Ankush Menat
7d50ef19d3 perf: Delete multiple keys in O(1) redis calls
Currently we call redis for each key, redis already supports deleting
multiple keys in one go.
2023-06-03 22:53:10 +05:30
Ankush Menat
4193a251a5 fix: Invalidate cache on rollback too
Steps:
    - Document modified
    - Document refetched from cache
    - Transaction rolled back
    - Cache now contains unmodified changes.
2023-06-03 22:53:10 +05:30
Ankush Menat
0b9dee4791 test: db callbacks 2023-06-03 18:44:19 +05:30