Commit graph

112 commits

Author SHA1 Message Date
David Arnold
6b2bb9a2ab
fix: add env overrides for service orchestration (#21577)
This avoids having to manipulate config files in brittle bash
entrypoints that need to react to dynamic service discovery.

This significantly improves the operability of various bench sites.
2023-07-14 11:39:01 +05:30
Ankush Menat
8a37d6d278
perf: reduce memory usage of background processes (#21467)
* perf: defer translation.py imports

This indirectly imports babel which isn't really required most of the
time.

* perf: defer gzip import

* perf: move validate_and_sanitize_search_inputs

This causes all sorts of indirect imports and increases memory usage

* perf: defer requests module imports

* perf: defer system settings import

* perf: defer LOG_DOCTYPES import

Causes many indirect imports

* perf: defer update_site_config

* perf: defer notifications import

* perf: remove unused import

* perf: defer safe exec import

* test: memory usage overhead
2023-06-23 12:51:45 +05:30
Sagar Sharma
a43ad15eab feat: Truncate QB function 2023-05-26 14:03:47 +05:30
Sagar Sharma
3f0b03c808 feat: Round QB function 2023-05-26 14:03:12 +05:30
Faris Ansari
c4bb732eaa fix: use dict syntax instead of string 2023-05-08 18:55:11 +05:30
Faris Ansari
41d7563aff feat: child_field[] syntax sugar for qb
- fetch child table rows in qb.get_query
- runs one query each for each child field
2023-05-08 18:55:11 +05:30
Rutwik Hiwalkar
69b22f3af2
chore: add ascending order to querybuilder (#20471) 2023-03-28 00:01:42 +05:30
Faris Ansari
fe13108eec fix: refactor
- move operator map in separate file
- remove unnecessary code
- organize functions
2023-01-10 16:15:33 +05:30
Faris Ansari
f982439eb9 fix: pass fields explicitly
- to prevent addition of default `name` field
- also, add fields only if it is a select query
2023-01-09 16:43:44 +05:30
Faris Ansari
be654eaa60
Merge branch 'develop' into refactor-qb-engine 2023-01-09 15:23:20 +05:30
Ankush Menat
075a2d778e feat: UnixTimestamp QB function 2023-01-09 12:05:52 +05:30
Faris Ansari
726fcfdb79 refactor: qb.engine
- simplify
- qb.engine.get_query -> qb.get_query
- qb.engine.build_conditions -> qb.get_query
2022-12-25 23:19:11 +05:30
Ankush Menat
b11793ab02 fix: set filename explicitly for safe_exec 2022-12-07 16:34:27 +05:30
Aradhya Tripathi
68e9fa639e
docs: Added query_builder docs (#19086)
* docs: Added query_builder docs

* docs: Added future works
2022-12-05 16:48:12 +05:30
Ritwik Puri
ba25e97d4e
Merge pull request #18756 from ankush/qb_simpler
refactor: qb builder class identification
2022-11-04 17:26:57 +05:30
Ankush Menat
f0197d35d6 refactor: qb builder class identification 2022-11-04 15:42:26 +05:30
Aradhya
8b73108270 feat: added PseudoColumnMapper for postgres support 2022-11-03 20:43:24 +05:30
mergify[bot]
002b27dbdd
fix: pypika does not parse datetime.time (backport #18184) (#18205)
* fix: pypika does not parse datetime.time

(cherry picked from commit b8f5a4304cccda954822cfe69d4805a18d9f07a7)

* fix: style - imports sort order fixed

(cherry picked from commit c3562c643ab9bff38d230202675d8ce1ae190912)

* test: add test for query builder parsing datetime.time

(cherry picked from commit 99889c270199da4b26b0d3858ae0f0a416d49d90)

* fix: format_time instead of format_datetime, test was failing!

(cherry picked from commit 5d697a22ac8cb98c8f8f44b3f99c57b932c5223c)

* fix(style): linter issues

(cherry picked from commit e231e1b0eadf63e84c9f81cfed9e2926e086fc44)

* test: add test for postgres

(cherry picked from commit 695591c43af9672cb83ce856a46c97e0bddfa9c3)

* fix: converting datepart and timepart to strings for Combinedatetime

(cherry picked from commit 1ebda943a607688bfb3db9fb42c80723a9050973)

* fix: style, linter issues

(cherry picked from commit c01262ad02b8831fdb0c5332b6973499e8adf6ad)

Co-authored-by: Anoop Kurungadam <anoop@earthianslive.com>
Co-authored-by: Aradhya <aradhyatripathi51@gmail.com>
2022-09-23 15:46:31 +05:30
Aradhya
85d1e41738 fix: fixed join query from fields 2022-08-10 23:42:01 +05:30
Aradhya
6c905233c5 feat: Added support for string filters in query 2022-07-28 23:55:58 +05:30
Aradhya
1ae3b7f16b feat: Added support for fields in Locate 2022-07-27 21:22:38 +05:30
Gavin D'souza
1ccdad811a Merge branch 'develop' of github.com:frappe/frappe into mariadb-client-refactor 2022-07-19 16:23:24 +05:30
Aradhya
c97562b736 feat: Added Timestamp support 2022-07-08 21:05:42 +05:30
Gavin D'souza
71b5c77e6e Merge branch 'develop' of github.com:frappe/frappe into mariadb-client-refactor 2022-07-05 14:55:02 +05:30
Aradhya
88faad6409 fix: pyupgrade 2022-07-01 13:30:18 +05:30
Aradhya Tripathi
ebd1d5b48c
Merge branch 'develop' into get-all-mod 2022-07-01 13:25:32 +05:30
Ankush Menat
81b37cb7d2
refactor: clean up code to py310 supported features (#17367)
refactor: clean up code to py39+ supported syntax

- f-strings instead of format
- latest typing support instead of pre 3.9 TitleCase
- remove UTF-8 declarations.
- many more changes

Powered by https://github.com/asottile/pyupgrade/ + manual cleanups
2022-07-01 11:51:05 +05:30
Gavin D'souza
fddd624a7c Merge branch 'develop' of github.com:frappe/frappe into mariadb-client-refactor 2022-06-30 14:31:52 +05:30
Ankush Menat
813dcc1848
Revert "feat: Adding support to Query engine" 2022-06-29 23:30:23 +05:30
Aradhya
58645998c0 fix: fixed fields with operators and added abs 2022-06-29 15:03:37 +05:30
Gavin D'souza
cf699fe40b Merge branch 'develop' of github.com:frappe/frappe into mariadb-client-refactor 2022-06-29 13:49:18 +05:30
gavin
91e76ff874
Merge branch 'develop' into get-all-mod 2022-06-28 19:47:20 +05:30
Aradhya
7732accded feat: Attached Engine object to qb & added dynamic type hints 2022-06-28 19:43:23 +05:30
Aradhya
960952cfc3 feat(qb-engine): Added Aggregation function support
* Added Min, Max
* Added tests
2022-06-28 19:41:39 +05:30
Aradhya
d0680941ad refactor: frappe.qb.engine
* feat: supporting empty iterables for Contains objects
* fix: explicitly setting empty iterables as tuples to support more operators
* feat: Added locate to frappe.qb Functions
* feat: Added support for functions passed as strings in fields
* feat: Included Criterion objects as fields
* fix: picking up only function intended fields to pass to get_function_objects
* feat: Added iterable for available functions, added support for Field objects
* fix: fixed * passed in fields in lists
2022-06-28 19:41:23 +05:30
Gavin D'souza
bfd51aa43a fix(qb): Use fallback og table if not found in schema mapper 2022-06-15 17:44:29 +05:30
Gavin D'souza
36fa2122a3 fix: Patch qb for different schemas in same process
You would want to switch schemas in the same process. Eversince the
change
64e5273764
we stopped patching on every frappe.init call which meant, if a MariaDB
site was initialized first, frappe._qb_patched would be set to True and
if a Postgres site was initialized after, _qb_patched would be lying as
the PG engine isn't patched yet. Sooooo we need a Dict instead to
maintain this record of patching. This issue caused weird errors lol -

Traceback:

  File "/home/frappe/Desktop/frappe-bench-dev/env/lib/python3.10/site-packages/click/decorators.py", line 21, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/frappe/Desktop/frappe-bench-dev/apps/frappe/frappe/commands/__init__.py", line 29, in _func
    ret = f(frappe._dict(ctx.obj), *args, **kwargs)
  File "/home/frappe/Desktop/frappe-bench-dev/apps/frappe/frappe/commands/site.py", line 524, in migrate
    SiteMigration(
  File "/home/frappe/Desktop/frappe-bench-dev/apps/frappe/frappe/migrate.py", line 169, in run
    self.setUp()
  File "/home/frappe/Desktop/frappe-bench-dev/apps/frappe/frappe/migrate.py", line 73, in setUp
    clear_global_cache()
  File "/home/frappe/Desktop/frappe-bench-dev/apps/frappe/frappe/cache_manager.py", line 102, in clear_global_cache
    clear_website_cache()
  File "/home/frappe/Desktop/frappe-bench-dev/apps/frappe/frappe/website/utils.py", line 374, in clear_website_cache
    clear_cache(path)
  File "/home/frappe/Desktop/frappe-bench-dev/apps/frappe/frappe/website/utils.py", line 369, in clear_cache
    for method in frappe.get_hooks("website_clear_cache"):
  File "/home/frappe/Desktop/frappe-bench-dev/apps/frappe/frappe/__init__.py", line 1440, in get_hooks
    hooks = _dict(_load_app_hooks())
  File "/home/frappe/Desktop/frappe-bench-dev/apps/frappe/frappe/utils/caching.py", line 57, in wrapper
    return_val = func(*args, **kwargs)
  File "/home/frappe/Desktop/frappe-bench-dev/apps/frappe/frappe/__init__.py", line 1407, in _load_app_hooks
    apps = [app_name] if app_name else get_installed_apps(sort=True)
  File "/home/frappe/Desktop/frappe-bench-dev/apps/frappe/frappe/utils/caching.py", line 57, in wrapper
    return_val = func(*args, **kwargs)
  File "/home/frappe/Desktop/frappe-bench-dev/apps/frappe/frappe/__init__.py", line 1374, in get_installed_apps
    installed = json.loads(db.get_global("installed_apps") or "[]")
  File "/home/frappe/Desktop/frappe-bench-dev/apps/frappe/frappe/database/database.py", line 917, in get_global
    return self.get_default(key, user)
  File "/home/frappe/Desktop/frappe-bench-dev/apps/frappe/frappe/database/database.py", line 921, in get_default
    d = self.get_defaults(key, parent)
  File "/home/frappe/Desktop/frappe-bench-dev/apps/frappe/frappe/database/database.py", line 938, in get_defaults
    defaults = frappe.defaults.get_defaults(parent)
  File "/home/frappe/Desktop/frappe-bench-dev/apps/frappe/frappe/defaults.py", line 88, in get_defaults
    globald = get_defaults_for()
  File "/home/frappe/Desktop/frappe-bench-dev/apps/frappe/frappe/defaults.py", line 218, in get_defaults_for
    frappe.qb.from_(table)
TypeError: 'Field' object is not callable
2022-06-15 15:55:43 +05:30
Gavin D'souza
64e5273764 perf: Patch qb only once - not on every init 2022-06-13 18:39:56 +05:30
gavin
6db1038501 refactor(qb): Rename term subqry to SubQuery
"SubQuery" conforms with terms naming in PyPika. Kept subqry that points
to same def for maintaining APIs.
2022-05-27 16:55:54 +05:30
saxenabhishek
1df3e8f5e7 test: test agg funtions 2022-05-16 15:20:23 +05:30
saxenabhishek
846b89a703 test: test cast funtion 2022-05-16 15:20:23 +05:30
saxenabhishek
e59c2f7b04 test: match edge case 2022-05-16 15:20:23 +05:30
Gavin D'souza
66655eb8ed Merge branch 'develop' of github.com:frappe/frappe into bg-rename_doc 2022-04-14 13:41:56 +05:30
Suraj Shetty
c0c5b2ebdd
style: format all python files using black (#16453)
Co-authored-by: Frappe Bot <developers@frappe.io>
2022-04-12 10:59:25 +05:30
Abhishek Saxena
5c8856d66e
refactor: db.sql calls to frappe.qb (#16107)
# Changes

- Introduces `subqry` class to use in where clause when there is a non-column condition. eg.
> .where(subqry(no_of_roles) == 0)
- Convert SQL queries to frappe.qb 

# Testing

Functions with query refactors
- frappe.boot.get_user_pages_or_reports() -> Same output of `get_bootinfo()` as develop
- frappe.boot.get_unseen_notes() -> Forms the same query as develop 
```sql
SELECT `name`,`title`,`content`,`notify_on_every_login`
FROM `tabNote` WHERE `notify_on_every_login`=1
AND `expire_notification_on`>'2022-03-30 01:10:53.393874'
AND (SELECT `nsb`.`user` FROM `tabNote Seen By` `nsb` WHERE `nsb`.`parent`=`tabNote`.`name`) NOT IN ('Administrator')
```
- frappe.installer._delete_doctypes() -> installed and uninsalled a dummy app to drop tables

### Not tested
- frappe.make_property_setter()
- frappe.realtime.get_pending_tasks_for_doc() [whitelist method]
- frappe.sessions.Session.start()
- frappe.twofactor.cache_2fa_data()
2022-04-12 05:07:25 +00:00
Gavin D'souza
2d806b5d6d fix: Ported subqry to branch for compatibility
Via https://github.com/frappe/frappe/pull/16107
2022-03-31 13:41:08 +05:30
Gavin D'souza
71d3f1c0d1 feat(minor): Add DateFormat function util for qb 2022-03-31 13:39:06 +05:30
Ritwik Puri
0d16c20f93
test: tests for Cast_ Function in qb (#16399)
* test: tests for Cast_ Function in qb

* test: remove unneeded sync

Co-authored-by: Mohammad Hasnain <hasnain2808@gmail.com>
2022-03-25 14:46:08 +05:30
Mohammad Hasnain Mohsin Rajan
fda544f424
refactor!: make automatically following documents optional (#16030)
* fix: make automatically following documents optional

* fix: optimize email triggers for document followed

* test: add tests for document follow settings

* test: sync global search before testing

* fix: extend pypika's cast function to mimic varchar cast in MariaDB

Co-authored-by: Suraj Shetty <13928957+surajshetty3416@users.noreply.github.com>
Co-authored-by: phot0n <ritwikpuri5678@gmail.com>
2022-03-25 02:01:15 +05:30
Ankush Menat
b029b255fe refactor: code duplication for sql whitelisting 2022-03-24 17:49:28 +05:30