Commit graph

346 commits

Author SHA1 Message Date
Shariq Ansari
fc1e01555b fix: added missing operators 2022-09-30 17:16:40 +05:30
Shariq Ansari
235171796d
fix: coalesce not in queries (#18099)
* fix: get workspaces with empty module fields

* Revert "fix: get workspaces with empty module fields"

This reverts commit 1f194be2c3642e31ebe2165e461b2f24be8cda4c.

* fix: always coalesce `not in` queries

Co-authored-by: Ankush Menat <ankush@frappe.io>
2022-09-13 12:08:07 +05:30
Ankush Menat
cdb0732646
perf: avoid coalescing wherever possible (#17920) 2022-08-24 10:30:51 +05:30
Faris Ansari
82fef72ce1
fix(db-query): parenttype condition on child table left join (#17865)
joining only on name is incorrect because parent primary keys are not unique across tables

How was this working all this time? 🥲
2022-08-18 09:04:40 +00:00
Sagar Vora
d6aa17cc14 chore: add deprecation warning everywhere verbose is used 2022-07-31 00:24:21 +05:30
Gavin D'souza
1dbc0b4d3c fix(db*): Use common fallback Datetime str
There existed inconsistencies between db_query & db's fallback for min
datetime in str format - missing decimal seconds places. Now, we're
storing the default string once and re-using it to reduce
inconsistencies or room for human errors.
2022-07-22 13:19:25 +05:30
Gavin D'souza
71b5c77e6e Merge branch 'develop' of github.com:frappe/frappe into mariadb-client-refactor 2022-07-05 14:55:02 +05:30
Ankush Menat
81b37cb7d2
refactor: clean up code to py310 supported features (#17367)
refactor: clean up code to py39+ supported syntax

- f-strings instead of format
- latest typing support instead of pre 3.9 TitleCase
- remove UTF-8 declarations.
- many more changes

Powered by https://github.com/asottile/pyupgrade/ + manual cleanups
2022-07-01 11:51:05 +05:30
Gavin D'souza
cf699fe40b Merge branch 'develop' of github.com:frappe/frappe into mariadb-client-refactor 2022-06-29 13:49:18 +05:30
Ankush Menat
c8ac3f922e Revert "fix: extra column in excel after exporting report with group by (#17126)"
This reverts commit 971b8160a3.
2022-06-28 18:40:27 +05:30
gavin
bb0283a1cf
Merge branch 'develop' into mariadb-client-refactor 2022-06-28 14:57:21 +05:30
Shariq Ansari
971b8160a3
fix: extra column in excel after exporting report with group by (#17126)
Co-authored-by: gavin <gavin18d@gmail.com>
2022-06-28 12:39:45 +05:30
gavin
73f6141e0d
Merge branch 'develop' into mariadb-client-refactor 2022-06-20 15:31:53 +05:30
Gavin D'souza
48243346e3 fix(get_contact_list): Don't pass conditions as query value
* Conditions passed are not valid prepared statement values. They
can be passed as string substitution since they're generated by DBQuery.
* Added typing hints & other improvements
* Removed seemingly pointless try-except block
2022-06-20 13:10:03 +05:30
Raffael Meyer
7b67e1f847
fix: translate doctype in error messages (#17239) 2022-06-20 10:35:57 +05:30
Gavin D'souza
9b4db43b84 perf(db_query): Maintain compiled pattern globally 2022-06-14 17:17:22 +05:30
Gavin D'souza
60ec324956 fix: Remove unwanted blacklist over fields
A field like 'count(`tabBOM Update Log`.name) as total_count' split by
whitespace loses it's meaning. Tried splitting it meaningfully but
didn't get the point of this tbh and stopped. I'm not sure what the code
before was trying to do and with what set of inputs.

Imagine the following fields: [count(`tabBOM Update Log`.name) as
total_count, `tabBOM Update Log`.name as update_name, `tabBOM Update
Log`.name, `tabBOM Update as Log`.name, tabBOM.name, name], I couldn't
see what the previous check was trying to protect - hence, didn't add
any equivalent functionality.
2022-06-14 16:40:41 +05:30
Gavin D'souza
32b4983197 fix(re): Pass flags on instantiation 2022-06-08 16:31:58 +05:30
Gavin D'souza
b696fa6da5 perf: Pre-compile and re-use regexp pattern
Converted all possible usages of re.* that weren't compiling the regex
separately and re-using it. Separated out the compiled patterns as
global variables. Repetitive patterns could be made DRY-er.

Would be nicer to have all regexes in a single module so that we could
re-use better, keep track of outdated, and keep checks for possible
reDos' etc
2022-06-08 14:07:38 +05:30
Faris Ansari
3d07a51442
fix: check read permission on link sub-fields (#17082) 2022-06-06 19:54:54 +05:30
Suraj Shetty
ed72fc5a0e
Merge pull request #17003 from netchampfaris/db-query-child-table-syntax-sugar 2022-05-30 12:21:35 +05:30
Faris Ansari
87ec6d4fb9 fix: initialize link_tables in constructor 2022-05-30 10:50:16 +05:30
Faris Ansari
507a45d8f2 fix: dot syntax support for link fields
 test
2022-05-28 13:57:44 +05:30
Faris Ansari
96b30e714c feat: table_field.fieldname field syntax for db_query 2022-05-27 16:04:22 +05:30
gavin
6dedf899ff
Merge pull request #16899 from ChillarAnand/imports
refactor: Removed unused imports & added flake8 check in CI
2022-05-23 15:53:18 +05:30
chillaranand
1d763a6659 refactor: Fix flake8 issues 2022-05-19 15:34:35 +05:30
gavin
7c77bedbf2 refactor: Simplify logic + Add typing hints 2022-05-18 16:57:54 +05:30
phot0n
7ec5e88278 chore: remove unnecessary groups from search regex in cast_name 2022-05-05 21:31:13 +05:30
phot0n
aa6a21fc2c chore: add regex to consider space in cast_name 2022-05-05 21:31:13 +05:30
Ritwik Puri
b41379c78b
fix: misc fixes (integer primary keys) (#16307)
* fix: misc fixes

local.x gets resetted on every request so switched to a simple dict

simplified is_val_used in set_next_val function for sequences

* chore: use multisql for sequence methods

* fix: fields not updating on form

* minor(base_input): removed unnecessary branching in update_input

* chore: remove prints and rename autoincremented_status_map

* chore: added proper type hint + comment + formatting

* fix: added searching in cast_name rather than handling it manually

* fix: share condition query + test_build_match_conditions

* fix: add cast_name to more places

* test: test for sequence

* fix: sequence functions

* fix: inherit frappetestcase

* minor: attach sequence methods to db context local

* chore: update sequence function names in Database

use frappe.db for sequences in naming.py

* fix: convert filename to str (for autoincremented doctypes)

* chore: better regex for modifying values for postgres

* minor: allow changing name column type (if no data is present in the doctype)

* refactor: validate_autoname

converted it to a simple function
enabled changing autoincrement autoname from customize form

* fix: use sql_ddl for change_column_type in postgres

* fix: use not null constraint in postgres when changing name type

* fix(test): updated test_autoincremented_doctype_transition with transitioning when no data is present

* fix(test): updated test_cast_name

probably messed up during rebase

* fix(test): used rollback upon error in transaction for postgres

* chore: use frappe.db.x methods for sequences

* minor: use temporary sequences in test

* minor: use generate_hash for sequence naming in sequence tests

* chore: replace sequence imports with frappe.db.x

* chore: move out casting name fields to a separate method

* refactor: cast_name

more explicit cases for casts and added docstring

* fix: added space in test_cast_name

* chore: fix linter

* chore: better naming for can_change_name_column_type

* chore: add comment for autoincremented_site_status_map

* chore: update/add docstrings
2022-04-29 15:06:03 +05:30
Suraj Shetty
c0c5b2ebdd
style: format all python files using black (#16453)
Co-authored-by: Frappe Bot <developers@frappe.io>
2022-04-12 10:59:25 +05:30
phot0n
8296d6e84a fix: use backticks for fieldname while preparing filters 2022-03-31 18:12:09 +05:30
phot0n
906f671d9c fix(db_query): ordering in extract_tables
this will allow the fields to be casted first
2022-03-11 23:46:00 +05:30
phot0n
4ea87fd9cc chore: rename cast_autoincremented_name to cast_name 2022-03-11 23:46:00 +05:30
phot0n
bebc8058b6 feat: integer primary keys 2022-03-11 23:46:00 +05:30
barredterra
745297a49d refactor: a not in b
Search: if not ([\w\d]*?) in ([\w\d]*?)
Replace: if $1 not in $2
2022-02-21 19:54:34 +01:00
barredterra
b1383df15a refactor: use is None instead of == None 2022-01-17 10:37:51 +01:00
Gavin D'souza
981564e544 Merge branch 'develop' into frappe-postgres-changes 2022-01-07 11:37:28 +05:30
Gavin D'souza
d9b571cf43 refactor(minor): DatabaseQuery.prepare_select_args
* Used regex to replace instead of re-iteration (for-loop)
* Used f-strings for better readability
* Renamed and re-ordered vars, code block
2022-01-07 11:35:52 +05:30
Conor
55392b64e7 fix: Revert changes made to fix collation differences
Due to collation differences in MariaDB and Postgres, tests gave
inconsistent results. This was to be handled in tests alone instead of
the application. It was unnecessary. Collation changes should be made at DBMS config
level only. Accomodating for those in the application will unnecessarily
degrade performance for everyone.

Other changes:
* use pluck in user_type
* revert ordering in nestedset
* revert parsing in order_field
* use preferred APIs & styling

Co-authored-by: gavin <gavin18d@gmail.com>
2022-01-07 10:45:10 +05:30
Conor
1c40881c41 fix: refactor postgres arg prep into its own function 2022-01-07 10:45:05 +05:30
Conor
40ba1ac9ba fix: Postgres Compatibility
* Handle inconsistencies in type handling in DatabaseQuery & Database
  APIs
* Update incompatible queries with frappe.qb notation
* Fixed use cases discovered by failing ERPNext CI tests

fix: db independent syntax for user_type

fix: handle postgres datetime values

feat: add ability to auto commit on db inserts

feat: add ability to escape underscore in postgres

fix: handle missing data in test runner bootstrapping

fix: db independent syntax for queries

fix: refactor to use qb

fix: update cache for language

fix: use pluck in email_queue

Co-authored-by: gavin <gavin18d@gmail.com>

fix: don't auto insert on tests for make_property_setter

fix: remove auto_commit in custom_field insertion

fix: remove auto_commit functionality

fix: review comments

fix: revert link validation

fix: style suggestion for readability

Co-authored-by: gavin <gavin18d@gmail.com>

fix: revert .lower() in link validation

fix: add rollback for setup_wizard

Revert "fix: add rollback for setup_wizard"

This reverts commit 83b3b0913db17718ccd5edae01858cff15603829.

Revert "feat: add ability to escape underscore in postgres"

This reverts commit 8ed9c2aa3306438e94bb813f60e65b416d0b947b.

fix: more concise representation of order fields

Co-authored-by: gavin <gavin18d@gmail.com>
2022-01-07 10:44:57 +05:30
Verequies
379342d0ca fix: Postgres Compatibility for Standard Views
Core/User DocType: Add 'group_by' for PostgreSQL

Signed-off-by: Verequies <hamishclaxton@gmail.com>

fix: Postgres Compatibility for Standard Views

Desk/Dashboard DocType: Fix lowercase table name

Signed-off-by: Verequies <hamishclaxton@gmail.com>

Desk/Page/Setup Wizard: Fix data not being commited to database

Signed-off-by: Verequies <hamishclaxton@gmail.com>

Model/db_query: Fix queries with order_by and group_by for PostgreSQL

Signed-off-by: Verequies <hamishclaxton@gmail.com>

Model/db_query: Fix order_by without table name for PostgreSQL

Signed-off-by: Verequies <hamishclaxton@gmail.com>
2022-01-06 14:02:23 +05:30
Gavin D'souza
0f71dd411b style: DatabaseQuery.execute's permission cond block 2021-12-24 14:01:26 +05:30
Gavin D'souza
33b7d7d74f fix: Handle custom child tables via check_parent_permission
Check `tabCustom Field` and `tabDocfield` for matching parent existence.
2021-12-24 13:28:40 +05:30
Aradhya
0ea0f7dfa3 fix: fixed default ordering in execute 2021-11-25 16:39:44 +05:30
Aradhya
30278a3939 refactor: removed no_order to support order_by None as valid input 2021-11-23 13:29:59 +05:30
Aradhya
ae59fd7c58 feat: added no_order to execute 2021-11-19 22:58:32 +05:30
Gavin D'souza
ccb53c0f5b fix: If column comparision is done, dont check for ifnull/coalesce 2021-10-29 14:19:28 +05:30
Gavin D'souza
9bf043eddf fix(db_query): Change fallback to min datetime 2021-10-29 13:24:28 +05:30