Commit graph

265 commits

Author SHA1 Message Date
AarDG10
fd5da930f3 fix(query): ensure aggregate queries without group_by trigger postgres sort normalization 2026-01-14 19:21:24 +05:30
AarDG10
50e675f009 refactor: update warning to apply only to select distinct queries 2026-01-14 18:33:40 +05:30
AarDG10
cb68c2df32 fix(query): aggregate order_field when used with select group_by 2026-01-14 14:14:59 +05:30
Akhil Narang
c2e2ca611a
fix(query): handle string argument with between
Prevents crash when filter value doesn't exist (example, invalid Fiscal Year)

Signed-off-by: Akhil Narang <me@akhilnarang.dev>
2026-01-08 16:34:27 +05:30
Akhil Narang
fb07cf7579
fix(query): don't crash for invalid value
Signed-off-by: Akhil Narang <me@akhilnarang.dev>
2026-01-08 16:33:40 +05:30
AarDG10
54da011c65 fix(query): check hooks for custom_operator definition 2026-01-07 20:42:24 +05:30
Akhil Narang
d02d89436e
fix(qb): support reportview alias format
Signed-off-by: Akhil Narang <me@akhilnarang.dev>
2026-01-06 19:30:14 +05:30
Sagar Vora
9a774de21e
fix: add permission conditions to where clause instead of join 2026-01-05 13:32:04 +00:00
Akhil Narang
0661fa3f60
Merge pull request #35538 from akhilnarang/ignore-filter-fifth-parameter
fix(query): ignore 5th filter parameter
2025-12-30 17:59:42 +05:30
Akhil Narang
92900d366c
refactor: simplify code using match
Signed-off-by: Akhil Narang <me@akhilnarang.dev>
2025-12-30 16:54:59 +05:30
Akhil Narang
038bd44d08
fix(query): ignore 5th filter parameter
It doesn't do anything, but many old charts, etc. still have it present, no point erroring out.

Signed-off-by: Akhil Narang <me@akhilnarang.dev>
2025-12-30 16:30:42 +05:30
Sagar Vora
c9147fb840 chore: improve docstring 2025-12-30 13:25:13 +05:30
Sagar Vora
9fd5106ad1 fix: specify correct doctype in _raise_permission_error 2025-12-30 13:21:49 +05:30
Sagar Vora
26b3e6363e chore: deprecate validate_filters 2025-12-30 13:02:49 +05:30
Sagar Vora
0a76f1fc36
Merge branch 'develop' into qb-fixes 2025-12-29 15:43:32 +05:30
Akhil Narang
3881a7ae7e
fix(query): check parent permissions when querying child table fields
Signed-off-by: Akhil Narang <me@akhilnarang.dev>
2025-12-22 12:48:35 +05:30
Sagar Vora
a13f02a240
Merge pull request #35343 from neox-d/fix-single-doctype-join 2025-12-22 12:27:59 +05:30
Sagar Vora
7ce888d5f1 fix: return early is parent doctype is single 2025-12-22 12:14:17 +05:30
Akhil Narang
5992baeaff
fix(query): respect validate_filters
Block dot notation for link field access, and automatic child table detection as well

Signed-off-by: Akhil Narang <me@akhilnarang.dev>
2025-12-19 19:04:10 +05:30
Akhil Narang
23cdeb7bf5
fix(query): remove redundant check
`_check_field_permission` already checks `self.apply_permissions`

Signed-off-by: Akhil Narang <me@akhilnarang.dev>
2025-12-19 19:04:10 +05:30
Akhil Narang
2056071f8e
fix(query): check permissions for backtick quoted fields
Signed-off-by: Akhil Narang <me@akhilnarang.dev>
2025-12-19 19:04:10 +05:30
Akhil Narang
0d92b8b195
fix(query): validate permissions before join
Signed-off-by: Akhil Narang <me@akhilnarang.dev>
2025-12-19 19:04:10 +05:30
Abhiraaj R C
584e629c5f fix: parent table join for child tables of Single DocTypes 2025-12-18 18:50:26 +05:30
Akhil Narang
81f7a247f0
fix(query): track aliases for DynamicTableField as well
Child tables, links for example.

Signed-off-by: Akhil Narang <me@akhilnarang.dev>
2025-12-18 17:26:52 +05:30
Ankush Menat
2505fa9c8c
Merge pull request #35308 from akhilnarang/qb-ifnull-optimization
perf: Split ifnull into two conditions
2025-12-18 10:31:30 +05:30
Sagar Vora
d1ffd99fe8 fix(query): wrap CombinedRawCriterion in parentheses for correct precedence
Without proper grouping, OR conditions from shared docs could bypass WHERE filters:
  WHERE filter=X AND perm_cond OR shared_cond  -- shared_cond ignores filter!

With proper grouping:
  WHERE filter=X AND (perm_cond OR shared_cond)  -- correct behavior
2025-12-17 22:28:52 +05:30
Sagar Vora
c7e9e01eb8 chore: fix linter issues 2025-12-17 19:04:00 +05:30
Sagar Vora
591d9a3535 fix: update logic for child tables 2025-12-17 19:03:15 +05:30
Ankush Menat
7be9fec036
perf: Split ifnull into two conditions
This produces better query plan with index intersection using 2
conditions instead of fulltable scan on dumb condition

Signed-off-by: Akhil Narang <me@akhilnarang.dev>
2025-12-17 17:47:08 +05:30
Sagar Vora
2b822cc63f refactor(query): simplify permission logic in add_permission_conditions
- Extract _raise_permission_error helper for reuse
- Simplify get_user_permission_conditions to return list[Criterion]
- Rewrite add_permission_conditions to match documented logic:
  - No role perms → apply only share permissions
  - Role perms → (owner OR user perms) AND query conditions
  - Shared docs OR other conditions when applicable
- Move apply_permissions check inside add_permission_conditions
2025-12-17 15:59:37 +05:30
Akhil Narang
54a0081cf3
fix(query): use AND if adding shared query
Signed-off-by: Akhil Narang <me@akhilnarang.dev>
2025-12-16 17:11:09 +05:30
Akhil Narang
c7f5ea837a
feat: implement field masking for query builder (#35230)
* feat: implement field masking for query builder

Signed-off-by: Akhil Narang <me@akhilnarang.dev>

* fix: add series and sessions to "core doctypes" list

This is so that we don't try to query their meta
This should also resolve #35030

Signed-off-by: Akhil Narang <me@akhilnarang.dev>

---------

Signed-off-by: Akhil Narang <me@akhilnarang.dev>
2025-12-16 12:08:13 +05:30
Akhil Narang
207ee7a367
fix(query): match between behaviour for datetime fields with db_query
Signed-off-by: Akhil Narang <me@akhilnarang.dev>
2025-12-08 20:06:32 +05:30
Aarol D'Souza
cf69e4bed1
fix(postgres): misc query building fixes + CI (#34831)
* fix(query): check standard field definitions

Signed-off-by: Akhil Narang <me@akhilnarang.dev>

* fix(postgres): fix order_by problem in pg

* fix(postgres): fix order_by in get_all for _test_connection_query

* fix: add check to a proper numeric fallback in _get_ifnull_fallback

* test(postgres): fix pg query used in assertion in test_permission_query

* fix(postgres): fix order_by in get_all for possible_link

* fix(postgres): fix order_by in get_all for set_modules

* fix(postgres): fix pg query count *

* fix(postgres): fix order_by in get_all for ask_pass_update

* fix(postgres): fix order_by statement in search_widget

* fix(postgres): fix order_by in get_list for get_stats

* test(postgres): normalize_sql for pg queries in test_arithmetic_operators_in_fields

* test(postgres): normalize_sql for pg queries in test_field_alias_in_group_by

* test(postgres): normalize_sql for pg queries in test_field_alias_permission_check

* test(postgres): fix order_by statement in get_all for test_db_keywords_as_fields

* test(postgres): fix order_by statement in get_all for test_prepare_select_args

* fix(treeview): use 0 instead of false to check since check field is an integer

* fix(postgres): fix order_by in get_all for sync_communication

* fix(postgres): fix order_by in get_all for get_references_across_doctypes_by_dynamic_link_field

* test(postgres): fix order_by in get_all for test_list_summary

* fix(postgres): fix order_by in get_all for email queries

* test(postgres): use order_by none and update assertion for postgres

* fix(postgres): use ILIKE to support case insensitive search in postgres

* test(test_query): update pg specific query assert to use ILIKE

* test(test_query): update test_nested_filters to use ilike instead for PG

* test(postgres): update pg query in assert to test updated qb query

* fix(search): update query to be db-agnostic

* test(postgres): normalize query for pg in test_build_match_conditions

* fix(postgres): suppress ORDER BY when SELECT DISTINCT in query for postgres specific behavior

* fix(postgres): suppress ORDER BY when GROUP BY is explicitly asked for pg specific behavior

* test(postgres): fix test behavior for pg ORDER BY drop when used with GROUP BY

* refactor: reducing noise in code by formatting code

* fix(query): use Star() to handle SQL wildcard character * correctly

* fix(postgres): display warning for ORDER BY fields that will be dropped

---------

Signed-off-by: Akhil Narang <me@akhilnarang.dev>
Co-authored-by: Akhil Narang <me@akhilnarang.dev>
2025-12-05 10:30:49 +05:30
Sagar Vora
d4cc9160e5 refactor: precompile CHILD_TABLE_FIELD_PATTERN regex 2025-12-02 15:02:29 +05:30
Sagar Vora
00dd0ba6c7
fix!: use Field instead of Column for field-to-field comparison in query filters (#35012) 2025-12-02 06:24:41 +00:00
Sagar Vora
7cb6cf9830 fix: improve functions error message 2025-12-02 02:17:08 +05:30
Sagar Vora
f2ac5babe1 fix: dont use comma pattern for group by splitting 2025-12-02 02:12:24 +05:30
Sagar Vora
261463e06a fix: keep Column compatibility, stricter regex for backtick parse 2025-12-02 01:59:16 +05:30
Sagar Vora
5122334b4b perf: cache get_permitted_fields across Engine instance
- Add permitted_fields_cache dict to Engine instance
- Create _get_cached_permitted_fields() method shared by _check_field_permission and apply_field_permissions
- Move OPTIONAL_FIELDS check earlier to avoid unnecessary work
- Reduces redundant expensive permission lookups for filter fields
2025-12-02 01:10:44 +05:30
Sagar Vora
e983da0136 chore: simplify function field validation 2025-12-02 01:06:12 +05:30
Sagar Vora
c40933dca7 fix: restrict '*' argument to COUNT function only
- Add STAR_ALLOWED_FUNCTIONS frozenset with COUNT
- Pass function_name through parse chain to validate '*' usage
- Prevents '*' in functions like SUM(*), AVG(*) where it's invalid
2025-12-02 01:00:16 +05:30
Sagar Vora
99039c23cb chore: remove unused import 2025-12-02 00:48:57 +05:30
Sagar Vora
768a141814 refactor: simplify string literal handling in SQLFunctionParser
- Inline string literal check and extraction (was _is_string_literal + _validate_string_literal)
- Remove dangerous_patterns validation - pypika handles escaping with wrap_constant
2025-12-02 00:48:28 +05:30
Sagar Vora
a14b11749b refactor: remove unnecessary sql_keywords validation in _validate_alias
pypika wraps aliases in backticks, so SQL keywords are safe to use as aliases
2025-12-02 00:40:23 +05:30
Sagar Vora
3dcd80326b perf: replace sqlparse-based _is_function_call with regex
- Remove sqlparse import (no longer used anywhere in query.py)
- Add FUNCTION_CALL_PATTERN regex for detecting SQL function calls
- Simplify _is_function_call from 6-line sqlparse parsing to 1-line regex match
2025-12-02 00:38:13 +05:30
Sagar Vora
1e05d6c326 perf: pre-compile regex patterns for field and alias validation
Added SIMPLE_FIELD_PATTERN and IDENTIFIER_PATTERN at module level
to avoid repeated regex compilation in hot paths like filter validation
and alias checking.
2025-12-02 00:27:11 +05:30
Sagar Vora
b5bae10dc8 perf: remove unnecessary sqlparse.format and comment stripping in _sanitize_field
ALLOWED_FIELD_PATTERN already restricts fields to simple identifiers
that cannot contain SQL comments, making the sqlparse.format(strip_comments=True)
and MARIADB_SPECIFIC_COMMENT.sub() calls unnecessary overhead.

Merged _sanitize_field into _validate_select_field.
2025-12-02 00:23:25 +05:30
Sagar Vora
bcac30836b refactor: remove redundant share check in check_read_permission
has_permission() already checks for shared documents internally when
role permissions do not grant access, so the separate frappe.share.get_shared()
call was dead code.
2025-12-02 00:19:44 +05:30
Sagar Vora
5297edc732 refactor: use FIELD_PARSE_REGEX instead of sqlparse for _parse_backtick_field_notation 2025-12-02 00:14:58 +05:30