Commit graph

125 commits

Author SHA1 Message Date
David Arnold
75377aaaf5
refactor(typing): type filters (#28218)
* chore(typing): type filters

* chore(typing): type filters for get_list et al

* fix: dashboard chart filter expression

* test: fix case with new-style right hand object to equality check

* chore: place new typed filter under typing verification

* chore: remove debug print statment

* chore: inverse logic of type guard

* fix: add float to filter value types

* chore: clarify value naming
2024-12-04 23:18:53 +00:00
Akhil Narang
e96ff7e65e
Merge pull request #28015 from marination/not-set-filter
fix: 'not set' equivalence in query conditions
2024-10-14 16:00:55 +05:30
David Arnold
83bc1f09e9
refactor: clarify test record dep management in test modules (#28060) 2024-10-09 13:44:27 +00:00
marination
6ab347d9f5 fix: 'not set' must translate to the same condition whether used via QB or not 2024-10-07 18:52:29 +02:00
David Arnold
c114e5fae8
refactor: unit vs integration treewide (#27992)
* refactor: constitute unit test case

* fix: docs and type hints

* refactor: mark presumed integration test cases explicitly

At time of writing, we now have at least two base test classes:

- frappe.tests.UnitTestCase
- frappe.tests.IntegrationTestCase

They load in their perspective priority queue during execution.

Probably more to come for more efficient queing and scheduling.

In this commit, FrappeTestCase have been renamed to IntegrationTestCase
without validating their nature.

* feat: Move test-related functions from test_runner.py to tests/utils.py

* refactor: add bare UnitTestCase to all doctype tests

This should teach LLMs in their next pass that the distinction matters
and that this is widely used framework practice
2024-10-06 09:43:36 +00:00
Ankush Menat
005e74b20d
perf: Avoid coalesce for between filters (#26531)
- Avoid on `between` + date
- Avoid on timestamp fields
- Avoid on `>` and `>=` comparisons
2024-05-22 09:32:59 +00:00
Ankush Menat
ff1f8ddbda
fix: allow any sequence container (#25664) 2024-03-27 06:31:15 +00:00
Ankush Menat
f642b1881f fix: invalid select star expansions 2024-03-20 16:31:02 +05:30
Ankush Menat
8a7beebf30
fix: handle distinct for fieldname (#25511)
`distinct count(fieldname)` is supported well but `count(distinct fieldname)` fails if fieldname contains full field with table name included. This PR just adds basic handling for it.

Needs to be rewritten entirely in QB __some day__.
2024-03-18 13:38:29 +00:00
Ankush Menat
a49fafbf8e feat: support countig till a limit
In InnoDB counting is essentially O(n) operation, it can be pretty fast
on indexes but when filters don't use any index it usually means doing a
full table scan.

Adding a limit will stop the scan as soon as that many records are
matched.
2024-03-12 13:17:28 +05:30
Ankush Menat
f7c0dd66fd refactor: migrate virtual doctypes to new API 2024-03-11 18:25:05 +05:30
Ankush Menat
4ab2ccd01f fix!: get_count accepts splatted arguments 2024-03-11 18:15:27 +05:30
Ankush Menat
daf43cff79
test: add test case for in ("") (#25189) 2024-03-02 06:00:02 +00:00
Akhil Narang
26ae0f3460
fix: ruff fixes
Signed-off-by: Akhil Narang <me@akhilnarang.dev>
2024-02-07 17:04:31 +05:30
Ankush Menat
de9ac89748 style: re-format with ruff 2024-02-05 18:53:33 +05:30
Ankush Menat
3bea50d519
fix: Return empty result if no perm level access (#24591) 2024-01-29 19:48:19 +05:30
Ankush Menat
5deabdde21 fix: skip virtual fields in perm level checks during DB Query
DB Query can't access virtual fields so it should ignore all virtual
fields.
2023-12-20 12:00:08 +05:30
Ankush Menat
687752359d
perf: Primary key is never nullable (#23788)
People ususally write queries like these...

```
frappe.get_all(doctype, {"name": ("in", list_of_docs))
```

Ocassionally, the `list_of_docs` is empty because it's dynamically
generated and in this case we end up doing full table scan to find... nothing!
2023-12-14 15:01:19 +00:00
Shariq Ansari
adbd88d394 test: fixed failing test 2023-12-08 15:20:50 +05:30
Ankush Menat
385fa8aaef
fix!: Correct between filtering (#22918)
* fix: Remove incorrect fallback

If you do +1 on date it will also start considering next date. This was
only done to accomodate date filter on datetime fields. Which also
doesn't really work.

* refactor: simplify fieldtype detection

* fix!: Correct datetime fallbacks for between filters

* chore: remove unncessary test

This is very specific and introduces flake when the job tests run before
it.
2023-10-26 06:23:14 +00:00
Shariq Ansari
e1c1e256ef
Merge pull request #21798 from pps190/frappe/fix/ambigus-table-join 2023-07-28 12:28:04 +05:30
Ankush Menat
8930d4b5e1
perf: drop ifnull from IS SET filter (#21822)
- Kinda confuses query planner (idk why it's not smart enough to
  understand but there are probably edge cases where it can't be done)
- `null != null` and `'' != null` both yield `null` which is falsy and
  won't be shown in results.

Alternate fix to https://github.com/frappe/frappe/pull/21817
2023-07-27 10:58:20 +05:30
Devin Slauenwhite
160f0b65fb
Merge branch 'develop' into fix-ambigus-table-join 2023-03-03 09:44:29 -05:00
gavin
3406b6d752
fix(db_query): selecting linked fields as alias (#20085)
* fix(db_query): Don't track link_tables separately

Treat all joined tables the same, expand the fieldnames whether Table or
Link type with their respective table names

* fix(db_query): Add link join conditions for all relevant fields

* Revert "fix(db_query): Add link join conditions for all relevant fields"

This reverts commit 79622ab4cea5aa73a24f4ba7afde8e83510a79fb.

* Revert "fix(db_query): Don't track link_tables separately"

This reverts commit b8364f781e52e7ffaa7faa8878ef409b023f2288.

* fix: Check link field tables permissions in permlevel checks

* test: Fix, add for test_permlevel_fields

Previous assertion was wrong :')
added extra to check if access is fine
2023-02-22 22:14:15 +05:30
Gavin D'souza
c4544cb37e test: test_get_count 2023-02-03 20:53:53 +05:30
Gavin D'souza
3cd781e316 Merge branch 'develop' into get_all-virtual-dts 2023-02-02 13:55:24 +05:30
Gavin D'souza
fdff6351cd test: Add test for DatabaseQuery for virtual doctypes 2023-02-02 13:45:35 +05:30
Gavin D'souza
c4061904da test: Split DBQuery & ReportView API tests into separate cases 2023-02-02 13:45:17 +05:30
Gavin D'souza
9efe84a644 fix(reportview): Remove aggregate_on_field added in fields
Essentially reverts changes added via
https://github.com/frappe/frappe/pull/14424 but works just the same
without the additional column :thonk:

The added column's data in mariadb was essentially garbage data. Wasn't
meaningful from what I could tell - couldn't play well with postgres
either
2023-01-20 19:26:56 +05:30
gavin
4be74bc013
Merge branch 'develop' into permlevel-apis 2023-01-19 11:00:15 +05:30
Gavin D'souza
f0a282e941 test: Add test for linked table permission check 2023-01-16 14:28:13 +05:30
Gavin D'souza
7e38d7fe63 test: Make test_child_table_join more resilient 2023-01-16 13:56:37 +05:30
Devin Slauenwhite
1fa9c15240 fix(test): test no pymysql.err.OperationalError 2023-01-13 14:39:21 -05:00
Devin Slauenwhite
af579e3192 test: ambiguous linked tables 2023-01-13 14:09:09 -05:00
Gavin D'souza
9012b95a9a fix(postgres): group_by in reportview, tests
Add approporiate group_by, order_by clauses in reportview and tests for
featureset compatibility with postgres
2023-01-11 12:42:45 +05:30
Gavin D'souza
d2ad86d2fe test: Add tests for permlevel handing in get_list 2023-01-10 17:44:06 +05:30
Faris Ansari
be654eaa60
Merge branch 'develop' into refactor-qb-engine 2023-01-09 15:23:20 +05:30
Faris Ansari
726fcfdb79 refactor: qb.engine
- simplify
- qb.engine.get_query -> qb.get_query
- qb.engine.build_conditions -> qb.get_query
2022-12-25 23:19:11 +05:30
gavin
6062d812a1
fix(db_query): Disallow usage of certain functions in *_by (#18981)
* fix(db_query): Disallow blacklisted functions in (order|group)_by

Changes:
- allow only functions that are not blacklisted in *_by clause:
  currently just sleep
- perf improvemnts: lower, in, split, strip & other low hanging  micro optimizations

Handle the following use cases:
- upper/lower case function usages
- spaces between function name and brackets

* test(db_query): Add tests for *_by checks
2022-11-28 12:42:29 +05:30
Gavin D'souza
1a5e5f546b fix: Move function check inside subquery 2022-11-24 16:27:54 +05:30
Gavin D'souza
1f913248aa test: Add more tests for illegal subquery and fn usage 2022-11-24 15:36:31 +05:30
Shariq Ansari
235171796d
fix: coalesce not in queries (#18099)
* fix: get workspaces with empty module fields

* Revert "fix: get workspaces with empty module fields"

This reverts commit 1f194be2c3642e31ebe2165e461b2f24be8cda4c.

* fix: always coalesce `not in` queries

Co-authored-by: Ankush Menat <ankush@frappe.io>
2022-09-13 12:08:07 +05:30
Ankush Menat
cdb0732646
perf: avoid coalescing wherever possible (#17920) 2022-08-24 10:30:51 +05:30
Ankush Menat
2eec621e95 chore: db.get_all -> get_all
Reduces 1 pointless function call.
Function calls are also not "super cheap" in python.
2022-08-22 11:35:14 +05:30
Faris Ansari
82fef72ce1
fix(db-query): parenttype condition on child table left join (#17865)
joining only on name is incorrect because parent primary keys are not unique across tables

How was this working all this time? 🥲
2022-08-18 09:04:40 +00:00
Ankush Menat
3e2d2a703a test: Use FrappeTestCase everywhere 2022-08-17 16:39:42 +05:30
phot0n
a52483f110 Revert "fix: remove integration request check from test_is_set_is_not_set"
This reverts commit b3f57f0e7774928df90bdf84c7ec23b5d3be53c8.
2022-07-26 23:17:04 +05:30
phot0n
91b04c2154 test: fix test_set_field_tables
removed amount_field from groupby and fields
2022-07-26 23:17:04 +05:30
phot0n
5c2cfdd1e9 fix: remove integration request check from test_is_set_is_not_set 2022-07-26 15:11:56 +05:30
Aradhya
4da5fdcd02 fix: fixed spaces in args
fix: lint
2022-06-29 17:37:49 +05:30