vassili/seitime-frappe
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1
51162 commits 1 branch 0 tags 587 MiB
Commit graph

984 commits

Author SHA1 Message Date
Ejaaz Khan
a52da6c8d6 fix(db_query): != condition not working in case of None 2025-09-30 13:23:16 +05:30
Akhil Narang
6ca4d4d167
refactor(treewide): ruff format 
Signed-off-by: Akhil Narang <me@akhilnarang.dev>
 2025-09-26 13:16:43 +05:30
Hussain Nagaria
9bd8dc8900 refactor: cleaner handling of length and precision 2025-09-17 23:54:06 +05:30
Hussain Nagaria
c230e86e2c fix: handle length more robustly 2025-09-17 13:27:13 +05:30
Hussain Nagaria
8f83dedbba feat: allow length change for decimal columns 2025-09-03 15:28:10 +05:30
Prafful S
bd4fc118e5
feat: Database SSL one way support (#33044) 
* feat: Support one-way SSL authentication for database connections

Updated MariaDB database connection to support one-way SSL authentication

* feat(database): Add support for MySQLClient one-way SSL connections

* feat: Format and pre-commit

* fix: Default option for hostname check

* chore: simplify

Signed-off-by: Akhil Narang <me@akhilnarang.dev>

---------

Signed-off-by: Akhil Narang <me@akhilnarang.dev>
Co-authored-by: Akhil Narang <me@akhilnarang.dev>
 2025-07-16 06:36:19 +00:00
Ankush Menat
1ed2447f6b
fix!: Always cast db.get_value for singles (#33276) 2025-07-11 07:32:41 +00:00
Akhil Narang
8cbd51eafa
fix(sqlite): accept chain for commit and rollback (#33250) 
Don't do anything with it for now

Signed-off-by: Akhil Narang <me@akhilnarang.dev>
 2025-07-08 07:14:29 +00:00
Sagar Vora
71428b7dcb
feat: utility to commit certain queries after sending response (#32978) 
* feat: utility to commit certain queries after sending response

* fix: add error handling

* refactor: subclass CallbackManager
 2025-07-08 05:02:27 +00:00
Ankush Menat
35c1cc0276
fix(DX): print execution time after printing query (#33207) 
Who reads it the other way around 🗿
 2025-07-03 09:18:19 +00:00
ruthra kumar
f19bd7c915
Merge pull request #33189 from ruthra-kumar/get_existing_procedures_and_functions 
refactor: utility method to fetch custom sql functions and procedure
 2025-07-03 10:22:51 +05:30
ruthra kumar
1ceea953fb refactor: utility method to fetch custom sql functions and procedure 2025-07-02 17:07:34 +05:30
Ankush Menat
f6567dabee
Revert: Keep None check (#33167) 
No idea why, but some numeric fields can have unspecified defaults.
 2025-06-30 12:45:38 +00:00
Corentin Forler
e94361e12b
fix(schema): Fix default value change detection for Time fields (#33142) 
* fix(schema): Handle "NULL" in default_changed_for_decimal

* fix(schema): Fix default value change detection for Time fields

* chore: format

* fix: Don't set zero default during change

This makes the check more aggresive. We by default set 0 on all numeric
fields, so we should ideally only migrate default between numbers and
not check `NULL` ever.

---------

Co-authored-by: Ankush Menat <ankush@frappe.io>
 2025-06-30 11:34:01 +00:00
Ankush Menat
1a1dc0a62c
fix: more bad migrations and sanity test (#33112) 
* test: prevent unnecessary migrations

* fix: Avoid resyncing JSON repeatedly

* fix: Varchar not nullable defaults should be casted

* fix: force cast to float before

Bad default values cause it to break.
 2025-06-26 10:31:55 +00:00
Ankush Menat
02ee722e5a
fix: Avoid unnecessary int/long-int migrations (#33109) 
Because size wasn't specified it kept syncing over and over again.

Specified default sizes for mariadb.
 2025-06-26 12:39:44 +05:30
Ankush Menat
40b465ee0d
fix: Avoid unnecessary syncing defaults (#33108) 
When default is `'0.0000'` (string) it gets synced again and again even
though it will end up being 0 again.
 2025-06-26 06:49:44 +00:00
Ankush Menat
2a1d3dd474
fix: Unnecessary migration from hard-coded not-nullable fields (#33107) 2025-06-26 06:25:14 +00:00
Faris Ansari
d584b8691d
Merge pull request #32381 from netchampfaris/api-v2-fixes 
feat!: Apply permissions in frappe.qb.get_query

Enhancements:
- Supports applying permissions
- Strict input parsing

BREAKING CHANGE:
```py
# Before
frappe.qb.get_query("Task", fields=["sum(is_completed) as count"])

# After
frappe.qb.get_query("Task", fields=[{"SUM": "is_completed", "as": "count"}])
```
 2025-06-23 15:50:25 +05:30
Faris Ansari
c2e08b3822 chore: remove unused code 2025-06-23 14:54:22 +05:30
Ankush Menat
b57eb60486
perf: chain db transactions (#33004) 
* perf: chain transactions

Frequently used rollback/commits can be modified to chain previous
transaction.

This reduces one query to DB in most requests.

* perf: chain transactions in requests
 2025-06-19 12:37:39 +05:30
Faris Ansari
f2a0724f9a feat: add back sql functions support with json syntax 
```
fields=['user_type', {'COUNT': 'name', 'as': 'total'}]
fields=[{"IFNULL": ["first_name", "'Unknown'"], "as": "safe_name"}]
```
 2025-06-18 16:15:25 +05:30
Faris Ansari
840e7991ce fix: dont allow partial backticks 
- add tests
 2025-06-18 16:14:32 +05:30
Faris Ansari
3f65806a0b fix: harden group by and order by inputs 
- only field, link_field.field, child_field.field allowed
- dont allow backticks
- add permlevel check
- add tests
 2025-06-18 16:10:52 +05:30
Faris Ansari
420e891d96 feat: remove support for sql functions in fields 2025-06-18 16:10:52 +05:30
Faris Ansari
9a84f20436 feat: add support for nested AND and OR conditions 2025-06-18 16:10:52 +05:30
Faris Ansari
b2a37f86b3 fix: filtering should only be allowed on permitted fields 2025-06-18 16:00:22 +05:30
Faris Ansari
f77a940582 fix: check permlevel for fields like "link_field.fieldname" 2025-06-18 16:00:22 +05:30
Faris Ansari
63afc0601b fix: restrict child table access if user has only "select" on parent 2025-06-18 16:00:22 +05:30
Faris Ansari
87664ad604 refactor: Enhance field and function parsing in query engine 
- Introduce `SqlFunctionParser` for robust parsing of supported SQL functions (e.g., `COUNT(*)`, `SUM(amount) as total`, `AVG(price - cost)`), replacing get_function_object and has_function.
- Refactor `DynamicTableField.parse` for improved handling of:
    - Aliases (case-insensitive `as`, quoted/unquoted).
    - `tabDocType.fieldname` notation (distinguishing child vs. main doctype refs).
    - Add validation and better error handling during parsing.
- Rewrite filter field validation (`_validate_and_prepare_filter_field`):
    - Disallow backticks (`) in filter field names.
    - Enforce specific patterns for dot notation (link/child fields only, reject `tabDoc.field`).
    - Validate character sets for simple field names.
- Update standard field parsing (`parse_string_field`, `ALLOWED_FIELD_PATTERN`, `FIELD_PARSE_REGEX`):
    - Support quoted table names potentially containing spaces (e.g., `tabTable Name`.`field`).
- Improve `parse_fields` and `_parse_single_field_item` logic:
    - Handle direct pypika `Field`/`AggregateFunction` inputs.
    - Reliably split comma-separated field strings.
```
 2025-06-18 16:00:22 +05:30
Faris Ansari
ddca77429c fix: secure query building 
Add strict validation using regex for fields in SELECT, filters, GROUP BY, and ORDER BY clauses to avoid potential SQL injection risks.

Refactor field parsing and validation logic into dedicated functions.
 2025-06-18 16:00:22 +05:30
Faris Ansari
8aa4c1030f fix: add support for AND, OR, NOT in RawCriterion 2025-06-18 15:56:11 +05:30
Faris Ansari
a94c143314 fix: add support for permission query conditions 2025-06-18 15:56:11 +05:30
Faris Ansari
f707cf5722 fix: raise PermissionError instead of ValidationError 2025-06-18 15:56:11 +05:30
Faris Ansari
f580cb3dad fix: add child query to allowed fields 2025-06-18 15:56:11 +05:30
Faris Ansari
471e001ebb feat: apply permissions in get_query 2025-06-18 15:56:11 +05:30
Ankush Menat
3a7db9cbb7
refactor: Default to mysqlclient (#32987) 2025-06-18 06:04:24 +00:00
Ankush Menat
7b8eb5d1b6
feat: MariaDB 11.8 support (#32289) 
* ci: Switch to MariaDB 11.8

* ci: Use mariadb's latest client libraries

Co-Authored-By: Sagar Vora <sagar@resilient.tech>

---------

Co-authored-by: Sagar Vora <sagar@resilient.tech>
 2025-06-12 10:11:24 +00:00
Sagar Vora
d35c1d958f
perf(query engine): create one less copy (#32889) 2025-06-11 12:51:55 +05:30
Ankush Menat
271fe0e47e
fix!: Dont silently ignore bad filters (#32871) 2025-06-10 12:06:44 +05:30
Ankush Menat
e4bc7f361b
Revert: DocRef (#32866) 
- Hardly used anywhere
- Too many hardcoded `__value__` calls without which it's not usable.
- Another type to worry about
 2025-06-10 05:20:56 +00:00
Ankush Menat
292646a5a7
Revert "Revert "perf: Make get_query query mutable (#32849)" (#32855)" (#32857) 
This reverts commit 85dc9e6981.
 2025-06-09 14:43:27 +00:00
Ankush Menat
85dc9e6981
Revert "perf: Make get_query query mutable (#32849)" (#32855) 
This reverts commit dda62ff784.
 2025-06-09 13:17:16 +00:00
Ankush Menat
dda62ff784
perf: Make get_query query mutable (#32849) 
pypika internally keeps copying query builder object because everything
is supposed to be immutable in pypika design, this however is terribly
slow. Often query generation takes more time than query execution.

This PR makes query builder mutable inside `get_query` function to avoid
copying while applying fields, filters, limit, order etc.

It's marked as immutable again when sending it back to users of the API.
 2025-06-09 13:46:40 +05:30
Ankush Menat
f1a03200ab fix: clear db.value_cache when clearing doctype cache 2025-06-05 14:46:03 +05:30
Ankush Menat
47a47a9b5d refactor!: Change internal datastructure of db.value_cache 
It's now a defaultdictionary of `[doctype][name/filters][fieldname]`

This allows us to implement granular clearing and improve usage of this cache.
 2025-06-05 14:46:03 +05:30
Ankush Menat
2d14918814
fix!: Change count(cache=True) implmentation (#32779) 
This makes cache implementation uniform for all methods on db API. It's
weird that this specific method was caching in redis, which defies
expectations.
 2025-06-04 19:15:27 +05:30
Sagar Vora
a212ca8be5 fix: better regex for extracting query type 2025-05-30 11:23:14 +05:30
Sagar Vora
8192a87d00 perf: prebuild types for type checking 2025-05-30 11:22:12 +05:30
Ankush Menat
e2d619504f
perf: batch bulk_insert (#32675) 
* fix: reduce bulk insert batch size

Back when this feature was added it used to lazily evaluate the input.
Now the iterator is consumed upfront so large batch sizes == huge memory usage.

* perf: bring back iterator for bulk_insert

Bulk insert used to support iterator for consuming arbitrarily large
amount of data and inserting it. Since child table support was added, it
can't do it anymore because that requires collecting values.

This change now brings back iterators by batching input iterator (by
default 1000) documents.

This is almost as good as original change from design POV. Performance
is still meh for flat documents.
 2025-05-26 13:36:53 +00:00