Commit graph

866 commits

Author SHA1 Message Date
Gavin D'souza
dbb37acedf fix: Transform queries with all types of values 2022-07-05 17:47:14 +05:30
Ankush Menat
5d2aedc698 fix(postgres): end transaction before DDL
closes https://github.com/frappe/frappe/issues/17210
2022-07-05 17:43:02 +05:30
Gavin D'souza
71b5c77e6e Merge branch 'develop' of github.com:frappe/frappe into mariadb-client-refactor 2022-07-05 14:55:02 +05:30
Ankush Menat
261fbfcd11
Revert "fix(doc)!: Always cast datetime, date and time fields"
Revert "fix(doc)!: Always cast datetime, date and time fields (#15891)"

This reverts commit d7789ab6ff.
2022-07-05 13:43:32 +05:30
Himanshu
d7789ab6ff
fix(doc)!: Always cast datetime, date and time fields (#15891)
### BREAKING CHANGE
#### Datetime, Date and Time fields will always be cast to respective objects in `setattr`, this will ensure uniformity while accessing the values, no more `getdate`, `get_datetime`, `to_timedelta` wrapper.
- While importing data, the framework does check for `set_only_once`.
- In normal case scenarios, this will work flawlessly since most date fields might not be set_only_once.
- But in Subscription, the date field is set to `set_only_once` and in `after_insert`, `document.save` is called, and while doing so, `set_only_once` is checked [here](1944a547f9/frappe/model/document.py (L566)).
-This works fine if the data imported is in the correct format.
- If the date's data is not in the correct format, the framework throws an error.
- for eg `06-02-2022 00:00:00 != 06-02-2022`
- fixes [Issue/#15370](https://github.com/frappe/frappe/issues/15370)

> no-docs
2022-07-05 07:07:16 +00:00
Ankush Menat
c12b52a259 refactor: use frappe.get_system_settings
Not sure why this needs YET ANOTHER separate cache.
2022-07-04 11:25:47 +05:30
Aradhya Tripathi
ebd1d5b48c
Merge branch 'develop' into get-all-mod 2022-07-01 13:25:32 +05:30
Ankush Menat
81b37cb7d2
refactor: clean up code to py310 supported features (#17367)
refactor: clean up code to py39+ supported syntax

- f-strings instead of format
- latest typing support instead of pre 3.9 TitleCase
- remove UTF-8 declarations.
- many more changes

Powered by https://github.com/asottile/pyupgrade/ + manual cleanups
2022-07-01 11:51:05 +05:30
Gavin D'souza
39d30ffeaa fix: Database._transform_query
* Handle dict substitutions in transformations too
* Allow list / tuple values
* Check for values inconsistencies before flattening step
2022-06-30 19:12:10 +05:30
Gavin D'souza
fddd624a7c Merge branch 'develop' of github.com:frappe/frappe into mariadb-client-refactor 2022-06-30 14:31:52 +05:30
Ankush Menat
813dcc1848
Revert "feat: Adding support to Query engine" 2022-06-29 23:30:23 +05:30
Aradhya
701bf2ede6 fix: fixed false operator placements in query 2022-06-29 22:34:43 +05:30
Aradhya
25bb945de7 feat: Added truediv & mul operators 2022-06-29 17:37:52 +05:30
Aradhya
4da5fdcd02 fix: fixed spaces in args
fix: lint
2022-06-29 17:37:49 +05:30
Gavin D'souza
daf3f05fea fix(db): Hanlde sequences in db.sql values 2022-06-29 16:32:00 +05:30
Aradhya
58645998c0 fix: fixed fields with operators and added abs 2022-06-29 15:03:37 +05:30
Gavin D'souza
cf699fe40b Merge branch 'develop' of github.com:frappe/frappe into mariadb-client-refactor 2022-06-29 13:49:18 +05:30
Ankush Menat
181833848f
Merge pull request #17337 from ankush/lint_cleanup
fix: remove bare exception catching
2022-06-29 11:39:05 +05:30
gavin
91e76ff874
Merge branch 'develop' into get-all-mod 2022-06-28 19:47:20 +05:30
Aradhya
303d94494d refactor: atomic functions & removed complicated checks
Co-authored-by: gavin <gavin18d@gmail.com>
2022-06-28 19:44:03 +05:30
Aradhya
6db6be1f3c refactor: frappe.qb.engine
* Small fixes in set_fields and clean code
* Optimize casefolds
* Fixed functions passed in List
* get_sql => get_query - more expressive, less confusion
* Updated tests
2022-06-28 19:43:57 +05:30
Aradhya
7732accded feat: Attached Engine object to qb & added dynamic type hints 2022-06-28 19:43:23 +05:30
Aradhya
960952cfc3 feat(qb-engine): Added Aggregation function support
* Added Min, Max
* Added tests
2022-06-28 19:41:39 +05:30
Aradhya
fca026927e refactor: moved all query logic to query class 2022-06-28 19:41:32 +05:30
Aradhya
d0680941ad refactor: frappe.qb.engine
* feat: supporting empty iterables for Contains objects
* fix: explicitly setting empty iterables as tuples to support more operators
* feat: Added locate to frappe.qb Functions
* feat: Added support for functions passed as strings in fields
* feat: Included Criterion objects as fields
* fix: picking up only function intended fields to pass to get_function_objects
* feat: Added iterable for available functions, added support for Field objects
* fix: fixed * passed in fields in lists
2022-06-28 19:41:23 +05:30
Aradhya
d1f5c49b02 refactor(qb-engine): frappe.db.query
* feat: Added support for True as filter and pluck in the query engine

* feat: Added join support

* fix: return if filters are None

* feat: Added support for sets as filters and additional conditions

* fix: fixed IS operator in query builder

* feat: Added support for 'as' in query engine

* fix: fixed 'as' for criterion objects passed directly

* fix: fixed frappe.db.count

* fix: fixed functions in fieldname

* feat: Added support for multiple fields passed as a single string :) fixed None fields in a list

* feat: Added support for "as" in single string fields

* fix: fixed queries with invalid syntax
2022-06-28 19:40:34 +05:30
Ankush Menat
d35d7ffbe2 fix: remove bare exception catching
A bare except catches lots of things (like generator iteration end) and should never be used.
2022-06-28 18:05:00 +05:30
Gavin D'souza
b8d2c195a6 fix: Disable connection pooling via bench commands unless specified 2022-06-23 19:42:12 +05:30
Gavin D'souza
ef078a4ab5 refactor(db-read_only): Track conn type in Database instance 2022-06-23 19:40:17 +05:30
gavin
574f160d3c
Merge branch 'develop' into mariadb-client-refactor 2022-06-23 16:13:03 +05:30
Gavin D'souza
14003e5ac9 refactor: DISABLE_DATABASE_CONNECTION_POOLING conf + var name 2022-06-20 15:31:15 +05:30
Gavin D'souza
a58a5bb848 fix: Improve _transform_query unique key identification 2022-06-20 13:10:13 +05:30
Gavin D'souza
1f1d91a056 fix: Make postgres' last_query lazy decodable 2022-06-18 14:04:38 +05:30
phot0n
f6c1eb10dd refactor(minor): modify_values
* don't truncate float(s)
2022-06-18 10:11:58 +05:30
phot0n
2ce6597a69 fix: cast list to tuple when sending parameterized query for postgres 2022-06-17 18:47:06 +05:30
Gavin D'souza
9b6a048bcd refactor(minor): Use db.db_type instead of conf.db_type 2022-06-17 12:49:12 +05:30
Gavin D'souza
3af8d5caea fix: Add fallbacks for values
Psycopg seems to like None over () and MariaDB - PyMySQL can't seem to
agree on anything - so this seems to keep everyone happy...a very
delicate balance :crie:
2022-06-17 12:47:44 +05:30
Gavin D'souza
90c716bce0 fix(sequence): Setup & use SequenceGeneratorLimitExceeded error 2022-06-17 12:10:43 +05:30
Gavin D'souza
a03bf6b0bb fix(db): Transform values only if not None 2022-06-17 11:31:55 +05:30
Gavin D'souza
1a772e304c fix(db): Store result of last executed query under frappe.db.last_result 2022-06-16 16:01:38 +05:30
Gavin D'souza
25b87a9d49 chore: NoneType alias for < PY310 2022-06-16 15:04:35 +05:30
Gavin D'souza
bc3780560e perf: Use lazy mogrified query for logging
For parameterized queries, there's an improvement of ~30% in query
execution via frappe.db.sql - from 58.6 µs ± 2.37 µs to 44.6 µs ± 1.56 µs
2022-06-16 14:26:34 +05:30
Gavin D'souza
effa942f4c refactor: frappe.db.log_query
* Mogrify queries and set them as frappe.db.last_query instead of
  directly interfacing with the clients
* This is required for now as the MariaDB client uses binary protocol to
  talk to the server and doesn't build the queries itself
* Add typing hints
* Imported Query object as FilterEngine - Query is too ambiguous lol

ref: https://jira.mariadb.org/browse/CONPY-208?focusedCommentId=226873&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-226873
2022-06-16 13:28:54 +05:30
Gavin D'souza
889ced3035 refactor: frappe.db.sql
* Move everything except _cursor.execute outside try-except block - This
  caused multiple traceback printing (by recursion of db.sql)
* Include values mogrifying & executing via client alone in execution
  time for query
* Reduce indentations lol
2022-06-16 12:39:30 +05:30
Gavin D'souza
0adf5e127a fix(db): Track query engine under db._filter_engine 2022-06-16 12:31:14 +05:30
Gavin D'souza
ded55fd98e fix(db): Skip transformation of query if no named params found 2022-06-15 17:43:44 +05:30
Gavin D'souza
5a29177e6b fix(db): Log queried tables through generated query 2022-06-15 17:42:52 +05:30
Gavin D'souza
b4578dc632 Merge branch 'develop' of github.com:frappe/frappe into mariadb-client-refactor 2022-06-15 16:40:29 +05:30
Gavin D'souza
57bd1b6031 fix(db): Import cast as cast_fieldtype to manevour ambiguity 2022-06-15 10:54:48 +05:30
Gavin D'souza
bbc90e6578 refactor!: frappe.db.get_singles_dict
* Don't cast values by default - only if cast kwarg is set
* Reverts breaking behaviour added via f74dc5023d
2022-06-15 10:33:06 +05:30