Commit graph

210 commits

Author SHA1 Message Date
Gavin D'souza
57bd1b6031 fix(db): Import cast as cast_fieldtype to manevour ambiguity 2022-06-15 10:54:48 +05:30
Gavin D'souza
bbc90e6578 refactor!: frappe.db.get_singles_dict
* Don't cast values by default - only if cast kwarg is set
* Reverts breaking behaviour added via f74dc5023d
2022-06-15 10:33:06 +05:30
Gavin D'souza
f74dc5023d refactor!: frappe.db.get_singles_dict
* Cast single's values as their fieldtypes before returning
* Support previously dead debug parameter
* Consider single with no meta as non-existent; skip query

Decided to go ahead with the breaking change given the nature of the
existing usages of get_singles_dict :crie:
2022-06-13 18:39:56 +05:30
Gavin D'souza
13cf4964a6 perf: Check query type via is_query_type 2022-06-13 18:39:56 +05:30
Gavin D'souza
b696fa6da5 perf: Pre-compile and re-use regexp pattern
Converted all possible usages of re.* that weren't compiling the regex
separately and re-using it. Separated out the compiled patterns as
global variables. Repetitive patterns could be made DRY-er.

Would be nicer to have all regexes in a single module so that we could
re-use better, keep track of outdated, and keep checks for possible
reDos' etc
2022-06-08 14:07:38 +05:30
gavin
44dba28159
Merge branch 'develop' into refactor-file 2022-06-08 12:43:35 +05:30
Ankush Menat
94920ab4ae
Merge pull request #16727 from ankush/pg_transaction_management
fix: explicitly start a new transaction
2022-06-07 12:46:23 +05:30
Gavin D'souza
50b0eb8cca perf: Initialize Query engine only if used 2022-06-06 15:42:59 +05:30
Ankush Menat
8861752675 fix: restart transaction after commit 2022-06-05 00:26:12 +05:30
Ankush Menat
654ded1a05 chore: print full stack on postgres DB errors 2022-06-04 22:42:03 +05:30
Gavin D'souza
5ec2690160 Merge branch 'develop' into refactor-file 2022-06-01 15:47:55 +05:30
gavin
7c77bedbf2 refactor: Simplify logic + Add typing hints 2022-05-18 16:57:54 +05:30
Suraj Shetty
1469339335 revert: "refactor: new query engine in get_count()"
Co-Authored-By: Gavin Dsouza <gavin18d@gmail.com>
2022-05-17 13:30:36 +05:30
gavin
d29641f734
Merge branch 'develop' into aks-feat-dbquery_update 2022-05-09 12:02:36 +05:30
Ritwik Puri
b41379c78b
fix: misc fixes (integer primary keys) (#16307)
* fix: misc fixes

local.x gets resetted on every request so switched to a simple dict

simplified is_val_used in set_next_val function for sequences

* chore: use multisql for sequence methods

* fix: fields not updating on form

* minor(base_input): removed unnecessary branching in update_input

* chore: remove prints and rename autoincremented_status_map

* chore: added proper type hint + comment + formatting

* fix: added searching in cast_name rather than handling it manually

* fix: share condition query + test_build_match_conditions

* fix: add cast_name to more places

* test: test for sequence

* fix: sequence functions

* fix: inherit frappetestcase

* minor: attach sequence methods to db context local

* chore: update sequence function names in Database

use frappe.db for sequences in naming.py

* fix: convert filename to str (for autoincremented doctypes)

* chore: better regex for modifying values for postgres

* minor: allow changing name column type (if no data is present in the doctype)

* refactor: validate_autoname

converted it to a simple function
enabled changing autoincrement autoname from customize form

* fix: use sql_ddl for change_column_type in postgres

* fix: use not null constraint in postgres when changing name type

* fix(test): updated test_autoincremented_doctype_transition with transitioning when no data is present

* fix(test): updated test_cast_name

probably messed up during rebase

* fix(test): used rollback upon error in transaction for postgres

* chore: use frappe.db.x methods for sequences

* minor: use temporary sequences in test

* minor: use generate_hash for sequence naming in sequence tests

* chore: replace sequence imports with frappe.db.x

* chore: move out casting name fields to a separate method

* refactor: cast_name

more explicit cases for casts and added docstring

* fix: added space in test_cast_name

* chore: fix linter

* chore: better naming for can_change_name_column_type

* chore: add comment for autoincremented_site_status_map

* chore: update/add docstrings
2022-04-29 15:06:03 +05:30
saxenabhishek
b1a0a3816b style: Applied Frappe linting 2022-04-28 17:10:31 +05:30
saxenabhishek
d39e6a7890 refactor: new query engine in get_count() 2022-04-28 17:10:31 +05:30
saxenabhishek
c9df86f9ef feat: multi table queries 2022-04-28 17:10:31 +05:30
gavin
2da85fa8da
Merge pull request #16698 from gavindsouza/traceback-with-context
feat: Traceback with context
2022-04-22 13:04:07 +05:30
Ankush Menat
dfef7192da
refactor: remove duplicate code from db.get_descendants (#16699) 2022-04-21 15:27:59 +05:30
Gavin D'souza
c691537e61 chore: Add typing for ease of development 2022-04-21 13:32:13 +05:30
Gavin D'souza
1d84483289 Merge branch 'develop' of github.com:frappe/frappe into refactor-file 2022-04-21 11:45:50 +05:30
Gavin D'souza
35c606707e fix(bulk_insert): Cast values as list before subscripting 2022-04-20 16:43:19 +05:30
Pruthvi Patel
8b010e1732
refactor: make frappe.db.bulk_insert work as expected (#16527)
## Issue

`frappe.db.bulk_insert` is not working as expected:
 - It will not insert any row if there are less than 3 values
 - It will not add 1st row at all. e.g if I'm adding 5 values, it will only add 4.
 - It will add values one by one after 2 values, instead it should have inserted items (in db) in chunk of 10000 (as per the code written before).

## Changes Made
 - Solved above issues 
 - use better way to chunk list
 - Added Postgres support for bulk_insert API
 
And now `bulk_insert` will only do **1 db call for each 10000** values.

Note: For testing purpose I made `Test Bulk Insert` doctype and keep chunk size of 100.

## Before
![image](https://user-images.githubusercontent.com/43115036/161979365-c1100745-7602-47d2-a9b8-62d797d2039f.png)

## After
![image](https://user-images.githubusercontent.com/43115036/161978344-3c17d56b-2195-40f4-b00c-e9478d4083f1.png)

nodocs
2022-04-19 14:47:22 +00:00
Gavin D'souza
b014ff8ce6 refactor: Simplify DB/QB API definitions 2022-04-19 19:09:10 +05:30
Gavin D'souza
08402a76f1 fix: Use query.run to execute query instead of db.sql 2022-04-19 19:06:43 +05:30
Gavin D'souza
97e911e069 Merge branch 'develop' of github.com:frappe/frappe into refactor-file 2022-04-13 10:49:26 +05:30
Suraj Shetty
c0c5b2ebdd
style: format all python files using black (#16453)
Co-authored-by: Frappe Bot <developers@frappe.io>
2022-04-12 10:59:25 +05:30
Gavin D'souza
ce5ac24b61 feat(db): Set frappe.db.debug to mass debug queries 2022-04-04 15:15:51 +05:30
gavin
3f6bbd6349
Merge pull request #16474 from resilient-tech/perf-get-doc-filters
perf: reduce query when calling `get_doc` with filters
2022-04-01 15:53:55 +05:30
Sagar Vora
e73c552632 fix: implement for_update for Single documents 2022-04-01 14:15:35 +05:30
Sagar Vora
493c7025f5 fix(get_value): make as_dict work for list-style filters 2022-04-01 00:33:15 +05:30
Ankush Menat
c4ca58fedf refactor: query stripping 2022-03-28 10:56:55 +05:30
Ankush Menat
456cba71cd refactor: expand unreadable one-liner in get_value 2022-03-28 10:56:55 +05:30
Ankush Menat
3e1df641da fix: force keyword arguments in new ORM args 2022-03-28 10:56:55 +05:30
Gavin D'souza
47cf46cd49 Merge branch 'develop' of github.com:frappe/frappe into refactor-file 2022-03-24 21:38:21 +05:30
gavin
a7f1639e65
Merge branch 'develop' into log-settings 2022-03-23 13:20:33 +05:30
Gavin D'souza
300227ba71 fix(set_single_value): Make value parameter optional
Similar to set_value for accepting multiple columns ot be updated for
the same Table through a Dict as the second positional arg

Misc: Added type hints
2022-03-23 13:11:55 +05:30
barredterra
44a7c0dd93 fix: copy dict before popping keys 2022-03-21 20:51:59 +01:00
Suraj Shetty
a1664330a7
Merge pull request #16344 from frappe/mergify/bp/develop/pr-16239 2022-03-21 20:49:33 +05:30
Shariq Ansari
5c6c9bb5c4
fix: resolved conflicts in database.py 2022-03-21 18:16:54 +05:30
Gavin D'souza
287543311c fix: Remove duplicate rollback observers and maintain order 2022-03-21 16:28:57 +05:30
Shariq Ansari
fbac6fbfb4 fix: only remove \n\t from start and end
(cherry picked from commit 7bb172365f2c9ae6cca98ccc4dfee7714b9c3f0c)

# Conflicts:
#	frappe/database/database.py
2022-03-21 05:50:58 +00:00
Shariq Ansari
b2c0bf7a4e fix: remove tab \t and newlines \n from start of query and remove from middle
(cherry picked from commit ac5effc7dd4d876d06daf945f0b8b77ecdd0c05f)

# Conflicts:
#	frappe/database/database.py
2022-03-21 05:50:57 +00:00
Ankush Menat
b97cfed6d7 perf: limit rows to 1 for get_value and exists 2022-03-21 11:19:28 +05:30
mergify[bot]
f47331cd1d
Merge pull request #16200 from alyf-de/refactor-db-exists
refactor: `frappe.db.exists`
2022-03-21 05:47:27 +00:00
barredterra
7e550e919c revert: don't check for doctype (861ff16) 2022-03-17 02:07:23 +01:00
phot0n
bebc8058b6 feat: integer primary keys 2022-03-11 23:46:00 +05:30
lapardnemihk1099
f4d3effaf2 fix: typo in database file for v12 - latest 2022-03-11 13:29:01 +05:30
barredterra
861ff16ac8 refactor: frappe.db.exists 2022-03-05 19:53:45 +01:00