Commit graph

191 commits

Author SHA1 Message Date
gavin
d29641f734
Merge branch 'develop' into aks-feat-dbquery_update 2022-05-09 12:02:36 +05:30
Ritwik Puri
b41379c78b
fix: misc fixes (integer primary keys) (#16307)
* fix: misc fixes

local.x gets resetted on every request so switched to a simple dict

simplified is_val_used in set_next_val function for sequences

* chore: use multisql for sequence methods

* fix: fields not updating on form

* minor(base_input): removed unnecessary branching in update_input

* chore: remove prints and rename autoincremented_status_map

* chore: added proper type hint + comment + formatting

* fix: added searching in cast_name rather than handling it manually

* fix: share condition query + test_build_match_conditions

* fix: add cast_name to more places

* test: test for sequence

* fix: sequence functions

* fix: inherit frappetestcase

* minor: attach sequence methods to db context local

* chore: update sequence function names in Database

use frappe.db for sequences in naming.py

* fix: convert filename to str (for autoincremented doctypes)

* chore: better regex for modifying values for postgres

* minor: allow changing name column type (if no data is present in the doctype)

* refactor: validate_autoname

converted it to a simple function
enabled changing autoincrement autoname from customize form

* fix: use sql_ddl for change_column_type in postgres

* fix: use not null constraint in postgres when changing name type

* fix(test): updated test_autoincremented_doctype_transition with transitioning when no data is present

* fix(test): updated test_cast_name

probably messed up during rebase

* fix(test): used rollback upon error in transaction for postgres

* chore: use frappe.db.x methods for sequences

* minor: use temporary sequences in test

* minor: use generate_hash for sequence naming in sequence tests

* chore: replace sequence imports with frappe.db.x

* chore: move out casting name fields to a separate method

* refactor: cast_name

more explicit cases for casts and added docstring

* fix: added space in test_cast_name

* chore: fix linter

* chore: better naming for can_change_name_column_type

* chore: add comment for autoincremented_site_status_map

* chore: update/add docstrings
2022-04-29 15:06:03 +05:30
saxenabhishek
b1a0a3816b style: Applied Frappe linting 2022-04-28 17:10:31 +05:30
saxenabhishek
d39e6a7890 refactor: new query engine in get_count() 2022-04-28 17:10:31 +05:30
saxenabhishek
c9df86f9ef feat: multi table queries 2022-04-28 17:10:31 +05:30
gavin
2da85fa8da
Merge pull request #16698 from gavindsouza/traceback-with-context
feat: Traceback with context
2022-04-22 13:04:07 +05:30
Ankush Menat
dfef7192da
refactor: remove duplicate code from db.get_descendants (#16699) 2022-04-21 15:27:59 +05:30
Gavin D'souza
c691537e61 chore: Add typing for ease of development 2022-04-21 13:32:13 +05:30
Gavin D'souza
35c606707e fix(bulk_insert): Cast values as list before subscripting 2022-04-20 16:43:19 +05:30
Pruthvi Patel
8b010e1732
refactor: make frappe.db.bulk_insert work as expected (#16527)
## Issue

`frappe.db.bulk_insert` is not working as expected:
 - It will not insert any row if there are less than 3 values
 - It will not add 1st row at all. e.g if I'm adding 5 values, it will only add 4.
 - It will add values one by one after 2 values, instead it should have inserted items (in db) in chunk of 10000 (as per the code written before).

## Changes Made
 - Solved above issues 
 - use better way to chunk list
 - Added Postgres support for bulk_insert API
 
And now `bulk_insert` will only do **1 db call for each 10000** values.

Note: For testing purpose I made `Test Bulk Insert` doctype and keep chunk size of 100.

## Before
![image](https://user-images.githubusercontent.com/43115036/161979365-c1100745-7602-47d2-a9b8-62d797d2039f.png)

## After
![image](https://user-images.githubusercontent.com/43115036/161978344-3c17d56b-2195-40f4-b00c-e9478d4083f1.png)

nodocs
2022-04-19 14:47:22 +00:00
Suraj Shetty
c0c5b2ebdd
style: format all python files using black (#16453)
Co-authored-by: Frappe Bot <developers@frappe.io>
2022-04-12 10:59:25 +05:30
Gavin D'souza
ce5ac24b61 feat(db): Set frappe.db.debug to mass debug queries 2022-04-04 15:15:51 +05:30
gavin
3f6bbd6349
Merge pull request #16474 from resilient-tech/perf-get-doc-filters
perf: reduce query when calling `get_doc` with filters
2022-04-01 15:53:55 +05:30
Sagar Vora
e73c552632 fix: implement for_update for Single documents 2022-04-01 14:15:35 +05:30
Sagar Vora
493c7025f5 fix(get_value): make as_dict work for list-style filters 2022-04-01 00:33:15 +05:30
Ankush Menat
c4ca58fedf refactor: query stripping 2022-03-28 10:56:55 +05:30
Ankush Menat
456cba71cd refactor: expand unreadable one-liner in get_value 2022-03-28 10:56:55 +05:30
Ankush Menat
3e1df641da fix: force keyword arguments in new ORM args 2022-03-28 10:56:55 +05:30
gavin
a7f1639e65
Merge branch 'develop' into log-settings 2022-03-23 13:20:33 +05:30
Gavin D'souza
300227ba71 fix(set_single_value): Make value parameter optional
Similar to set_value for accepting multiple columns ot be updated for
the same Table through a Dict as the second positional arg

Misc: Added type hints
2022-03-23 13:11:55 +05:30
barredterra
44a7c0dd93 fix: copy dict before popping keys 2022-03-21 20:51:59 +01:00
Suraj Shetty
a1664330a7
Merge pull request #16344 from frappe/mergify/bp/develop/pr-16239 2022-03-21 20:49:33 +05:30
Shariq Ansari
5c6c9bb5c4
fix: resolved conflicts in database.py 2022-03-21 18:16:54 +05:30
Shariq Ansari
fbac6fbfb4 fix: only remove \n\t from start and end
(cherry picked from commit 7bb172365f2c9ae6cca98ccc4dfee7714b9c3f0c)

# Conflicts:
#	frappe/database/database.py
2022-03-21 05:50:58 +00:00
Shariq Ansari
b2c0bf7a4e fix: remove tab \t and newlines \n from start of query and remove from middle
(cherry picked from commit ac5effc7dd4d876d06daf945f0b8b77ecdd0c05f)

# Conflicts:
#	frappe/database/database.py
2022-03-21 05:50:57 +00:00
Ankush Menat
b97cfed6d7 perf: limit rows to 1 for get_value and exists 2022-03-21 11:19:28 +05:30
mergify[bot]
f47331cd1d
Merge pull request #16200 from alyf-de/refactor-db-exists
refactor: `frappe.db.exists`
2022-03-21 05:47:27 +00:00
barredterra
7e550e919c revert: don't check for doctype (861ff16) 2022-03-17 02:07:23 +01:00
phot0n
bebc8058b6 feat: integer primary keys 2022-03-11 23:46:00 +05:30
lapardnemihk1099
f4d3effaf2 fix: typo in database file for v12 - latest 2022-03-11 13:29:01 +05:30
barredterra
861ff16ac8 refactor: frappe.db.exists 2022-03-05 19:53:45 +01:00
barredterra
745297a49d refactor: a not in b
Search: if not ([\w\d]*?) in ([\w\d]*?)
Replace: if $1 not in $2
2022-02-21 19:54:34 +01:00
Ankush Menat
1439e667b2 chore: add debug status temporarily 2022-02-09 16:25:39 +05:30
phot0n
266e1f95ed chore: adding back idx column
This is needed for ordering in display of links as it keeps
track of the clicks on a particular link item
2022-02-07 16:40:01 +05:30
phot0n
b31f3c24f6 refactor: remove parent, parenttype, parentfield, idx columns from non-child table doctypes
* feat: add parent, parenttype, idx, parentfield columns to doctypes when transitioning from normal -> child table

* fix: remove parent, parenttype, parentfield, idx from DocType DocType
2022-02-07 15:12:37 +05:30
mergify[bot]
819202f5da
Merge pull request #15560 from gavindsouza/set_value-refactor
refactor: frappe.db.set_value
2022-02-02 06:41:58 +00:00
Saqib Ansari
bd8ac90286
fix(ux): show report button on too many writes error (#15614)
* feat: log errors on too many writes to ease debugging

* fix(ux): extract and show which app/file/function caused too many writes

* fix: postgres error checking assumes exception is postgres exception

* fix: better default for skipping frames

Typically stack is like: some_app -> your_function -> this function. So last 2 frames need to be skipped.

* fix: postgres error checking assumes exception is postgres exception

* fix: better default for skipping frames

Typically stack is like: some_app -> your_function -> this function. So last 2 frames need to be skipped.

* refactor: show Report button instead of logging

* revert: unnecessary new functionality

* test: assert exact exception

Co-authored-by: Ankush Menat <ankush@frappe.io>
2022-01-31 18:57:32 +05:30
Gavin D'souza
b64dc65088 fix: Set cache=True to maintain old behaviour consistency 2022-01-11 16:25:10 +05:30
Gavin D'souza
d81fe8699c feat: frappe.db.set_single_value
Alias to frappe.db.set_value without having to pass the doctype name
twice or teh second one as None

Other changes:
* set/get db APIs for single DocTypes
* Make use of redundant cache key in frappe.db.get_single_value
2022-01-10 21:03:14 +05:30
Gavin D'souza
479f637f76 fix(single:set_value): Cast using sbool instead of str for all 2022-01-10 20:59:50 +05:30
Gavin D'souza
3052808048 fix: Don't cast to str if None or Falsy
* refactor: use get_single_value instead of set_value(blah, None, blah1)
* User.reload in test_home_page to combat what happens locally
2022-01-10 15:05:57 +05:30
Gavin D'souza
123bcc9509 fix: Clear cache on set_value 2022-01-07 14:16:45 +05:30
Gavin D'souza
b0c30363cf fix: Cast values as str for all single doctypes
Re-arranged block for simplicity. Type casting doesn't change anything
it seems. On get_value, values are casted properly.
2022-01-07 11:49:49 +05:30
Gavin D'souza
697d6f2d7a fix: Run query even if no documents found for_update
If no docnames are found, then use NULL as fallback. I tested this on PostgreSQL & MariaDB and it seems to work as expected

    In [1]: from frappe.query_builder import Field

    In [2]: frappe.db.set_value('ToDo', Field("creation") > Field("modified"), 'description', 'change 2', for_update=True, debug=True)
    SELECT "name" FROM "tabToDo" WHERE "creation">"modified"
    Execution time: 0.0 sec
    UPDATE "tabToDo" SET "description"='change 2',"modified"='2021-10-22 09:38:40.110481',"modified_by"='Administrator' WHERE "name" IN (NULL)
    Execution time: 0.0 sec
2022-01-07 11:47:49 +05:30
Gavin D'souza
aefb8e3b06 fix: Expand iterable to get rid of extra brackets
Although valid SQL, MariaDB didn't like double brackets on VALUES. It
raised ERROR 1241: Operand should contain 1 column(s)
2022-01-07 11:46:56 +05:30
Gavin D'souza
44f603fc6e refactor: frappe.db.set_value
* Simplified logic
* Perf enhancements (removed unnecessary conditional computations)
* Use query builder and ORM to build queries instead of juggling
2022-01-07 11:45:27 +05:30
mergify[bot]
96a45473ab
Merge pull request #15492 from resilient-tech/fix-clear-cache-in-set-value
fix: clear cache for all docs matched with the filters passed as `dn` param in `set_value`
2022-01-05 07:06:07 +00:00
Ankush Menat
80d456ef7e fix: avoid cached results for table_exists during install 2022-01-05 11:54:46 +05:30
Ankush Menat
08464553f3 fix: define check_transaction_status for postgres
To implement repeatable reads it has to respect transactions.
2022-01-04 13:59:22 +05:30
Ankush Menat
0fddbafd09 fix(postgres)!: dont silently rollback on db exception 2022-01-04 11:25:08 +05:30