Commit graph

438 commits

Author SHA1 Message Date
gavin
0fe764c9b8 fix: Depend on replica details being there for Replica ConnectionPool
This logic mirror how replica connections are handled
2022-05-24 19:19:53 +05:30
gavin
aaef732581 fix!: Remove frappe.db.create_help_table
Help table has been deprecated for a while. It's safe to get rid of the
API too.
2022-05-24 19:19:53 +05:30
gavin
f021250308 refactor: Move connection & pool management out, inherit instead 2022-05-24 19:19:53 +05:30
gavin
993b935097 fix: Separate pool for replica setup
_SITE_POOLS[frappe.local.site].read_only will hold ConnectionPool for
replica database connections. _SITE_POOLS[frappe.local.site].default
will hold pool for connections that allow read+writes
2022-05-24 19:19:53 +05:30
gavin
363708d7f2 fix: frappe.conf.disable_database_connection_pooling
For the times you don't want to use pooling ;) Calling
get_connection/connect will close all pooled connections
for the current process / worker!
2022-05-24 19:19:53 +05:30
gavin
dae57d0de2 fix(db)!: Base methods should throw NotImplementedError 2022-05-24 19:19:53 +05:30
gavin
0d80f6ac52 feat: mariadb connection pooling
Start initial pool of 4 (_POOL_SIZE) connections for a given site. When
the pool is exhausted, generate new connections upon request and add
them to the pool. Max allowed size for each pool is 64 (_MAX_POOL_SIZE)
connections. However, you may have more than 64 active connections, just
that they won't be pooled but destroyed upon job completion/end of
request/similar cycle.
2022-05-24 19:19:53 +05:30
gavin
a17c978065 refactor!: frappe.db.get_database_list
* Drop mandatory unused arg in method call
* Use pluck instead of list comprehension + subscripting
2022-05-24 19:19:53 +05:30
gavin
756e385362 refactor: Move exception detection & abstraction in separate class
This change has only been done to separate and club only like methods /
utils together
2022-05-24 19:19:53 +05:30
gavin
1fe3624c43 fix: DB error detection API usage
* Make all methods static
* Add typing hints
* Don't safe fetch attribute value for errno - MariaDB exceptions if
  raised will have errno attr in them. If the class doesn't have one,
  it's not meant to be passed in these methods.
2022-05-24 19:19:53 +05:30
gavin
d3900485a6 refactor: Use mariadb client for creating connections
Move to use MariaDB's official Python client written in C instead of
the PyMySQL library. This change doesn't rid Frappe of the PyMySQL
library. Instead, it continues to utilize it for the ER module and
converter methods until the MariaDB library adds support for the same.

Ticket: https://jira.mariadb.org/projects/CONPY/issues/CONPY-203
2022-05-24 19:19:53 +05:30
gavin
15af545acf chore: Update typing + noqa - flake8 failures 2022-05-23 18:55:55 +05:30
gavin
6dedf899ff
Merge pull request #16899 from ChillarAnand/imports
refactor: Removed unused imports & added flake8 check in CI
2022-05-23 15:53:18 +05:30
gavin
a20800ee2b fix: mport module from line N shadowed by loop variable
Flake8 F402 reported by sider
2022-05-19 17:45:22 +05:30
gavin
db11af2a5c feat(wip): Custom filters in db.query engine
* Added provision for semi-implemneted version
    * Hard to fix it completely given it's broken on develop / desk
* Added TODO for adding nestedset related filters
2022-05-19 17:31:35 +05:30
chillaranand
1d763a6659 refactor: Fix flake8 issues 2022-05-19 15:34:35 +05:30
gavin
72ad582b8a fix(db.query): Casefold key to match operators 2022-05-18 16:59:53 +05:30
gavin
5ebaf42468 feat(db.query): Add support for timespan operator 2022-05-18 16:59:10 +05:30
gavin
7c77bedbf2 refactor: Simplify logic + Add typing hints 2022-05-18 16:57:54 +05:30
Aradhya Tripathi
7a0dc40693
feat!: Added support for IS in frappe qb (#16877)
Fixes #16869
Using 53148c3f90
2022-05-18 09:09:22 +00:00
Suraj Shetty
84122c6eec fix: Pass field object instead of string 2022-05-17 18:27:19 +05:30
Suraj Shetty
1469339335 revert: "refactor: new query engine in get_count()"
Co-Authored-By: Gavin Dsouza <gavin18d@gmail.com>
2022-05-17 13:30:36 +05:30
Himanshu
70409a3c7b
fix: set translated text to link field (#15451)
- Fixes [Issue/15324](https://github.com/frappe/frappe/issues/15324)
- When selecting a value for Link Field, the English text was set as link field value.
- This PR aims to fix it by setting the translated text as link field value on selection.

Todo:

- [x] Show translated text in the select dropdown
- [x] Set translated value to Link field on select
- [x] Show original value when link field is in focus
- [x] Add option to toggle this behaviour

### Behaviour

- Link field loses focus: show the translated name.
- Link field is focused
    - If old value is present
        - If options **are** one of [Role, DocType]: show the translated name
        - Else: show the name (untranslated) to enable search in untranslated values
    - Else: show what the user typed (untranslated) to enable search in untranslated values
- Value is selected: link field loses focus

## Demo

### Link to UOM in a custom Item DocType

The UOM names are in english, so the search needs to happen in english. When possible, the translation is displayed.

#### Before

https://user-images.githubusercontent.com/14891507/156415248-e5e80d05-53dc-4ca8-89c7-998986ff6e99.mov

#### After

https://user-images.githubusercontent.com/14891507/156410386-a874430c-f340-43ed-9c3a-92e8d4d50fc9.mov

### Link to DocType in Customize Form

The DocType names get translated before being searched. This is a preexisting hack in the framework for DocType and Role. In this case, we can search in the translations.

#### Before

https://user-images.githubusercontent.com/14891507/156414648-8e505f8c-9dee-4358-8182-3b358c28bb62.mov

#### After

https://user-images.githubusercontent.com/14891507/156411881-c4ca22e1-1397-4e13-9768-5e16b72f8d6d.mov

https://docs.erpnext.com/docs/v13/user/manual/en/customize-erpnext/customize-form/edit?wiki_page_patch=fdafee2715
2022-05-16 13:21:42 +00:00
Ritwik Puri
968343b302
Merge branch 'develop' into autoinc-customize-form-fix 2022-05-13 00:41:58 +05:30
phot0n
f6dac70033 refactor(minor): convert change_name_type_and_make_sequence function to setup_autoincrement_and_sequence method
* chore(Customize Form): remove autoincrement naming option from autoname description

* chore: move sequence cache comment from schema.py to database.py

* chore: added docstrings to some functions
2022-05-13 00:03:19 +05:30
gavin
d29641f734
Merge branch 'develop' into aks-feat-dbquery_update 2022-05-09 12:02:36 +05:30
phot0n
cfc905b567 fix: use cast when chaninging name type from varchar to bigint in postgres 2022-05-05 21:31:13 +05:30
phot0n
9f0c40dbba fix: create sequence onchange from any other autoname whcih is not autoincrement
* chore: use class variable for determining sequence cache

* chore: use sql_ddl when creating sequence
2022-05-05 21:31:13 +05:30
Ritwik Puri
b41379c78b
fix: misc fixes (integer primary keys) (#16307)
* fix: misc fixes

local.x gets resetted on every request so switched to a simple dict

simplified is_val_used in set_next_val function for sequences

* chore: use multisql for sequence methods

* fix: fields not updating on form

* minor(base_input): removed unnecessary branching in update_input

* chore: remove prints and rename autoincremented_status_map

* chore: added proper type hint + comment + formatting

* fix: added searching in cast_name rather than handling it manually

* fix: share condition query + test_build_match_conditions

* fix: add cast_name to more places

* test: test for sequence

* fix: sequence functions

* fix: inherit frappetestcase

* minor: attach sequence methods to db context local

* chore: update sequence function names in Database

use frappe.db for sequences in naming.py

* fix: convert filename to str (for autoincremented doctypes)

* chore: better regex for modifying values for postgres

* minor: allow changing name column type (if no data is present in the doctype)

* refactor: validate_autoname

converted it to a simple function
enabled changing autoincrement autoname from customize form

* fix: use sql_ddl for change_column_type in postgres

* fix: use not null constraint in postgres when changing name type

* fix(test): updated test_autoincremented_doctype_transition with transitioning when no data is present

* fix(test): updated test_cast_name

probably messed up during rebase

* fix(test): used rollback upon error in transaction for postgres

* chore: use frappe.db.x methods for sequences

* minor: use temporary sequences in test

* minor: use generate_hash for sequence naming in sequence tests

* chore: replace sequence imports with frappe.db.x

* chore: move out casting name fields to a separate method

* refactor: cast_name

more explicit cases for casts and added docstring

* fix: added space in test_cast_name

* chore: fix linter

* chore: better naming for can_change_name_column_type

* chore: add comment for autoincremented_site_status_map

* chore: update/add docstrings
2022-04-29 15:06:03 +05:30
saxenabhishek
36de21c059 feat: left join tables by default 2022-04-28 17:10:31 +05:30
saxenabhishek
b1a0a3816b style: Applied Frappe linting 2022-04-28 17:10:31 +05:30
saxenabhishek
3498445733 test: multiple_tables_in_filters 2022-04-28 17:10:31 +05:30
saxenabhishek
d39e6a7890 refactor: new query engine in get_count() 2022-04-28 17:10:31 +05:30
saxenabhishek
c9df86f9ef feat: multi table queries 2022-04-28 17:10:31 +05:30
Suraj Shetty
ccc82e2c4c
Merge pull request #15538 from noahjacob/phone_field_control
feat: Phone Control Type
2022-04-28 10:27:52 +05:30
gavin
085900328b
Merge pull request #16688 from phot0n/multiple_order_by
fix: support for multiple order by in add_conditions
2022-04-26 16:09:29 +05:30
phot0n
4418c03237 fix: remove default null definition for name column in tabSeries
for postgres
2022-04-25 20:13:10 +05:30
gavin
2da85fa8da
Merge pull request #16698 from gavindsouza/traceback-with-context
feat: Traceback with context
2022-04-22 13:04:07 +05:30
Ankush Menat
dfef7192da
refactor: remove duplicate code from db.get_descendants (#16699) 2022-04-21 15:27:59 +05:30
Gavin D'souza
c691537e61 chore: Add typing for ease of development 2022-04-21 13:32:13 +05:30
phot0n
6405b0510a chore: fix linter 2022-04-20 17:56:48 +05:30
phot0n
da191390a5 fix: support for multiple order by in add_conditions 2022-04-20 17:56:48 +05:30
Gavin D'souza
35c606707e fix(bulk_insert): Cast values as list before subscripting 2022-04-20 16:43:19 +05:30
Pruthvi Patel
8b010e1732
refactor: make frappe.db.bulk_insert work as expected (#16527)
## Issue

`frappe.db.bulk_insert` is not working as expected:
 - It will not insert any row if there are less than 3 values
 - It will not add 1st row at all. e.g if I'm adding 5 values, it will only add 4.
 - It will add values one by one after 2 values, instead it should have inserted items (in db) in chunk of 10000 (as per the code written before).

## Changes Made
 - Solved above issues 
 - use better way to chunk list
 - Added Postgres support for bulk_insert API
 
And now `bulk_insert` will only do **1 db call for each 10000** values.

Note: For testing purpose I made `Test Bulk Insert` doctype and keep chunk size of 100.

## Before
![image](https://user-images.githubusercontent.com/43115036/161979365-c1100745-7602-47d2-a9b8-62d797d2039f.png)

## After
![image](https://user-images.githubusercontent.com/43115036/161978344-3c17d56b-2195-40f4-b00c-e9478d4083f1.png)

nodocs
2022-04-19 14:47:22 +00:00
Suraj Shetty
6b7fda495b Merge branch 'develop' of https://github.com/frappe/frappe into phone_field_control 2022-04-14 14:19:35 +05:30
Shridhar Patil
39f8267a15
feat: added support for data type json (#16187)
> Please provide enough information so that others can review your pull request:

Added json support for postgres and mariadb



> Explain the **details** for making this change. What existing problem does the pull request solve?
https://github.com/frappe/frappe/projects/4#card-50160428


> Screenshots/GIFs
![json](https://user-images.githubusercontent.com/11792643/156367383-8f8492c2-3817-449d-a2dd-c983eeadbb48.gif)



---

**Previous attempts:** 

https://github.com/frappe/frappe/pull/8128
https://github.com/frappe/frappe/pull/7096


Docs: https://frappeframework.com/docs/v13/user/en/basics/doctypes/fieldtypes#json
2022-04-14 04:21:41 +00:00
Noah Jacob
c2f2fc10e7 Merge branch develop into phone_field_control 2022-04-12 20:40:29 +05:30
Suraj Shetty
c0c5b2ebdd
style: format all python files using black (#16453)
Co-authored-by: Frappe Bot <developers@frappe.io>
2022-04-12 10:59:25 +05:30
Noah Jacob
bf92b363e2
Merge branch 'develop' into phone_field_control 2022-04-07 15:04:47 +05:30
Gavin D'souza
ce5ac24b61 feat(db): Set frappe.db.debug to mass debug queries 2022-04-04 15:15:51 +05:30