Commit graph

178 commits

Author SHA1 Message Date
Akhil Narang
ae9ae40bad
Merge pull request #23168 from blaggacao/refactor/centralize-python-shell-interface-for-database-binaries
refactor: give cli invocations of dbtools a dedicated interface
2023-11-17 17:02:41 +05:30
David Arnold
283e91fd43
fix: minor fixes
Co-authored-by: Akhil Narang <me@akhilnarang.dev>
2023-11-17 10:01:53 +01:00
David Arnold
5b33608fe3
fix: shell escaping in execute_in_shell helper 2023-11-16 10:52:11 +01:00
David Arnold
0dc2a9c2f3
refactor: normalize function signature 2023-11-10 15:12:59 +01:00
David Arnold
b2f9ff09aa
feat(db): boostrap only option if resource management is exogenous
The database resource management including privileges granting can be
done by external secops. It can be undesirable to grant the grant option
to the framework user.

This commit implements the possibility to remove user resource
management from frappe entirely and only boostrap the database with its
target user.
2023-11-09 17:11:39 +01:00
David Arnold
40a2daf84c
chore: give cli invocations of dbtools a dedicated interface 2023-11-09 12:14:41 +01:00
David Arnold
d5a390cd95
chore: compose postgres conn settings in the same way as with mariadb 2023-11-09 10:00:24 +01:00
Ankush Menat
0d5f28d569 feat: rename custom field 2023-09-25 12:25:28 +05:30
Ankush Menat
730e906dfd
refactor!: remove implicit primary key from logs (#22209) 2023-08-26 16:01:47 +05:30
mergify[bot]
fd10ab25cc
Merge pull request #22003 from gavindsouza/refactor-doc-bits
refactor: Use single query to delete child rows on doc.save
2023-08-11 06:45:07 +00:00
Gavin D'souza
28e657b603
fix: Check if fieldtype in set rather than str comparison 2023-08-11 11:45:42 +05:30
barredterra
88c8baa9ee refactor: for append to extend, merge list extend
Replace a for append loop with list extend.
Create the list with values instead of creating
an empty list and extending it with another list.
2023-08-09 13:25:39 +02:00
David Arnold
a926e64ec9
fix: procure db config from single authority (#21578)
* fix: procure db config from single authority

ensures that configuration is uniformely procured from local.conf
instead of making use of hard to audit multilevel fallback logic

Implementation Note:
- `get_db(host, port, user, password)` was stripped of any optional
  argument and therefrom all errors where fixed.
- All occurances of `grep 'frappe.db.db_'` where changed to
  `frappe.conf.db_`

* fix: revert unnecessary breaking changes
2023-07-24 10:48:29 +05:30
David Arnold
6b2bb9a2ab
fix: add env overrides for service orchestration (#21577)
This avoids having to manipulate config files in brittle bash
entrypoints that need to react to dynamic service discovery.

This significantly improves the operability of various bench sites.
2023-07-14 11:39:01 +05:30
Ankush Menat
4104e7d733
feat: show db table utilization on doctype (#21193)
* feat: show db table utilization on doctype

* fix: nicer error message with docs
2023-05-31 16:28:44 +05:30
Ritwik Puri
d099c9376b
feat(minor): db.get_column_type for postgres (#21125) 2023-05-26 16:03:28 +05:30
Sagar Vora
4ea39d1a68
chore!: drop device support from session (#18729)
* chore!: drop device support from session

* chore: remove `session_expiry_mobile` field

* fix: remove extra `%s`
2022-12-05 16:37:32 +05:30
Ankush Menat
ce360b6fce feat: Set default SQL statement timeouts 2022-11-08 14:51:43 +05:30
Ankush Menat
8cca50bf26
refactor: remove naming case feature (#18672)
This has never worked since 2016... which can mean two things:
- No one really uses this.
- If I fix this now suddenly people will find different behaviour in
  naming because `name_case` is selected in some doctypes (but never
  tested)
2022-10-31 11:23:49 +05:30
Ankush Menat
f96505fae0 feat: wrap read only mode SQL errors 2022-09-09 17:34:44 +05:30
Himanshu
3faaf5f361
Merge branch 'develop' into merge_translated_doctypes 2022-08-12 13:10:49 +01:00
Gavin D'souza
91a47c2602 fix: Deprecate setup help database APIs
APIs have been unused since ~v4/6? Don't see any need to keep them in.
2022-08-12 12:52:19 +05:30
hrwx
b01929405d refactor: translatable doctypes 2022-08-05 21:38:10 +01:00
Ritwik Puri
cb6438158b
fix: don't use cache for sequence in mariadb (#17640)
* fix: don't use cache for sequence in mariadb

* chore: update sequence related comments
2022-07-27 22:46:56 +05:30
Gavin D'souza
bcfa8c276e fix: Set default port attribute for Database classes
db.default_port wil be available as a class attribute to hold defaults
for DB types.

Usage: frappe.conf.db_port or frappe.db.default_port
Why: I couldn't run the mariadb command because the defaults aren't set
for my system. server is remote / containerized. Setting port in
equivalent mysql command fixes this.
2022-07-22 13:16:26 +05:30
Gavin D'souza
1a610e135d fix(db): Use sentinel object for default values paramters 2022-07-22 12:11:01 +05:30
gavin
43f378d848
Merge branch 'develop' into mariadb-client-refactor 2022-07-07 16:00:38 +05:30
Ankush Menat
5d2aedc698 fix(postgres): end transaction before DDL
closes https://github.com/frappe/frappe/issues/17210
2022-07-05 17:43:02 +05:30
Gavin D'souza
71b5c77e6e Merge branch 'develop' of github.com:frappe/frappe into mariadb-client-refactor 2022-07-05 14:55:02 +05:30
Ankush Menat
261fbfcd11
Revert "fix(doc)!: Always cast datetime, date and time fields"
Revert "fix(doc)!: Always cast datetime, date and time fields (#15891)"

This reverts commit d7789ab6ff.
2022-07-05 13:43:32 +05:30
Himanshu
d7789ab6ff
fix(doc)!: Always cast datetime, date and time fields (#15891)
### BREAKING CHANGE
#### Datetime, Date and Time fields will always be cast to respective objects in `setattr`, this will ensure uniformity while accessing the values, no more `getdate`, `get_datetime`, `to_timedelta` wrapper.
- While importing data, the framework does check for `set_only_once`.
- In normal case scenarios, this will work flawlessly since most date fields might not be set_only_once.
- But in Subscription, the date field is set to `set_only_once` and in `after_insert`, `document.save` is called, and while doing so, `set_only_once` is checked [here](1944a547f9/frappe/model/document.py (L566)).
-This works fine if the data imported is in the correct format.
- If the date's data is not in the correct format, the framework throws an error.
- for eg `06-02-2022 00:00:00 != 06-02-2022`
- fixes [Issue/#15370](https://github.com/frappe/frappe/issues/15370)

> no-docs
2022-07-05 07:07:16 +00:00
Ankush Menat
81b37cb7d2
refactor: clean up code to py310 supported features (#17367)
refactor: clean up code to py39+ supported syntax

- f-strings instead of format
- latest typing support instead of pre 3.9 TitleCase
- remove UTF-8 declarations.
- many more changes

Powered by https://github.com/asottile/pyupgrade/ + manual cleanups
2022-07-01 11:51:05 +05:30
gavin
574f160d3c
Merge branch 'develop' into mariadb-client-refactor 2022-06-23 16:13:03 +05:30
Gavin D'souza
1f1d91a056 fix: Make postgres' last_query lazy decodable 2022-06-18 14:04:38 +05:30
phot0n
f6c1eb10dd refactor(minor): modify_values
* don't truncate float(s)
2022-06-18 10:11:58 +05:30
phot0n
2ce6597a69 fix: cast list to tuple when sending parameterized query for postgres 2022-06-17 18:47:06 +05:30
Gavin D'souza
3af8d5caea fix: Add fallbacks for values
Psycopg seems to like None over () and MariaDB - PyMySQL can't seem to
agree on anything - so this seems to keep everyone happy...a very
delicate balance :crie:
2022-06-17 12:47:44 +05:30
Gavin D'souza
90c716bce0 fix(sequence): Setup & use SequenceGeneratorLimitExceeded error 2022-06-17 12:10:43 +05:30
Gavin D'souza
effa942f4c refactor: frappe.db.log_query
* Mogrify queries and set them as frappe.db.last_query instead of
  directly interfacing with the clients
* This is required for now as the MariaDB client uses binary protocol to
  talk to the server and doesn't build the queries itself
* Add typing hints
* Imported Query object as FilterEngine - Query is too ambiguous lol

ref: https://jira.mariadb.org/browse/CONPY-208?focusedCommentId=226873&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-226873
2022-06-16 13:28:54 +05:30
Gavin D'souza
b4578dc632 Merge branch 'develop' of github.com:frappe/frappe into mariadb-client-refactor 2022-06-15 16:40:29 +05:30
Gavin D'souza
b696fa6da5 perf: Pre-compile and re-use regexp pattern
Converted all possible usages of re.* that weren't compiling the regex
separately and re-using it. Separated out the compiled patterns as
global variables. Repetitive patterns could be made DRY-er.

Would be nicer to have all regexes in a single module so that we could
re-use better, keep track of outdated, and keep checks for possible
reDos' etc
2022-06-08 14:07:38 +05:30
Ankush Menat
8861752675 fix: restart transaction after commit 2022-06-05 00:26:12 +05:30
gavin
8ffdc2d465 refactor(minor): PostgresDatabase's exceptions
Focus on Non-blocking Postgres exception checking by:

* Use safe getattr to fetch pgcode instead
* Use psycopg errorcodes module to use named variables instead of direct
  codes...for way superior readability xD

Also, moved exceptions out of the main class - just code separation, no
namespace change.
2022-05-25 14:09:31 +05:30
gavin
aaef732581 fix!: Remove frappe.db.create_help_table
Help table has been deprecated for a while. It's safe to get rid of the
API too.
2022-05-24 19:19:53 +05:30
gavin
a17c978065 refactor!: frappe.db.get_database_list
* Drop mandatory unused arg in method call
* Use pluck instead of list comprehension + subscripting
2022-05-24 19:19:53 +05:30
chillaranand
1d763a6659 refactor: Fix flake8 issues 2022-05-19 15:34:35 +05:30
Himanshu
70409a3c7b
fix: set translated text to link field (#15451)
- Fixes [Issue/15324](https://github.com/frappe/frappe/issues/15324)
- When selecting a value for Link Field, the English text was set as link field value.
- This PR aims to fix it by setting the translated text as link field value on selection.

Todo:

- [x] Show translated text in the select dropdown
- [x] Set translated value to Link field on select
- [x] Show original value when link field is in focus
- [x] Add option to toggle this behaviour

### Behaviour

- Link field loses focus: show the translated name.
- Link field is focused
    - If old value is present
        - If options **are** one of [Role, DocType]: show the translated name
        - Else: show the name (untranslated) to enable search in untranslated values
    - Else: show what the user typed (untranslated) to enable search in untranslated values
- Value is selected: link field loses focus

## Demo

### Link to UOM in a custom Item DocType

The UOM names are in english, so the search needs to happen in english. When possible, the translation is displayed.

#### Before

https://user-images.githubusercontent.com/14891507/156415248-e5e80d05-53dc-4ca8-89c7-998986ff6e99.mov

#### After

https://user-images.githubusercontent.com/14891507/156410386-a874430c-f340-43ed-9c3a-92e8d4d50fc9.mov

### Link to DocType in Customize Form

The DocType names get translated before being searched. This is a preexisting hack in the framework for DocType and Role. In this case, we can search in the translations.

#### Before

https://user-images.githubusercontent.com/14891507/156414648-8e505f8c-9dee-4358-8182-3b358c28bb62.mov

#### After

https://user-images.githubusercontent.com/14891507/156411881-c4ca22e1-1397-4e13-9768-5e16b72f8d6d.mov

https://docs.erpnext.com/docs/v13/user/manual/en/customize-erpnext/customize-form/edit?wiki_page_patch=fdafee2715
2022-05-16 13:21:42 +00:00
phot0n
f6dac70033 refactor(minor): convert change_name_type_and_make_sequence function to setup_autoincrement_and_sequence method
* chore(Customize Form): remove autoincrement naming option from autoname description

* chore: move sequence cache comment from schema.py to database.py

* chore: added docstrings to some functions
2022-05-13 00:03:19 +05:30
phot0n
cfc905b567 fix: use cast when chaninging name type from varchar to bigint in postgres 2022-05-05 21:31:13 +05:30
phot0n
9f0c40dbba fix: create sequence onchange from any other autoname whcih is not autoincrement
* chore: use class variable for determining sequence cache

* chore: use sql_ddl when creating sequence
2022-05-05 21:31:13 +05:30