Commit graph

203 commits

Author SHA1 Message Date
David Arnold
7bec22e238
Merge remote-tracking branch 'origin/develop' into chore/cleanup-low-level-db-interface 2024-01-18 13:22:33 +01:00
David Arnold
9a1af12a41
chore: more concise code 2024-01-18 10:20:12 +01:00
David Arnold
bd357f41d3
feat: add defaults to db superuser prompt 2024-01-17 20:25:27 +01:00
David Arnold
f889d09d0b
fix: set defaults at the highest level interface 2024-01-17 18:59:34 +01:00
Akhil Narang
265c0b0e01
fix(drop-site): set db root credentials in frappe.flags like how new-site is done
Signed-off-by: Akhil Narang <me@akhilnarang.dev>
2024-01-16 12:35:01 +05:30
Akhil Narang
f876aa385c
fix(setup_db): use the correct user
Signed-off-by: Akhil Narang <me@akhilnarang.dev>
2024-01-16 12:35:01 +05:30
Akhil Narang
1e6eeba9b7
fix: add in missing logic, add in a test for creating a site with a specific user
Signed-off-by: Akhil Narang <me@akhilnarang.dev>
2024-01-16 12:35:01 +05:30
Akhil Narang
c721fcf8c4
refactor: allow setting a db user name different from the database name
Signed-off-by: Akhil Narang <me@akhilnarang.dev>
2024-01-16 12:35:01 +05:30
Ankush Menat
a2525e545a
perf: Unbuffered cursors for large result sets (#24365)
If you're reading 1000s of rows from MySQL, the default behaviour is to
read all of them in memory at once.

One of the use case for reading large rows is reporting where a lot of
data is read and then processed in Python. The read row is hoever not
used again but still consumes memory until entire function exits.

SSCursor (Server Side Cursor) allows fetching one row at a time.

Note: This is slower than fetching everything at once AND has risk of
connection loss. So, don't use this as a crutch. If possible rewrite
code so processing is done in SQL.
2024-01-16 11:00:12 +05:30
Ankush Menat
d5b27061ad perf: avoid duplicate copies of result set
When as_list, as_dict is done we hold on to original result set until
next query is performed. This can be HUGE for large queries.
2024-01-15 12:32:07 +05:30
David Arnold
3fa76b827c
chore: name current db handle properly 2024-01-14 17:14:21 +01:00
Xiaoguang Sun
68d4a5ad82
Do not call begin in updatedb after syncing (#23934)
Do not call begin in updatedb after syncing as commit does it already.

Signed-off-by: Xiaoguang Sun <sunxiaoguang@gmail.com>
2023-12-26 12:12:19 +05:30
Hussain Nagaria
8d2137c265 docs: consistent doc strings 2023-12-18 18:27:39 +05:30
barredterra
c35476256f refactor: simplify conditional logic
Command: `sourcery review --fix --enable de-morgan .`
2023-12-05 11:14:41 +01:00
Akhil Narang
fbc88a4d24
refactor(treewide): code cleanup
Drop redundant bool conversion

Signed-off-by: Akhil Narang <me@akhilnarang.dev>
2023-11-23 13:57:51 +05:30
Ankush Menat
d715b0ad31
Merge pull request #22804 from akhilnarang/non-nullable-fields
feat: allow setting fields as not nullable
2023-11-20 12:46:02 +05:30
Akhil Narang
ae9ae40bad
Merge pull request #23168 from blaggacao/refactor/centralize-python-shell-interface-for-database-binaries
refactor: give cli invocations of dbtools a dedicated interface
2023-11-17 17:02:41 +05:30
Akhil Narang
5598fe241e
fix: use default default if a custom default isn't specified
Signed-off-by: Akhil Narang <me@akhilnarang.dev>
2023-11-16 14:51:57 +05:30
Akhil Narang
558ac413ce
fix(mariadb/schema): use correct default value
Signed-off-by: Akhil Narang <me@akhilnarang.dev>
2023-11-16 14:51:57 +05:30
Akhil Narang
8d540963c8
refactor(mariadb/schema): use sql_ddl() method
Signed-off-by: Akhil Narang <me@akhilnarang.dev>
2023-11-16 14:51:57 +05:30
Akhil Narang
ccc2bdad52
fix: go back to original transaction handling mechanism
This reverts the following:
07acfeed47 chore: move check_implicit_commit() call back to its original location
fc38a0b503 chore: simplify transaction handling
1e29e81543 chore: rework implicit commit checks
ae0a3fd202 feat: allow ignoring implicit commit warning

Signed-off-by: Akhil Narang <me@akhilnarang.dev>
2023-11-16 14:51:57 +05:30
Akhil Narang
4855ec76c2
chore: use index to access table column instead of getattr
Signed-off-by: Akhil Narang <me@akhilnarang.dev>
2023-11-16 14:51:57 +05:30
Akhil Narang
806748b063
chore: simplify transaction handling
Signed-off-by: Akhil Narang <me@akhilnarang.dev>
2023-11-16 14:51:57 +05:30
Akhil Narang
fe549ecc49
chore: don't check for semgrep here
Signed-off-by: Akhil Narang <me@akhilnarang.dev>
2023-11-16 14:51:57 +05:30
Akhil Narang
58b8b26b93
chore(mariadb): switch to using query builder for update statement
Signed-off-by: Akhil Narang <me@akhilnarang.dev>
2023-11-16 14:51:57 +05:30
Akhil Narang
e41fa2dfc7
feat: migrate columns to be non-nullable if required
Signed-off-by: Akhil Narang <me@akhilnarang.dev>
2023-11-16 14:51:57 +05:30
David Arnold
0dc2a9c2f3
refactor: normalize function signature 2023-11-10 15:12:59 +01:00
David Arnold
b2f9ff09aa
feat(db): boostrap only option if resource management is exogenous
The database resource management including privileges granting can be
done by external secops. It can be undesirable to grant the grant option
to the framework user.

This commit implements the possibility to remove user resource
management from frappe entirely and only boostrap the database with its
target user.
2023-11-09 17:11:39 +01:00
David Arnold
40a2daf84c
chore: give cli invocations of dbtools a dedicated interface 2023-11-09 12:14:41 +01:00
Ankush Menat
0d5f28d569 feat: rename custom field 2023-09-25 12:25:28 +05:30
Sagar Vora
7e9b08a040 fix: convert value column to longtext for Singles 2023-09-06 16:23:01 +05:30
Ankush Menat
730e906dfd
refactor!: remove implicit primary key from logs (#22209) 2023-08-26 16:01:47 +05:30
barredterra
88c8baa9ee refactor: for append to extend, merge list extend
Replace a for append loop with list extend.
Create the list with values instead of creating
an empty list and extending it with another list.
2023-08-09 13:25:39 +02:00
David Arnold
a926e64ec9
fix: procure db config from single authority (#21578)
* fix: procure db config from single authority

ensures that configuration is uniformely procured from local.conf
instead of making use of hard to audit multilevel fallback logic

Implementation Note:
- `get_db(host, port, user, password)` was stripped of any optional
  argument and therefrom all errors where fixed.
- All occurances of `grep 'frappe.db.db_'` where changed to
  `frappe.conf.db_`

* fix: revert unnecessary breaking changes
2023-07-24 10:48:29 +05:30
Ankush Menat
fa6dc03cc8
refactor: frappe.cache() usage to frappe.cache (#21282) 2023-06-08 11:47:17 +05:30
Ankush Menat
4104e7d733
feat: show db table utilization on doctype (#21193)
* feat: show db table utilization on doctype

* fix: nicer error message with docs
2023-05-31 16:28:44 +05:30
Ankush Menat
48539dc0a3
feat: log all DDL queries (#21107) 2023-05-31 16:00:59 +05:30
Ritwik Puri
d099c9376b
feat(minor): db.get_column_type for postgres (#21125) 2023-05-26 16:03:28 +05:30
Ankush Menat
024faff025
build: bump pymysql (#20478)
Actual fix for this bandaid fix: https://github.com/frappe/frappe/pull/20475

Keeping pymysql hard pinned until we have better way to get last full
query.
2023-03-28 13:04:27 +05:30
Ankush Menat
dad0e5cbba fix: drop table if exists for action and links
closes https://github.com/frappe/frappe/issues/19712
2023-02-01 22:37:43 +05:30
Sagar Vora
db9b25ec0e
fix(MariaDBTable): dont attempt to drop index twice (#19783) 2023-01-27 09:27:11 +05:30
Gavin D'souza
d357af1533 refactor: Add a maxsplit limit to string splits 2023-01-24 19:22:51 +05:30
Ankush Menat
894f0affcc
fix(DX): Print alter query if it fails during migration (#19552) 2023-01-10 16:55:29 +05:30
Ankush Menat
95039a1265
fix: db ssl connection (#19308)
ref: https://pymysql.readthedocs.io/en/latest/modules/connections.html

[skip ci]
2022-12-19 11:21:15 +05:30
Ankush Menat
8df845ca35 fix: duplicate unique index when column is altered 2022-12-15 13:36:28 +05:30
Sagar Vora
4ea39d1a68
chore!: drop device support from session (#18729)
* chore!: drop device support from session

* chore: remove `session_expiry_mobile` field

* fix: remove extra `%s`
2022-12-05 16:37:32 +05:30
Ankush Menat
186b88a8cb
build: Bump minimum required MariaDB version (#19116) 2022-12-05 16:24:35 +05:30
Ankush Menat
9c104a8e5e fix: Warn about incompatible mariadb versions 2022-12-03 12:20:35 +05:30
Ankush Menat
11cdd21b88 fix: drop mariadb 10.2 checks 2022-12-03 12:20:35 +05:30
Ankush Menat
263055e247
fix: site creation using non-root users (#19014) 2022-11-26 12:45:57 +05:30