Commit graph

826 commits

Author SHA1 Message Date
Akhil Narang
06db7a2b2b
fix: can't compare str with int
Signed-off-by: Akhil Narang <me@akhilnarang.dev>
2024-03-13 12:52:54 +05:30
David
61e54eac17
fix: postgres version check 2024-03-12 09:21:34 +01:00
Ankush Menat
13c74ec231 fix: Use debug log to log DB queries 2024-03-10 12:49:19 +05:30
Ankush Menat
62afa419f8 fix: Add is_virtual in bootstrap schema 2024-03-10 12:19:37 +05:30
Ankush Menat
b60c64cacc
fix: rollback invalid customize form changes (#25198)
* fix: rollback invalid customize form changes

* fix: commit before doing DDL
2024-03-03 17:17:12 +05:30
Ankush Menat
b4fe7223c1 fix(postgres): treat LockNotAvailable as timeout
It's a lock timeout in a way.
2024-02-29 18:04:05 +05:30
Ankush Menat
e810fb7eca feat: nowait to skip blocking locks 2024-02-29 16:30:24 +05:30
Shankarv19bcr
437beaa735 fix: typo 2024-02-21 17:19:32 +05:30
Akhil Narang
3f1e19de85
refactor(treewide): enable RUF rules
Signed-off-by: Akhil Narang <me@akhilnarang.dev>
2024-02-21 16:20:28 +05:30
Raffael Meyer
68eb2d978d
fix: keep order in get_values_from_single (#24907)
* fix: keep order in `get_values_from_single`

* fix: add test for destructuring

---------

Co-authored-by: Ankush Menat <ankush@frappe.io>
2024-02-19 05:35:35 +00:00
Ankush Menat
4350f94616
refactor: add_column (#24867)
- Skip cache which can be invalid
- Do check at db level
2024-02-12 07:45:18 +00:00
Suraj Shetty
803c02258a fix: Ignore "Invalid Doctype" validation if ignore flag is passed 2024-02-09 14:08:24 +05:30
Akhil Narang
26ae0f3460
fix: ruff fixes
Signed-off-by: Akhil Narang <me@akhilnarang.dev>
2024-02-07 17:04:31 +05:30
Ankush Menat
de9ac89748 style: re-format with ruff 2024-02-05 18:53:33 +05:30
Ankush Menat
c02f5d5876
fix: Skip fulltext indexes during sync (#24728)
These are not managed by framework's migration system so we shouldn't
delete them.
2024-02-05 05:30:44 +00:00
Akhil Narang
f19c25ec42
Merge pull request #24432 from blaggacao/chore/cleanup-frappe-connect-2
chore: cleanup frappe connect II/II
2024-02-02 16:53:19 +05:30
David Arnold
3abbde3284
fix: shell escaping on external commands 2024-01-30 18:23:05 +01:00
David Arnold
b57dbe2dbc
Merge remote-tracking branch 'origin/develop' into chore/cleanup-frappe-connect-2 2024-01-30 10:47:19 +01:00
Akhil Narang
06ed2f5246
fix: allow site creation to work on FC again
FC creates a temporary non-root user to use as the root user here, which doesn't have `SELECT` permissions on `mysql.user`
Try to work around it by relying on `CREATE USER IF NOT EXISTS`

Signed-off-by: Akhil Narang <me@akhilnarang.dev>
2024-01-24 18:50:23 +05:30
David Arnold
94378b6566
chore: simplify frappe.connect and use site config for bootstrapping 2024-01-19 13:49:36 +01:00
Akhil Narang
8a3f14d990
chore: restore original order of parameters - add the new one to the end
Signed-off-by: Akhil Narang <me@akhilnarang.dev>
2024-01-19 16:28:56 +05:30
David Arnold
a1a653632f
fix: avoid changing argument order 2024-01-19 10:31:57 +01:00
David Arnold
7bec22e238
Merge remote-tracking branch 'origin/develop' into chore/cleanup-low-level-db-interface 2024-01-18 13:22:33 +01:00
David Arnold
9a1af12a41
chore: more concise code 2024-01-18 10:20:12 +01:00
David Arnold
bd357f41d3
feat: add defaults to db superuser prompt 2024-01-17 20:25:27 +01:00
David Arnold
f889d09d0b
fix: set defaults at the highest level interface 2024-01-17 18:59:34 +01:00
Ankush Menat
484049c5ae
fix: simultenous sessions ux (#24387)
* fix: set 2 as simultaneous_sessions by default

* fix: Correct offset for simultaneous_sessions

* refactor: use freeze_time instead of patching

* chore: misleading docstring

* test: set lower simultaneous_sessions for test
2024-01-16 13:36:05 +00:00
Akhil Narang
fff866417d
fix: format
Got messed up during rebase - forgot to run pre-commit before `git rebase --continue`

Signed-off-by: Akhil Narang <me@akhilnarang.dev>
2024-01-16 12:35:01 +05:30
Akhil Narang
9b18289b33
fix: postgres uses double quotes for identifiers, especially if case-sensitive
(single quotes would force it to lowercase)

Also unconditionally create a new user while running tests - we're gonna be dropping the test site

Don't pass a `None` value to bench cli

Signed-off-by: Akhil Narang <me@akhilnarang.dev>
2024-01-16 12:35:01 +05:30
Akhil Narang
265c0b0e01
fix(drop-site): set db root credentials in frappe.flags like how new-site is done
Signed-off-by: Akhil Narang <me@akhilnarang.dev>
2024-01-16 12:35:01 +05:30
Akhil Narang
f876aa385c
fix(setup_db): use the correct user
Signed-off-by: Akhil Narang <me@akhilnarang.dev>
2024-01-16 12:35:01 +05:30
Akhil Narang
1e6eeba9b7
fix: add in missing logic, add in a test for creating a site with a specific user
Signed-off-by: Akhil Narang <me@akhilnarang.dev>
2024-01-16 12:35:01 +05:30
Akhil Narang
d1906b3c16
refactor: add in a fallback to use db name as db user name
Signed-off-by: Akhil Narang <me@akhilnarang.dev>
2024-01-16 12:35:01 +05:30
Akhil Narang
c721fcf8c4
refactor: allow setting a db user name different from the database name
Signed-off-by: Akhil Narang <me@akhilnarang.dev>
2024-01-16 12:35:01 +05:30
Ankush Menat
a2525e545a
perf: Unbuffered cursors for large result sets (#24365)
If you're reading 1000s of rows from MySQL, the default behaviour is to
read all of them in memory at once.

One of the use case for reading large rows is reporting where a lot of
data is read and then processed in Python. The read row is hoever not
used again but still consumes memory until entire function exits.

SSCursor (Server Side Cursor) allows fetching one row at a time.

Note: This is slower than fetching everything at once AND has risk of
connection loss. So, don't use this as a crutch. If possible rewrite
code so processing is done in SQL.
2024-01-16 11:00:12 +05:30
Ankush Menat
d5b27061ad perf: avoid duplicate copies of result set
When as_list, as_dict is done we hold on to original result set until
next query is performed. This can be HUGE for large queries.
2024-01-15 12:32:07 +05:30
Ankush Menat
588157df74 feat: frappe.db.sql results as iterator
- Also avoid self.last_result that holds on to large result set reference.
2024-01-15 12:32:02 +05:30
David Arnold
3fa76b827c
chore: name current db handle properly 2024-01-14 17:14:21 +01:00
Ankush Menat
e45e313bfb
feat: Skip locked rows while selecting (#24298) 2024-01-13 09:49:27 +05:30
Akhil Narang
58a13cfb3e
refactor: set pipefail in shell before running piped backup/restore commands
Drop the complicated logic behind storing pid and killing process if first stage of pipe fails

Signed-off-by: Akhil Narang <me@akhilnarang.dev>
2024-01-04 18:41:37 +05:30
Ankush Menat
4996b8ac7b refactor: postgres version check 2023-12-27 16:12:40 +05:30
hhharsha36
445da4319b
fix: error - permission denied for schema public from Postgres >= 15 during initial DB setup (#23799)
Error from Postgres:
ERROR:  permission denied for schema public at character 14

Error from Frappe:
psql:/home/frappe/frappe-bench/apps/frappe/frappe/database/postgres/framework_postgres.sql:72: ERROR:  permission denied for schema public

Error Causer:
Starting Postgres version >= 15, all users will have the `CREATE` permission revoked by default.

Fix:
Grant relevant privileges to the database in question for the user. In this case, DB owner privilege.

Note: the below two permission attempts were unsuccessful and still caused the same public schema permission error

GRANT USAGE, CREATE ON SCHEMA public TO {frappe.conf.db_name};
GRANT ALL ON SCHEMA public TO {frappe.conf.db_name}

References:
https://stackoverflow.com/questions/74110708/postgres-15-permission-denied-for-schema-public
https://stackoverflow.com/questions/67276391/why-am-i-getting-a-permission-denied-error-for-schema-public-on-pgadmin-4
2023-12-27 16:04:21 +05:30
Ankush Menat
2dfef73ae8
Merge pull request #22915 from akhilnarang/dont-extract-backup-unconditionally
perf: don't extract gzipped backups
2023-12-26 12:19:10 +05:30
Xiaoguang Sun
68d4a5ad82
Do not call begin in updatedb after syncing (#23934)
Do not call begin in updatedb after syncing as commit does it already.

Signed-off-by: Xiaoguang Sun <sunxiaoguang@gmail.com>
2023-12-26 12:12:19 +05:30
Akhil Narang
6e8f32af58
chore: don't pipe output through pv
No point if we're using `execute_in_shell()`

Signed-off-by: Akhil Narang <me@akhilnarang.dev>
2023-12-26 11:09:08 +05:30
Akhil Narang
3fe840fb31
fix(postgres): make use of common helper
Signed-off-by: Akhil Narang <me@akhilnarang.dev>
2023-12-26 11:09:08 +05:30
Akhil Narang
76c9fbd0cd
fix: use source before changing its contents
Signed-off-by: Akhil Narang <me@akhilnarang.dev>
2023-12-26 11:09:08 +05:30
Akhil Narang
012b0fdb7e
fix(postgres/setup): use gzip to get backup contents if the file is an archive
Signed-off-by: Akhil Narang <me@akhilnarang.dev>
2023-12-26 11:09:08 +05:30
Akhil Narang
0b508e2a96
feat(db_manager): avoid extraction of DB dump if gzipped
Use `gzip -cd` to directly get the contents onto stdout and pipe to mariadb

Signed-off-by: Akhil Narang <me@akhilnarang.dev>
2023-12-26 11:09:08 +05:30
Hussain Nagaria
ecdd7b3e37 docs: consistency 2023-12-20 14:02:32 +05:30