Commit graph

48 commits

Author SHA1 Message Date
Rushabh Mehta
8707ed1f27 fix(minor): remove "Custom Link" and add patch 2020-10-26 10:02:04 +05:30
Rushabh Mehta
7f9284f3ef fix(tests): postgres gotcha, errors will rollback? 2020-10-26 10:02:04 +05:30
Rushabh Mehta
a0a3606a7f fix(tests): add test cases for custom_link and custom_action 2020-10-26 10:02:04 +05:30
Rushabh Mehta
0192bcf45f fix(minor): frappe.db.set_value to allow multiple values 2020-08-17 22:19:44 +05:30
Rushabh Mehta
56853a0533 fix(linting): sider 2020-08-17 11:10:48 +05:30
Rushabh Mehta
b71acfd5fa fix(minor): print query and values directly if mogrify fails 2020-08-17 10:45:58 +05:30
Rushabh Mehta
b09beb9147 fix(tests): cleanup 2020-08-17 09:41:54 +05:30
Rushabh Mehta
aad2375cf3 fix(major): make document events more efficient, fix bug in cache_manager.get_doctype_map 2020-08-17 09:41:54 +05:30
Rushabh Mehta
26a9230fdd fix(tests): more debugging 2020-08-17 09:41:54 +05:30
Rushabh Mehta
e262ef6add fix(minor): logging fix in document.py 2020-08-17 09:41:54 +05:30
Rushabh Mehta
ef1d1b0dc4 fix(minor): typo 2020-08-17 09:41:54 +05:30
Rushabh Mehta
c83a9a142b fix(minor): mogrify 2020-08-17 09:41:54 +05:30
Rushabh Mehta
eba6dfa12a fix(major): frappe.db.set_value will lock the row 2020-08-17 09:41:54 +05:30
Rushabh Mehta
e421bc19f6 fix(major): frappe.db.set_value will lock the row 2020-08-17 09:41:54 +05:30
Rushabh Mehta
4d97635ab4 fix(major): frappe.db.set_value will lock the row 2020-08-17 09:41:54 +05:30
Rushabh Mehta
14d9c28eb7 fix(test): try with print_sql 2020-08-17 09:41:54 +05:30
Suraj Shetty
ed105eb001 fix: Clear db_tables cache whenever a table is created or dropped 2020-04-30 14:16:29 +05:30
Chinmay D. Pai
2f821f9c2d
chore: fix deepsource issues
* use set comprehension instead of generator
* remove redundant else statement

Signed-off-by: Chinmay D. Pai <chinmaydpai@gmail.com>
2020-04-29 21:58:00 +05:30
Suraj Shetty
a73be732fd
refactor: Fix redundant cache value set 2020-04-29 16:43:01 +05:30
Suraj Shetty
bf3a80f05d fix: Directly used set comprehension
As indicated by deepsource
2020-04-29 14:39:14 +05:30
Suraj Shetty
031aad0d85 perf: Cache get_tables values
- To avoid redundant db calls
2020-04-29 12:28:08 +05:30
Dinuka Salwathura
e3dfb148e3
Change mysql host from localhost to 127.0.0.1
Having localhost makes mandatory to connect via mysqld.sock
changing host to 127.0.0.1 makes it to connect via tcp
2020-03-16 19:32:24 +05:30
Faris Ansari
488c68df96
Merge branch 'develop' into refactor-scheduler 2019-12-02 13:16:45 +05:30
Faris Ansari
2dac4f8432
fix(db.sql): Print query in debug mode (#8767) 2019-11-07 17:12:11 +05:30
Rushabh Mehta
fea34e3f6b fix(tests): test_scheduler 2019-11-07 12:14:09 +05:30
Himanshu
bf57a1d79a Patch(Tags): Check if column exists (#8682)
* fix: tags patch

* fix: auto_commit_on_many_writes

* fix: check if tag exists

* fix: check if tag or tag link exists

* fix: check if column exists

* fix: set autocommit false

* fix: use ignore in insert query for bulk insert

* fix: add option to ignore duplicates
2019-10-29 09:52:38 +00:00
Suraj Shetty
86263e05f4 perf: Cache db columns to avoid redundant database calls (#8543)
* perf: Cache db columns to avoid redundant database calls

* fix: Move cache clearing code from doctype to schema.py

* fix: self.table_name instead of self.name

* fix: Cache columns in  "table_columns" key

`table_columns` was cached in meta but columns were also getting accessed
directly using frappe.db.get_table_columns. Now, it is cached at
`frappe.db` layer

Co-authored-by: Suraj Shetty <surajshetty3416@gmail.com>
2019-10-07 13:39:40 +00:00
Faris Ansari
f926a2aa48
Merge pull request #8488 from hrwX/global_tags
feat(Desk): Global Tags
2019-10-07 19:05:28 +05:30
John Clarke
118bba4a83
report the name of the DocType when the associated table is missing
Reported here https://discuss.erpnext.com/t/bench-migrate-fails-with-pymysql-err-programmingerror-it-doesnt-print-what-is-the-error/53540
2019-10-06 10:34:19 -06:00
Himanshu Warekar
02cebdd7d3 fix: use frappe.tags 2019-10-03 16:43:36 +05:30
Suraj Shetty
7fe35a2d3c
fix: set default value in set_value as None to avoid error.
```
frappe.db.set_value('Call Log', log.name, {
			fieldname: doc.name,
			display_name_field: doc.get_title()
		}, update_modified=False)
```
this code should not fail if value is not pass because value is passed in the dict.
2019-09-23 15:54:46 +05:30
Faris Ansari
c8a80a2e4d fix: frappe.db.field_exists
- field_exists didn't work correctly
- The call to field_exists itself was wrong
2019-09-19 15:41:17 +05:30
Faris Ansari
64f078a686 fix: Validation for invalid field name 2019-07-16 13:58:14 +05:30
Anurag Mishra
570fd44248 fix: Translating Errors and Messages 2019-07-03 11:52:47 +05:30
Aditya Hase
2c93dc74fe fix: Update regex for capturing touched tables from query (#7588)
Previous regex used to yield false positives and false negatives
for queries like

UPDATE tabToDo SET description = "something"

Instead of yielding "tabToDo" it used to yield "tabToDo SET".

Now two separate regexes handle single word and multi-word names
In case of multi-word surrounding quotes are a must
2019-05-29 14:51:19 +05:30
Rushabh Mehta
7e42520554 fix(minor): frappe.conf.db_type in sql exception 2019-05-28 12:29:39 +05:30
Rushabh Mehta
1c893e44c0 fix: frappe.conf.db_type -> frappe.db.db_type 2019-05-24 11:44:28 +05:30
Faris Ansari
7cd94e9323
fix: Print query in console if syntax error (#7371) 2019-04-29 21:37:44 +05:30
Rushabh Mehta
fce9989e1f
Merge pull request #7326 from oshosanya/allow-database-port-config
feat: Allow user to specify database port
2019-04-25 11:40:54 +05:30
Michael Oshosanya
5e97e04000 Resolve conversation 2019-04-23 06:41:33 +01:00
Aditya Hase
c29fcb02d6 feat: Maintain list of tables touched during migrate
This can be used to selectively restore changed tables from backup
after migrate failure
2019-04-23 10:31:44 +05:30
Michael Oshosanya
a0f8d3f21c Allow user to specify database port 2019-04-21 19:43:49 +01:00
Rushabh Mehta
0f1d8c7622 fix(for-patch): cache manager to ignore missing tables 2019-04-20 16:48:30 +05:30
Suraj Shetty
56dfe365ed fix: Add missing is_column_missing method (#7150) 2019-03-28 10:27:50 +05:30
Frappe Bot
bfc25ad1e5 Merge branch 'master' into develop 2019-02-15 10:21:47 +00:00
Faris Ansari
c687d7e8b7 fix: Cast numeric values to int 2019-01-01 18:27:56 +05:30
Ameya Shenoy
edaeaec5f3
Merge branch 'staging' into develop 2018-10-31 13:50:21 +00:00
Rushabh Mehta
2e6a202652 Postgres support for Frappe (#5919)
* [start] postgres

* [wip] started refactoring db_schema

* Add psycopg2 to requirements.txt

* Add support for Postgres SQL

- Separate frameworkSQL, database, schema, setup_db file for
mariaDB and postgres
- WIP

* Remove quotes from sql to make it compatible with postgres as well

* Moved some code from db_schema to database.py

* Move code from db_schema to schema.py

Add other required refactoring

* Add schema chages

* Remove redundant code in file

* Add invalid column name exception class to exceptions.py

* Add back tick in query wherever needed and replace ifnull with coalesce

* Update get_column_description code in database.py file

* Remove a print statement

* Add keys to get on_duplicate query

* Add bactick wherever necessary

- Remove db_schema.py file

* Remove DATE_SUB as it is incompatible with postgres

- Fix prepare_filter_condition

* Add backtick and quotes wherever necessary
- Move get_database_size to frappe.db namespace
- fix some left out bugs and errors

* Add code to create key and unique index
- added mysql and posgres in their respective database.py

* Add more bacticks in queries and fix some errors
- Pass keys to on_duplicate_update method
- Replace MONTH with EXTRACT function
- Remove DATEDIFF and CURDATE usage

* Cast state value to int in toggle_two_factor_auth
- since two_factor_auth has the datatype of Int

* Refactor
- Replace Timediff with normal arithmetic operator
- Add MAX_COLUMN_LENGTH
- Remove Redundant code
- Add regexp character constant
- Move create_help_table to database.py
- Add get_full_text_search_condition method
- Inherit MariaDBTable from DBTable

* Replace Database instance with get_db method

* Move db_manager to separate file

* Refactor
- Remove some unwanted code
- Separate alter table code for postgres and mysql
- Replace data_type with column_type in database.py

* Make fulltext search changes in global_search.py

* Add empty string check

* Add root_password to site config

* Create cli command for postgres console

* Move setup of help database to setup_db.py

* Add get_database_list method

* Fix exception handling
- Replace bad_field handler with missing_column handler

* Fix tests and sql queries

* Fix import error

* Fix typo db -> database

* Fix error with make_table in help.py

* Try test for postgres

* Remove pyhton 2.7 version to try postgres travis test

* Add test fixes

* Add db_type to the config of test_site_postgres

* Enable query debug to check the reason for travis fail

* Add backticks to check if the test passes

* Update travis.yml
- Add postgres addon

* Try appending 'd_' to hash for db_name
- since postgres does not support dbname starting with a number

* Try adding db_type for global help to make travis work

* Add print statements to debug travis failure

* Enable transaction and remove debug flag

* Fix help table creation query (postgres)

* Fix import issue

* Add some checks to prevent errors
- Some doctypes used to get called even before they are created

* Try fixes

* Update travis config

* Fix create index for help table

* Remove unused code

* Fix queries and update travis config

* Fix ifnull replace logic (regex)

* Add query fixes and code cleanup

* Fix typo
- get_column_description -> get_table_columns_description

* Fix tests
- Replace double quotes in query with single quote

* Replace psycopg2 with psycopg2-binary to avoid warnings
- http://initd.org/psycopg/docs/install.html#binary-install-from-pypi

* Add multisql api

* Add few multisql queries

* Remove print statements

* Remove get_fulltext_search_condition method and replace with multi query

* Remove text slicing in create user

* Set default for 'values' argument in multisql

* Fix incorrect queries and remove few debug flags
- Fix multisql bug

* Force delete user to fix test
- Fix Import error
- Fix incorrect query

* Fix query builder bug

* Fix bad query

* Fix query (minor)

* Convert boolean text to int since is_private has datatype of int
- Some query changes like removed double quotes
and replace with interpolated string to pass multiple
value pass in one of the query

* Extend database class from an object to support python 2

* Fix query
- Add quotes around value passed to the query for variable comparision

* Try setting host_name for each test site
- To avoid "RemoteDisconnected" error while testing data migration test
- Update travis.yml to add hosts
- Remove unwanted commit in setup_help_database

* Set site hostname to data migration connector (in test file)
- To connect the same site host

* Fix duplicate entry issue
- the problem is in naming series file.
In previous commits I unknowingly changed a part of a series query
due to which series were not getting reset

* Replace few sql queries with orm methods

* Fix codacy

* Fix 'Doctype Sessions not found' issue

* Fix bugs induced during codacy fixes

* Fix Notification Test

- Use ORM instead of raw sql

* Set Date fallback value to 0001-01-01

- 0000-00-00 is invalid date in Postgres
- 0001-01-01 works in both

* Fix date filter method

* Replace double quotes with single quote for literal value

* Remove print statement

* Replace double quotes with single

* Fix tests

- Replace few raw sql with ORM

* Separate query for postgres

- update_fields_to_fetch_query

* Fix tests

- replace locate with strpos for postgres

* Fix tests

- Skip test for datediff
- convert bytes to str in escape method

* Remove TestBot

* Skip fieldname extraction

* Replace docshare raw sql with ORM

* Fix typo

* Fix ancestor query test

* Fix test data migration

* Remove hardcoded hostname

* Add default option and option list for db_type

* Remove frappe.async module

* Remove a debug flag from test

* Fix codacy

* fix import issue

* Convert classmethod to static method

* Convert few instance methods to static methods

* Remove some unused imports

* Fix codacy

- Add exception type
- Replace few instance methods with static methods
- Remove unsued import

* Fix codacy

* Remove unused code

* Remove some unused codes

- Convert some instance methods to static function

* Fix a issue with query modification

* Fix add_index query

* Fix query

* Fix update_auth patch

* Fix a issue with exception handling

* Add try catch to a reload_doc

* Add try-catch to file_manager_hook patch

* import update_gravatar to set_user_gravatar patch

* Undo all the wrong patch fixes

* Fix db_setup code 😪
- previously it was not restoring db from source SQL
which is why few old patched were breaking
(because they were getting different schema structure)

* Fix typo !

* Fix exception(is_missing_column) handling

* Add deleted code
- This code is only used in a erpnext patch.
Can be moved to that patch file

* Fix codacy

* Replace a mariadb specific function in a query used in validate_series

* Remove a debug flag

* Revert changes (rename_parent_and_child)

* Fix validate_one_root method

* Fix date format issue

* Fix codacy
- Disable a pylint for variable argument warning
- Convert an instance method to static method

* Add bandit.yml

The Codacy seems to use Bandit which generates
warning for every subprocess import and its usage during pytest
Since we have carefully used subprocess (avoided user input),
warnings needs to be avoided.
This can be removed if we have any alternative for subprocess usage.

* Skip start_process_with_partial_path check

* Fix typo

* Add python 2.7 test

* Move python versions in travis.yml

* Add python versions to jobs

* Overwrite python version inheritance for postgres in travis.yml

* Add quotes around python version in .travis.yml

* Add quotes around the name of the job

* Try a travis fix

* Try .travis.yml fix

* Import missing subprocess

* Refactor travis.yml

* Refactor travis.yml
- move install and tests commands to separate files
- Use matrix to build combination of python version and db type

* Make install.sh and run-tests.sh executable

* Add sudo required to travis.yml to allow sudo cmmands in shell files

* Load nvm

* Remove verbose flag from scripts

* Remove command-trace-print flag

* Change to build dir in before script

* Add absolute path for scripts

* Fix tests

* Fix typo

* Fix codacy
- fixes - "echo won't expand escape sequences." warning

* Append (_) underscore instead of 'd' for db_name

* Remove printf and use mysql execute flag
2018-09-21 10:20:48 +05:30
Renamed from frappe/database.py (Browse further)