## Issue
`frappe.db.bulk_insert` is not working as expected:
- It will not insert any row if there are less than 3 values
- It will not add 1st row at all. e.g if I'm adding 5 values, it will only add 4.
- It will add values one by one after 2 values, instead it should have inserted items (in db) in chunk of 10000 (as per the code written before).
## Changes Made
- Solved above issues
- use better way to chunk list
- Added Postgres support for bulk_insert API
And now `bulk_insert` will only do **1 db call for each 10000** values.
Note: For testing purpose I made `Test Bulk Insert` doctype and keep chunk size of 100.
## Before
## After
nodocs
Similar to set_value for accepting multiple columns ot be updated for
the same Table through a Dict as the second positional arg
Misc: Added type hints
* feat: log errors on too many writes to ease debugging
* fix(ux): extract and show which app/file/function caused too many writes
* fix: postgres error checking assumes exception is postgres exception
* fix: better default for skipping frames
Typically stack is like: some_app -> your_function -> this function. So last 2 frames need to be skipped.
* fix: postgres error checking assumes exception is postgres exception
* fix: better default for skipping frames
Typically stack is like: some_app -> your_function -> this function. So last 2 frames need to be skipped.
* refactor: show Report button instead of logging
* revert: unnecessary new functionality
* test: assert exact exception
Co-authored-by: Ankush Menat <ankush@frappe.io>
Alias to frappe.db.set_value without having to pass the doctype name
twice or teh second one as None
Other changes:
* set/get db APIs for single DocTypes
* Make use of redundant cache key in frappe.db.get_single_value
If no docnames are found, then use NULL as fallback. I tested this on PostgreSQL & MariaDB and it seems to work as expected
In [1]: from frappe.query_builder import Field
In [2]: frappe.db.set_value('ToDo', Field("creation") > Field("modified"), 'description', 'change 2', for_update=True, debug=True)
SELECT "name" FROM "tabToDo" WHERE "creation">"modified"
Execution time: 0.0 sec
UPDATE "tabToDo" SET "description"='change 2',"modified"='2021-10-22 09:38:40.110481',"modified_by"='Administrator' WHERE "name" IN (NULL)
Execution time: 0.0 sec
This takes care of the following scenario.
During restore, before tabDefaultValue is created if a request/command executes frappe.db.get_tables, then the cached value db_tables is populated with partial list of tables.
bootstrap_database then checks if this partial list contains tabDefaultValue (as a sanity test to see if the restore worked). This check fails even though the database is correctly restored.
