seitime-frappe/frappe/database
Ankush Menat e2d619504f
perf: batch bulk_insert (#32675)
* fix: reduce bulk insert batch size

Back when this feature was added it used to lazily evaluate the input.
Now the iterator is consumed upfront so large batch sizes == huge memory usage.

* perf: bring back iterator for bulk_insert

Bulk insert used to support iterator for consuming arbitrarily large
amount of data and inserting it. Since child table support was added, it
can't do it anymore because that requires collecting values.

This change now brings back iterators by batching input iterator (by
default 1000) documents.

This is almost as good as original change from design POV. Performance
is still meh for flat documents.
2025-05-26 13:36:53 +00:00
..
mariadb fix: handle snapshot isolation errors better (#32318) 2025-04-28 06:18:38 +00:00
postgres perf: compute query type only once 2025-03-15 13:39:57 +05:30
sqlite chore(sqlite): set connection timeout to 15 seconds and set synchronous pragma 2025-04-15 13:59:17 +05:30
__init__.py fix: support sqlite 2025-04-15 13:59:16 +05:30
database.py perf: batch bulk_insert (#32675) 2025-05-26 13:36:53 +00:00
db_manager.py refactor: use a database file directly, instead of an SQL file 2025-04-15 13:59:16 +05:30
operator_map.py perf: Avoid Coalesce in QB is implementation 2025-05-02 19:09:51 +05:30
query.py feat: init sqlite 2025-04-15 13:59:16 +05:30
schema.py fix(add_column): adjust for sqlite 2025-04-15 13:59:16 +05:30
sequence.py style: re-format with ruff 2024-02-05 18:53:33 +05:30
utils.py feat: init sqlite 2025-04-15 13:59:16 +05:30