seitime-frappe/frappe/tests
Ankush Menat 9b79dfeb7b
perf: "random" naming to improve concurrency and locality (#30053)
This feels overengineered and it kinda is, but other efforts to
inroduce sequential naming/UUID naming haven't been that fruitful
either.

10 character random "hash" i now changed to.

1. first character - last character in UUID4 ID of request/job
2. three characters - derived from current timestamp.
4. 6 characters - random data.

This satisfies all three requirements:

1. Readers - temporal locality should result in spatial locality on disk. (fewer pages accessed)
2. Single writer - temporal locality should result in spatial locality. (fewer dirty pages)
3. Multiple writers - temporal locality should NOT result in spatial locality. (less lock contention)

Mostly concludes https://github.com/frappe/frappe/pull/25309 and https://github.com/frappe/frappe/pull/28349

Rough probabiliy numbers

Assumptions:
- Unique per worker prefix - 16 (uuid's base16 version)
- Rough time spent generating names - 10% of request (very very conservative estimate)

Probability(collision) = P(at least one prefix collision) * P(time collision)
Probability(collision) = (1 - p(all different)) * 10%
Probability(collision) = (1 - (16! / 16-N! )/ 16^N ) * 10%

| N (concurrency) | Probability(collision) |
| 1  |    0.0% |
| 2  |    0.6% |
| 3  |    1.8% |
| 4  |    3.3% |
| 5  |    5.0% |
| 6  |    6.6% |
| 7  |    7.9% |
| 8  |    8.8% |
2025-01-25 07:04:09 +00:00
..
classes test: ttl and maxsize 2025-01-06 18:57:57 +05:30
data test: Add test for lazy_import 2022-08-11 23:53:21 +05:30
utils test: reduce noise in test output (#28862) 2024-12-23 06:11:47 +00:00
__init__.py refactor: organize test contextmanagers (#28041) 2024-10-09 02:09:19 +02:00
README.md refactor: Structure frappe.test.utils (green to green) (#28038) 2024-10-08 15:10:24 +00:00
test_api.py refactor: unit vs integration treewide (#27992) 2024-10-06 09:43:36 +00:00
test_api_v2.py feat(APIv2): Add comment via REST API (#25889) 2024-04-10 10:53:22 +00:00
test_assign.py refactor: unit vs integration treewide (#27992) 2024-10-06 09:43:36 +00:00
test_auth.py feat: add allowed referrers to loosen csrf prevention (#27841) 2024-11-15 12:09:53 +05:30
test_background_jobs.py refactor: fixup with ruff 0.8.1 2024-12-04 13:18:04 +05:30
test_base_document.py refactor: unit vs integration treewide (#27992) 2024-10-06 09:43:36 +00:00
test_boilerplate.py test: adjust param for get_user_input_stream 2024-01-29 15:18:08 +01:00
test_boot.py perf: cache unseen notes 2025-01-14 19:43:22 +05:30
test_caching.py fix: only cache if called directly 2025-01-24 19:08:04 +05:30
test_child_table.py refactor: unit vs integration treewide (#27992) 2024-10-06 09:43:36 +00:00
test_client.py test: avoid flaky behaviour on test ordering 2025-01-04 11:54:37 +05:30
test_client_cache.py fix: Avoid mixing client cache and redis_wrapper 2025-01-14 15:20:47 +05:30
test_config.py refactor: unit vs integration treewide (#27992) 2024-10-06 09:43:36 +00:00
test_cors.py refactor: unit vs integration treewide (#27992) 2024-10-06 09:43:36 +00:00
test_dashboard_connections.py refactor: unit vs integration treewide (#27992) 2024-10-06 09:43:36 +00:00
test_db.py perf: misc client cache improvements (#29070) 2025-01-07 16:14:43 +05:30
test_db_query.py refactor(typing): type filters (#28218) 2024-12-04 23:18:53 +00:00
test_db_update.py refactor: unit vs integration treewide (#27992) 2024-10-06 09:43:36 +00:00
test_defaults.py refactor: unit vs integration treewide (#27992) 2024-10-06 09:43:36 +00:00
test_deferred_insert.py refactor: unit vs integration treewide (#27992) 2024-10-06 09:43:36 +00:00
test_doc_ref.py chore(docref): fix circular imports (#28282) 2024-10-24 22:31:12 +02:00
test_docstatus.py refactor: unit vs integration treewide (#27992) 2024-10-06 09:43:36 +00:00
test_document.py chore(printview): change error message 2024-12-23 13:29:44 +05:30
test_document_locks.py refactor: unit vs integration treewide (#27992) 2024-10-06 09:43:36 +00:00
test_document_ro_mode.py fix: make read only mode thread safe (#28359) 2024-11-18 15:00:50 +01:00
test_domainification.py refactor: unit vs integration treewide (#27992) 2024-10-06 09:43:36 +00:00
test_dynamic_links.py refactor: unit vs integration treewide (#27992) 2024-10-06 09:43:36 +00:00
test_email.py refactor: clarify test record dep management in test modules (#28060) 2024-10-09 13:44:27 +00:00
test_exporter_fixtures.py refactor: unit vs integration treewide (#27992) 2024-10-06 09:43:36 +00:00
test_fixture_import.py refactor: unit vs integration treewide (#27992) 2024-10-06 09:43:36 +00:00
test_fmt_datetime.py refactor: unit vs integration treewide (#27992) 2024-10-06 09:43:36 +00:00
test_fmt_money.py refactor: unit vs integration treewide (#27992) 2024-10-06 09:43:36 +00:00
test_form_load.py refactor: clarify test record dep management in test modules (#28060) 2024-10-09 13:44:27 +00:00
test_formatter.py refactor: unit vs integration treewide (#27992) 2024-10-06 09:43:36 +00:00
test_frappe_client.py refactor: unit vs integration treewide (#27992) 2024-10-06 09:43:36 +00:00
test_global_search.py refactor: unit vs integration treewide (#27992) 2024-10-06 09:43:36 +00:00
test_goal.py refactor: unit vs integration treewide (#27992) 2024-10-06 09:43:36 +00:00
test_hooks.py perf: cache hooks in client-side cache 2025-01-06 20:24:27 +05:30
test_linked_with.py refactor: unit vs integration treewide (#27992) 2024-10-06 09:43:36 +00:00
test_listview.py refactor: unit vs integration treewide (#27992) 2024-10-06 09:43:36 +00:00
test_local_proxy.py perf: faster LocalProxy (#29181) 2025-01-20 12:49:55 +00:00
test_model_utils.py refactor: unit vs integration treewide (#27992) 2024-10-06 09:43:36 +00:00
test_modules.py refactor: unit vs integration treewide (#27992) 2024-10-06 09:43:36 +00:00
test_monitor.py refactor: unit vs integration treewide (#27992) 2024-10-06 09:43:36 +00:00
test_naming.py perf: "random" naming to improve concurrency and locality (#30053) 2025-01-25 07:04:09 +00:00
test_nestedset.py refactor: unit vs integration treewide (#27992) 2024-10-06 09:43:36 +00:00
test_non_nullable_docfield.py refactor: unit vs integration treewide (#27992) 2024-10-06 09:43:36 +00:00
test_oauth20.py refactor: fixup with ruff 0.8.1 2024-12-04 13:18:04 +05:30
test_password.py refactor: unit vs integration treewide (#27992) 2024-10-06 09:43:36 +00:00
test_password_strength.py test: retry flaky password_strength test thrice 2023-12-20 13:57:48 +05:30
test_patches.py refactor: unit vs integration treewide (#27992) 2024-10-06 09:43:36 +00:00
test_pdf.py refactor: unit vs integration treewide (#27992) 2024-10-06 09:43:36 +00:00
test_perf.py perf: misc client cache improvements (#29070) 2025-01-07 16:14:43 +05:30
test_permissions.py refactor: clarify test record dep management in test modules (#28060) 2024-10-09 13:44:27 +00:00
test_printview.py refactor: unit vs integration treewide (#27992) 2024-10-06 09:43:36 +00:00
test_project_meta.py test: impose tax on __init__.py (#28940) 2024-12-28 10:16:33 +05:30
test_query.py refactor: unit vs integration treewide (#27992) 2024-10-06 09:43:36 +00:00
test_query_builder.py chore(communication): cleanup unused code (#28199) 2024-11-14 00:12:14 +01:00
test_query_report.py refactor: organize test contextmanagers (#28041) 2024-10-09 02:09:19 +02:00
test_rate_limiter.py perf: No need to set expiry for rate limiter key everytime (#28956) 2024-12-30 07:31:47 +00:00
test_rating.py refactor: unit vs integration treewide (#27992) 2024-10-06 09:43:36 +00:00
test_recorder.py refactor: Structure frappe.test.utils (green to green) (#28038) 2024-10-08 15:10:24 +00:00
test_redis.py refactor: unit vs integration treewide (#27992) 2024-10-06 09:43:36 +00:00
test_rename_doc.py chore: formatting 2024-12-02 13:40:16 +05:30
test_reportview.py refactor: unit vs integration treewide (#27992) 2024-10-06 09:43:36 +00:00
test_safe_exec.py fix: make shared jenv thread-safe 2025-01-15 11:53:00 +05:30
test_scheduler.py refactor: unit vs integration treewide (#27992) 2024-10-06 09:43:36 +00:00
test_search.py refactor: unit vs integration treewide (#27992) 2024-10-06 09:43:36 +00:00
test_seen.py refactor: unit vs integration treewide (#27992) 2024-10-06 09:43:36 +00:00
test_sequence.py refactor: unit vs integration treewide (#27992) 2024-10-06 09:43:36 +00:00
test_sitemap.py refactor: unit vs integration treewide (#27992) 2024-10-06 09:43:36 +00:00
test_test_utils.py refactor: unit vs integration treewide (#27992) 2024-10-06 09:43:36 +00:00
test_trace.py refactor: organize test contextmanagers (#28041) 2024-10-09 02:09:19 +02:00
test_translate.py test: make test translation independent from real ones 2024-12-12 00:57:14 +01:00
test_twofactor.py test: avoid possible flake in OTP tests (#29214) 2025-01-17 07:32:17 +00:00
test_utils.py test: igore edge case of invalid strings 2025-01-15 12:07:06 +05:30
test_virtual_doctype.py refactor: unit vs integration treewide (#27992) 2024-10-06 09:43:36 +00:00
test_webform.py perf: cache hooks in client-side cache 2025-01-06 20:24:27 +05:30
test_website.py perf: cache hooks in client-side cache 2025-01-06 20:24:27 +05:30
tests_geo_utils.py refactor: unit vs integration treewide (#27992) 2024-10-06 09:43:36 +00:00
translation_test_file.txt feat: allow splitting _() function call on multiple lines 2022-03-26 16:18:35 +05:30
ui_test_helpers.py fix(test): fix tests 2024-08-14 17:08:01 +05:30

Frappe Test Framework

This README provides an overview of the test case framework available in Frappe. These utilities are designed to facilitate efficient and effective testing of Frappe applications.

This is different from the frappe.testing module which houses the discovery and runner infrastructure for CLI and CI.

Directory Structure

The test framework is organized into the following structure:

frappe/tests/
├── classes/
│   ├── context_managers.py
│   ├── unit_test_case.py
│   └── ...
├── utils/
│   ├── generators.py
│   └── ...
├── test_api.py
├── test_child_table.py
└── ...

Key Components

  1. Test case classes (UnitTestCase and IntegrationTestCase)
  2. Framework and class specific context managers
  3. Utility functions and generators
  4. Specific test modules for various Frappe components

Test Case Classes

UnitTestCase (classes/unit_test_case.py)

Import convention: from frappe.tests import UnitTestCase

This class extends unittest.TestCase and provides additional utilities specific to the Frappe framework. It's designed for testing individual components or functions in isolation.

Key features include:

  • Custom assertions for Frappe-specific comparisons
  • Utilities for HTML and SQL normalization
  • Context managers for user switching and time freezing

IntegrationTestCase (classes/integration_test_case.py)

Import convention: from frappe.tests import IntegrationTestCase

This class extends UnitTestCase and is designed for integration testing. It provides features for:

  • Automatic site and connection setup
  • Automatic test records loading
  • Automatic reset of thread locals
  • Context managers that depend on a site connection
  • Asserts that depend on a site connection

For a detailed list of context managers, please refer to the code.

Utility Functions and Generators (utils/generators.py)

This module contains utility functions for generating test records and managing test data.

Specific Test Modules

Various test modules (e.g., test_api.py, test_document.py) contain tests for specific Frappe core components and functionalities.

Note that Document tests are collocated alongside each Document module.

Usage

To use these test utilities in your Frappe application tests, you can inherit from the appropriate test case class:

from frappe.tests import UnitTestCase

class MyTestCase(UnitTestCase):
    def test_something(self):
        # Your test code here
        pass

Contributing

When adding new test utilities or modifying existing ones:

  1. Place them in the appropriate directory based on their function.
  2. Update this README to reflect any significant changes in the framework structure or usage.
  3. Ensure that your changes follow the existing coding style and conventions.

Remember to always refer to the actual code for the most up-to-date and detailed information on available methods and their usage.