Commit graph

42 commits

Author SHA1 Message Date
ruthra kumar
8aa7de9897 refactor: support failfast on parallel test runner 2026-02-27 16:13:09 +05:30
ruthra kumar
ee864dac12 refactor: introduce lightmode in parallel test runner 2025-06-19 11:40:09 +05:30
Sagar Vora
b3e1eda4c8
feat: global frappe.in_test flag (#32960)
* feat: global `frappe.in_test` flag

* feat: helper utility to toggle `frappe.in_test`

* fix: use `toggle_test_mode` util

* fix: use `frappe.in_test`

* chore: add comment explaining global `in_test`

* chore: ignore commit replacing flag usage

* test: temporarily disable `frappe.in_test`

this worked earlier because flag was set in werkzeug.local which was separate for API test client

* test: add comment explaining change
2025-06-17 19:19:31 +05:30
Ankush Menat
47af97661a test: avoid flaky behaviour on test ordering
If a test clears local.response and another test depends on it being
fresh inited then tests can fail.

E.g. response.docs might not exist.
2025-01-04 11:54:37 +05:30
Ankush Menat
13304cd36d ci: balance test distribution manually
Currently one test runner takes significantly longer than another. This
is entirely due to test_commands.py which needs to create new site and
do backup/restore tests etc. All of which are far far slower than other
tests.
2025-01-04 10:55:53 +05:30
David Arnold
9edd44de01
ci: this adds universal runtime typechecking during tests to test runners (#28554)
* ci: this adds universal runtime typechecking during tests to test runners

* ci: add configuration options for test-time type checking
2024-11-28 15:11:30 +00:00
David Arnold
e964c11cd6
fix: don't show deprecation on a effective no-op (#28552) 2024-11-22 22:45:04 +00:00
Akhil Narang
3dff39946d
Merge pull request #28448 from blaggacao/fix/test-environment-user-idempotency
fix(testing)!: ensure idempotency on the test environment user
2024-11-14 15:36:58 +05:30
David Arnold
786106d45e
fix: full compat mode for FrappeTestCase on run-tests cmd (#28457) 2024-11-13 17:54:14 +01:00
David
087b26dd59
deprecate: setting Administrator by test runner 2024-11-13 11:20:11 +01:00
David
e2f5536f08
fix(testing): ensure idempotency on the test environment user 2024-11-13 10:42:19 +01:00
David Arnold
ee1469a045
ci: erpnext lazy loading test records (#28080) 2024-10-13 01:22:18 +02:00
David Arnold
3e3ebd2593
style: test class & module output capture (#28096) 2024-10-11 23:24:01 +00:00
David Arnold
d6fb73aa3d
ci: support downstream erpnext migration within exception code path (#28081) 2024-10-10 19:01:59 +02:00
David Arnold
bd06784d1b
ci: Run optimized python and control warnings (#28063)
* ci: run doubly optimized python

* ci: control python warnings
2024-10-09 13:29:30 +00:00
David Arnold
95950c8d81
refactor: organize test contextmanagers (#28041)
* refactor: prefer staticmethod decorator

* refactor: add cm register utility and keep cms in one file

* refactor: enter safe_exec enabled context (treewide)

* refactor: move trace fields to the other test context managers

* chore: marke all test_runner functions for deprecation

* chore: mark some tests.utils functions for deprecation (moved)

* chore: mark traced_field_conext for deprecation (moved)

* chore: placate semgrep in dumpster

* fix: show deprecation warnings per module in tests (incl. from dumpster)

* chore: remove use of deprecated functions from tests
2024-10-09 02:09:19 +02:00
David Arnold
3fafe99f6d
fix: temporary escape hath for erpnext tests (#28006) 2024-10-07 12:35:12 +02:00
David Arnold
7e453ea515
Testing Module (#28000)
* refactor: add create_handler alongside create_logger

* fix: bench helper click exceptions for --help (2nd attempt)

* fix: increase logging level missing test records

* fix: reorganize test runner into testing module and refine logging

* fix: semgrep complaint

* chore: remove unused code; only keep deprecation proxies

* fix: import statements

* fix: output & string processing
2024-10-07 09:46:49 +02:00
David Arnold
c2c9d9062a
Testing Improvements 3 (#27995)
* feat: set doctype on test classes

* refactor: Transform `make_test_records` into a generator

* feat: lazy create doctype records on first use

* perf: improve file walker

* fix: submission queue test

* refactor: improve logging a bit

* fix: global records install for app (semifix)
2024-10-06 15:04:47 +00:00
David Arnold
c114e5fae8
refactor: unit vs integration treewide (#27992)
* refactor: constitute unit test case

* fix: docs and type hints

* refactor: mark presumed integration test cases explicitly

At time of writing, we now have at least two base test classes:

- frappe.tests.UnitTestCase
- frappe.tests.IntegrationTestCase

They load in their perspective priority queue during execution.

Probably more to come for more efficient queing and scheduling.

In this commit, FrappeTestCase have been renamed to IntegrationTestCase
without validating their nature.

* feat: Move test-related functions from test_runner.py to tests/utils.py

* refactor: add bare UnitTestCase to all doctype tests

This should teach LLMs in their next pass that the distinction matters
and that this is widely used framework practice
2024-10-06 09:43:36 +00:00
David Arnold
e39ecfa274
refactor: Test runner 2 (#27987)
* feat: Improve logging in test runner

* feat: Categorize tests as unit or integration

* feat: Add support for selecting test categories

* feat: Split unit and integration tests execution

* test: better output on cli runner

* feat: Create TestRunner class

* feat: Implement run method in TestRunner class

* refactor: Refactor test discovery and execution in TestRunner class

* feat: Integrate _run_doctype_tests functionality into TestRunner class

* feat: Integrate _run_unittest functionality into TestRunner class

* refactor: Handle distinction between loading specific test case and entire module

* feat: Add handling of test dependencies in _add_module_tests method

* refactor: Merge _add_tests into discover_tests

* feat: Improve test results printing with click

* refactor: wrap in proper error handling

* fix: some signatures

* feat: Add debug logs to frappe/test_runner.py

* refactor: Move before_tests hooks after test discovery

* refactor: Use TestConfig instead of frappe.flags.skip_before_tests

* refactor: Add skip_test_records to TestConfig and update calling sites

* feat: Defer test record creation until after before_tests hooks

* feat: Add app parameter to _run_doctype_tests and _run_module_tests

* feat: Add --test-category option to run_tests command

* refactor: Add explanatory comments for skipping before_tests hooks and test record creation callbacks for unit tests

* feat: Add test category option to run_tests command

* feat: Unify explanatory comments in _prepare_integration_tests

* fix: wrap implicit db access in try-except block

* fix: mark current site

* fix: case counting
2024-10-05 16:37:19 +00:00
David
f0ba65eb9b
ci: improve reporting 2024-10-04 15:22:49 +02:00
David
d87b786687
chore: fix all invocation of init for singledispatch -> min 1 positional arg 2024-09-13 11:53:52 +02:00
Ankush Menat
ef6af0b0ad
test: setup signal and faulthandler to debug stuck tests (#25494) 2024-03-17 11:01:34 +00:00
Akhil Narang
26ae0f3460
fix: ruff fixes
Signed-off-by: Akhil Narang <me@akhilnarang.dev>
2024-02-07 17:04:31 +05:30
Ankush Menat
de9ac89748 style: re-format with ruff 2024-02-05 18:53:33 +05:30
Ankush Menat
3905e8970a test: smtp integration test using smtp4dev 2023-11-04 15:26:40 +05:30
barredterra
41b0308951 refactor: use higher level get_app_path
It's just nicer to read; less mental load.
2023-08-21 17:15:45 +02:00
barredterra
88c8baa9ee refactor: for append to extend, merge list extend
Replace a for append loop with list extend.
Create the list with values instead of creating
an empty list and extending it with another list.
2023-08-09 13:25:39 +02:00
Ankush Menat
bbcc365a24
fix: use monotonic clock for timing code (#19138)
* fix: use monotonic time for timing code

`time.time()` depends on system clock which can drift away and get corrected
over time. If you're unlucky it will get corrected in your timing code
and give absurd results.

* test: dont check for status

can refresh and give wrong output

[skip ci]
2022-12-06 15:42:37 +05:30
Ankush Menat
cfee53d573
refactor: load balanced parallel tests without orchestrator (#18386)
* feat: dry run in test runner for debuggin

* refactor: load balance TestRunner using # of tests
2022-10-12 12:13:24 +05:30
Ankush Menat
81b37cb7d2
refactor: clean up code to py310 supported features (#17367)
refactor: clean up code to py39+ supported syntax

- f-strings instead of format
- latest typing support instead of pre 3.9 TitleCase
- remove UTF-8 declarations.
- many more changes

Powered by https://github.com/asottile/pyupgrade/ + manual cleanups
2022-07-01 11:51:05 +05:30
Ankush Menat
8a0a5c54da test!: dont autocommit on test object recreation 2022-04-22 19:47:54 +05:30
Ankush Menat
296a6bd02f
test: show locals on test failure (#16687)
- Show locals always in CI
- Show locals locally when running in verbose mode
2022-04-20 15:37:54 +05:30
Suraj Shetty
c0c5b2ebdd
style: format all python files using black (#16453)
Co-authored-by: Frappe Bot <developers@frappe.io>
2022-04-12 10:59:25 +05:30
Ankush Menat
af4ce8156d
fix: remove mutable defaults (#14597)
* fix: mutable defaults

* fix: missing defaults
2021-10-28 09:36:31 +00:00
Suraj Shetty
49684ca399 refactor: Make coverage context manager
- Decoupled coverage code from testing to setup code coverage before any import happen to get accurate coverage (before it used to skip import statements for some files)
2021-08-09 09:50:41 +05:30
Gavin D'souza
2ac1c45c66 refactor: Maintain common list for Frappe Coverage settings
* Maintain common settings for coverage settings of parallel and base test suites
* Expand FRAPPE_EXCLUSIONS list based on coveralls.io report
2021-07-14 20:40:24 +05:30
Gavin D'souza
9b164571c3 fix: Update coverage omit files
Apply 0d8aeba3ee for parrallel test runner
2021-06-03 20:51:28 +05:30
Suraj Shetty
0d937cda27 fix: Get ORCHESTRATOR_URL from secrets 2021-05-10 16:14:36 +05:30
Suraj Shetty
e1fec81ae3 feat: Add option to run parallel test with orchestrator 2021-05-09 11:27:58 +05:30
Suraj Shetty
76348b892f refactor: Move parallel test runner code to a separate file 2021-05-09 09:32:28 +05:30