Unit Testing
1.Introduction
Frappe provides some basic tooling to quickly write automated tests. There are some basic rules:
- Test can be anywhere in your repository but must begin with
test_and should be a.pyfile. - Tests must run on a site that starts with
test_. This is to prevent accidental loss of data. - Test stubs are automatically generated for new DocTypes.
- Frappe test runner will automatically build test records for dependant DocTypes identified by the
Linktype field (Foreign Key) - Tests can be executed using
bench run-tests - For non-DocType tests, you can write simple unittests and prefix your file names with
test_.
2. Running Tests
This function will build all the test dependencies and run your tests. You should run tests from "frappe_bench" folder. Without options all tests will be run.
bench run-tests
If you need more information about test execution - you can use verbose log level for bench.
bench --verbose run-tests
Options:
--app <AppName>
--doctype <DocType>
--test <SpecificTest>
--module <Module> (Run a particular module that has tests)
--profile (Runs a Python profiler on the test)
--junit-xml-output<PathToXML> (The command provides test results in the standard XUnit XML format)
2.1. Example for app:
All applications are located in folder: "~/frappe-bench/apps". We can run tests for each application.
- frappe-bench/apps/erpnext/
- frappe-bench/apps/erpnext_demo/
- frappe-bench/apps/frappe/
bench run-tests --app erpnext
bench run-tests --app erpnext_demo
bench run-tests --app frappe
2.2. Example for doctype:
frappe@erpnext:~/frappe-bench$ bench run-tests --doctype "Activity Cost"
.
----------------------------------------------------------------------
Ran 1 test in 0.008s
OK
2.3. Example for test:
Run a specific case in User:
frappe@erpnext:~/frappe-bench$ bench run-tests --doctype User --test test_get_value
.
----------------------------------------------------------------------
Ran 1 test in 0.005s
OK
2.4. Example for module:
If we want to run tests in the module:
/home/frappe/frappe-bench/apps/erpnext/erpnext/support/doctype/issue/test_issue.py
We should use module name like this (related to application folder)
erpnext.support.doctype.issue.test_issue
EXAMPLE:
frappe@erpnext:~/frappe-bench$ bench run-tests --module "erpnext.stock.doctype.stock_entry.test_stock_entry"
...........................
----------------------------------------------------------------------
Ran 27 tests in 30.549s
2.5. Example for profile:
frappe@erpnext:~/frappe-bench$ bench run-tests --doctype "Activity Cost" --profile
.
----------------------------------------------------------------------
Ran 1 test in 0.010s
OK
9133 function calls (8912 primitive calls) in 0.011 seconds
Ordered by: cumulative time
ncalls tottime percall cumtime percall filename:lineno(function)
2 0.000 0.000 0.008 0.004 /home/frappe/frappe-bench/apps/frappe/frappe/model/document.py:187(insert)
1 0.000 0.000 0.003 0.003 /home/frappe/frappe-bench/apps/frappe/frappe/model/document.py:386(_validate)
13 0.000 0.000 0.002 0.000 /home/frappe/frappe-bench/apps/frappe/frappe/database.py:77(sql)
255 0.000 0.000 0.002 0.000 /home/frappe/frappe-bench/apps/frappe/frappe/model/base_document.py:91(get)
12 0.000 0.000 0.002 0.000
2.6. Example for XUnit XML:
How to run:
bench run-tests --junit-xml-output=/reports/junit_test.xml
Example of test report:
<testsuite tests="3">
<testcase classname="foo1" name="ASuccessfulTest"/>
<testcase classname="foo2" name="AnotherSuccessfulTest"/>
<testcase classname="foo3" name="AFailingTest">
<failure type="NotEnoughFoo"> details about failure </failure>
</testcase>
</testsuite>
It’s designed for the CI Jenkins, but will work for anything else that understands an XUnit-formatted XML representation of test results.
Jenkins configuration support:
- You should install xUnit plugin - https://wiki.jenkins-ci.org/display/JENKINS/xUnit+Plugin
- After installation open Jenkins job configuration, click the box named “Publish JUnit test result report” under the "Post-build Actions" and enter path to XML report: (Example: reports/*.xml)
3. Tests for a DocType
3.1. Writing DocType Tests:
- Records that are used for testing are stored in a file
test_records.jsonin the doctype folder. For example see the Event Tests. - Test cases are in a file named
test_[doctype].py - To provide the test records (and dependencies) call
test_records = frappe.get_test_records('Event')in your test case file.
Example (for test_records.json):
[
{
"doctype": "Event",
"subject":"_Test Event 1",
"starts_on": "2014-01-01",
"event_type": "Public"
},
{
"doctype": "Event",
"starts_on": "2014-01-01",
"subject":"_Test Event 2",
"event_type": "Private"
},
{
"doctype": "Event",
"starts_on": "2014-01-01",
"subject":"_Test Event 3",
"event_type": "Private",
"event_individuals": [{
"person": "test1@example.com"
}]
}
]
Example (for test_event.py):
# Copyright (c) 2015, Frappe Technologies Pvt. Ltd. and Contributors
# MIT License. See license.txt
import frappe
import frappe.defaults
import unittest
# load test records and dependencies
test_records = frappe.get_test_records('Event')
class TestEvent(unittest.TestCase):
def tearDown(self):
frappe.set_user("Administrator")
def test_allowed_public(self):
frappe.set_user("test1@example.com")
doc = frappe.get_doc("Event", frappe.db.get_value("Event", {"subject":"_Test Event 1"}))
self.assertTrue(frappe.has_permission("Event", doc=doc))
def test_not_allowed_private(self):
frappe.set_user("test1@example.com")
doc = frappe.get_doc("Event", frappe.db.get_value("Event", {"subject":"_Test Event 2"}))
self.assertFalse(frappe.has_permission("Event", doc=doc))
def test_allowed_private_if_in_event_user(self):
frappe.set_user("test1@example.com")
doc = frappe.get_doc("Event", frappe.db.get_value("Event", {"subject":"_Test Event 3"}))
self.assertTrue(frappe.has_permission("Event", doc=doc))
def test_event_list(self):
frappe.set_user("test1@example.com")
res = frappe.get_list("Event", filters=[["Event", "subject", "like", "_Test Event%"]], fields=["name", "subject"])
self.assertEquals(len(res), 2)
subjects = [r.subject for r in res]
self.assertTrue("_Test Event 1" in subjects)
self.assertTrue("_Test Event 3" in subjects)
self.assertFalse("_Test Event 2" in subjects)