Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🧪 Testing of JS scripts using Cypress integrated into PyTest #1051

Merged
merged 2 commits into from
Nov 2, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .github/workflows/ci.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,12 @@ jobs:
uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python-version }}
- name: Use Node.js
uses: actions/setup-node@v3
with:
node-version: 18
- name: Install Cypress Test Framework
run: npm install cypress
- name: Install Nox Dependencies
run: |
python -m pip install poetry nox nox-poetry pyparsing==3.0.4
Expand Down
3 changes: 2 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,8 @@ docs/_images/need_bar_*.*
.dockerignore

.benchmarks/

tests/cypress/
cypress/

need_bar_*.png
need_pie_*.png
Expand Down
8 changes: 6 additions & 2 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -10,11 +10,15 @@ lint:

.PHONY: test
test:
poetry run pytest -n auto --tb=long --ignore=tests/benchmarks tests/
poetry run pytest -n auto -m "not jstest" --tb=long --ignore=tests/benchmarks tests/

.PHONY: test
test-short:
poetry run pytest -n auto --tb=long --ignore-glob="*official*" --ignore=tests/benchmarks tests/
poetry run pytest -n auto -m "not jstest" --tb=long --ignore-glob="*official*" --ignore=tests/benchmarks tests/

.PHONY: test
test-js:
poetry run pytest -n 1 -m "jstest" --tb=long --ignore-glob="*official*" --ignore=tests/benchmarks tests/

.PHONY: benchmark-time
benchmark-time:
Expand Down
97 changes: 97 additions & 0 deletions docs/contributing.rst
Original file line number Diff line number Diff line change
Expand Up @@ -121,6 +121,103 @@ These snapshots can be updated by running:

pip install -r docs/requirements.txt

Running JS Testcases with PyTest
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

**Setup Cypress Locally**

* Install Node JS on your computer and ensure it can be accessed through the CMD.
* Install Cypress using the npm package manager by running ``npm install cypress``. Visit this link for more information on `how to install Cypress <https://docs.cypress.io/guides/getting-started/installing-cypress#npm-install>`_.
* Verify if Cypress is installed correctly and is executable by running: ``npx cypress verify``. Get out this page for more information about `Cypress commandline <https://docs.cypress.io/guides/guides/command-line>`_.
* If everything is successful then we can use Cypress.

**Enable Cypress Test in Python Test Files**

Under the ``js_test`` folder, you can save your Cypress JS test files (files should end with: ``*.cy.js``). For each Cypress JS test file, you will need to write the Cypress JS test cases in the file. You can read more from the `Cypress Docs <https://docs.cypress.io/>`_. You can also check the ``tests/js_test/sn-collapse-button.cy.js`` file as reference.

In your Python test files, you must mark every JS related test case with the marker - ``jstest`` and you must include the ``spec_pattern`` key-value pair as part of the ``test_app`` fixture parameter.
Also, you need to pass the ``test_server`` fixture to your test function for it to use the automated HTTP test server. For example, your test case could look like this:

.. code-block:: python

# tests/test_sn_collapse_button.py

import pytest


@pytest.mark.jstest
@pytest.mark.parametrize(
"test_app",
[
{
"buildername": "html",
"srcdir": "doc_test/variant_doc",
"tags": ["tag_a"],
"spec_pattern": "js_test/js-test-sn-collapse-button.cy.js"
}
],
indirect=True,
)
def test_collapse_button_in_docs(test_app, test_server):
...

.. note::

The ``spec_pattern`` key is required to ensure Cypress locates your test files or folder. Visit this link for more info on how to set the `spec_pattern <https://docs.cypress.io/guides/guides/command-line#cypress-run-spec-lt-spec-gt>`_.

After you set the ``spec_pattern`` key-value pair as part of the ``test_app`` fixture parameter, you can call ``app.test_js()`` in your Python test case to run a JS test for the ``spec_pattern`` you provided. For example, you can use ``app.test_js()`` like below:

.. code-block:: python

# tests/test_sn_collapse_button.py

import pytest


@pytest.mark.jstest
@pytest.mark.parametrize(
"test_app",
[
{
"buildername": "html",
"srcdir": "doc_test/variant_doc",
"tags": ["tag_a"],
"spec_pattern": "js_test/js-test-sn-collapse-button.cy.js"
}
],
indirect=True,
)
def test_collapse_button_in_docs(test_app, test_server):
"""Check if the Sphinx-Needs collapse button works in the provided documentation source."""
app = test_app
app.build()

# Call `app.test_js()` to run the JS test for a particular specPattern
js_test_result = app.test_js()

# Check the return code and stdout
assert js_test_result["returncode"] == 0
assert "All specs passed!" in js_test_result["stdout"].decode("utf-8")

.. note::

``app.test_js()`` will return a dictionary object containing the ``returncode``, ``stdout``, and ``stderr``. Example:

.. code-block:: python

return {
"returncode": 0,
"stdout": "Test passed string",
"stderr": "Errors encountered,
}

You can run the ``make test-js`` command to check all JS testcases.

.. note::

The ``http_server`` process invoked by the ``make test-js`` command may not terminate properly in some instances.
Kindly check your system's monitoring app to end the process if not terminated automatically.

Linting & Formatting
--------------------

Expand Down
1 change: 1 addition & 0 deletions noxfile.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@
TEST_DEPENDENCIES = [
"pytest",
"pytest-xdist",
"pytest-xprocess",
"syrupy",
"responses",
"lxml",
Expand Down
Loading