Testing tutorial

This continues where the previous Writing rules tutorial left off.

One of the chief problems with writing codemods is being able to succinctly test them. Because ick is built around modifying sets of files, the tests for a rule are files showing the before and after states expected.

The ick test-rules command will run tests for your rules. We haven’t written any tests yet, so it has nothing to do:

$ ick test-rules
testing...
  move_isort_cfg: <no-test> PASS

DETAILS
move_isort_cfg: no tests in /tmp/foo/tests/move_isort_cfg

The ick output shows where the tests should go.

In your rule directory, create a tests subdirectory with another subdirectory named for your rule: tests/move_isort_cfg. In there each additional directory will be a test. Create a tests/move_isort_cfg/no_isort directory. In there, the input directory will be the “before” state of the files, and the output directory will be the expected “after” state of the files. Running the test checks that the files in input are transformed to match the files in output when the rule runs.

Create two files input/pyproject.toml and output/pyproject.toml with the same contents:

[project]
name = "foo"

Your directory structure should look like this:

├── ick.toml
├── isort.cfg
├── move_isort_cfg.py
├── pyproject.toml
└── tests/
    └── move_isort_cfg/
        └── no_isort/
            ├── input/
            │   └── pyproject.toml
            └── output/
                └── pyproject.toml

This is a simple test that checks that if there is no isort.cfg file, the pyproject.toml file will be unchanged. Run ick test-rules:

$ ick test-rules
testing...
  move_isort_cfg: . PASS

Now make a more realistic test. Create a change_made directory in the tests/move_isort_cfg directory. Create these files:

change_made/input/isort.cfg:

[settings]
line_length = 88
multi_line_output = 3

change_made/input/pyproject.toml:

[project]
name = "foo"

change_made/output/pyproject.toml:

[project]
name = "foo"

[tool.isort]
line_length = "88"
multi_line_output = "3"

Now ick test-rules shows two tests passing:

$ ick test-rules
testing...
  move_isort_cfg: .. PASS

Now that we have two tests, the full directory structure looks like this:

├── ick.toml
├── isort.cfg
├── move_isort_cfg.py
├── pyproject.toml
└── tests/
    └── move_isort_cfg/
        ├── change_made/
        │   ├── input/
        │   │   ├── isort.cfg
        │   │   └── pyproject.toml
        │   └── output/
        │       └── pyproject.toml
        └── no_isort/
            ├── input/
            │   └── pyproject.toml
            └── output/
                └── pyproject.toml

This can seem intricate, but it establishes a good structure: a directory can have more than one rule, each rule can have more than one test.