Make sure informative file name make it to the XML report

[silviot]

Currently black tests have BLACK as a node name. Unfortunately this means test reports are not very informative. For instance this is Azure Pipelines showing results from this plugin:
.

This PR changes the node name from BLACK to ${FILENAME}-BLACK so that reports are more informative.

[paddycarey]

Hey @silviot, thanks for this. I'm not sure I fully understand this change though.

From what I can see, the node name is already output as ${FILENAME}::BLACK, is Azure Pipelines treating the :: specially and ignoring the part before that?

The :: seems pretty standard across other pytest plugins like flake8: https://github.com/tholo/pytest-flake8/blob/master/pytest_flake8.py#L90 and I'm a little reluctant to break convention (but very willing to do so if it solves a real problem).

[silviot]

Sorry, I didn't include enough information.

To be honest, I'm not familiar with the internals of pytest so the solution I'm proposing here might not be optimal.

Maybe the optimal solution would be to revisit the strategy of how data is pulled out of test nodes and put into an XML JUnit file, but I lack knowledge of pytest internals to do that. So I just followed the path in reverse from the Azure Pipelines, to the JUnit XML file, to the python code that generates it, to the pytest-black code that puts data there in the first place.

Then I created a test that makes sure the file name is somewhere in the name attribute of the testcase tag of the generated JUnit XML file.

More in detail:

pytest constructs the XML file putting the last element of the :: separated path in the name attrribute of the test XML tag.

This is the XML report of the test in this PR before the change:

<?xml version="1.0" encoding="utf-8"?>
<testsuite errors="0" failures="0" name="pytest" skipped="0" tests="1" time="0.218">
    <testcase classname="" file="test_names.py" line="-1" name="BLACK" time="0.186"/>
</testsuite>

and this is after the change:

<?xml version="1.0" encoding="utf-8"?>
<testsuite errors="0" failures="0" name="pytest" skipped="0" tests="1" time="0.218">
    <testcase classname="" file="test_names.py" line="-1" name="test_names.py-BLACK" time="0.186"/>
</testsuite>

I really wanted the filename to appear in the <testcase name= attribute, and the change in this PR sounded reasonable.

I haven't looked into pytest-flake8 yet (I want to integrate that too, so I'll do that) but I suspect it suffers from the same problem and needs some kind of solution.

Another possible solution is to prepend BLACK, like this:

self._nodeid = "BLACK::" + self._nodeid

so that BLACK appears in the "path" but the rightmost node is also informative and apt to end up the name tag in the XML.

[jaraco]

Please consider contributing this to coherent-oss/pytest-black.