Add code testing via phpunit and/or cypress

[parapente]

This is an effort to add automated testing of the code to ensure that the extension works whenever new patches are added to the codebase or the joomla framework changes.

There are few phpunit tests (ported from older tests that were present in the codebase) but almost all fail for now. It will need quite some effort to refactor the extension code to be able to test it effectivly using phpunit.

Cypress on the other hand can help us test the extension as it is running on an joomla instance like we would manually. I started adding some basic tests that are run in batches. I have created folders for each part of the package in /cypress/e2e that eventually will be populated with the tests.

I will add a checklist for everything that needs to be done and of course the last step would be to add github actions to run the tests whenever we push code to master.

[JLTRY]

Hello it seems to be a big work.
I wanted to try to use the tests.
Are there ready yet ?
I see that the file FOR_DEVELOPERS.md was modified for PHPUnit
Is it usable ?
I see also docker files for joomla cypress novnc , what are the files used for ?

[parapente]

For phpunit:

• You need to run composer install (or composer update if you get a warning about the lock file)
• composer test (to run the tests)

For cypress:

• Requirement: You need to have docker compose available on your system (with docker desktop on windows and mac, with docker desktop or docker repository in linux) Install instructions
• Copy the .env.4.4.example to .env (This will add joomla 4.4 with all the necessary requirements for the tests. The other 2 example files are for joomla 5.4 and 6 respectively. I am still testing with 4.4)
• docker compose --profile testing up -d
• If all went well you now can open http://localhost:8090 and connect to novnc which will have cypress desktop opened. You select end 2 end tests and the browser of your choice (I have only tested chrome for now) and then select a spec file to run the tests. It only contains a small amount of tests but I will eventually add a lot more.
• Using http://localhost:8080 you can access the joomla site and if you run docker compose up -d phpmyadmin you will have on http://localhost:8888 phpmyadmin available to access the database.

[JLTRY]

@parapente for phpunit I have an error at st step 1)

Tests\Unit\Helper\AttachmentsInstallTest::testInstallAttachmentsPermissionsMethodExists
PHPUnit\Framework\MockObject\Generator\ReflectionException: Class "Joomla\CMS\User\User" does not exist

[JLTRY]

I fixed last error by adding link from temp\joomla\libraries\src to joomla 4.x library src path

[JLTRY]

Next error is

Tests\Unit\Helper\AttachmentsPermissionsTest::testGetActionsSuperAdmin
Error: Call to a member function load() on false

F:\Sites\site OVH JLT local\joomla_4.0\libraries\src\User\User.php:819`

[parapente]

@JLTRY yes that is the error I'm getting too. The summary gives:

Tests: 222, Assertions: 265, Errors: 75, Deprecations: 16

but most of those that pass as quite trivial tests. Another thing that will be of importance if we continue to work with phpunit is that the joomla framework packages have different numbering from the joomla cms releases:

Joomla Framework	Joomla CMS
2.x	4.x
3.x	5.x
4.x	6.x

So for the packages that I have added as requirements in composer.json we should use the joomla 6 code in /temp/joomla.

[JLTRY]

I have made a link to joomla_6 libraries/src. It still fails
I commented the lines in bootstrap

/*if (class_exists('\\Joomla\\CMS\\Factory')) {
   // If running within a Joomla installation, use the Joomla factory
   // This is for when tests are run from within a Joomla instance
} else {
   // Setup Joomla framework for standalone testing
   // This is where we'd initialize mock services if needed
}*/

I have got this error
Error: mysqli object is not fully initialized
Let me know if you go further

[parapente]

@JLTRY I managed to make the tests run correctly:

PHPUnit 10.5.58 by Sebastian Bergmann and contributors.

Runtime:       PHP 8.4.11
Configuration: /home/DIDE/teo/Projects/attachments/phpunit.xml

..........WW.WWWWWWWWW.........................................  63 / 222 ( 28%)
............................................................... 126 / 222 ( 56%)
............................................................... 189 / 222 ( 85%)
.................................                               222 / 222 (100%)

Time: 00:00.125, Memory: 14.00 MB

OK, but there were issues!
Tests: 222, Assertions: 440, Warnings: 1, Deprecations: 1.

I am not 100% happy with the state of the code but it at least works. I will review the code on the weekend to untangle the mess I created just to make it work. Joomla has many interconnected parts that makes it difficult to run such tests without running a web server & database. Feel free to test on your end for any issues.

[JLTRY]

@parapente
I have the integration tests that are failing

1) Tests\Integration\Component\Admin\Import\ImportAttachmentsTest
Joomla\Database\Exception\UnsupportedAdapterException: PDO Extension is not available.

F:\Sites\repos\attachments\vendor\joomla\database\src\Pdo\PdoDriver.php:184
F:\Sites\repos\attachments\vendor\joomla\database\src\Sqlite\SqliteDriver.php:80
F:\Sites\repos\attachments\vendor\joomla\database\src\Sqlite\SqliteDriver.php:361
F:\Sites\repos\attachments\vendor\joomla\test\src\DatabaseTestCase.php:55

Any advice ?

[parapente]

@JLTRY You need to install the php sqlite extension to be able to run the tests. By the way I forgot to mention that there are some data missing from the repository that were used to initialize some tables with data. They are not really necessary but this is the reason you will see some hard coded values in the code. I will need to extract those values in a seperate structure.

[JLTRY]

Runtime: PHP 8.3.3
Configuration: F:\Sites\repos\attachments\phpunit.xml

..........WW.WWWWWWWWW......................................... 63 / 222 ( 28%)
............................................................... 126 / 222 ( 56%)
............................................................... 189 / 222 ( 85%)
................................. 222 / 222 (100%)

Time: 00:03.729, Memory: 16.00 MB

OK, but there were issues!
Tests: 222, Assertions: 440, Warnings: 1, Deprecations: 1.
same as you !!!

[JLTRY]

@parapente To be noticed that we can tell phpunit to be more verbose by setting
displayDetailsOnTestsThatTriggerDeprecations="true"
displayDetailsOnTestsThatTriggerErrors="true"
displayDetailsOnTestsThatTriggerNotices="true"
displayDetailsOnTestsThatTriggerWarnings="true"
displayDetailsOnPhpunitDeprecations="true"
For Warning I have got always the same one
`

1. F:\Sites\site OVH JLT local\joomla_5.0\libraries\src\Plugin\PluginHelper.php:190
   foreach() argument must be of type array|object, null given

Triggered by:

• Tests\Integration\Component\Admin\Import\ImportAttachmentsTest::testImportAttachmentsFromCSVFile#0
  F:\Sites\repos\attachments\tests\Integration\Component\Admin\Import\ImportAttachmentsTest.php:154
  `
  It is releated to no (null) plugins found in database

[parapente]

I did check the warnings/deprecations using the switches --display-warnings and --display-deprecations but the ones that are left don't really matter much. They are not in our code but due to the way the code is running some joomla function fails to return an array and that is why we get the warning/deprecation. While refactoring the test code I will revisit the warning in case I can make the test skip this joomla plugin function.

[parapente]

@JLTRY I tried to rebuild the joomla_db.xml that was missing from the codebase and use sqlite for querying 'real' data instead of mocking everything. This adds more tests to the phpunit suite. I will continue with cypress tests next.

[JLTRY]

I checked the unit tests and it is OK for me