introduce unit, integration and ui tests#164
Conversation
- Implemented HtmlContentHelpers class with methods to scrub dates from strings and normalize HTML content using Jsoup.
- Added regex pattern to identify date formats (e.g., "Fri 25 Dec 2025") and replace them with a placeholder ("Mon 01 Jan 2000"). This makes the output HTML deterministic.
- Created unit tests for the scrubDates method to ensure correct functionality.
|
Interesting, the inline base64 encoded png's differ slightly when run again today. Not the status icons, they remain the same, just the charts. Something is non-deterministic. The font is slightly less legible in one run than the other. Run 1: Run 2: Diff Run 1 vs Run 2 (🔵 This image below is a visual diff of the two images above highlighting the differences found 🔵) Base 64 diff The status icons at the top of this diff are exactly the same base64 but the chart's base64's are all slightly different. 
|
|
The appearance of fonts in java depends on which font packages are installed on the host OS, and which source of java is being used. It's a bit of a pain since linux users often need to install additional OS packages to get proper functionality, even if they have the java package installed. That's probably the root of the initial differences you saw. No idea what's going on in the one with the messed up colours! |
|
Thanks! I was beginning to suspect as much because it was at the same time I was experimenting with installing Chrome and Firefox andI think it adjusted the fonts available. The one will the odd colours is a visual diff just showing where the two prior images differ. It's easier than playing 'spot the difference' with two similar images. I will see if I can stabilise the fonts available in the devcontainet and add some tests around the image generation so it's at least simpler to see what's going on. |
|
The tests are stable within the devcontainer and I believe that will do as a stable reproducible environment for now. |
|
This looks really nice @PeterKneale! What do you think about adding a GitHub Actions workflow that runs the tests for pull_requests, and pushes to Could tie in nicely with #156 👀 |
|
Yes absolutely - I saw your work on that and its definately the right direction. Also I found that its possible to render individual ui components to a PNG for use in approval tests which looks quite useful. I'm working my way through a pipeline making sure I really understand the tools i'm using and will be taking a look at your test data for multiqc soon too - very nice product by the way! |
|
@ewels done |
…all types of tests can use the same test data set.
…nt in the fastq files
- Added missing dependencies to the postCreateCommand in devcontainer.json. - Renamed 'quick' to 'fast' in TestCases.java for consistency. - Created a new readme.md file for tests, detailing test data usage and running instructions.
|
The ui test automation looks to be working well in the devcontainer. Its good to see my tiny hand typed fastq files being rendered out nicely.
|
PeterKneale
changed the title
|
@s-andrews @ewels It was interesting finding out that I can render out individual I dont understand the bioinformatics like you guys but the above two screen shots of the basic stats module have what I understand to be the right numbers of sequences / bases / lengths etc. |
4 participants
Hi @s-andrews, like we discussed briefly I've put together a PR introducing unit, integration and approval tests.
I tried to keep a few constraints in mind while doing this.
testfolder/libfolder, but didnt move the existing ones, can move those should you okay it.In case Approval tests are a bit new I've borrowed this from the below video..They are essentially BlackBox tests that execute FastQC, capture its generated files and compare them to prevous snapshots.
Approval Test:
https://www.youtube.com/watch?v=3viG9KFZ3L0
Further reading
Steps:
jUnit 5andApprovalTestsjars to the lib folder and referenced where necessary.build.xmlfile and verifiedant clean build testworks.test/integration/climinimalandcomplexto the/test/integration/datafolderfastqc_data.txtandfastqc_report.htmlfastqc_data.txtdiffers from those approved copies.scrubberbecause the report date in the html will shift over time. I'll test this tomorrow and make sure that this isnt marked as a regression if i generate another report with a new date.pretty formattingmakes it easier to read.Should you like this direction, I can also look at adding tests using
https://playwright.dev/which can be used todrivea web browser like FireFox,Chrome etc to the html file and validate that each of the links work. Full browser automation is pretty straight forward these days as is adding headless chrome to a dockerised environment.@ewels I liked your design revamp and was reading though your PR and noticed you mentioned backward compatibilty and thought this might help validate your changes too.