IwTest Release Notes ==================== v1.1.0 (10 May 2020) -------------------- * Redesigned look & feel * Added full test suite editing functionality in the UI * Test cases can now be duplicated from the UI, including the possibility to define extra stubbed services * Test cases can now be run immediately from the Test Suites page * Test cases can now be disabled * Test suites now can define a setup and/or a teardown service, optionally specifying an input file * Added 'About' page, including license upload facility v1.2.0 (13 June 2020) --------------------- * Test cases can now be 'repaired', preserving any regular expressions. * Bugfix: pipeline is now saved with the result when a callback fails * Usability: assertions are now sorted in the results page v1.2.1 (16 August 2020) ----------------------- * Bug fix: fields in saved pipelines could not be edited because of an NPE. * Bug fix: Error messages in the GUI popped up if a service under test set the HTTP return code != 2xx * Cosmetic: The full width of the browser window is now used for listing test suites and recording services * Bug fix/feature: New parameter 'record.services.use.service.signature introduced analogous to 'record.stubs.use.service.signature': option to only record advertised inputs and outputs v1.2.2 (21 August 2020) ---------------------- * Bug fix: if 'ui.distributed.enabled' was set to true, then when a test case executes, server wide stubs were created on all known Remote Agents, but they were not removed. Now they are, plus, they're only created if callbacks are defined. v1.3.0 (19 October 2020) ----------------------- * Gave the UI a more modern, cleaner look * Bug fix: Loading a newer version of IwTest & IwTestAgent now doesn't require a restart anymore * Bug fix: Repairing test cases failed with an NPE if the input or output contained a string list or table containing a null value. This has been resolved * Performance: the list of test suite service is now cached. The 'Test Suite' page now loads much faster on IS's with many packages * Empty IData arrays are now also flagged as data types that cannot be represented in JSON * Added links from test case on the Results page to their definitions. * Extra icon for expanding/collapsing test case details v1.3.1 (20 October 2020) ------------------------ * Bug fix: Test suites would not get listed if only one of them referred to a non-existing stub directory. This has been fixed. v1.3.2 (13 December 2020) ----------------------- * Bug fix: After running all test suites, sometimes the statistics were incorrect. * Bug fix: If a test suite pointed to a non-existing folder, it would not show up in the 'Test Suites' page. * Bug fix: an NPE was thrown if a stub was not fully configured. Now a clear exception is thrown. * Bug fix: the input of a setup/teardown service can now be edited through the GUI; Also, an empty input pipeline is now created if a setup/teardown service is selected. * Bug fix: file browser now scans the directory everytime it's activated, instead of showing the previously shown contents. v1.4.0 (10 January 2021) ------------------------ * UI: all dialogs are now modal; sharper look * Feature: Setup/teardown services can now be executed individually and against remote servers. Plus, they're now only executed if the whole test suite is executed. * Feature: introduced the concept of 'environments': a collection of remote servers that support distributed test scenario's. * Feature: in the 'About' page one can now manually check whether a new version is available and read the release notes. One can subsequently download it and install. One can also enable an automatic daily update check. * Feature: One can now drag-and-drop a license file into the upload area. * Feature: one can now edit the inputs *before* duplicating a test case. v1.4.1 (17 January 2021) ------------------------ * Bug fix: Test Suites page: no jump to top anymore when clicking on 'delete' or 'execute' test suite * Cosmetic: the 'Configuration' page has been split into a 'Settings' and an 'Environment page. v1.4.2 (31 January 2021) ------------------------ * Bug fix: the automatic 'check for updates' feature, if enabled, now correctly checks for updates * Cosmetic: added XML syntax highlighting for editing raw pipelines * Upgraded to JSONEditor v9.1.9. v1.4.3 (7 March 2021) --------------------- * Logback is now used as the logging backend, making it independent of IS logging (SAG replaced log4j in 10.5) * Bug fix: byte arrays - a data type supported by IDataXMLCoder - were not properly compared. Now they are. v1.4.4 (11 April 2021) --------------------- * Bug fix: If a pipeline input was specified for a 'service' stub, then reading it during execution failed. Now it's properly read. * The 'remote-this-alias' is no longer saved on the Agent, but rather in IwTest itself. * The mechanism that implements the callback functionality does not generate notifier services anymore. v1.5.0 (16 May 2021) -------------------- * Feature: Utility for migrating WmTestSuite files * Optimization: when generating dynamic stubs, duplicate stub instances are filtered out * Optimization: if there is only one choice for a dynamic stub, the 'match' file is cleared generated v1.5.1 (24 May 2021) -------------------- * Bug fixes for the WmTestSuite migration utility: paths now always contain forward slashes; option for 'lax' validation; corrected erroneous logging statement; better reporting of warnings and errors. * Bug fix: the explicit 'exception' stub now longer wraps a com.wm.app.b2b.server.ServiceException in a com.wm.util.ServerException. *Warning* This might invalidate test cases that use explicit exception stubs! v1.5.2 (04 July 2021) --------------------- * Feature: regular expressions are now also supported in inline exceptions * Feature: the repair functionality now also works on exceptions * *Warning*: The exception passed by the callback mechanism back into the test case is now the 'cause' of the exception, if there is any. This may invalidate test cases that test for exceptions that occur in asynchronously executed code. Example: a divide by zero exception is reported as: java.lang.ArithmeticException: / by zero not as: com.wm.app.b2b.server.ServiceException: java.lang.ArithmeticException: / by zero * Bug fix: for remotely called services that drop inputs, those inputs were still visible in IwTest. This has been corrected. v1.5.3 (14 July 2021) --------------------- * Bug fix: fixed NPE that occurred when recording a (stubbed) service with no declared input or output fields * Bug fix: the message saying that an expected exception was not as expected, contained the class name twice. This has been corrected. * Feature: descriptions for test cases and test suites can now be edited through the UI v1.5.4 (25 October 2021) ------------------------ * Bug fix: Arrays of Shorts, Integers, Longs, Floats and Doubles can now correctly be validated. * Bug fix: When running multiple test suites, it could happen that in the GUI it appeared as if the tests were still running, but in fact had already finished. This has been corrected. * Enhancement: Test Suites and Results are now shown collapsed by default. * Enhancement: There is now easily run all test suites for one package from the Test Suites page: there is now 'run' button for each package. * Enhancement: The setting 'ui.update.check.enabled' has been renamed to 'ui.update.check' with these possible values: update (default), check, off * Enhancement: There is now a clearer visual distinction between stubs and callbacks in the test suite editor by using different background colors. v1.6.0 (13 February 2022) ------------------------- * Feature: a new facility has been added to the 'record' page that let's you execute a local or remote (flow) service. * Feature: you can now validate JSON input and output data * Feature: you can generate random, but structurally correct, input data * Enhancement: on the 'record' page you can now show and hide services that are configured for recording * Enhancement: not all GUI elements on the 'edit test suite' panel were visible for test suites with many test cases. Now they are. * Enhancement: after editing a callback or changing the 'Lax' flag the test suite is saved immediately without the need to click the 'save'-button'. * Bug fix: changing a stubbed service (type 'dynamic') did not change the folder name on the file system. Now it does. v1.6.1 (29 April 2022) ---------------------- * Bug fix: fixed an NPE in iw.test.agent.manage.schema:get that occurred when a service did not have an input or output defined * Bug fix: due to restructuring of the logging facility, the previously defined log-level of iw.test was not set on restored on startup. Now it is. * Bug fix: the timer that checks for updates was not removed when the IwTest package is unloaded. Now it is. * Enhancement: IwTest now also can produce sample data for string tables v1.6.2 (2 May 2022) ------------------ * Bug fix: fixed an NPE in iw.test.generate.service:createTestSuiteService that occured when generating a test suite from recorded data. v1.6.3 (8 May 2022) ------------------ * Performance: The test results page now loads much faster if there are many test case and assertions. Also opening a test suite for editing which has many assertions in the current results now occurs much faster. * Bug fix: the 'Check for update' button on the 'About' page did not return a message. Now it does. * Enhancement: now when the assertion mode is 'lax', mismatches in array length do not result in a validation error anymore. v1.7.0 (20 August 2022) ----------------------- * Feature: global code coverage. There is now a new page that shows for all services whether there any test cases defined for them, and if so, how many and where they are located. There are graphs for the overall coverage and the per package coverage. * Enhancement: the directory where initially test runs are recorded, configured by the setting 'record.storage.location' will now be created, provided the path is relative. * Bug fix: the link from the 'results' page to the test suite definition did not work correctly if the service is tested by two or more test suites. This has been fixed. v1.7.1 (27 September 2022) -------------------------- * Feature: extra option to clear the cache for the service or for all services before executing a test case. * Bug fix: Changing the event type of a callback was not working properly. Now it is. * Bug fix: Fixed an NPE that occurred if a null value was expected, but not present in the actual results. * Enhancement: Empty arrays were not correctly reported in the assertions. Now they are v1.7.2 (24 October 2022) ----------------------- * Enhancement: UI controls for removing/duplicating dynamic stub entries * Enhancement: When generating a test case with a dynamic stub that has more than one entry, a 'default' entry is created as a 'catch-all'. This prevents 'Cannot find matching stub entry' -errors. * Enhancement: Extra 'Close' button on the inline results panel. * Enhancement: One can now also record 'system' services. * Bug fix: Sometimes an output was not recorded if child service was separately recorded. Now the output is always recorded. v1.7.3 (8 November 2022) ------------------------ * Bug fix: the assertion logic in 'lax' mode did not report missing expected array elements. Now it does: expected data must always be present in the actual data. WARNING: existing passing test cases may fail. v1.8.0 (31 January 2023) ------------------------ * Bug fix: Fixed a ClassCastException that occurred if a String[] was encountered instead of a String * Feature: Dynamic Code Coverage. One can now get an assessment of how much flow code was covered by the test cases. There is a separate page that shows in detail which paths were executed. v1.8.1 (12 March 2023) ---------------------- * Bug fix: Dynamic Code Coverage now also works for IntegrationServers running on Java 9 or higher. Warning: a server restart may be required. v1.8.2 (31 August 2023) --------------------- * Bug fix: Reduced the connection timeout for unreachable hosts. This used to be two minutes, now it is a second. * Bug fix: In the Static Code Coverage page the colored progress bar was not always visible. Now it is. * Enhancement: Added a summary table for the Dynamic Code Coverage page. v1.8.3 (14 January 2024) ------------------------ * Enhancement: one can now download the Dynamic Code Coverage report. * Enhancement: the list of test suites is now presented as a tree * Bug fix: IwTest is now insensitive to the Extendend Setting watt.server.http.jsonFormat * Enhancement: one can now use a regular expression to validate the class name of an exception * Enhancement: the 'record' page now remembers which services were selected for stubbing after a recording session was stopped. v1.8.4 (10 February 2024) ------------------------- * Bug fix: Fixed a StackOverflow error when generating sample data for complex document types with multi-level recursive definitions. * Bug fix: A downloaded input/output/error file had a space in its name; now a dot appears instead. * Bug fix: If the 'Base Directory' of a test suite started with './packages' (instead of 'packages'), then the path was considered to be relative to the package, instead of the the IS working directory. * Bug fix: Under certain circumstances a regular expression in an expected output was not honoured. Now it is. v1.8.5 (9 August 2024) --------------------- * Bug fix: iw.test.agent.manage.testrun:delete does not throw an exception anymore if one tries to delete testruns for a service that was not even registered for recording. * Bug fix: In the code coverage report BRANCH and LOOP now say on what they branch or loop over. * Bug fix: Massively reduced the number of calls to iw.test.logging:log. * Enhancement: Comments on a test suite are now carried over to the comments of the corresponding flow service * Enhancement: There appears now a link to the test suite in the message after generating a test suite. * Enhancement: The base directory of a test suite can now be relative to the package it resides in. * Enhancement: There is a now flag 'Ignore data type difference' [yes/no/defer to global default] on the expected results. v1.8.6 (15 November 2024) ------------------------- * Bug fix: The 'Ignore data type difference'-flag now also works on callbacks. * Bug fix: Fixed an NPE that occurred when a service was executed in the 'Record'-page, that had references in its input or output to non-existing document types. * Enhancement: If the 'distributed'-mode is enabled, one can now edit the properties of the remote agents. * Enhancement: The number of the days to retain old log files can now be changed in the 'Settings'-page. * Enhancement: When recording a service (and its defined stubs), their cache is now cleared to force a recording. * Enhancement: One can now execute a test case/suite with its stubs disabled. v1.9.0 (7 January 2025) ----------------------- * Bug fix: An invalid JSON-schema was produced by the 'iw.test.agent.manage.schema:get' service if the input or output defined duplicate fields. This has been corrected. * Bug fix: The downloaded and extracted dynamic code coverage report could not be viewed because of missing js-file. This has been corrected. * Buf fix: The 'Start'-button on the panel for registering a service for recording would become unreacheable if the service implementation was large. This has been corrected. * Enhancement: IwTest now records disabled test cases as 'skipped' if they happened to be included in a test run. This information is also carried over to the result in JUnit format. * Enhancement: One can now easily change a 'normal' expected output into an 'exception' and vice versa. * Enhancement: One now gets a warning when repairing a test case and the failing assertions are inlined. Also if one tries to accept an exception as the expected output, but a normal result was expected (and vice versa). * Enhancement: One now gets a confirmation dialog when trying to delete a stub or a callback. * Enhancement: A new parameter 'record.recordings.max' was introduced in order to limit the number of recorded test runs and as such to prevent accidentally filling up the file system. * Enhancement: Complete overhaul of the Logs page: introduced paging, ordering (ascending/descending), percentage seek and the possibility to view the logs of remote agents. v1.9.1 (8 February 2025) ------------------------ * Bug fix: iw.test.agent.schema:get would not return any sample data if all fields of an input or output structure were optional. Now it does. * Bug fix: When repairing a callback, IwTest would add a field named '$alias' to the pipeline, which during runtime would be absent and then lead to a validation error. This now works correctly, i.e. '$alias' is not added anymore. * Bug fix: dynamic code coverage would not automatically be enabled after a reload of IwTest or a restart of the IntegrationServer, if it previously was enabled in Configuration -> Settings -> execute.coverage.enabled. Now it is. * Bug fix: after downloading the dynamic code coverage results, the archive did not contain all files due to a bug in iw.test.util.file:copy. This has been corrected. * Enhancement: The last opened/edited test suite is now made bold in the test suite definitions overview.