Merge pull request #69 from codefori/feature/improve-test-docs

Document library list setup, execution configuration, test stub preferences

Author: worksofliam

.astro/astro/content.d.ts

@@ -589,5 +589,5 @@ declare module 'astro:content' {
 
 	type AnyEntryMap = ContentEntryMap & DataEntryMap;
 
-	export type ContentConfig = typeof import("../../src/content/config.js");
+	export type ContentConfig = typeof import("./../../src/content/config.js");
 }

.astro/settings.json

@@ -1,5 +1,5 @@
 {
 	"_variables": {
-		"lastUpdateCheck": 1750695278030
+		"lastUpdateCheck": 1752178270354
 	}
 }

src/content/docs/developing/testing/Testing_LibraryValidation.png

@@ -4,7 +4,7 @@ title: Configuring Tests
 
 import { Aside, CardGrid, Card, Tabs, TabItem, FileTree, Steps } from '@astrojs/starlight/components';
 
-## Test Configuration
+## Compile Configuration
 
 The process of defining how to compile your tests is greatly simplified using a configuration file called `testing.json`. This file allows you to specify the parameters for the RPGUnit compile commands (`RUCRTRPG` and `RUCRTCBL`) and the code coverage command (`CODECOV`) in JSON format. The scope in which these parameters are applied is determined by the location of the `testing.json` file and depends on whether you are working with local files or source members.
 
@@ -72,14 +72,15 @@ Use the following steps to create a default configuration for your project:
       {
           "rpgunit": {
               "rucrtrpg": {
-                  "tgtCcsid": 37,
+                  "tgtCcsid": "*JOB",
                   "dbgView": "*SOURCE",
+                  "rpgPpOpt": "*LVL2",
                   "cOption": [
                       "*EVENTF"
                   ]
               },
               "rucrtcbl": {
-                  "tgtCcsid": 37,
+                  "tgtCcsid": "*JOB",
                   "dbgView": "*SOURCE",
                   "cOption": [
                       "*EVENTF"
@@ -96,10 +97,35 @@ Use the following steps to create a default configuration for your project:
 
 ## Execution Configuration
 
-When it comes time to executing the tests, the extension essentially runs the `RUCALLTST` command provided by RPGUnit. The parameters for this command are configured in the VS Code settings.
+When it comes time to executing the tests, the extension essentially runs the `RUCALLTST` command provided by RPGUnit. The parameters for this command modify how the tests are executed, such as the order to run the tests and whether to reclaim resources after each test case. These parameters can be configured in the either the VS Code settings or in the `testing.json` configuration file.
+
+### VS Code Settings
+
+Configuring the execution parameters via the VS Code settings is the suggested option if you prefer to treat these parameters as preferences that will allow each developer to configure for how tests are executed. These settings can be found by going to the extension's settings and searching for **IBM i Testing** (from **Run Order** to **Reclaim Resources**).
 
 ![](Testing_ExecutionConfiguration.png)
 
+### Configuration File
+
+The alternative option is to configure the execution parameters in the `testing.json` configuration file (the same one used for compile command configuration). This is useful if you want to enforce specific execution parameters to be used by all developers working on a project.
+
+Here is an example with all execution parameters set in the `testing.json` file:
+
+```json
+{
+    "rpgunit": {
+        "rucalltst": {
+            "order": "*API",
+            "libl": "*CURRENT",
+            "jobD": "*DFT",
+            "detail": "*BASIC",
+            "output": "*ALLWAYS",
+            "rclRsc": "*NO"
+        }
+    }
+}
+```
+
 ## Compile, Execution, and Coverage Parameters
 
 <Tabs>
@@ -121,7 +147,7 @@ When it comes time to executing the tests, the extension essentially runs the `R
     |`compileOpt` |Pre-Compiler COMPILEOPT |Refer to the `COMPILEOPT` parameter in `CRTSQLRPGI` command help.                                                                                                                                                           |        |
     |`tgtRls`     |Target release          |Refer to the `TGTRLS` parameter in `CRTSRVPGM` command help.                                                                                                                                                                |        |
     |`incDir`     |Include directory       |Specifies one or more directories to add to the search path used by the compiler to find copy files. The compiler will search the directories specified here if the copy files in the source program can not be resolved.   |        |
-    |`tgtCcsid`   |Target CCSID            |Specifies the CCSID that the compiler uses to read the source files.                                                                                                                                                        |37      |
+    |`tgtCcsid`   |Target CCSID            |Specifies the CCSID that the compiler uses to read the source files.                                                                                                                                                        |*JOB    |
     |`wrapperCmd` |Wrapper Command         |Specifies a custom command to wrap the `RUCRTRPG` command.                                                                                                                                                                  |        |
   </TabItem>
   <TabItem label="RUCRTCBL" >
@@ -141,7 +167,7 @@ When it comes time to executing the tests, the extension essentially runs the `R
     |`compileOpt` |Pre-Compiler COMPILEOPT |Refer to the `COMPILEOPT` parameter in `CRTSQLCBLI` command help.                                                                                                                                                           |        |
     |`tgtRls`     |Target release          |Refer to the `TGTRLS` parameter in `CRTSRVPGM` command help.                                                                                                                                                                |        |
     |`incDir`     |Include Directory       |Specifies one or more directories to add to the search path used by the compiler to find copy files. The compiler will search the directories specified here if the copy files in the source program can not be resolved.   |        |
-    |`tgtCcsid`   |Target CCSID            |Specifies the CCSID that the compiler uses to read the source files.                                                                                                                                                        |37      |
+    |`tgtCcsid`   |Target CCSID            |Specifies the CCSID that the compiler uses to read the source files.                                                                                                                                                        |*JOB    |
     |`wrapperCmd` |Wrapper Command         |Specifies a custom command to wrap the `RUCRTCBL` command.                                                                                                                                                                  |        |
   </TabItem>
   <TabItem label="CODECOV" >
@@ -157,41 +183,56 @@ When it comes time to executing the tests, the extension essentially runs the `R
     |Run Order               |Specifies the order for running the test procedures. Useful to check that there is no dependencies between test procedures.                                                                                                                                                           |*API     |
     |Library List            |Specifies the library list for executing the specified unit test.                                                                                                                                                                                                                     |*CURRENT |
     |Job Description         |Specifies the name of the job description that is used to set the library list, when the `IBM i Testing: Library List` setting is set to `*JOBD`. `*DFT` can be used here to indicate the library of the unit test suite (service program) is searched for job description `RPGUNIT`. |*DFT     |
-    |Job Description Library |Specifies the library that is searched for the job description. `*LIBL` can be used here to indicate all libraries in the user and system portions of the job's library list are searched until the first match is found.                                                             |         |
     |Report Detail           |Specifies how detailed the test run report should be.                                                                                                                                                                                                                                 |*BASIC   |
     |Create Report           |Specifies whether a report is created.                                                                                                                                                                                                                                                |*ALLWAYS |
     |Reclaim Resources       |Specifies when to reclaim resources. Resources, such as open files, can be reclaimed after each test case or at the end of the test suite. This option is useful if the test suite calls OPM programs, which do not set the `*INLR` indicator.                                        |*NO      |
   </TabItem>
 </Tabs>
 
-    <Aside type="note">
-        The `wrapperCmd` parameter in the `RUCRTRPG` and `RUCRTCBL` sections are not actual parameters of the RPGUnit commands, but rather are provided by the extension to allow wrappering them. This is primarily used when integrating RPGUnit with other vendor tools.
-
-        **Example `testing.json` configuration**:
-        ```json
-        {
-            "rpgunit": {
-                "rucrtrpg": {
-                    "tgtCcsid": 37,
-                    "dbgView": "*SOURCE",
-                    "cOption": [
-                        "*EVENTF"
-                    ],
-                    "wrapperCmd": {
-                        "cmd": "MYLIB/MYCMD TSTCMD",
-                        "params": {
-                            "desc": "My description"
-                        }
-                    }
-                }
-            }
-        }
-        ```
-        **Resolved command**:
-        ```
-        MYLIB/MYCMD TSTCMD(RPGUNIT/RUCRTRPG TSTPGM(MYUSER/TEMPDET)
-        SRCSTMF('/home/MYUSER/builds/ibmi-company_system/qtestsrc/empdet.test.sqlrpgle')
-        TGTCCSID(37) DBGVIEW(*SOURCE) COPTION(*EVENTF) RPGPPOPT(*LVL2)
-        INCDIR('''/home/MYUSER/builds/ibmi-company_system''')) DESC('My description')
-        ```
-    </Aside>
+<Aside type="note">
+    The `wrapperCmd` parameter in the `RUCRTRPG`/`RUCRTCBL`/`RUCALLTST` specifications are not actual parameters of the RPGUnit commands, but rather are provided by the extension to allow wrappering them. This is primarily used when integrating RPGUnit with other vendor tools.
+
+    **Example `testing.json` configuration**:
+    ```json
+    {
+      "rpgunit": {
+          "rucrtrpg": {
+              "tgtCcsid": "*JOB",
+              "dbgView": "*SOURCE",
+              "cOption": [
+                  "*EVENTF"
+              ],
+              "wrapperCmd": {
+                  "cmd": "MYLIB/MYCMD TSTCMD",
+                  "params": {
+                      "desc": "My description"
+                  }
+              }
+          },
+          "rucalltst": {
+              "wrapperCmd": {
+                  "cmd": "MYLIB/MYCMD CALLCMD",
+                  "params": {
+                      "desc": "My description"
+                  }
+              }
+          }
+      }
+    }
+    ```
+    **Resolved compile command**:
+    ```js
+    MYLIB/MYCMD TSTCMD(RPGUNIT/RUCRTRPG TSTPGM(MYUSER/TEMPDET)
+    SRCSTMF('/home/MYUSER/builds/ibmi-company_system/qtestsrc/empdet.test.sqlrpgle')
+    TGTCCSID(*JOB) DBGVIEW(*SOURCE) COPTION(*EVENTF) RPGPPOPT(*LVL2)
+    INCDIR('''/home/MYUSER/builds/ibmi-company_system''')) DESC('My description')
+    ```
+
+    **Resolved execution command**:
+    ```js
+    MYLIB/MYCMD CALLCMD(RPGUNIT/RUCALLTST TSTPGM(MYUSER/TEMPDET) ORDER(*REVERSE)
+    DETAIL(*ALL) OUTPUT(*ALLWAYS) LIBL(*CURRENT) JOBD(*DFT) RCLRSC(*NO)
+    XMLSTMF('/tmp/testing/vscode-ibmi-testing/RPGUNIT/TEMPDET_1751646914682.xml'))
+    DESC('My description')
+    ```
+</Aside>

src/content/docs/developing/testing/Testing_TestStubPreferences.png

@@ -30,12 +30,12 @@ The [IBM i Testing](https://marketplace.visualstudio.com/items?itemName=IBM.vsco
 The following extensions can be installed from the Visual Studio Code Marketplace or Open VSX Registry:
   * [IBM i Testing](https://marketplace.visualstudio.com/items?itemName=IBM.vscode-ibmi-testing) ***(Required)***
   * [Code for IBM i](https://marketplace.visualstudio.com/items?itemName=HalcyonTechLtd.code-for-ibmi) ***(Required - `2.16.0` minimum)***
-  * [RPGLE](https://marketplace.visualstudio.com/items?itemName=HalcyonTechLtd.vscode-rpgle) ***(Required - `0.30.0` minimum)***
+  * [RPGLE](https://marketplace.visualstudio.com/items?itemName=HalcyonTechLtd.vscode-rpgle) ***(Required - `0.31.0` minimum)***
   * [COBOL](https://marketplace.visualstudio.com/items?itemName=bitlang.cobol) *(Optional - Install if writing COBOL unit tests)*
 
 ### 2. RPGUnit
 
-The `RPGUNIT` library must be be installed on your IBM i in order to leverage the framework and assertions that it provides. It can be installed easily via the extension using the steps below which is the recommended approach or manually using the instructions [here](https://irpgunit.sourceforge.io/help/).
+The `RPGUNIT` library ***(`5.1.0-beta.005` minimum)*** must be be installed on your IBM i in order to leverage the framework and assertions that it provides. It can be installed easily via the extension using the steps below which is the recommended approach or manually using the instructions [here](https://irpgunit.sourceforge.io/help/).
 
 #### Preconditions
 RPGUnit requires OS400 7.5 for the latest features. It can also be installed on 7.4 and 7.3 if the following PTFs have been installed:

src/content/docs/developing/testing/Testing_TestStubPreview.png

@@ -4,6 +4,14 @@ title: Running Tests
 
 import { Aside, CardGrid, Card, Steps } from '@astrojs/starlight/components';
 
+## Setup
+
+### Library List
+
+Before running any tests, it is recommended that `RPGUNIT.LIB` and `QDEVTOOLS.LIB` are added to your library list to avoid issues with resolving include files. In the case you are working with local test files and are using `iproj.json`, you can add this to your `preUsrlibl` in `iproj.json`. In the case of source members, you can add these libraries via the **User Library List** view in Code for IBM i. You may see the following warnings in the case these libraries are not on your library list:
+
+![](Testing_LibraryValidation.png)
+
 ## Test Explorer
 
 For tests to be discovered by the extension, you must first [connect to an IBM i](../../../quickstart/). Once connected, the extension will automatically discover any RPG/COBOL tests as local files from your workspace or as source members from your library list. These tests will all be listed in the built-in **Test Explorer** view in VS Code (look for the beaker icon).
@@ -14,9 +22,9 @@ From here you can hover on any test case, test suite, folder, library, or source
 
 <Steps>
   1. **Deployment *(Local Files Only)***: The contents of your workspace folder will be deployed to a directory in the IFS. The default deployment directory suggested will be of the following format: `/home/<user>/builds/<workspace-folder>/`. This is the same deployment process that takes place when you compile any local files via Code for IBM i.
-  2. **Read Test Configuration**: To customize the parameters used to compile your tests, a global and directory specific test configuration will be read. Learn more about this in the [Test Configuration](../configuring/#test-configuration) section.
-  3. **Compilation**: Given the test configuration and tests selected for execution, the tests will be compiled via the `RUCRTRPG` or `RUCRTCLB` command. This step will be skipped for tests which have had no change since the last compilation.
-  4. **Execution**: The compiled test program will be executed via the `RUCALLTST` command. The parameters for this command are configured as VS Code settings. Again, learn more about this in the [Execution Configuration](../configuring/#execution-configuration) section.
+  2. **Read Compile Configuration**: To customize the parameters used to compile your tests, a global and directory specific test configuration file will be read. Learn more about this in the [Compile Configuration](../configuring/#compile-configuration) section.
+  3. **Compilation**: Given the compile configuration and tests selected for execution, the tests will be compiled via the `RUCRTRPG` or `RUCRTCLB` command. This step will be skipped for tests which have had no change since the last compilation.
+  4. **Execution**: The compiled test program will be executed via the `RUCALLTST` command. The parameters for this command are configured in the VS Code settings or the test configuration file. Learn more about this in the [Execution Configuration](../configuring/#execution-configuration) section.
 </Steps>
 
 <Aside type="note">

src/content/docs/developing/testing/configuring.mdx

@@ -49,6 +49,18 @@ After selecting an option, a test suite will be created if it does not already e
 
 ![Generated Test Stub](Testing_GeneratedTestStub.png)
 
+### Test Stub Preferences
+
+There are various preferences that can be used to configure how the test stubs are generated. To access these preferences, go to the extension's settings and search for **Test Stub Preferences**.
+
+![Test Stub Preferences](Testing_TestStubPreferences.png)
+
+### Test Stub Preview
+
+One notable preference is the **Show Test Stub Preview** which controls whether to show a preview of the test stub before adding it to the file or source member. Once this preference is enabled, the **Refactor Preview** view will be populated with a set of edits to insert the control options & directives, includes, prototypes, and the test cases. You will need to first select the checkboxes next to the items you want to insert and then click on any item to bring up a preview of the code in the editor. Once you are satisfied with the preview, you can click the **Apply** button to insert the code into your file or source member.
+
+![Test Stub Preview](Testing_TestStubPreview.png)
+
 ## API Reference
 
 In order to use the APIs provided by RPGUnit, you need to add the `TESTCASE` copybook to your test suite: