Suittest Manual PDF

Módulo 1: Instalación, Solución de Problemas y Configuración Instalación, Solución de Problemas y Configuración del Runner Installation Suittest was developed to be easy to install and try, but depending on the technology under testing, more configurations may be necessary The first step is to...

Módulo 1: Instalación, Solución de Problemas y Configuración Instalación, Solución de Problemas y Configuración del Runner Installation Suittest was developed to be easy to install and try, but depending on the technology under testing, more configurations may be necessary The first step is to log onto Suittest, click in Download Runner Once the download is complete, simply extract the ZIP file into any folder in the machine Execute the Suittest.exe (you can create a shortcut if you want) After opening, the Runner will update for the first time (other updates will happen automatically) After the installation, the default technologies (Web,APP,Mobile,Database Webservices and others) will be available For Web Service testing using SoapUI, it is required to install SoapUI Troubleshooting Video Record If your Video Evidence Recording is not working, make sure that: Windows Media Feature Pack is installed Visual C++ Redistributable 2015 is installed  For Laptops with both Dedicated (usually Intel) and other GPUs (NVidia, AMD, etc) In the control panel of the High end GPU (NVidia, AMD, etc), set the Runner to work on the dedicated GPU If the error "Could not load file or assembly 'ScreenRecorderLib.dll' or one of its dependencies. The specified module could not be found" is shown in the log after a video fails to create, donwload the DependenciesGui software to check what's missing https://github.com/lucasg/Dependencies Install/Update Runner Another common issue is OneDrive (or other Cloud saving applications that replace system folders) can cause conflicts with the Runner installing/Updating/Executing tests This issue is caused by these Cloud Platforms "locking" necessary files to fix this issue, make sure that the Cloud Platforms either don't have access to the Documents and Downloads folder, or create an exception for the Folder where all the Logs/Video Evidences are stored (by default it's in the Documents/TestingTool folder) for OneDrive in particular another issue may arise: OneDrive can replace the system's default Documents folder with a Documents fodler created by OneDrive (from C:\Users\USER\Documents to C:\Users\USER\OneDrive\Documents) If this happens, it may be needed to restore the Documents folder to it's default state Configuration To configure the Runner, after it is launched it is necessary to fill the fields Server is the Suittest link Username is the user to be logged Password for the user password After the fields, click on Authenticate, if everything is correct, the Project Combobox will now have options, select the desired project and the Runner is connected For Web and Desktop testing, some configurations are recommended The Desktop screen text size must be 100% Browser zoom (both app under testing and Suittest web app must be 100% (IE only) Internet Explorer Security zone setting must have Enabled protected mode equal for all 4 (enabled or disabled) (IE only) Internet Explorer must have tabs open a new tab by default Getting Started You are now logged, and your runner is configured, so this means you can create and execute tests To have something to execute, follow these steps: 1 - Go to Test Resources -> Applications 2 - Create an Application (Web/Mobile/Desktop App) 3 - Go to Test Plans o Hint: to keep both Tabs Open, click Test Plans with the middle button on your mouse 4 - Create a Test Case 5 - Add a Test Step 6 - Add a Launch Operation (choose the application you created before) 7 - Click on Debug 8 - Record new Steps Instalación en Android e iOS Android Installation for Local Testing Android can be used in 3 different ways: Local Mode Remote Mode Xamarin Mode o NOTE: THIS ENGINE IS DEPRECATED, it will still receive basic support to keep it functional, but no new features are planned for the future, it is highly recommended to use Local Mode engine for LTS Setting up Local Mode Pre requisites for Local Mode JAVA SDK Installed ANDROID SDK Installed System Environment variable must be set o JAVA_HOME the value will be: C:\Program Files\Java\JDK VERSION o ANDROID_HOME the value will be: %ANDROID SDK INSTALL FOLDER% o PATH two lines must be added: %ANDROID_HOME%\tools and %ANDROID_HOME%\platform-tools Node.JS installed Setting up Appium: Run the following NPM Command: npm i --location=global appium Install the Appium Driver with the command: appium driver install uiautomator2 Starting the server (will be required each time you intend to automate a test) by running the command: appium o in the command line, the available servers should be visible, both the localhost and the public. these can be used in the Host field in Test Resources- >Appication->Host NOTE: for Local and Xamarin Mode, when a real device is used, it must be in developer mode  NOTE: for Local Mode, in order to install and configure Appium, Admin Rights might be necessary Setting up Remote Mode Pre requisites for Remote Mode Access to a Remote Farm: Supported Farms are: o BrowserStack o SauceLabs To Set up the Remote Devices and Farm settings, see Setting up Android Testing, in the Remote Section Setting up Xamarin Mode Pre requisites for Xamarin Mode JAVA SDK Installed ANDROID SDK Installed System Environment variable must be set o JAVA_HOME the value will be: C:\Program Files\Java\JDK VERSION o ANDROID_HOME the value will be: %ANDROID SDK INSTALL FOLDER% o PATH two lines must be added: %ANDROID_HOME%\tools and %ANDROID_HOME%\platform-tools INSTALLATION Download and Extract the Android zip file from an Android application in Suittest Resources Running in Admininistrator mode may be necessary, to stop requiring Admin user, try the following command: netsh http add urlacl url=http://+:31994/ user=Everyone listen=yes (in the argument "user", the value must be in the user's machine language) iOS Installation for Local Testing iOS can be used in 2 different ways: Local Mode Remote Mode Setting up Local Mode Pre requisites for Local Mode minimum macOS: 14.1 minimum Xcode: 15.0.1 Setting up Appium: Run the following Command: npm i --location=global appium Install the Appium Driver with the command: appium driver install xcuitest Starting the server (will be required each time you intend to automate a test) by running the command: appium o in the command line, the available servers should be visible, both the localhost and the public. these can be used in the Host field in Test Resources- >Appication->Host NOTE: for Local Mode, when a real device is used, it must be in developer mode Setting up Remote Mode Pre requisites for Remote Mode Access to a Remote Farm: Supported Farms are: o BrowserStack o SauceLabs To Set up the Remote Devices and Farm settings, see Setting up iOS Testing, in the Remote Section Setting up iOS Testing iOS Testing can be achieved in 2 ways: Local Mode: requires Appium (See iOS Installation -> Appium) Remote Mode: Connects to a Remote Farm o NOTE: both Local Mode and Remote Mode use the same base engine, so scripts should be compatible in each mode Included Remote Farm support: BrowserStack SauceLabs To use a Remote Device: Create a device in Test Resources -> Mobile Devices Associate Farm accounts in the User Account page o Device Farm accounts are connect at User level (each user can have a device farm account) To set the iOS App as available for Remote Farm, select 'Remote' in the Engine field, this will indicate that the application is intended for Remote Farms Extra Configurations: IPA/ZIP Path: the machine path to the IPA/ZIP to be installed (in case of remote, it can be the File URL that the remote farm provides after uploading the app) o If the test is run on a simulator, a ZIP is required, if it's a device, it's the IPA Device: the device to interact with o Local: the Device ID, this field is optional, if left blank, it will use the available connected device/simulator o Remote: the Remote device created in Remote Devices page, right clicking the field will display all the iOS devices available Host: the IP and Host of the Appium Server on Local Engine mode o This field is optional, if left blank, it will use the localhost in the machine  o This field is optional Start as Webview: Apps may a NATIVE and WEB context, setting this field will "force" the app to launch in WEB mode (if the app supoprts it) Setting up Android Testing Android Testing can be achieved in 3 ways: Local Mode: requires Appium (See Android Installation -> Appium) Remote Mode: Connects to a Remote Farm o NOTE: both Local Mode and Remote Mode use the same base engine, so scripts should be compatible in each mode Xamarin Mode: uses a different engine and a specific Android Runner (See Android Installation -> Xamarin) o NOTE: THIS ENGINE IS DEPRECATED, it will still receive basic support to keep it functional, but no new features are planned for the future, it is highly recommended to use Local Mode engine for LTS Included Remote Farm support: BrowserStack SauceLabs To use a Remote Device: Create a device in Test Resources -> Mobile Devices Associate Farm accounts in the User Account page o Device Farm accounts are connect at User level (each Suittest user can have a device farm account) To set the Android App as available for Remote Farm, select 'Remote' in the Engine field, this will indicate that the application is intended for Remote Farms Extra Configurations:  APK Path: the machine path to the APK to be installed (in case of remote, it can be the File URL that the remote farm provides after uploading the app) Device: the device to interact with o Local and Xamarin: the Device ID (can be seen in running the ADB command: adb devices), this field is optional, if left blank, it will use the available connected device/emulator o Remote: the Remote device created in Remote Devices page, right clicking the field will display all the Android devices available Host: the IP and Host of the Appium Server on Local Engine mode or Xamarin based Runner o This field is optional, if left blank, it will use the localhost in the machine Package: the android app package o This field is optional Start as Webview: Apps may a NATIVE and WEB context, setting this field will "force" the app to launch in WEB mode (if the app supoprts it) Módulo 2: Creación y Ejecución de Pruebas, Operaciones y Verificaciones Creación y Ejecución de Pruebas Executing Tests Executions can be schedules on: The logged user: The execution is scheduled in the logged user account Any agent: THe first available agent picks up the execution Specific Agent pool: Only the agents on the pool will pick up the execution An user can only execute or run a debug session, both can't be done by the same user at the same time If enabled, Run Options allows the user to specify rules for the execution, as of now, the available options are:  JIRA Test Plan: Enabled by having a Jira integration and the Request JIRA Test Plan checked JIRA Component: Enabled by having a Jira integration and the Request JIRA Component checked Checking Evidences After an execution, it is possible to check the evidences in 4 different formats Online Report: All steps executed with image evidences PDF Report: All steps executed with image evidences in a downloadable format Video Evidence: Video of the execution Log: Detailed text format log of the execution with detailed information Test Plan Test Plans are essentially folders used to organize tests New Test Plans can be added, edited and removed (if they're empty) Tests can be moved from a folder to another Test Cases Tests are the main resource in Suittest, this is where the user sets the steps to execute Tests can be created with the types: Automated: Create an Automated test that will run a script Manual: Create a manual test that requires a manual execution, evidences can be placed manually as well Performance: Create a Performance test that will execute a performance script Tests supports the following actions:  Save Test: Saves the header fields of the test Create Data Driven: Creates a data driven for the test Disable/Enable: Toggles the status of the test Export: Export the test in JSON or Excel format Move Test: Move the test to another test plan Delete: Deletes the test Time Machine: Check the test history of changes Test Flow: Check the test flow of the test Analytics: Check the test analytics Messages: Test discussion board Copy: Creates a new test based on the test Run: Execute the test (local or agent) Debug: Start a debug session Tests can contain several test steps, in where the operations are placed Tests can also be created by uploading a JSON file (in the same format of the Export) Debug The Debug Feature allows the user to create in real time a test usiong the application under test The Debug requires the Runner for the logged user to be in the ONLINE state While a Debug session is active, the user can't execute tests on the same runner The following features are available Create Session:Creates a new Spy Session End Session:Ends the current Spy Session Refresh Screen:Refreshes the screenshot of the opened application Toggle Size:Toggles the size of the screenshot Show/Hide Logs:Toggles the disply of the session logs Start Record:Starts the record feature Debug contains a Record feature where the user can simulate actions  Debug also allows to insert a brand new step, even allowing to execute it before adding to the test For zone mapping, the user can select an area and all objects inside will be mapped When a step is generated using the Record feature, the object will also be generated, unless it already exists, in that scenario, it will use the existing object When new steps are created, they are placed in a temporary list, where the user can choose which ones to add to the test Record Mode For Record, two modes exist: Screenshot and Screen Record For Screenshot record, a screenshot preview of the application is displayed, and can be interacted with to simulate actions For Screen Record, the user can interact with the application and the steps will be automatically added to the Debug Screen Record is only available with Web Applications To ensure best results, recreate the steps slowly to give time for the Debug session to correctly load them It is recommended that, for best results, the Screen record to be used with two monitors (keeping the application under test isolated, as not to "missclick" when not needed) Operaciones Básicas Operations Operations are the actions used to move the test forward, these operations interact directly with the application, verify certain conditions or create variables to save values Operation Attach Back BreakWhile BrowserPopup Click ClickAt ClickEvent ClickGridControl ClickPopup ClickTable ClickTreeNode Close CloseIFrame ClosePowerShell CloseWindow ConnectIFrame DeleteCookies DoubleClick DoubleClickTable DoubleTap Download Drag DragScreen Else EndIf EndTest EndWhile FileOperation Focus Forward Hold Home HorizontalScroll Hover JavaScript JumpStep Keyboard KeyboardDesktop Launch Maximize Minimize MobileSettings Navigate NextWindow OpenFile OpenPowerShell PowerShellCommand PreviousWindow Refresh RenameExcelSheet Resize RestoreApp RightClick RunKeyword RunQuery SaveJSON SaveXML Select SelectChildTreeNode SelectGridRow SelectParentTreeNode SelectSiblingTreeNode SelectTableColumn SelectToolbar SelectTreeNode Set SetContext SetLocation SetOrientation Submit Swipe SwipeObject SwitchApp Tap Upload VerticalScroll WaitExit Write WriteClipboard WriteExcel WriteJSON WriteTable WriteXML ZoomIn ZoomOut Checked CompareScreenshot Contains CreateEvidence Enabled Equals Exists FileExists Greater HasValue ImageExists ImageNotExists IsEmpty Less MultipleVerification NotChecked NotContains NotEnabled NotEquals NotExists NotVisible SelectHasValue Valid Visible Count FindExcelRow FindTableRow GetAllContext GetContext GetExcel GetSize GetTableColumn GetTableValue GetTreeNodeProperty GetTreeSelectedNodes LoadJSON LoadXML ObjectOCR PositionOCR RandomNumber ReadCell ReadExcel ReadFile ReadJSON ReadXML RunWebServiceTest SaveAlertVariable SaveCellLocation SaveDataVariable SaveObjectVariable SaveObjectVariableAt SavePartialText SaveProperty SaveURL Name Operation Type  Description Requires Object Requires Data Expected Value Requires Additional Data Expected Value Tips File Operations The Operation FileOperation contains sub operations, those are DeleteFile Deletes a file. The Data Field is the file name or the DataDriven file. The Additional Data Field is the Folder location (empty if the DataDriven File was placed) RenameFile Renames a file in the same directory. The Data Field is the path to the original file. The Additional Data Field is the new file name (with extension) CreateFile Creates a new File. The Data Field is the file name or full path (if only the file name is provided, it will be saved in the Downloads folder), the Additional Data Field is the data to be inserted inside the file (supports variables) CopyFile Copy a file into a new one. The Data Field is the source file (can be an existing file in the downloads folder, full file name, DataDriven File or a Base64 value), the Additional Data Field is the target file name (the file will be saved in the Downloads folder) Zip Zips a folder into a new file. The Data Field is the new file name or full name with path. The Additional Data Field is the folder to be zipped UnZip Unzips a file into a folder location. The Data Field is the file name, the Additional Data is the folder location The Folder Location allows special values Downloads to direct to the Downloads folder Desktop to direct to the Desktop Documents to direct to the My Documents folder Keyboard Operations the KeyboardDesktop Operations commands are normal letters and number, and the special keys: SHIFT CONTROL ALT LEFT_ALT RIGHT_ALT RETURN RIGHT BACKSPACE LEFT ESCAPE TAB HOME END UP DOWN INSERT DELETE CAPS F1 F2 F3 F4 F5 F6 F7  F8 F9 F10 F11 F12 F13 F14 F15 F16 F17 F18 F19 F20 F21 F22 F23 F24 PAGEUP PAGEDOWN PRINT PRINTSCREEN SPACE NUMLOCK SCROLL LWIN RWIN the Keyboard Operation for web commands are normal letters and number, and the special keys: Alt ArrowDown ArrowLeft ArrowRight ArrowUp Backspace  Cancel Clear Command Control Decimal Delete Divide. Down End Enter Equal Escape F1 F10 F11 F12 F2 F3 F4 F5 F6 F7 F8 F9 Help Home Insert Left LeftAlt LeftControl LeftShift Meta Multiply  Null NumberPad0 NumberPad1 NumberPad2 NumberPad3 NumberPad4 NumberPad5 NumberPad6 NumberPad7 NumberPad8 NumberPad9 PageDown PageUp Pause Return Right Semicolon Separator Shift Space Subtract Tab Up Environemnt Variables Environment Variables can be used by using the separator #(ENVIRONEMNT_VAR)#. The available values are: CurrentDirectory MachineName  OSVersion ProcessorCount SystemDirectory UserDomainName UserName Version Verifications Operations can be used as Verifications, to check the status of a test To create a verification, use one of the Verification operations (Equals, HasValue, etc), and in the final field, write VERIFY Verifications can be used as Conditions, instead of VERIFY, place the Condiftion to use (IF or WHILE) If a Verification fails, the test fails as well! Variables y Condiciones Test Variables Tests can create Variables to be used during the execution of a test There are operations that will create those Variables (SaveDataVariable, SaveObjectVariable, etc) for verifications based on value (HasValue, Equals, etc) Variables can be used as objects, they will appear in the combo box of objects when they can be used  They can also be used similarly like Data Driven values. To call a variable value, the syntax is: %(VARNAME)% Variables can be treated as text, use more than once on the same field, and appear in macros as well Variables generated during an execution can be seen in the Report, both Online and PDF Conditions conditions are based on verifictions to influence the flow of a test These conditions are: While If/Else Conditions are created using the verifications available (Equals, Exists, etc) Conditions can be created inside other conditions (IFs inside IFS, While inside Whiles, IFs inside Whiles, etc), with no limit It is advised to use them in the same context, for instance, it is not advised to start a condition in the test, and end it inside a keyword IF/Else IF Conditions allow the test to have some behaviour logic based on the conditions of the application If conditions work on an IF X THEN Y ELSE Z condition All Verifications can be used as an IF condition to create a IF condition, in the final field of a Verification, "IF" must be written to end the condition, a new step with the operation ENDIF must be created Example: IF Equals...(operations inside IF) ENDIF ELSE Condition It is also possible to set an ELSE condition, by using the ELSE operation, else is an optional part of the IF condition Example: IF Equals...(operations inside IF) ELSE...(operations inside ELSE) ENDIF While WHILE Conditions allow the test to have some behaviour logic based on the conditions of the application WHILE conditions work on an WHILE X then Y condition All Verifications can be used as WHILE condition to create a WHILE condition, in the final field of a Verification, "WHILE" must be written to end the condition, a new step with the operation ENDWHILE must be created a While cycle can be ended at any time by using the BREAKWHILE operation Example: WHILE Equals...(operations inside WHILE) ENDWHILE as a safety measure, the maximum amount of WHILE cycles are 200, in order to avoid an infinite loop Macros Macros allow the user to transform text They can be placed in the data field, during execution the value will be transformed Macros usually have a number of arguments, those arguments can be filled by raw text, variables or data driven Macros are called as: /MACRONAME arg1,arg2,... Add Adds two numbers, return the result /add 1,2 /add %(VAR1)%,%(VAR2)% Subtract Subtracts two numbers, return the result /subtract 5,2 /subtract %(VAR1)%,%(VAR2)% Multiply Multiples two numbers, return the result /multiply 5,2 Divide Divides two numbers, return the result /divide 5,2  Combine joins two string, return the result /combine 5,2 /combine %(VAR1)%,%(VAR2)% Replace Replaces words from a text, return the result /replace TEXT,T,M will return MEXM /replace %(ORIGINAL)%,%(VAR1)%,%(VAR2)% First returns the first X characters from a text /first TEXT,2 returns TE Last returns the last X characters from a text /last TEXT,2 returns XT Between returns the characters between X and Y (index starts at 0) /between TEXT,0,3 returns TEX /between TEXT,1,3 returns EX /between TEXT,2,3 returns X Split Splits a text into different values, splitting from X, return the Y index /split TTAMM,A,1 returns TT Date Creates a Date value, allows date modifications and return format /date DATE,INCREMENTs,FORMAT,CULTURE (INCREMENTS,FORMAT and Culture are optional) /date Today returns the current date /date Today,1D returns the current date + one day /date Today,-1Y returns the current date - one year /date Today,1D&1Y returns the current date + one day and one year /date Today,,yyyyMMddHHmmss returns the current date in the yyyyMMddHHmmss format more info about date formats: https://docs.microsoft.com/en- us/dotnet/standard/base-types/custom-date-and-time-format-strings more info about Culture: https://docs.microsoft.com/en- us/openspecs/windows_protocols/ms-lcid/a9eac961-e77d-41a6-90a5-ce1a8b0cdb9c Date increments supported are Y Adds Days (example 1D or -1D) M Adds Months (example 1M or -1M) D Adds Days (example 1D or -1D) WD Adds Week Days (example 1WD or -1WD) H Adds Hours (example 1H or -1H) MI Adds Minutes (example 1MI or -1MI) S Adds Seconds (example 1S or -1S) Query Returns the result from a Query /query TABLEVAR,column;row  Decimal Converts a number into a decimal format, with X amount of decimal places /decimal 10.123,2 returns 10.12 Variable Variable macros allow the user to navigate the properties of a xml or JSON variable /variable %(JSONVAR)%,//xpath returns the value in the node Upper/Lower Upper/Lower macros allow the user to transform the texto into upper or lower case /uppercase %(VAR)% /lowercase %(VAR)% returns the transformed value Test Resources Test Resources are items that assist the project management These Resources are manageable by the team to support the creation of tests, managing applications, and so on The available resource types are: Applications Remote Devices Keywords Data Driven Databases Web Services  Variables Applications Applications are the test subjects of a project Applications can be of the following technologies WEB APP ANDROID IOS Applications have Objects that represent the objects that can be manipulated during a test (button, text fields, etc) Objects can be mapped manually in the Application Object table, or during a Spy or Debug Session Manually added objects are supported for Web and Mobile technologies These objects can be created with either a Xpath or Text o Xpath: the location of an object in an application o Text: The Text of an object, this text can be either a button text, an input placeholder, an object's label, etc. The system will try to generate the best object (can be used in a shift left approach to create tests before the app is available) It is advised to map with the Spy and Debug functionality, since it will be the app itself that generates the mapping object value, instead of being manually, thus making duplicated objects a less likely occurrence Applications supports the following actions: Save App:Updates the aplication header fields Export:Exports the Application and it's objects in a JSON format Spy:Starts a Spy Session for the application Delete:Removes the application  Web Application can import Browser Extensions by placing the Extension CRX file in the WebExtensions folder in the Runner folder (folder should be created automatically after any Web Application execution) Applications also supports Environments, in the secondary list, Environments can be added (DEV,QUAL,etc), which allow a different URL (in Web Applications for instance), and each Object can have a different property per environment Applications of type APP can have two different secondary technologies: Default: used for UWP applications SAP: used for SAP applications SAP Applications SAP applications can have some differences on how to operate the Connection field represents the default SAP Logon connection to interact with (can be overritten by the tests) the GenerateTree on the Spy and Debug is available for SAP applications for more on how SAP objects work, and properties (in particular for the SaveProperty action) available, please consult the SAP GUI Scripting API from SAP Spy The Spy Feature allows the user to inspect an application in order to map objects The Spy requires the Runner for the logged user to be in the ONLINE state While a Spy session is active, the user can't execute tests on the same runner The following features are available Default Page: Sets the default page name for new objects Browser:(Web Only) Set the browser for the spy Create Session:Creates a new Spy Session End Session:Ends the current Spy Session Refresh Screen:Refreshes the screenshot of the opened application  Toggle Size:Toggles the size of the screenshot Show/Hide Logs:Toggles the disply of the session logs Create/Hide Zone:TOggles the mapping type between click and zone Run Action:Executes a predefined action on the opened application For mapping, the Spy allows mapping by click or by zone For zone mapping, the user can select an area and all objects inside will be mapped For Click mapping, the user simply needs to click on the screenshot to map n object When a new object is identified, it is placed in the list Objects can be edited to change the page name, name and property If the objects already exists in the application, the line will have a different color and will display a warning message informing the user that the mapped object can be a duplicate of a previouly created one During Zone mapping, the application may also generate Array objects Remote Devices Suittest supports Remote Devices for Testing These devices can be Android or iOS These devices can be from third party vendors: BrowserStack SauceLabs After creating a remote device, these devices can be added to Android and iOS Applications (mandatory for iOS) Keywords Keywords are resources that encapsulate operations in order for tests to call, calling the keyword instead of all the operations. Any set of operations that are usually replicated in several tests should become a keyword (they can be created from the test itself) A test can call a Keyword by using the RunKeyword Operation A good example of a keyword is the login function on any website, since most tests will start with a login, it makes sense to place it on a keyword, thus if any maintenance is needed, only the keyword needs to be updated Some advantages of Keywords are: Reduce the size of tests Reduce the amount of duplicated combinations of operations Ease of maintenance Keywords allow arguments, this means that values placed in the Object or Data fields don-t need to be fixed, they can be placeholders for arguments that the test sends to place a argument in any field, simply write &(X)& on the respective field, where X is the argument number (starts from 1), this means that is a field contains &(1)&, the first argument when a test calls a keyword, will replace this token Arguments can be set in Object and Data fields NOTE: arguments in the Object field must either be variables or objects, "fixed" values are not allowed (a way to work around this, for values instead of objects, is to send a value, and create a variable inside the keyword to receive the argument) For ease of understand, arguments can have names, on the side table, this will update the Keyword to see the argument name instead of the token, the test will also show the argument and the current value Keyword supports the following actions:  Save Keyword:Updates the keyword header fields (Name, Descriptions, Expected Result) Copy Keyword:Creates a new Keyword based on the previous one, with the selected new name Delete:Deletes the keyword Export:Exports the keyword in a JSON format Time Machine:Displays a history of changes of the keyword Locate:Locates the tests that use the keyword Generate Arguments:Genertes a list of arguments in the argument list Keywords can call other keywords, but they can't call themselves, otherwise the tests runs the risk of entering an infinite loop Data Driven Suittest supports DataDriven Testing DataDriven allows the user to save form fields outside the test themselves, in order to improve the test scalability DataDriven functions as if it was a spreadsheet The number of rows and columns are unlimited DataDriven also supports files, not just text A DataDriven offers some advantages All Arguments can be stored outside the test Ease of maintenance Repeated arguments (credential for instance) don-t need to be repeated Default values can be added to Data Driven cells, this will generate dummy data for testing purposes Add a Data Type to a Column in a column, or row, click on Generate Missing Data  This ameans that, for instance, if a platform credentials are changed, the only change the user is required to make is to update the DataDriven To call a DataDriven value, in the test, instead of the normal value, the user call teh DataDriven value with the following formula: $(DATADRIVEN.COLUMNNAME.ROWNUMBER)$ The row number is optional, by default it will be 1 The row number can also be overwritten by RunSuites, but if the number is selected in the test, that will be the value Databases Suittest supports Database testing To add a Database, create a new record in the list, the following fields will be mandatory An object Name for the database, so the tests can know eachthe database to use Database Technology The Database Server The Database Name The Database Username The Password To execute a Database Test, in the test case, select the operation RunQuery, the result wil be saved in a variable that will contain the result table Using the ReadCell Operation, the user can get the value of a particular cell of the table (by searching the column and row number) THe Spy feature is also available for Database testing, allowing the user to test queries and check the database schema Web Services Suittest supports Web Service testing The types of web services supported are: REST SOAP SOAPUI Integration The user can select the return type of a web service call Raw Text JSON XML WebServices are avaiable in the Spy feature, allowing the user to tests their calls REST To create and execute REST Web Services, create a WebService and choose REST as a technology in the button + New REST Call, the user can create new REST Calls, by filling the fields: Name: the name of the call, which will be used in the tests (can't be duplicated) Method: the type of call (GET,POST,PATCH,etc) URL: the URL of the call BODY: the body of the call (xml, json). this field is optional Headers: the REST Headers necessary to make that call. this fields are optional REST Calls are supported by variables, they can be placed in the URL, BODY and Header fields. SOAP To create and execute SOAP Web Services, create a WebService and choose SOAP as a technology the WSDL can be added in the text field, and saved using the + Update WSDL button To generate the new Calls, save the WSDL following the instructions above, and execute the Spy feature, a button called "Generate SOAP Structure" will be available, by clicking it, all the calls inside the WSDL will be added to the WebService, and the Body auto-filled with a template new calls can't be made, but the user can modify and copy existing calls Name: the name of the call, which will be used in the tests (can't be duplicated) SOAP Action: the SOAP Action that will execute (not recommended to change) SOAP Service: the URL of the SOAP Call (not recommended to change) BODY: the body of the call (xml). this field is optional SOAP Calls are supported by variables, they can be placed in the BODY field. SOAPUI Integration Suittest supports Web Service testing, with intetgration with SoapUI Projects To add a SoapUI Project, the user uploads a.xml SoapUI Project, it will display a list of TestSuites and TestCases of the Project SOAPUI Integration supports the following operations: Upload:Uploads a SoapUI Project To execute a WebService Test, in the test case, select the operation RunWebServiceTest, the data will be TestCase;Property1=X;Property2=Y;... (properties are optional) The Global properties have to be defined in SoapUI, using the Global Properties Menu, adn in the test steps on soapui, the parameter will be ${PropertyName} Variables Variables are objects created by the user during a test execution, that contain a value There are several actions that end with the creation of a Variable (SaveObjectVariable, SaveDataVariable, ReadExcel, etc), these operations usually have the last Data field open with the following placeholder: variable name A variable value can be the text of an object, the result of a macro created by the user, or simply a value saved using a normal operation Variables can be Local or Global Local Variables are used by the test that created them Global Variables are used by every test in the project It's possible to see a variable's history Variables can also be created as Arrays calling a variable as %(VAR)% will return the first index of the variable o Variable index starts at 0 Variables can also be used as the index value (example: %(VAR[index])% Saving a value in a index works by adding the index in the variable name, same as when reading Módulo 3: Ejecución y Reportes, Gestión de Proyectos Ejecución de Pruebas Page for Test Plans Introduction Test Plans stores your Test Cases, you can organize them by folders (or sub-folders), you can delete, copy, enable/disable and other settings. All tests have a Test Key that displays the test number, this number can be used to quickly use them in other modules Tests are shared between your team, and updated in real time, so any member can contribute to the test Folders Test Folders allow you to organize your tests, to create a Folder, click the Add Test Plan button to create a new folder Tests can be filtered in the search bar Once a folder is added, you can click the left button to manage that folder: Create Test -> Creates a new Test Case inside that folder Create Sub Test Plan -> Creates a new folder inside the original folder (creating a sub-folder system) Move Test Plan Under... -> Moves the selected folder inside a previously created folder (selecting --- will place it in the top of the tree) Edit Test Plan -> Updates the folder name Send Tests to Run Suite -> Sends all the tests inside the folder to an existing Run Suite, choosing the "Replace Tests" option makes it that the run suite will override all the tests with the new ones Remove -> Removes the folder, will not work if there are tests or folders inside To open a test, simply choose the folder, the folder will expand and show all the tests and folder inside it, click on the test to open the Test Case Test Case The Test Case will be separated by two zones, the header and the script. The header will show you all the test information, and the test options. The test information consists of: Name of the test Description of the test Creator of the test Key of the test Priority of the test Risk of the test Status of the test for the Status of the test, it will determine how the test operates, also, the status history is available In Implementation the users can implement the test In Acceptance locks the test, only a Team Leader can re-open the test Done the test is complete, it can be re-open by a Team Leader Reopen the test was marked as Done, but was reopened for further edits In the top of the header the possible operations are displayed: Save: Saves the test options of a test (the test script part saves automatically) Add Data Driven/Data Driven: Creates a data driven connected to the test, if the data driven already exists, it redirects to the data driven page Enable/Disable: Toggle the Enable/Disable option for the test, if disabled, no changes can be made Integrations: Opens the integration panel for the test, allowing the user to connect a test to an external platform (JIRA/AZURE/PEGA, etc). only available if the Project has a connection Export: Export the Test Case into Excel format, JSON or Code Delete: Deletes the Test Case (IRREVERSIBLE) Time Machine: Tracks all the changes to the test script, and allows the user to "go back in time" and restore a specific version Flow: Shows a graphic demonstration the script work flow, including Keywords Analytics: Tracks the last executions and performance Messages: Message board for the Test Case  Copy: Copy a test into a new one, the user can choose how much they want to copy Run: Opens the run module, allowing the user to execute a test in the user machine, any agent available or a specific agent pool Debug: Creates a debug session for the Test Case Test Script The test script is the executable part of the test, in order to simulate the appearance of a manual test, the script is composed of Test Steps, and inside of each Test Step, the required operations to fulfill the Test Step For example, supposing that a Test Step requires the user to perform a login, the Test Case would be the login procedure, and the test operations would be Launch Application, Write Username, Write password and Click the login button Test Steps A Test Case can have has many Test Steps has the user wants Test steps contains two descriptions fields, the Description and Expected Result After creating/opening a Test Step, the script will be shown to the side The Test Step can be deleted, or copied to another Test Case Test Operations Test Operations are the actions the script will automate, they can be Actions: operations that interact with an application Verifications: operations that verify the status of an application or variables Variables: operations that save values Operations can be Edited, Reordered, Enabled/disabled, duplicated or removed in the operations header, a list of more actions are available:  Create Keyword: creates a keyword with the selected operations, the user can choose if they want the data field, orbjects or both become arguments, the user can also choose if they want the test to replace the steps with the keyword Copy to Test: Copies the selected operations to another test Copy to Keyword: Copies the selected operations to a keyword Convert to Multiple Verification: Groups the selected verifications into a new Multiple Verification Move to Step: Moves the operations to a new Test Step, if the Test Step does not exist, it will be created Duplicate Steps: Duplicates all the selected operations Enable Steps: Enables all the selected operations Disable Steps: Disables all the selected operations Remove Steps: Removes all the selected operations Test Operations also support Response Timers, to verify if the execution time is above or below the set response timer Response Timers are set in miliseconds (ms) THey can also be setup to fail the test if the execution is above the response timer (optional feature) Run Suites Run Suites allow the user to create a lists of test executions Each RunSuite can have multiple tests RunSuite alows for the following actions: Save Run Suite: Updates the name of the Run Suite Delete: Remves the Run Suite Rules: Shows the list of rules of the Run Suite Disable/Enable: Toggles the active status of the Run Suite Run Suite: Prepares the execution of the Run Suite  Run Suites can be set to run concurrently or sequential (concurrency only applies to executions on agents) For the Run Suite execution, some actions can be set before hand What happens On Run Fail, if this is set as Stop, the Run Suite will stop here after a test fails, if Continue, it will continue to the next test The order of tests can be switched by reordering the tests (Drag and Drop) Tests can be disabled or removed In the Row field, it places the default Data Driven Row if they can be overwritten (check the DataDriven Page) Run Suite Overrides let the user select pieces of the test they want to override with another, this can be a text override, data driven or Environment Run Suite Rules can be added to create flows between several Run Suites Release Cycles Release Cycles can be added to create a sprint system for your executions The default range of days for a cycle can be changed in the Project Settings Each Release supports several cycles After creating a cycle, Run Suites can be added to the Cycle When executin a Run Suite that is inside a current cycle, the execution will be added in the cycle Cycle completion is measured in % of run suites inside the cycle that have the last execution completed with success Example: If a cycle has 10 Run Suites, all have the last execution as a success, represents a 100% completion Cycles can be copied to a new cycle Revisión de Evidencias Test Runs The Test Runs page shows the full list of executions in the project The information is updated on realtime The user can select several filters to see the information they which to see It will also be possible ro reschedule and check evidences of the executed tests from this page Gestión de Proyectos Tags and Coverages Tags and Coverages can be added to Test Cases To do so: Open a Test Case in the Tag or Coverage field, type the name If it exists, a auto suggest will show, and select the tag/coverage If not, press ENTER to save Tags and Coverages can also have a link (right click on the object, or do it in Tags and Coverages Tab) in the Tags and Coverage page it's possible to create and remove new objects, change the link, view the tests associates, and in the case of coverages, assign to a requirement Requirements Requirements can be added to the project Requirements can have a Risk, Status and Priority Requirements also support attachments To Assign coverages to a requirement, go to the Tags and Coverage Page in the Dashboard, the RBT Coverage Analysis will display the requirement status based on coverages, and test case risk, displaying the status of the test cases executions Runners The Runners page allows the user to check the state of the runners for both the Users and the Agents It also shows a minimized history of executions (the last 20 executions from that User/Agent) The User can reschedule an execution, check the evidences and follow a real time tracker of running executions Schedule Tests and Run Suites can be scheduled to execute at a certain time, instead of a Run ASAP model When Scheduling, the user can choose if they want to schedule a test or a Run Suite The User can also see a calendar showing the scheduled execution, filter by user, and also show in a list format This page is also the place to cancel any execution that hasn't started yet When scheduling the user can: Select the Test/Run Suite Select teh date/time  Select where to execute (User or Agent) Set the recurrence settings (How often to repeat, and how many times) Schedule from API Run Suites can be triggerd to execute using the API This allow integrations with CI/CD tools such as Jenkins to trigger an execution, the API command is POST SERVERURL/rest/api/runsuite/{projectCode}/{userName}/{suiteKey} (example: http://127.0.0.10/rest/api/runsuite/DEMOPROJECT/SAMPLEUSERNAME/1) projectCode is the project code where the Run Suite is userName is the user name of the user (or agent) that will execute suiteKey is the run suite key to execute Run Options can be added as JSON in the message body, JSON Example { "JIRA_TestPlan":"PLAN", "JIRA_Component":"COMP", "JIRA_Project":"PROJ", "TFS_TestPlanId":"TFSTP", "TFS_SuiteId":"TFSS" } The API Result will contains a JSON response with the Overall Status, Execution Key and all the Tests status to verify the status of an execution, the API command is GET SERVERURL/rest/api/runsuiteexecutionstatus/{projectCode}/{executionKey} projectCode is the project code where the Run Suite is executionKey is the run suite execution key (this value is available in the response from the trigger execution call) The API Result will be the same as the trigger execution call Suittest API Suittest supports some API calls to help with the management phase These calls can be used for Scheduling (see Schedule) or Data modification for Data Modification, the calls are: UpdateApplication: this call will update values from the application o to trigger the call, the API command is POST SERVERURL/rest/api/updateapplication/{projectCode} (example: http://127.0.0.10/rest/api/updateapplication/DEMOPROJECT) ▪ projectCode is the project code where the Application is o Body JSON Example { "AppName":"APPNAME", "AppLocation":"NEWLOCATION", } Page for Test Integrations Introduction The Test Integrations page displays the current integration status of all the tests in the project This page is only available if an Integration has been created in the project All test cases are displayed in the list, regardless if they have an integration or not Integrations Test integrations can be managed for the entire project in this page An integration can be created, removed, edited just like it can be turned on or off Page for Defects Introduction The Defects page shows all the defects generated by executions Defects Defects are generated automatically after each execution By default, the system filters defects by not generation duplicates (if a test fails twice for the same reason, a second defect is not generated Defects can have three stages: Open: the defect is currently open Closed: the defect as been closed Rejected: the defect was rejected (false positive) Page for Run Suites Introduction In the Run Suites you canset up execution batteries to execute several tests together You can set up specific data per execution, you can change interact with data driven files, override values and set up rules Run Suites After creating a Run Suite, you can setup the execution You can create Rules, that can interact with your project after an execution You can also choose how the Run Suite will execute, if sequential (running in order), or dynamically (running as fast as possible, regardless of order) Dynamic executions are only available if you choose to execute in an agent (or agent pool) To add tests, you can add them from the table, or from Test Plan page, by selecting a Test Plan Folder, and sending all the tests into a Run Suite When you add a Test in the Run Suite, you have the option to link with a Data Driven (the test must have it's own Data Driven), when it's linked, an execution will be added per Data Driven row Similar with Test Cases, the user can execute a Run Suite on their User, Any Agent or a specific Agent Pool Additionally, a Run Suite can also be triggered from an API (see User Manual) Run Suite Executions Next to the Tests, there is a list of the Run Suite Executions All executions are displayed, and a report can be seen with the executions details Run Suite Overrides Run Suites support value overrides which can change values, application environments and even Data Driven Creating an override, the options will be: Text: simple text override, will replace the Original value with the New Value Data Driven: replaces the Original Data Driven name with the New Data Driven name Environment: Replaces an application default environment with the new environment (in the Original field place the Application name, in the New Value place the desired environment) Run Suite Rules Run Suites support rules that trigger actions depending on events Run Suite Events are: On Success: triggers when a Run Suite executes with success On Fail: triggers when a Run Suite executes and fails After the event, it can trigger one of the actions: Execute Run Suite: Executes another run suite o Arguments: Run Suite Name;Target (User or Agent Pool) Schedule Run Suite: Executes another run suite o Arguments: Run Suite Name;Date Format;Target (User or Agent Pool) Cancel Run Suite: cancels another run suite o Arguments: Run Suite Name Enable Run Suite: enables another run suite o Arguments: Run Suite Name Disable Run Suite: disables another run suite o Arguments: Run Suite Name For Date Format, it is accepted to use /date and date increments such as 1D, 1H, 1Y (D=Days, H=Hours, Y=Years), example: /date 3D means current date +3 days Page for Releases Introduction Run Suites can be aggregated in Releases and Cycles Releases Releases can be added as many as wanted Each Release can have several cycles Cycles can't intersect each other in terms of dates Cycles in the same timeline can exist, as long as they belong in different Releases The default Cycle Range can be modified in the Project Settings How Cycles Work Add Run Suites to Cycles (cycles can also copy Run Suites to another cycle in the release) After a Cycle has a Run Suite, everytime that Run Suite executes, if there is (at least one) a cycle ongoing, the eexecution will be automatically added to the cycle If a Run Suite belongs to multiple ongoing cycles, it will be added to all Completion is determined by how many Run Suites executed successfully in the cycle (counts the last execution during the cycle)  for example, if all Run Suites have executed and have "Passed" status, the cycle will diplay 100% Page for Test Runs Introduction The Test Runs page displays all the executions in the project Executions Executions can be filtered by several filters Executions from Run Suites display the run suite name by pressing the Key filed in an execution, the progress of the execution is displayed step by step, if the test is currently running, the trackers updates in real time Executions in progress can be stopped or paused Evidences Completed executions have a set of evidences: Logs: technical evidence of the execution in text format Online: Online report with image evidences PDF: Detailed report that can be downloaded Video: Video the execution Page for Runners Introduction The Runners module shows the active users and agents in the project Displayes the latest executions, and machine specs Users Users are displayed in the left column Users can have three different status: Offline (Red): the user is currently offline Online (Green): the user is currently online Busy (Blue): the user is executing a test, debug or spy session Agents The Runners module separates users and agents Selecting the Agents tab displays the available agents in the project, similar with the users, their status is also available Page for Requirements Introduction The Requirements page shows all requirements in the project Requirements Requirements can be grouped in folders Requirements have Priority, Risk and Status Requirements also support attachments Coverages can be assigned in the Tags and Coverage tab, displaying the total tests that meet a requirement Page for Tags and Coverages Introduction The Tags and Coverages page shows all the tags and coverages in the project Tags/Coverages Tags and Coverages can be used to group tests A Link can be added for external access Coverages can be assigned to a requirement Page for Schedule Introduction Executions can be scheduled to execute at any time Both Test Cases and Run Suites can be scheduled Scheduling Test Cases and Scheduling works in the same way When creating a schedule, the date, execute machine can be chosen additionally, a recurrence can be added (optional), this recurrence will determine if the execution will be repeated based on time frame (hour, day, week, month)