Working With Scripting Languages

TestMaker includes Wizards and Recorders to facilitate creating tests. While these are powerful tools in their own right there is only so much one can do with them. For instance, suppose you need to write a test that analyzes the results of a server response according to some complex business logic and takes subsequent action based on the results. At moments like these we recommend you use TestMaker's dynamic scripting capabilities.

Writing A Test Script
Tutorial 1: Running A Test Script
Tutorial 2: Using Agentbase In A Web Test Script
Tutorial 3: Jump Start A Web Test Script
When Is It Appropriate To Use TestGen4Web and the Recorder?
Tutorial 4: Using TOOL Protocol Handlers In A Script
Tutorial 5: Using Scripts in TestScenarios
JSR 223 and Class Instances

Writing A Test Script

Test scripts running in the TestMaker environment have several advantages:

Test scripts are programs. They have input parameters, output values, access to objects and frameworks, and may be version controlled like any other software program.
TestMaker supports a variety of languages, including Java, Jython, Groovy, PHP, Ruby, and many others. TestMaker does so by using the ScriptEngine (JSR 223) that appeared in Java 6.
TestMaker provides the Test Object Oriented Library (TOOL) to enabled the test scripts you write with abilities to speak native SOA, SOAP, Web, telephony, and email protocols.
TestMaker provides a base library of functions (called Agentbase) to make it easier to write a Web-oriented test using HTTP GET and POST commands and to emulate typical browser functions such as image caching.
TestMaker runs test scripts locally for functional and unit testing and turns test scripts into load tests, regression tests, service monitors and more using the TestScenario system and TestNodes.

This tutorial will show a series of examples to demonstrate the TestMaker advantages. We will show writing a test script in the Jython language. Jython is the Python language implemented entirely in Java. Our intent will be to make the concepts presented in these tutorials applicable to all of the supported scripting languages, including Groovy, Ruby, PHP, JavaScript and many others.

Tutorial 1: Running A Test Script

Please follow these steps to create and run your first script on the local TestMaker console.

Open TestMaker and click the Create A New Test button in the QuickStart Helper or click the New Test icon in the icon bar.
Click the Generic Jython TestCase Script button.
Use the Save file selector to choose a name and path for the new script. Since this is a Jython script please choose a file name that ends in .py.
A skeleton of a test script opens in the editor panel. The skeleton implements a JUnit TestCase class, the most basic of unit tests of a service or application. TestCase classes have a setUp, runTest, and tearDown method.
Using the editor change the skeleton to be as follows.

'''





Agent name: Examples1.py





Created on: July 18, 2007





Created by: Frank Cohen, PushToTest











For details on TestMaker see http://www.pushtotest.com





'''











from junit.framework import TestCase





import sys, re, time





from java.lang import Exception











class example1:











    def __init__( self ):





        ''' Initialize the test case '''





        print "Example1 here: Initializing"











    def setUp( self ):





        ''' Add any needed set-up code here. '''





        pass





        





    def runTest( self ):





        ''' Run the test '''





        pass











    def tearDown( self ):





        ''' Add any needed code to end the test here. '''





        pass

                                 






'''





Convenience main method for running this test by itself





otherwise, plug this into XSTest to turn it into a scalability





and load test, and the Service Monitor System (SMS) for





a Quality of Service (QOS) monitor.





'''











if __name__ == 'main':





    print '======================================================='





    print 'example 1: Functional test'





    print '======================================================='





    print 'Test created by TestMaker from http://www.pushtotest.com'





    print





    print











    test = example1()





    test.setUp()





    test.runTest()





    test.tearDown()











    print "done"

Please make a change to the skeleton as highlighted above. This prints an "... Initializing..." message when the script operates and the example1 class runs. The changed skeleton comes with TestMaker in TestMaker_home/example_agents/scriptTutorial/example1.py
Run this script by choosing the Run Agent command in the Agents drop-down menu.
Results of the script appear in the Output panel

=======================================================





example 1: Functional test





=======================================================





Test created by TestMaker from http://www.pushtotest.com

















Example1 here: Initializing





done

In this tutorial we create a script that implements a jUnit TestCase class. We modified the skeleton script slightly and then learned how to run the script locally to your TestMaker computer. In the next tutorial we will learn how to use the functions of the Agentbase library to build a Web test script.

Tutorial 2: Using Agentbase In A Web Test Script

In the previous tutorial we created a script that implements a jUnit TestCase class. In this tutorial we will show how to use the Agentbase library to write a Web test script. Agentbase provides a base set of functions to interact with a Web site over HTTP protocols.Agentbase is found in TestMaker_home/lib/agentbase.py.

This tutorial will show how to operate the HTTP_Example.py script found in in TestMaker_home/example_agents/HTTP_Example.py.

From TestMaker open the HTTP_Example.py script in the TestMaker/example_agents directory. Use the navigation panel (to the left of the editor panel) to view the example_agents directory. Double click HTTP_Example.py. The script appears in the editor panel.
There are some facets of the HTTP_Example script that we should point out first.

Notice that the definition for the HTTP_Example class identifies the JUnit TestCase and AgentBase libraries. This is the way Jython uses inheritence from abstract classes and frameworks. Without having to write any additional code the HTTP_Example class inherits the methods defined in the Agentbase module class.
Read the __init__ method. Notice that this method takes several input values. Look at the bottom of this script to see details on the input values. All of these input values are optional in that the script will use default values if none are sent when the class instantiates.

Read the setUp method. Notice that this method makes a call to the self.config() method. config() is provided in the Agentbase library to establish internal values and parameters to work with a Web site in a test defined in HTTP_Example.
Read the runTest method. Notice the use of the get and post methods to interact with Web applications. Also notice the self.params value being set before several get and post method calls. self.params passes parameters to the target host.

Run this script by choosing the Run Agent command in the Agents drop-down menu.
Observe the results in the Output panel.

Agentbase takes much of the coding out of a test script by providing several immediately useful methods. It also provides emulation of a Web browser cache and checks Web references to images and other Web resources. For instance, while processing a 'get' command Agentbase also checks that all the images referenced in the received Web page actually make it to the browser. Here is a summary of the user selectable parameters for a script that uses Agentbase.

log level = 0- no logging, 1- informational messages, 2- detailed messages, 3- Everything
log destination =
console - to the screen,
file - to a file,
response - to the log file in xml form with the host response,
database - to a JDBC datasource
follow_redirects = 0 - do not follow HTTP 302 response codes, 1 - follow them automatically
successcodes = Regular Expression (regex) defining HTTP success response codes
logpath = path and file name to store log file
sleeptime_min = minimum amount of time in seconds to sleep between requests to the host
sleeptime_max = maximum amount of time to sleep between requests to the host
imagesleeptime = time in seconds to sleep between requests for <img> tag references
loadimgtags = 1 - load <img> tag references, 0 - skip them
imagecache = 1 - emulate a browser cache of <img> tag references, 0 - load all <img> tags
logfirst = 1 - add <testmaker> element to head new log files, 0 - ignore

Agentbase also provides cookie handling for state management. In the HTTP_Example class all of the get and post requests will share a single HTTP protocol handler and store and send cookies for statement management.

Agentbase provides setProxy() method to operate the test in environments using a proxy server.

In this tutorial we showed the many functions Agentbase adds to your ability to rapidly write test scripts using TestMaker. Agentbase provides a base set of functions to interact with a Web site over HTTP protocols. In the next tutorial we will show how to use the TestGen4Web addon to Firefox to jump start a Web-oriented test script.

Tutorial 3: Jump Start A Web Test Script

In the previous tutorial we learned how the Agentbase library provides many functions to accelerate your ability to write test scripts in TestMaker. There are two additional ways to jump start a Web test script: Use the TestGen4Web add-on to the FireFox Web browser and use the Recorder. This tutorial shows how to use both.

Transform A TestGen4Web Test Into A Jython Test Script

TestMaker comes with the TestGen4Web add-on to the FireFox Web browser to record use of a Web application to create a unit test of the Web application. TestGen4Web creates a test in an XML file format that TestMaker operates using a langtype of testgen4web. For instance:

<run name="MyTest" testclass="PTT_Examples_UnitTest.testgen4web" method="testGen4Web" langtype="testgen4web"/>

The test runner in TestMaker that operates a TestGen4Web test plays-back the recorded test. To add more advanced functions to the test we recommend you create a Jython script from the recorded TestGen4Web test file.

Select the Tools drop-down menu -> Transform TestGen4Web Test command.
A file dialog appears. Choose the TestGen4Web file.
A save dialog appears. Choose the name and path of the new Jython script.
The new Jython script appears in the script editor panel.

The new Jython script implements a class that operates the test just as the TestGen4Web XML file format document would but now you have all the richness of the Jython scripting language to customize and make the test more advanced.

Use The Recorder To Write a Jython Test Script

The Recorder is an alternative technique to using TestGen4Web. The Recorder watches you drive a Web-enabled application using your browser and writes a test agent script for you. You may play-back the test agent script in TestMaker or from the command-line to perform a functional (unit) test. Recorded tests may be used in a TestScenario to conduct scalability, performance and service montoring.

The Recorder is built around a smart proxy server that watches for HTTP traffic between your browser and the service. The proxy decodes HTTP GET and POST commands from the browser and the responses from the server. The proxy then writes the Jython and TOOL script commands necessary to replay your use of the Web-enabled application. This design supports HTTP 1.0 and 1.1 compliant browsers, including the use of JavaScript, plug-ins, and ActiveX objects, and Java applets.

The Agent Recorder does not support HTTPS connections. By the time the TestMaker proxy receives the request from the browser, the request is encrypted and TestMaker is not able to decode the body of the HTTP protocol. The only workaround at this time is to temporarily host your application with SSL encryption turned-off while you record the test and then turn on SSL encryption when you play-back the test.

Before using the Recorder you will need to configure your browser to communicate through the Recorder proxy. By default, TestMaker sets the proxy port to 8090. Your browser preferences is usually the place to set the used proxy port to be used. Each browser is different at controlling the proxy server. For example, in Microsoft Internet Explorer 6.0 for Windows 2000, the proxy server settings are found by choosing the Internet Options command in the Tools drop-down menu. Choose the Connections tab and click the LAN Settings... button. The lower portion of the dialog that appears controls the Proxy server settings.

If port 8090 is already in use on your system then change the TestMaker proxy server to use a different port number. In TestMaker choose the Preferences command in the Help drop-down menu and then choose the Recorder tab.

With the proxy settings configured, every request from the browser will go through TestMaker's proxy server. This has a side effect in that TestMaker needs to be running for you to use your browser.

Next we will Record a test of the Web-enabled application hosted at examples.pushtotest.com. The application is a simple servlet that responds with Web pages. Depending on the parameters send to the servlet the response can be a Web page containing random content, responses to HTTP Post commands with HTML forms, and provides a Web site with HTML links our recorded test will follow.

Follow these steps:

1. Configure your browser to use Proxy port 8090.

2. Start TestMaker.

3. Choose File -> New -> HTML Agent Recorder or click the Recorder button.

4. TestMaker displays a dialog asking for the name of the new test agent. Type MyFirstTest.

5. Click the Start Recording button.

6. Point your browser to this URL: http://examples.pushtotest.com/responder/htmlresponder

7. Click the file2.html link. You should find the link about halfway down the Web page.

8. The browser will display a new Web page that contains two forms. In the top-most form, enter your first and last name, and your phone number. Then enter 1081 as the account number and 75.36 in the amount field. Click the Transfer Funds button.

9. Your browser will show a new Web page that echoes the HTML form information you submitted on the previous page.

10. Return to TestMaker and click the Stop Recording button.

11. TestMaker will display a standard file selector dialog. Navigate to the directory you wish to store the new agent, enter a file name for the agent, and click the Save button.

12. The new test agent script will appear in the Editor panel in TestMaker.

Congratulations. You recorded your first test agent script.

To play-back the new agent script, click the Run icon in the Execution panel. Alternatively you may choose the Run command in the Agent drop-down menu.

The agent will replay the steps you took while using the browser to drive the Web-enabled application on the examples.pushtotest.com domain. You will see the results of the play-back in the Output panel.

Advanced Options

The Recorder includes several advanced options to make the recorded scripts act in real-world manner. The advanced options include the ability to load image references embedded in an retrieved Web page, follow 302-redirect responses from the Web host, and much more. The Recorder's advanced options are viewed by clicking the Advanced Options button from the Recorder window. Click the Advanced Options button to expand the window to see the following interface:

The Recorder offers the following advanced options:

Log To

As TestMaker runs a test agent script it outputs logged informational and results information. This controls the destination of the logged data.

Output Panel - Displays logged data into the output panel of the TestMaker graphic interface (the lower right corner.)
File - Writes logged data to the file defined in "Log path/name".
File with response - Writes the logged data in XML format to the file define in "Log path/name". The XML file format is defined in testmaker_home/docs/results.xsd. The file format saves the request header and body and the response header and body.
Database - Writes the logged data to a relational database using your provided JDBC driver. This requires you to edit testmaker_home/lib/agentbase.py to configure the JDBC driver.

Log Path/Name

Destination file name and path to write the logged data TestMaker generates while running a test agent. The Browse button displays a standard file selector to graphically navigate the file system and directories.

Log Level

Controls the amount of information logged by the test agent, including:

0 - No logging
1 - Major operations. This logs information messages only. For example, the test agent logs informational notices on when the test has finished setting-up for a test.
2 - Details. This instructs the test agent to display the contents of an HTTP request and response.
3 - Debug. The test agent displays all information and detailed information, plus additional details that may be useful while debugging test agent scripts.

Sleep Time

This setting controls the minimum and maximum amount of time, in seconds, to pause between Web page requests. You set the minimum and maximum values and the test agent chooses an amount of time between the two values to pause.

Success Responses

This is a Regular Express (regex) defining the HTTP response codes that indicate the request was accepted by the Web host. By default this regex considers HTTP response codes of 200-299, and 300-304, and others to be successful responses. HTTP response codes of 500, for example, are not part of the regex and so the test agent throws an exception.

Follow HTTP 302 Redirects

Web hosts may return a 302 response code to direct a Web browser to load another page. Checking this field tells the test agent at runtime to follow 302 reponse codes and load them automatically.

Load <IMG> Tag References

When checked the recorded test agent will automatically parse a retrieved Web page for <img> image tag references and make HTTP requests to load the images. If File with response is checked then the test agent will store the URL, time it takes to load the image, and image size in bytes for each image in the Web page.

TestMaker uses the TagSoup library to parse HTML pages for <img> tag references. Instead of parsing well-formed or valid XML, TagSoup parses HTML as it is found in the wild: nasty and brutish world of the Web. It makes a best effort to parse a Web page. The test agent will show errors in the logged data when they occur.

Load <IMG> Tag References

This determines the number of seconds the test agent will pause before loading the next <img> tag reference.

Emulate Browser Image Caching

Web browsers usually store downloaded images in a local cache. When checked this instructs the test agent to avoid loading the same <img> tag references more than once.

When Is It Appropriate To Use TestGen4Web and the Recorder?

You may be wondering when it is appropriate to use the TestGen4Web add-on to Firefox or the Recorder? Here is a summary of the advantages of each approach.

	TestGen4Web	Recorder
Record secure Web pages using HTTPS protocols	Yes	No
Transforms recorded test into Jython script	Yes	Yes
Transform script underlying framework	HTMLUnit	TOOL
Edit and reorganizate recorded steps in graphical interface	Yes	No
Emulate browser functions (browser image caching, testing of image tag validity, log levels)	No	Yes

Tutorial 4: Using TOOL Protocol Handlers In A Script

In the previous tutorial we used two techniques to jump start a Web test script: the tutorial first showed the TestGen4Web add-on to the FireFox Web browser and then showed the Recorder. Both of these create test use cases implemented as a Jython class - although there are several differences. The nature of XML, Service Oriented Architecture (SOA,) Web Services, Databases, and Applications means we need to be prepared for new and inventive protocols to interoperate with services and applications. In this tutorial we will show how to use a TestMaker Test Object Oriented Library (TOOL) protocol handler libraries to create test scripts that use many prootocols.

TestMaker comes with several example test agent scripts to show you immediately useful examples to show you many of the other facets of writing test scripts using the TOOL and agentbase libraries. For instance, TestMaker_home/example_agents/SOAP_Message_Example.py is a Jython script that uses the SOAP protocol handler in TOOL to interoperate with a Web Service. The following are excerpts from this script.

from com.pushtotest.tool.protocolhandler import ProtocolHandler, SOAPProtocol, SOAPBody, SOAPHeader




from com.pushtotest.tool.response import Response




   . . .









self.protocol = ProtocolHandler.getProtocol("soap")




self.body = SOAPBody()




self.protocol.setBody(self.body)




self.protocol.setUrl("http://examples.pushtotest.com/axis/services/MessageService")




   . . .









self.doc="<hello>Jack</hello>"




   . . .









self.body.setDocument( self.doc )




self.response = self.protocol.connect()




   . . .

The above excerpts create a new SOAP protocol handler, create and attach a new SOAP body, set the URL for the target service, set the message body (self.doc) document, and then connects to the service. The response variable holds the response object and has its own set of APIs to analyze and understand the result.

TOOL is an extensible protocol handler library. For instance, one TestMaker users needed a SIP protocol handler to test an Astrix telephony application. He spent less than a day to add a SIP protocol handler to TOOL. See the developer's guide for more information.

Tutorial 5: Using Scripts in TestScenarios

TestMaker's TestScenario facilitty turns any of these scripts into a functional test, load test, and service monitor. See the TestScenario Reference for details and the Tutorials page for examples. TestScenario documents identify the test scripts to operate during a test. For instance, the following are portions of a TestScenario that operate a test written in a Java method:

. . .










<resources>










    <module name="myjar" path="myclasses.jar"/>










</resources>










. . .










<run name="test1" testclass="com.pushtotest.example.mytest" method="runtest" langtype="java">










. . .

The <resources> identifies the JAR file containing my testclass. The <run> instantiates a mytest object and calls the runtest method. Here is what the mytest class looks like.

package com.pushtotest.example.mytest;

import org.junit.TestCase;

public class Console implements TestCase{

public void setUp()
{
. . .
}

public void runTest()
{
System.out.println("Running the test.");
. . .
}

public void tearDown()
{
. . .
}

}

The langtype attribute in <run> identifies the language type to be used. TestMaker supports these language types:

Language Name	langtype
Java	java
Jython (the Java implementation of the popular Python language)	jython
JRuby (the Java implementation of the Ruby language)	jruby
Groovy	groovy
Rhino (the open-source implementation of the JavaScript language)	rhino

A more complete list of scripting languages appears here.

The <resources> section of a TestScenario identifies the test script. The following is an example using JRuby:

<resources>




  <module name="rubyModule" path="./example_agents/JRubyExample/example.rb"/>




</resources>




. . .






<run module="rubyModule" name="calc" testclass="Calculator" method="average_of_3" langtype="jruby"/>




...

and for Jython use the following:

<resources>




  <module name="myJythonScript" path="./example_agents/HTTP_Example.py"/>




</resources>




. . .






<run module="

myJythonScript

" name="test1" testclass="TestClass" method="runTest" langtype="jython"/>




...

See the Example Agents for examples of running JRuby, Jython, and other script language test scripts.

JSR 223 and Class Instances

TestMaker uses the Java 6 and greater JSR 223 ScriptEngine to operate tests written in the supported languages. The ScriptEngine is not able to instantiate a class. It may call a method or invoke a method in a class already instantiated. To do that, it needs a method that returns the object. To accomodate the ScriptEngine test scripts must include an "entry point" method that returns an instance of the test class referenced in the <run> <setup> or <teardown> methods of a test. TestMaker follows a convention that the entry point method is of the form get<classname>(). The following shows an instance of this for a Jython script.

class DPLExample( agentbase.agentbase,  junit.framework.TestCase ):







    . . .

    def runTest( self, dpl_provided_argument_value ):







        ''' Run the test '''







        self.log( 1, "test: runTest" )







        self.params = [ [ '''testinput''', dpl_provided_argument_value ] ]







        self.get( '''http://examples.pushtotest.com/responder/htmlresponder''', self.params)








        . . .

def getDPLExample():

       '''







       Returns an instance of this object for the JSR 223 ScriptEngine in TestMaker







       '''

       return DPLExample()

Where To Go From Here

Add a Data Production Library to your script for dynamic data at runtime

Additional documentation, product downloads and updates are at www.PushToTest.com. While the PushToTest TestMaker software is distributed under an open-source license, the documentation remains (c) 2007 PushToTest. All rights reserved. PushToTest is a trademark of the PushToTest company.