Product Information
Product Pricing, Costs, Support
Installation and Running
TestMaker Capabilities
Creating Tests
Technology Questions
Company Information
Everything Else
PushToTest is a solutions provider to enable enterprises and institutions to monitor in real-time information services and provide immediate root-cause analysis and remediation for performance bottlenecks and downtime at development time, QA time, and runtime. PushToTest offers free open-source test software, free performance kits, and global services solutions that test, monitor and automate Web service systems for reliability, functionality, scalability and performance. The result is the right size datacenter, qualification of software patches and updates, and happy users.
PushToTest was founded in 2001 by Frank Cohen. PushToTest is a privately held small business. PushToTest offices are in Campbell, California, Washington DC, San Jose, Costa Rica, and London UK. PushToTest is a small business with 135,000 users, 22 engineers, and 5 managers. PushToTest was founded by Frank Cohen.
PushToTest
offers the TestMaker service governance and test automation platform.
TestMaker is a platform for software developers, QA groups, and IT
management to test, monitor, and govern information systems. At any
given time new software needs to be installed, existing software
modules and database software need to be patched, application software
and databases need to be tuned and optimized, and the root-causes of
crashes, downtime, and performance bottlenecks needs to be analyzed and
remediatedquickly.
Software developers use PushToTest Version 5 to turn their unit tests
into functional tests in a test automation platform that runs on their
development machine. PushToTest Version 5 includes Wizards and
Recorders to automatically build tests and supports a variety of
languages, including Java, .NET, Jython, Groovy, PHP, Ruby, and many
others. Plus PushToTest Version 5 supports SOA, Web Service, AJAX, and
REST services using HTTP, HTTPS, SOAP, XML-RPC, and the email protocols.
The PushToTest Version 5 runtime environment automatically turns these same functional tests into load tests, scalability and performance tests, regression tests, and service monitors for QA technicians, IT operations managers, and CIOs. PushToTest Version 5 runtime load tests and service monitors integrate into Service Registry/Repository products and database and application performance optimization and root-cause analysis tools.
PushToTest offers a variety of free and for-pay support options to assist you with TestMaker and TestNetwork. These are described on the Support Page. We pride ourselves on building exceptional relationships with customers and partners. PushToTest offers professional services to help solve difficult enterprise problems.
PushToTest distributes TestMaker under dual licenses:
Please contact us by telephone at (408) 871-0122 (USA, California Pacific time) or by email at sales@pushtotest.com.
General Motors, European Union, Lockheed Martin, US Army, Sun, IBM, BEA, Microsoft, CapeClear, Software AG
PushToTest TestMaker and TestNetwork provide full support for complex SOAP data types. See http://www-106.ibm.com/developerworks/webservices/library/ws-complex.html
PushToTest TestMaker provides direct support for making SOAP calls over HTTP, HTTPS and JMS.
PushToTest solutions are designed for Java, .NET, J2EE, Spring, Plain Old Java Object (POJO) and legacy systems that use open-standards protocols. PushToTest TestMaker is a 100% Java application and runs on any platform that supports Java.
PushToTest TestMaker provides direct support for SOAP RPC, Document Literal, SOAP RPC Literal and many other Web Service encoding formats.The Test Object Oriented Library (TOOL) in TestMaker is an extensible library of protocol handlers. Support for other data types is easily added using TOOL APIs.
PushToTest TestMaker is implemented as an extensible SWING-based application written 100% in Java. The Test Object Oriented Library (TOOL) in TestMaker is an extensible library of protocol handlers. Support for other data types is easily added using TOOL APIs. Full source code and support are offered by PushToTest.
The PushToTest Version 5 runtime environment automatically turns these same functional tests into load tests, scalability and performance tests, regression tests, and service monitors for QA technicians, IT operations managers, and CIOs. PushToTest Version 5 runtime load tests and service monitors integrate into Service Registry/Repository products and database and application performance optimization and root-cause analysis tools.
Click here to download TestMaker.
From Freshmeat.net:
PushToTest TestMaker is a platform for real-time monitoring and
governance of information systems. Software developers use TestMaker
turn their unit tests into functional tests that run on their
development machine. TestMaker includes Wizards and Recorders to
automatically build tests and supports a variety of languages to build
tests, including Java, .NET, Jython, Groovy, PHP, Ruby, and many
others. Plus TestMaker supports SOA, Web Service, AJAX, and REST
services using HTTP, HTTPS, SOAP, XML-RPC, and the email protocols. The
TestMaker test runtime environment automatically turns these same
functional tests into load tests, scalability and performance tests,
regression tests, and service monitors for QA technicians, IT
operations managers, and CIOs.
TestNetwork enables businesses to automate system tests to check for correctly operating functions, to handle increasing user loads, and continuously monitor systems to proof service levels. TestNetwork builds on TestMaker by delivering these features:
TestMaker comes with everything needed to build and run intelligent test agents. TestMaker requires a Java 1.6 or higher virtual machine.
Instructions on installation are found in the Installation Instructions page.
TestMaker is available for free download from the PushToTest Web site and through anonymous FTP download. See the downloads page on the PushToTest web site to download TestMaker using a browser.
For FTP download:
ftp.pushtotest.com
user name: anonymous
user password: enter-your-email-address
The file TestMaker.zip (capitalization is important) is always the latest release of TestMaker.
TestMaker comes with start-up scripts for Windows and Unix systems, including Linux, Mac OS X and Solaris. To start TestMaker on Windows, run the testmaker_home/TestMaker.bat script; for Unix, run the testmaker_home/TestMaker.sh script. These scripts are simple shell scripts that build a Java classpath containing the libraries TestMaker comes with, and executes Java telling it the TestMaker object to run. Additional instructions and a tutorial on using TestMaker are found in the Tutorial.
TestMaker normally requires very little configuration to install and run. The TestMaker distribution comes with an intstaller application. The installer requires Java 1.6 or higher runtime environment. Sun provides the Java runtime environment at java.net. You may install either the Java Runtime Environment (JRE) or the Java Software Development Kit (JSDK) depending on your intended use of TestMaker. To install TestMaker run or double-click one of these installers:
These installers create a directory with the TestMaker files. In this directory you will find the TestMaker.bat (for Windows) and TestMaker.sh (for Unix machines, including Linux, Mac OS X, and Solaris) launcher scripts. The script builds a Java classpath, executes Java, and tells Java to run the TestMaker application.
The launcher script and many other Java utilities depend on a system environment variable named JAVA_HOME to points to the Java installation. For UNIX machines set the JAVA_HOME variable in the.cshrc, .bashrc, or other shell configuration file. For Windows systems set the JAVA_HOME by following these instructions:
As far as we know there is no single product that will conduct stress
tests in all of these methods. Even the expensive commercial products
would have a hard time doing it all. PushToTest
TestMaker
is a framework and utility for building such a test. TestMaker's
integrated Test
Object Oriented Library (TOOL) is an extensible API that
provides protocol handlers with which to write intelligent test agent
scripts. So, TOOL can certainly be extended to provide support for ftp,
telnet, and NETBIOS. Under TestMaker's open-source license you can do
the work yourself, or you can contract to PushToTest Global
Services to extend TestMaker for you.
Yes, TestMaker is appropriate for testing Java Client/Server applications. For example, Sun Microsystems engaged PushToTest to use TestMaker to test its bug tracking system. Sun deploys a Java application to provide a Swing-based user interface. The application makes SOAP calls to a middle-tier that then makes calls to a Siebel host, Oracle database, and LDAP server. TestMaker drives the bug tracking system by making calls directly to the command objects in the Java application. By varying the number of concurrent requests and implementing the behavior of archetypal users, the TestMaker system can report on the functionality, scalability, and performance of the bug tracking system.
TestMaker function tests, load tests, and service monitors may be run from the command-shell. See the TestMaker_home/TestMaker.sh and TestMaker_home/TestMaker.bat scripts for details.
TestMaker 4 offers no support for WS-Security directly at this time. However, because TestMaker supports Java and the Jython scripting language you can easily import and use an Java library that implements the security library and profile you need.
TestMaker comes with the Eviware soapUI utility to analyze a WSDL file and visually create a TestSuite. Use the Tools menu, Start soapUI command to start soapUI from within TestMaker.
TestMaker operates unit tests as load tests. TestMaker provides a scalability index chart showing transactions per second (TPS), a distribution chart showing response times during the test, and a monitor of CPU, network, and memory utilization as the test operates. TestMaker logs all transactional data to a set of XML-formatted documents for import into other results anlalysis products.
TestMaker is architected to operate tests under multiple IP addresses using the distributed TestNetwork environment.
TestMaker
does not offer any specific P2P protocol handlers (such as JXTA.)
However, TestMaker's Test Object Oriented Library (TOOL) was build
around an extensible architecture. That makes it easy to extend TOOL to
provide a JXTA protocol handler. PushToTest can provide you with this
or support your own effort.
TestMaker is distributed under a dual license. The PushToTest TestMaker source code is licensed under the GPL version 2 license. PushToTest users may download pre-build, tested, and ready-to-run TestMaker software that is licensed under a commercial license where the first 200 concurrent virtual users (CVUs) and 10 service monitors (CSMs) are free. PushToTest sells a license for additional CVUs and CSMs.
TestMaker's home is at the PushToTest Web site. The download page contains the most up-to-date version of TestMaker. TestMaker is usually updated few months to incorporate bug fixes, documentation updates and new features. Between releases the modules, object libraries and working-in-progress code is available at the PushToTest CVS server.
TestMaker packages bug fixes, documentation updates and new features into binary distributions that are ready-to-run. Between releases the modules, object libraries and working-in-progress code is available at the PushToTest CVS server. CVS is a version control system that enables developers to work on TestMaker as a team.
The PushToTest CVS server allows anonymous download of the TestMaker code. Updates and patches should be sent to Frank Cohen, principal maintainer for TestMaker, for consideration. With enough support Frank may choose to give you CVS commit privileges. Current there are 4 committers.
From Unix systems:
$
export
CVSROOT=:pserver:anon@cvs.pushtotest.com:/var/cvsroot
From Windows we recommend using a CVS client:
C:>set CVSROOT=:pserver:anon@cvs.pushtotest.com:/var/cvsroot
In either case, the checkouts are:
$cvs login(Logging in to anonymous@cvs.pushtotest.com)
CVS password: <ENTER>$cvs co tm5cvs server: Updating tm5
...
The tm5 module includes:
TestMaker - The stand-alone Java application code for TestMaker
tool - the Test Object Oriented Library (TOOL) provides protocol handler objects
j-0-1.21-tmbranch - The TestMaker branch of the J text editor
maxq-tmbranch - The TestMaker brand of the MaxQ Proxy Recorder
TestNetwork - The distributed test environment of TestNodes
To receive email announcements when new TestMaker code is changed in cvs subscribe to the cvscommits email list. Discussion among the TestMaker developers is on dev@lists.pushtotest.com. This is a good place to post questions and suggest new features.
Modern systems today use technology that enables software applications to communicate over private intranets and the Internet. The protocols (HTTP, HTTPS, SOAP, XML-RPC) are accepted standards and supported by the entire computer industry. We call these systems Web Services. Enterprise adoption of Web services has sparked an explosion of new software projects, all of which have interoperability and connectivity requirements. A Web browser may retrieve a Web page, a pager may receive an email, a portal may collect data from many services to present a single Web page, and many systems located in various places may be integrated into a productive system.
Server software needs to exchange data with other server software. Past attempts for server data exchange include Corba, DCOM, RMI, EDI and and host of other protocols. SOAP is an emerging industry standard that has a better chance than previous attempts. SOAP is to servers what HTTP and HTML are to the Web, a lightweight means to communicate.
Here is the SOAP definition from the draft W3C specification: SOAP is a lightweight protocol for exchange of information in a decentralized, distributed environment. It is an XML based protocol that consists of three parts: an envelope that defines a framework for describing what is in a message and how to process it, a set of encoding rules for expressing instances of application-defined datatypes, and a convention for representing remote procedure calls and responses.
The SOAP specification is at: http://www.w3.org/TR/SOAP/
SOAP uses serializer and deserializer objects to convert data in XML-encoded request and response documents into Java objects that your objects may use. TestMaker uses the Apache SOAP library to make SOAP calls. Detailed instructions are found in the IBM developerWorks article titled Complex Data Types and Axis, Apache Soap and TestMaker by Frank Cohen. Additionally, we recomment checking the Apache SOAP Web site for details on adding custom SOAP data types to TestMaker. Also, the docs.pushtotest.com site has documentation on implementing custom serializers.
Click here for the PushToTest privacy policy.
docs.pushtotest.com is your single source for additional information, articles, books and news on TestMaker and Web Services.
Frank Cohen, principal maintainer for TestMaker, is author of FastSOA (Morgan Kaufmann 2006,) Java Testing and Design: From Unit Tests to Automated Web Tests (Prentice Hall 2004.) The book covers many of the experiences Frank has had in designing and testing Web Services in many varied environments using TestMaker.
Many articles on PushToTest technology and products have appeared on Web sites, in magazines and journals. These links take you to articles that teach more about Web Services and TestMaker:
Performance testing SOAP-based applications
Myths and Misunderstandings surrounding SOAP
Using Web services for e-Commerce single sign-in
IBM's article on PushToTest and Load.
IBM jStart Web Services Program
Additionally, IBM, Sun, BEA and CapeClear have partnered with PushToTest because we each have a common goal to establish the Web Services standard. Each company offers tutorials, documentation, software and utilities at little or no cost.
TestMaker makes an ideal environment for students learning new technologies, including Java, software test methodologies and Web Services. TestMaker includes example test agents which may be used to form a course featuring practical examples of new technologies in action. The docs.pushtotest.com site offers many free downloads which may help you construct a course. Frank Cohen is available to speak at conferences, classes and lectures.
TestMaker
comes with a a User Guide, Tutorial, Sample Agents and Javadoc
documentation of the test objects. You will find all of these in the
docs directory that is installed with TestMaker. The docs.pushtotest.com
Web site is home to these same documents and more.
SOAP-based Web Services use XML-encoded request and response data. TestMaker comes with JDOM, a Java framework for working with XML data. The JDOM objects and methods are accessed through the TestMaker scripting language. Look at the Book_price.a test agent for examples of how to use JDOM methods to parse and validate XML-encoded request and response data.
Chapter 6 of Java Testing and Design: From Unit Tests to Automated Web Tests covers SOAP and Web Services in detail, with lots of example TestMaker scripts.
PushToTest
TestMaker includes the SpikeSource TestGen4Web recorder as a FireFox
add-on that watches you operate a Web browser-based application and
creates a unit test automatically. These unit tests support dynamic Web
pages, JavaScript, and Ajax.
PushToTest TestMaker includes the SpikeSource TestGen4Web recorder as a FireFox add-on that watches you operate a Web browser-based application and creates a unit test automatically. Details are in the docs/tutorial.html document
PushToTest TestMaker includes the SpikeSource TestGen4Web recorder as a FireFox add-on that watches you operate a Web browser-based application and creates a unit test automatically. TestGen4Web creates tests that support SSL.
TestMaker and Load use the JSSE library from Sun to implement HTTPS connections over SSL protocols to the server. JSSE has certificate handlers for Verisign and Thawte certificates by default. The keytool utility that comes with JSSE is used to install additional certificates. Details on using keytool are on the TestMaker Security wiki page. Additional information on keytool is available on the java.sun.com site.
Follow
these steps to use the Network Monitor to watch network traffic:
PushToTest TestMaker supports MIME encoding for email tests. Email messages use MIME encoding to attach files, icons, and binary data to a message body. The Mail protocol handler supports MIME encoding too. The agent script below illustrates how to send an email message with a file attachment: Click here for details on the Mail protocol handler.
TestMaker comes with the Jython scripting language. Jython is a 100% Java implementation of the popular Python language. Jython's unique design gives you access to all of Python's objects and any Java object that is on the TestMaker classpath. This means you may use your own custom Java objects from within a TestMaker test agent. To use your Java objects in TestMaker 4.x, follow these steps:
1)
Create the following Java class file.
package com.pushtotest.myapplication;public class myApp
{
public String getName()
{
return "Frank";
}
}
2) Compile the class into myApp.class and put it into a myApp.jar file.
3)
With
TestMaker 4.0, modify testmaker_home/TestMaker.bat (for Windows) or
testmaker_home/TestMaker.sh (for Unix/Linux) to add myApp.jar to the
classpath.
4)
Create a new TestMaker script as follows:
from com.pushtotest.myapplication import myApp
a = myApp()
print "myApp says: Thank you", a.getName(), \
", I will buy a support agreement."
The
output window will display:
myApp says: Thank you, Frank, I will buy a support agreement.Additional information on using Java objects from within a test agent script is found at http://www.jython.org/docs/usejava.html.
Proxy servers provide an IT team with a way to manage network traffic.A proxy server is a router that forwards network traffic to a host. If your network uses a proxy server chances are TestMaker will give you errors when you try to run a test agent script or use the New Agent Wizard to record a script. You may see errors like these:
To use a proxy server from a TestMaker test agent script use setProxyHost, setProxyPort, setProxyUser, setProxyPass methods of the protocol handler. Below is an example script showing an HTTPProtocol handler using a proxy server:
protocol = ProtocolHandler.getProtocol("https")
protocol.setProxyHost( "localhost" )
protocol.setProxyPort( 9001 )
protocol.setProxyUser( "frank" ) #optional
protocol.setProxyPass( "password" ) #optionalMany times the proxy server does not require a user or password. In this case, setProxyUser and setProxyPass are optional.
The TestMaker New Agent Wizard supports proxy servers too. When using the New Agent Wizard to record a test agent script, set the proxy information for the recorder by clicking the Help drop-down menu and choosing the Preferences command. Click on the Recorder tab. Enter the values for your proxy server.
Software developers, QA technicians, and IT managers often need to see the HTTP, SOAP, and XML-RPC messages being used to invoke Web applications and web services along with the results of those messages. The network monitor provides a way to monitor the messages being used.To start the network monitor use the Tools -> Network Monitor command in TestMaker. The network monitor window will appear and the administrative tab will appear by default. Click here for directions on using the network monitor.
To launch PushToTest TestMaker on a Macintosh computer running Mac OS X version 10.4 or greater follow these instructions:
1) Make certain your computer has Java version 1.6 or greater. To do so open a Terminal shell window from the Applications directory. Type java -version and press enter. The Java version number will appear. If you have an older version of Java use the Software Update feature found in the System Preferences icon on the Dock to download the new Java runtime environment. Details on the Java virtual machine provided by Apple Computer are found at http://developer.apple.com/java/
2) From a Terminal shell window change the current directory to the location of the TestMaker directory. Type ./TestMaker.sh and press the enter key.
We are unaware of Java runtime environments for Macintosh operating systems prior to Mac OS version 10.4. If you figure out how to run on Mac OS 9.2, for example, then please let us know by sending email to info@pushtotest.com.
J is an open-source text editor. TestMaker embeds J as a test agent script editor. J offers many features and functions that are described in the J User's Guide.
Typically applets are used for presentation and communication with hosts in some fashion that is not easily possible using HTML pages in a browser. TestMaker does not provide special functions to test an applets ability to present data. However, TestMaker does fully support all of the major Web protocols (HTTP, SOAP, XML-RPC, SMTP, etc.) an applet would use to request something from a host. Additionally, TestMaker's Recorder feature automatically detects an applet's HTTP communication with a host and writes a TestMaker intelligent test agent script that emulates the request from the applet to a host.
PushToTest recommends the following books:
If you know of additional books, please let us know by sending email to info@pushtotest.com and we will list them here.
PushToTest
tests may be run from the command-line. This enables you to schedule
scripts using external tools and "cron" jobs.
We do not currently have a scheduler built into TestMaker and have one
on the product
road map.
PushToTest comes with the PushToTest As A Service (PAAS)
interface to
SOA registry/repository products. For instance, SoftwareAG X-Registry
may start a PushToTest test scenario as the result of a service policy
being triggered and then receive the results and transactional
statistics in a Governance Statistics Result Set (GSRS) XML response
that is stored as metadata in the repository.
PushToTest
test scenario output is in a set of charts (Scalability
index, transaction distribution, and resource utilization for CPU,
Network, and memory use during the test) that are output in PNG format,
the statistical data is output in XML format, and the transactional
data is output in XML format.
The logging capability outputs to HTML format as well and is
customizable by you or PushToTest's global services consultants.
PushToTest TestMaker test scripts implement a JUnit TestCase and then are orchestrated as functional tests, load tests, and service monitors using a TestScenario XML document. TestMaker supports JUnit TestCases implemented in Java, Jython, JRuby, Rhino (JavaScript,) PHP, and several other dynamic scripting languages. All TestMaker TestScenario scripts are human readable using a browser, the notepad, or the text editor.
PushToTest TestMaker tests are composed of individual files that may be
checked into a configuration management system as code.
PushToTest TestMaker provides the Test Object Oriented Library (TOOL)
that delivers SOAP and other protocol handlers. TOOL provides APIs to
check SOAP messages for mandatory custom headers.
PushToTest TestMaker provides the Test Object Oriented Library (TOOL) that delivers HTTP, HTTPS, XML-RPC, SOAP, IMAP, SMTP, POP, SIP and other protocol handlers. A JMS driver for TOOL is planned.
PushToTest TestMaker is build on the Java platform and supports HTTP Bound 1-way and 2-way digital certificates through the Jave Security (JSSE) library, including x509 XML encrypting support and 2-way SSL authentication.
PushToTest
TestMaker integrates the Eviware
soapUI utility to visually create tests. You provide a WSDL
definition, soapUI visually displays a skeleton of the SOAP request
document, you fill it in with data, and the utility shows you the SOAP
response from the service. This requires no programming experience.
PushToTest
supports ANT and MAVEN. An ANT build script TestMaker operates a
TestMaker test from a custom ANT
task. PushToTest TestMaker supports MAVEN through
a command-line call.
PushToTest TestMaker test scripts implement a JUnit TestCase and then are orchestrated as functional tests, load tests, and service monitors using a TestScenario XML document. TestMaker supports JUnit TestCases implemented in Java, Jython, JRuby, Rhino (JavaScript,) PHP, and several other dynamic scripting languages.
PushToTest
has plans to provide a Web interface to allow people to review,
orchestrate, and operate tests. This will appear in an upcoming version
of TestMaker. To enable this, PushToTest TestMaker is architected to
support deployment as a service. For
example, we now offer integration with Software AG's X-Registry.
PushToTest runs as a Web service, receiving a PushToTest
As A Service
(PAAS) request and providing a Governance Statistics Result Set
(GSRS)
response. This will be the basis for offering a Web interface to
operate tests.
PushToTest TestMaker provides a protocol handler library called the Test Object Oriented Library (TOOL.) TOOL provides a SOAP protocol handler, including APIs to set custom headers and methods to validate response headers. PushToTest TestMaker provides multiple XML parsers, including an XPath and an XQuery engine, to validate results. (And also to compare XML parser performance.) PushToTest TestMaker provides Data Production Libraries (DPL) to work with data stored in CSV, RDBMS, and custom DPLs implemented by any developer to validate response data.
There are many ways to get your question answered. Send us an email message at info@pushtotest.com, call us at (408) 871-0122 (California USA time,) or use the Ask A Question form on the PushToTest Web site.
|
