See the readme.html document for the changes in this version of PushToTest(tm) TestMaker.
| Note: This documentation accompanies the "beta testing" distribution of PushToTest Test Version 5. During "beta testing" all of the features are implemented, bugs and issues still exist, and the documentation is incomplete. |
The enemy is downtime, and PushToTest TestMaker version 5 is the solution. PushToTest is a platform for software developers, QA groups, and IT management to test, monitor, and govern information systems. At any given time new software needs to be installed, existing software modules and database software need to be patched, application software and databases need to be tuned and optimized, and the root-cuases of crashes, downtime, and performance bottlenecks needs to be analyzed and remediatedquickly.
Software developers use PushToTest Version 5 to turn their unit tests into functional tests in a test automation platform that runs on their development machine. PushToTest Version 5 includes Wizards and Recorders to automatically build tests and supports a variety of languages, including Java, .NET, Jython, Groovy, PHP, Ruby, and many others. Plus PushToTest Version 5 supports SOA, Web Service, AJAX, and REST services using HTTP, HTTPS, SOAP, XML-RPC, and the email protocols.The PushToTest Version 5 runtime environment automatically turns these same functional tests into load tests, scalability and performance tests, regression tests, and service monitors for QA technicians, IT operations managers, and CIOs. PushToTest Version 5 runtime load tests and service monitors integrate into Service Registry/Repository products and database and application performance optimization and root-cause analysis tools.
To jump right-in, follow one of the QuickStart Tutorials, or view a video tutorial and presentation on PushToTest Testmaker.
|
Introduction For TestMaker 4 Users | |
Building Load Tests, Functional Tests, Service Monitors | |
Advanced Test Scenarios | |
Governance Using PushToTest As A Service |
PushToTest TestMaker is an open-source end-to-end service governance and test automation platform. PushToTest provides many options to build test:
This user guide intends to teach you these facets of TestMaker:
This section describes how to install PushToTest TestMaker, and how to get started quickly, and how PushToTest TestMaker enables black-box and grey-box testing.
The
PushToTest
Company distributes pre-built and ready-to-run binary of TestMaker
under a commercial license found in license.txt. The commercial license
is free and authorizes you to run up to 200 concurrent virtual users
and 10 concurrently running service monitors. Additional virtual users
and monitors are available for purchase from PushToTest. PushToTest
TestMaker source code is free and distributed under a GPL v2 license.
The PushToTest TestMaker installer requires the Java 1.6 or higher
runtime environment. Sun Microsystems makes the Java runtime
environment available at http://www.java.net.
Please run the following installers. Normally you accomplish this by
running or double-clicking one of the following files:
The installers will ask you for the installation directory. The TestMaker documentation will refer to testmaker_home as the directory into which the TestMaker files are installed.
The testmaker_home directory contains these files:
PushToTest TestMaker comes with everything needed to build and run tests. PushToTest TestMaker requires a Java 1.6 or higher compliant virtual machine already installed. The TestNode software may run under Java 1.4 and 1.5 for compatibility with tests that are compiled under those older Java versions.
From time-to-time we update PushToTest TestMaker to solve bugs and introduce new features. The PushToTest Web site home page shows the most recent version number. Downloads are available there too.
Answers to common installation problems are found on the Frequently Asked Questions page.
PushToTest TestMaker is appropriate for testing Web applications, Service Oriented Architecture (SOA,) Web Services, Email systems, Java applications, and more. The following is a table showing our suggestions on how to get started with PushToTest TestMaker.
 |
|
|
|
|
|
|
| Web application test | Web Service and SOA test | Email system test | Java application test | |||
|
Web applications provide users with functions accessible through a Web browser. PushToTest TestMaker features record and playback technology for Web applications. PushToTest TestMaker watches your use of a browser and writes a test agent script. To the Web application the test agent appears to be a Web browser user. The test agent script is a functional test (also referred to as a unit test.) PushToTest TestMaker includes HTTP and HTTPs protocol handlers. These handlers provide support for cookies, redirect commands, authentication and HTML parsing. To begin using PushToTest TestMaker in a Web application test environment follow the tutorial. To learn more about PushToTest TestMaker in a Web application test environment we recommend any of the following: look at the Web sample test agents and read the tutorials. |
Web Services and Service Oriented Architecture (SOA) provide server-to-server and application-to-application XML communication. PushToTest TestMaker features the soapUI utility to reads a Web Services Description Language (WSDL) definition of a service and builds a unit and function test. To the Web Service the test appears to be a server or application making a request and expecting a well formed response. PushToTest TestMaker includes SOAP and XML-RPC protocol handlers. These handlers support SOAP document-style (message) encoding, SOAP RPC-encoding, HTTP authentication, SOAP over HTTPS communication, and cookies. PushToTest TestMaker provides XML parsing and query capabilities through included JDOM, XPATH, Xerces, and JAXB libraries. To begin using PushToTest TestMaker in a Web Service test environment, use the Tools -> Start soapUI command to create a unit and functional test. Then run the test in a TestScenario as a functional and load test and service monitor. To learn more about PushToTest TestMaker in a Web application test environment: look at the Web Service sample test agents. Then view one of the tutorials.
|
Email systems provide access to email messages stores through email client software and Web browser based email access portals. PushToTest TestMaker includes SMTP, POP3 and IMAP protocol handlers to communicate with an email store. PushToTest TestMaker features record and playback technology to use Web browser-based email access portals. PushToTest TestMaker email protocol handlers support MIME attachments, UTF encoding support, and authentication. To begin using PushToTest TestMaker in an email test environment, read the email sample test agent. Then read and understand XSTest to turn the same test into a scalability/load test. Then read the Service Monitor System (SMS) documentation to turn the test into a Quality of Service monitor.
|
Many companies deploy Java applications to provide an intelligent graphical interface to a service. For example, Sun Microsystem's provides Java desktop application to its engineers to enable bug tracking and change management in its products. The application makes SOAP requests to a service to manage bugs. PushToTest TestMaker - being a 100% Java application itself - enables test agent scripts make direct calls to a Java application's methods. The application responds in the same way it would to a real user. PushToTest TestMaker runs a test of a Java application using the same functional test (unit test) framework as Web application, Web Service, and email tests. To begin using PushToTest TestMaker to test a Java application, read the Java application sample test agent. Then read and understand the TestScenario tutorial to understand how to turn a unit test into a regression, function, and load test, and a service monitor.
|
Testing And Remediation
The test environments described in the previous section show how PushToTest TestMaker provides a framework and utility for building tests that check services for regression, function, scalability, performance, and reliability. PushToTest TestMaker is a unique solution to test and remediate issues that keep a system from achieving its performance and functional goals. For instance, PushToTest TestMaker generates load on a system by executing a Test Sceanrio and it correlates the running tests with database and application server monitoring and analysis tools to identify costly transactions, bottlenecks to good performance, and optimizations that prevent downtime.
In a black-box test environment, a test environment generates load on the service by making requests to the service and observing the service's response. Black-box scalability testing observes the service's ability and speed to respond to requests as the number of concurrent requests increases.These observations facilitate capacity planning and identify the service's ability to "go live" in a production environment. The PushToTest TestMaker test environment conducts black-box tests and goes further.
PushToTest TestMaker is also appropriate for grey-box tests. In a grey-box test the service under test may be queried for internal information that will help to uncover functional and scalability problems. The service under test provides state information from the internal operation of the service, including performance information from the application and database tiers of the service environment. For example, the service responds to special debugging service requests with CPU utilization, I/O utilization, memory allocation, and transaction states. This extra information allows the Test Environment to provide actionable knowledge in three categories:
This section described how to install PushToTest TestMaker, how to get started with PushToTest TestMaker quickly, and how PushToTest TestMaker enables testing and remediation.
This section provides a high-level view of PushToTest TestMaker. The section after this one describes PushToTest TestMaker in-depth.
Software
developers, QA technicians, and IT managers have many architectural
patterns to build software applications. PushToTest test provides
testing and remediation against these popular architectural patterns
and many more.
 |
The Domain Model uses an application server, servlet engine, EJB/Plan Old Java Object, Model View Controller, Workflow, and Object-Relational Mapping technology to coordinate data stored in a relational database. The user interface is a Web browser using HTML/HTTP protocols. PushToTest drives Domain Model applications using HTTP protocols. | |
 |
The Enterprise Service Bus Model (ESB) uses a centralized service bus to transform and deliver messages between a set of installed and reusable services. Each service uses the most appropriate protocol (for instance, soap, pop/imap, http, ajax) for its application. PushToTest drives ESB applications using multiple native protocols and accessing services through standard bus adaptors. | |
 |
The Web 2.0 Model uses enhanced Web browser functions to provide an enhanced user experience and new functions. For instance, a Web browser provides live maps using Ajax protocols between a browser, servlet interface, XML parser, and POJO. PushToTest drives Web 2.0 applications using multiple native protocols, including Ajax, RSS, and HTTP protocols. | |
 |
The Virtualization model uses shared virtual containers to operate multiple services. For instance, in one virtual container a applications operates an instance of the Domain Model and in a second container operates an ESB Model. PushToTest provides a distributed set of TestNodes to drive applications and services deployed in a virtualized model. |
The PushToTest environment provides scalability, performance, and service monitor tests for the Domain Model, ESB Model, Web 2.0 Model, and Virtualization Model.
Software developers introduce extensions to the Domain Model, ESB Model, and Web 2.0 Model everyday, including new protocols, new user interfaces, and new encoding styles to message between consumers and services. The PushToTest environment provides an open-source platform and runtime to build tests for these models. The environment is fully extensible to support new protocols, interfaces, encoding styles, and libraries.
The
PushToTest
runtime uses a TestScenario XML document to define a scalability,
performance, and service monitor test. The Governance platform stores
the TestScenario XML document. The TestScenario is the key to turning
unit tests into functional, regression, and load tests, and service
monitors.
The
tests use the
native protocols of the services under test. PushToTest runtime
provides an extensible protocol handler library (TOOL) that provides
protocol handlers for HTTP, HTTPS, SOAP, XML-RPC, and the email
protocols.
TODO: Add illustration of TestNode environment running in QA lab, on distributed networks, and as service monitors.
PushToTest is a distributed test environment. A single modern computer, for instance equipped with an Intel 3 Ghz processor and Linux, operates 300 or more concurrent virtual users depending on the complexity of the test. PushToTest TestMaker operates test agents remotely on a grid of test services (called TestNodes.) The distributed test environment enables you to run up to hundreds-of-thousands of virtual test agents concurrently by installing multiple TestNodes in a rack of servers in a QA lab. Equally powerful is operating a test on a set of TestNodes distributed across several parts of your network to determine which network provider delivers the best service. PushToTest TestMaker is the graphic console to operate tests across a network of TestNodes.
The
PushToTest runtime deploys monitors to track CPU, Network, and Memory
utilization in the servers operating each service.
The combination of these features is very powerful. The PushToTest runtime offers a SOAP-based Web Service request using the PushToTest As A Service (PAAS) protocol. PushToTest executes the test and returns a Governance Statistics Results Set (GSRS) response document. Use PAAS and GSRS to integrate PushToTest test and monitor data with analysis tools. For instance, PushToTest integrates with the WebMethods X-Registry to provide service governance through test automation and integrates with DBA InfoPower to provide insight and remediation of database bottlenecks.
PushToTest TestMaker delivers a rich environment for developing tests. PushToTest TestMaker comes with:
Many first time PushToTest TestMaker users find that running the included sample test agents is a quick way to learn the PushToTest TestMaker system. Additionally, PushToTest provides recorded screen-cast tutorials. The QuickStart guide, documentation, and tutorials will help you learn PushToTest TestMaker.
PushToTest products are designed for software developers, QA technicians and IT managers. Software developers need a framework to conduct system-level unit tests to check functionality. PushToTest TestMaker provides a functionality test framework by enabling construction of unit tests using a variety of languages, including Java, Jython, Groovy, Ruby, Javascript and many others.PushToTest TestMaker agent scripts developed by software developers for unit and functionality tests are then reused by QA technicians to run scalability and performance tests. IT managers run the same agent scripts in the PushToTest TestMaker environment as quality-of-service monitors. This unique approach from PushToTest provides a new way to leverage any enterprises' software development, QA and IT efforts resulting in new huge cost-savings.
PushToTest TestMaker is written in Java and runs everywhere Java runs, including Windows, Linux, Solaris, Macintosh and more.
The
PushToTest Company distributes this pre-built and ready-to-run binary
of TestMaker under a commercial license
found in license.txt. The commercial license is free and authorizes you
to run up to 200 concurrent virtual users
and 10 concurrently running service monitors. Additional virtual users
and monitors are available for purchase from PushToTest. PushToTest
TestMaker source code is free and distributed under a GPL v2 license.
The PushToTest Company provides a meeting place for the hundreds of
developers that contribute to the open-source project and the 130,000
users that share their knowledge and answer questions about PushToTest
TestMaker. PushToTest provides professional support services, technical
support, and Global Services.
A list of frequently asked questions (FAQ)
Known problems and incomplete features in this release of PushToTest TestMaker
Read the book: Java Testing and Design: From Unit Tests to Automated Web Tests
Read the documentation on the docs.pushtotest.com site
Building test agents to check email services for scalability and functionality
Learn about the changes and new features since the previous PushToTest TestMaker release.
Additionally, check the testmaker_home/docs/lessons directory for tutorials and articles on using PushToTest TestMaker.
|
