web2test - The new generation of automated functional tests for complex web applications

 

Modern Internet technologies found their way in almost all areas of the economic life. Together with the increasing use and the continuous enhancement existing business processes become “Internet enabled” and new business areas evolve. That is because a lot of those solutions offer functionalities which cannot be realized with desktop applications. However when developing and refining web applications one has to consider “ROI” and “time to market” on one hand and meet one’s own and especially the customers quality standards on the other hand. The ideal balance between those requirements is the key to success. To avoid leaving this to chance there are efficient testing procedures and corresponding efficient test tools available. Because especially for quality assurance testing the respective application is indispensable – and this means in general an additional investment of time and money. The software and system house itCampus has now developed and brought to market the test software web2test which fulfills all requirements which come along with the testing of modern web applications and at the same time reduces the effort for creation and maintenance of test scripts. The new test software can be downloaded under web2test.

 

Manual vs. automated testing

Before actually starting to test an important question has to be answered: “Do I want to test manually or automated?” Due to the increasing complexity of web applications manual testing is – if at all – only possible with a huge effort of time and manpower. Modern web applications are characterized by a large number of elements and possibilities of interaction which all have to be verified and included in the test. In addition to the high overall testing effort especially complex user interfaces with a lot of GUI elements when tested manually tend to push testers to their limits. There are for instance tasks where the same or similar data has to be entered manually in hundreds of input masks – a situation which requires full concentration but does not pose an intellectual challenge and thus quickly leads to a decrease of motivation and attention. That is why it is not surprising that in practice when employing manual tests the application is often only partially tested or in the worst case not tested at all.

A better alternative are automated tests. For automated testing a test tool is used to create different test scripts which can be replayed automatically. This reduces the required time and manpower and avoids that trivial work flows have to be repeated over and over. Furthermore tests can be executed several times without great additional effort.

In spite of these advantages which clearly indicate the need automate testing the majority of web applications are tested manually. This situation is due to the particularities of modern web applications: A lot of automated approaches fail because of dynamic content which has almost become the rule when using AJAX or they cause a maintenance effort which is way too high because layout changes or different browser/platform combinations require constant adjustment of already created test scripts.

There are different approaches which can be considered for automated testing depending on what hast to be tested and to what extent. For a simple verification of geometry or web content displayed in a browser in practice mostly tools are being used which compare screenshots. The problem is that these tests are static. If the layout of the website content page changes compared to its state at the time of recording, tests directly fail. The change of a single value in a row of a table is sufficient for it. In order to avoid this all preconditions have to be set and ensured exactly before the test – which is very complex and difficult in most cases.

Besides variations in layout one can also check the complete functionality of websites, portals and web based applications. There are different approaches for automated web testing without having to modify the system under test:

 

• HTTP Unit Test
• HTTP Request/Response
• Control and monitoring at GUI level

 

figure 1: chronological classification of manual and automated tests with and without web2test in the developing process

 

Functional testing via the user interface of the application = real user simulation

The new test software of itCampus - web2test - is testing the GUI of an application using Capture & Replay. Why? Because with this approach the functionality is checked right at the user interface level of the application in contrary to HTTP Unit Test and HTTP Request/Response which both start below the presentation layer. Thus even a change of state of the user interface can be taken into account and functional tests as well as layout checks are possible without any difficulty.

In addition web2test records all user interaction based on components. This avoids possible events which could be accidentally intercepted by the operating system or other applications – e.g. a mouse click on fixed position on the screen. For such events may not only constrain the test but they can also lead to unwanted reactions of the application.

web2tests combination of component based recording and the replay at the user interface level provides a realistic user simulation. And it represents the best way to ensure the full support of AJAX and dynamic HTML (DHTML). What usually makes the testing of AJAX and DHTML so difficult: both technologies allow to reload or even change parts of the user interface of a website dynamically. Since these changes are only implemented after the arrival of the HTML document in the browser they can only be tested after the completion of the page in the browser – and that is exactly what is being realized with the user simulation of web2test.

To be able to record user interactions component based the components have to be clearly identified. Most tools use either the inflexible structure information of the HTML documents or unique names or IDs for this purpose. However both approaches soon reach their limits when it comes to AJAX and DHTML: A dynamic adjustment of the user interfaces leads to a change of structure. In addition names and IDs are often also dynamically attributed when using these technologies. But web2test is different: Several different heuristics have been integrated in the itCampus test software which allow for an intelligent recognition of components even in the described cases. The algorithms take the hierarchical structure of the GUI as well as all available component information (e.g. title, description) and last but not least the component geometry in account. In this context the visibility of the elements which have to be displayed also matters. If for a functional test all elements of a website which have to be displayed are being examined it is difficult to verify whether user interactions are actually possible or allowed during certain states of the website. web2test however disposes of numerous algorithms which clearly analyze the information concerning the visibility of every element.

But the intelligent recognition of components also solves another problem besides the already mentioned: The late beginning of the automated testing process compared to manual testing (see figure 1). Despite all known advantages this objection was often the striking argument for a decision against automated testing. And so far it was completely justifiable since the layout of a web application had to be fully completed before starting with the creation of test scripts.

 

Intelligent recognition of components, tolerance towards layout changes

With web2test this objection can be refuted to a large extend. Thanks to the different heuristics for the recognition of components, test script which have been created using web2test are tolerant towards layout changes. The user has hence the possibility to start with the creation of test cases even before the layout is completed. This reduces project time while keeping the same quality level (see figure 1). With this feature web2test provides an added value which meets the increasing need to enter the market as early as possible. In other words: The date of completion can determine whether a project is successful or not: the more you save in time the more you also save in money.

Another aspect which indicates the need of automated functional testing using web2test: Test scripts are notably more robust if they are tolerant towards layout changes. If the GUI is changed at a later date, tests or at least parts of them can be reused. Hence the main objection to the Capture & Replay approach which states that tests have to be re-recorded after layout changes does not apply anymore (see figure 2). Thanks to web2test this is not necessary any longer.

figure 2: conventional CR based testing procedure (figure taken from C. Sensler, M. Kunz, P. Schnell Testautomatisierung mit modellgetriebener Testskript-Entwicklung; (http://www.sigs.de/publications/os/2006/03/kunz_sensler_OS_03_06.pdf last accessed 09/04/2008)

 

Functional tests without restrictions on browser and platform

Another problem of GUI based functional tests of web applications numerous differences when using miscellaneous platforms and browsers. There are for example known problems of the IE 6 when displaying transparent PNGs, under Windows XP IE7/8 can optionally render content using ClearType and under Windows Vista ClearType can be configured local in the browser as well as global for the operating system. But also with identical edge conditions on different windows systems content is to some extend interpreted in different ways. In certain cases there are even slight variances in geometry. If one extends the tests and uses different browsers besides the IE, e.g. FireFox or Seamonkey, there are also differences in rendering between the miscellaneous browsers in addition to the already mentioned differences which were related to the operating systems. These variances are increased if invalid HTML is being used – which does not pose a problem from the user’s point of view as long as content can be displayed without an error message.

Therefore every browser/platform combination has to be tested to ensure that there are no differences in rendering which could lead to a malfunction of the web application. For automated testing this means that theoretically for every possible combination a separate test suite has to be created. To avoid the additional effort web2test also offers - besides the support of a multitude of different browsers – a platform and browser independent approach. For an optimal consideration of rendering differences a operating and test environment which is as real as possible should be available for testing. That is why web2test uses the browsers already installed on the system without employing additional plug-ins or miscellaneous extensions unlike other test tools. Thus even dynamical browser settings like proxies, existing cookies or localization can be adjusted for every session. Advanced requirements e.g. the access to DOM and browser events can be realized via the Jython and Groovy scripting interface.

figure 3: User interface of web2test

 

Despite the extensive scripting options attention was paid that tests scripts can be created fast, modular and without programming knowledge thanks to an intuitive user interface (see figure 3).

 

About web2test

web2test has been developed as a co-operation project between itCampus and Quality First Software (QFS) and is based on QFS’s leading Java GUI test tool QF-Test.

Thanks to web2test the full range of capabilities and features of QF-Test concerning job sequencing and test case management are now also available for automated web testing.

 

Features of web2test at a glance

• GUI-related system tests
• tests are tolerant to GUI changes
• intuitive interface
• quick start due to „capture and replay“
• no programming knowledge necessary for creation and maintenance of test scripts (drag & drop/copy & paste)
• clear structuring of complex test cases
• flexible reporting
• integrated test debugger
• scripting Interface (Jython, Groovy)
• comprehensive handbook and tutorial
• excellent price- performance ratio
• free evaluation
• direct access to the DOM of the browser
• browser and platform independent test cases
• real user simulation (100% AJAX support)
• uses browsers installed on the test system - no additional browser plugins necessary
• permits to dynamically customize the browser settings (proxy, cookies, locale, etc.)
• Supports all major web application frameworks (e.g. GWT, SAP WD, JSF, JSP, ASP.NET)
• Integration with common test management tools (e.g. HP TestDirector for Quality Center)
• Data driven testing
• Distributed testing

 

References

[1] DUSTIN, E.; RASHKA, J.; Paul, J.: Automated Software Testing: Introduction, Management, and Performance, Addison Wesley, 1999.
[2] ANDRAE, C.: Erstellung von Softwarekomponenten zur Testautomatisierung von webbasierten Anwendungen, Berufsakademie Sachsen, Staatliche Studienakademie Dresden, 2004.
[3] MAX, T.: Entwicklung eines Testframeworks zum Test eines Co-Browsing-Systems, Universität Leipzig, 2005.
[4] TYMOSHENKO, V.: Entwicklung einer dynamischen und plattformunabhängigen Bibliothek für die externe Anbindung von Web-Browsern, Hochschule für Technik, Wirtschaft und Kultur Leipzig, 2006
[5] LANDMANN, J.: Evaluation und Konzeption eines automatisierten Ergonomie-Testwerkzeugs für die Software-Entwicklung, Universität Halle, 2007
[6] HERRON, D.: AJAX Application Testing, http://weblogs.java.net/blog/robogeek/archive/2007/07/ajax_applicatio.html, 2007
[7] MARKIEWICZ, B. : Abstraktion von GUI-Widgets zur Optimierung der Plattformunterstützung von C/R-Testautomatisierungen, Hochschule für Technik, Wirtschaft und Kultur Leipzig, 2007