Answer
# 2 |
Framework automation with IBM Rational Functional Tester:
Keyword-driven
Level: Intermediate
Michael Kelly (Mike@MichaelDKelly.com), Consultant,
www.MichaelDKelly.com
This third article in a three-part series on test automation
frameworks with IBM Rational Functional Tester looks at
creating a keyword-driven framework. A test automation
framework is a set of assumptions, concepts, and practices
that provide support for automated software testing. The
first two types of frameworks covered in this series are
modularity and data-driven.
Editor's Note: This article was written using the IBM®
Rational® Functional Tester for Java™ and Web 6.1 and
Microsoft® Windows® XP Professional SP2. Code examples will
be in Java, but all the concepts apply to the .NET version
of Rational Functional Tester as well.
This article is the third in a three-part series on test
automation frameworks with IBM Rational Functional Tester. A
test automation framework is a set of assumptions, concepts,
and practices that provides support for automated software
testing. The three types of frameworks covered in this
series are modularity, data-driven, and keyword-driven. This
article looks specifically at creating a keyword-driven
framework.
Keyword-driven testing (sometimes called table-driven
testing) refers to an automation framework where data tables
and keywords are used to drive the test automation. It's
subtly different than data-driven testing (and often used in
conjunction with data-driven testing), in that the focus of
keyword-driven testing is on actions and not data. In a
keyword-driven framework, testing is driven using tables of
data with predefined keywords. Keyword-driven tests look
similar to manual test cases.
A quick example
I often find it easier to demonstrate a keyword-driven
framework than to talk about one. So let's jump right in
with an example using the Windows Calculator, shown in Figure 1.
Figure 1. The Windows Calculator
If you were to map out the actions you perform with the
mouse when executing a test for the Windows Calculator, you
could create Table 1.
Table 1. A keyword table for the Windows Calculator
Window Control Action Argument
Calculator Menu
Calculator Pushbutton Click 1
Calculator Pushbutton Click +
Calculator Pushbutton Click 3
Calculator Pushbutton Click =
Calculator Verify result 4
Calculator Clear
Calculator Pushbutton Click 6
Calculator Pushbutton Click -
Calculator Pushbutton Click 3
Calculator Pushbutton Click =
Calculator Verify result 3
The table has four columns:
· Window: the application you're processing
· Control: the object you're processing on the window
· Action: the action you're performing on that object
· Argument: the data you require to perform that action
In this example, the Window column contains the name of the
application window where you perform the mouse action (in
this case, they all happen to be in the Calculator window).
The Control column names the type of control the mouse
clicks. The Action column lists the action taken with the
mouse (or by the tester). Finally, the Argument column names
a specific control (1, 2, 3, 5, +, -, and so on). This table
represents one complete test; you can make more as needed in
order to represent a series of tests.
Once you have a table like Table 1, all you need to do is
write driver code to parse the table and perform the actions
listed on the objects referenced. Listing 1 shows some
pseudocode for the Windows Calculator example.
Listing 1. Pseudocode for Windows Calculator
Main Script / Program
{
Connect to data tables.
Read in row and parse out values.
Pass values to appropriate module.
Close connection to data tables.
}
Menu Module
{
Set focus to window.
Select the menu pad option.
Return.
}
Pushbutton Module
{
Set focus to window.
Push the button based on argument.
Return.
}
Verify Result Module
{
Set focus to window.
Get contents from label.
Compare contents with argument value.
Log results.
Return.
}
From this example, you can see that this framework requires
relatively little code to generate many test cases. The data
tables are used to generate the individual test cases, while
the same code is reused.
One of the selling points of keyword-driven testing is that
it often allows nonautomated testers to write automated
tests. If you look at this Windows Calculator example --
without even seeing the code that parses the data and runs
the test -- you can probably imagine writing automated test
cases for the Windows Calculator application without knowing
anything more than you know now.
As Danny Faught points out in his article, Keyword-Driven
Testing (see Resources), this level of abstraction separates
the interface driver from the application interface.
"Though the sample script above uses terms suggesting a GUI
interface, the interface could be an application program
interface (API), web service, or anything else."
Danny points out that it's best to avoid embedding any
assumptions about the design of the user interface in your
choice of keywords. This way, the automated testers can
change interface drivers (for example, the pseudocode shown
in Listing 1) without changing the keyword script.
Traditionally, manual testers or business analysts write the
keyword tables, while the automated testers write the driver
code. From the driver code perspective, the process is
similar to code for data-driven testing, only now you have
if statements and case statements that look for keyword
values in the data.
Implementing a keyword-driven framework in Rational
Functional Tester
Now let's look at how you might implement a keyword-driven
framework in Rational Functional Tester. This example uses
www.BookPool.com. I'll start with a simple record and
playback test, and then make it keyword-driven. This is not
necessarily the process I would use to create an entire
keyword-driven framework (I would normally code it from
scratch), but it's a good way to get started with a short
example, and it will quickly create some code to work with.
For the first test, perform a search for a book on Ruby, add
it to your cart, and then click checkout. Listing 2 contains
the record and playback code for that test case.
Listing 2. Test case record and playback code
package Scripts;
import resources.Scripts.TestCaseOneHelper;
import com.rational.test.ft.*;
import com.rational.test.ft.object.interfaces.*;
import com.rational.test.ft.script.*;
import com.rational.test.ft.value.*;
import com.rational.test.ft.vp.*;
public class TestCaseOne extends TestCaseOneHelper
{
public void testMain(Object[] args)
{
startApp("www.BookPool.com");
// Document: Bookpool: http://www.bookpool.com/
text_qs().click(atPoint(77,11));
browser_htmlBrowser(document_bookpoolDiscountCompu(),
DEFAULT_FLAGS).inputChars("Ruby");
button_search_btnGif().click();
// Document: Bookpool: Books Found
button_addToCart2Gif().click();
// Document: Bookpool: Shopping Basket
button_proceedToCheckOutsubmit().click();
//...code to check out...
browser_htmlBrowser(document_bookpoolDiscountCompu2(),
MAY_EXIT).close();
}
}
Next, "variable-ize" the search so that when you create a
table for your keywords, you can pass in your search
criteria. To do that, use the Find Literals and Replace with
Datapool Reference command by following these steps.
1. Double-click Ruby in the test script and, from the Script
menu, select Find Literals and Replace with Datapool
Reference. Note: If the command is not enabled, you need to
associate a datapool with the script. This opens the
Datapool Literal Substitution dialog, shown in Figure 2.
Figure 2. Datapool Literal Substitution dialog
2. Enter Arguments for the Datapool Variable name.
3. Click Replace.
4. Click Close.
You should now have code that looks similar to that in
Listing 3. Note the removal of the Ruby search text and the
addition of the datapool call.
Listing 3. Test case record and playback code
package Scripts;
import resources.Scripts.TestCaseOneHelper;
import com.rational.test.ft.*;
import com.rational.test.ft.object.interfaces.*;
import com.rational.test.ft.script.*;
import com.rational.test.ft.value.*;
import com.rational.test.ft.vp.*;
public class TestCaseOne extends TestCaseOneHelper
{
public void testMain(Object[] args)
{
startApp("www.BookPool.com");
// Document: Bookpool: http://www.bookpool.com/
text_qs().click(atPoint(77,11));
browser_htmlBrowser(document_bookpoolDiscountCompu(),
DEFAULT_FLAGS).inputChars(dpString("Arguments"));
button_search_btnGif().click();
// Document: Bookpool: Books Found
button_addToCart2Gif().click();
// Document: Bookpool: Shopping Basket
button_proceedToCheckOutsubmit().click();
//...code to check out...
browser_htmlBrowser(document_bookpoolDiscountCompu2(),
MAY_EXIT).close();
}
}
The script in Listing 3 always selects the third search
result returned. That's fine for the purposes of this
example, but later I'll show you a way that you might build
out the framework to accommodate other choices. Next, you
need to build a keyword table for this script and then map
the script to that table. You can do this any number of
ways. It's important to work closely with the users of this
table to define meaningful keywords that make sense to them.
For all intents and purposes, this is a full-fledged
software development project for those users. You need to
gather their requirements and review the framework with them
on a regular basis as you build it. Table 2 is an example
that would work.
Table 2. A keyword table for www.BookPool.com
Action Argument
Start
Search Ruby
Select result 3
Check out
... etc. ...
Close
If you make Table 2 into the datapool for the test script,
it should look like Figure 3.
Figure 3. Example keyword Test Datapool
Next, add some code to the test script to process this
datapool. Loop through the datapool and, for each action you
encounter, call the code that performs that action. Listing
4 loops through the datapool using a while loop and
continues until dpDone() returns TRUE, indicating that there
are no more rows in the datapool. While in the loop, perform
a case-insensitive comparison to look for each action
keyword. If you don't find a keyword that you're expecting,
you should log an error to let users know that they have
used an incorrect keyword. You can do that using logError().
When you reach the end of the loop, you need to increment
the row in the datapool using dpNext(). When all is said and
done, you might have code that looks similar to Listing 4.
Listing 4. Loop through the datapool and look for keywords
String action = "";
//Run until the datapool is empty
while(!dpDone())
{
action = dpString("Action");
if(action.equalsIgnoreCase("Start"))
{
}
else if(action.equalsIgnoreCase("Search"))
{
}
else if(action.equalsIgnoreCase("Select Result"))
{
}
else if(action.equalsIgnoreCase("Check Out"))
{
}
else if(action.equalsIgnoreCase("...etc..."))
{
}
else if(action.equalsIgnoreCase("Close"))
{
}
else
{
logError("Invalid action passed: = " + action);
}
// Increment the datapool
dpNext();
}
Next, start adding code into each of these if-block
sections. You can do that by adding the couple lines of code
needed to perform that action, or you can call a class or
helper script (see the article on modularity for examples).
This example uses the simplest code possible to get things
working, as shown in Listing 5.
Listing 5. Copy and paste original script code
Package Scripts;
import resources.Scripts.TestCaseOneHelper;
import com.rational.test.ft.*;
import com.rational.test.ft.object.interfaces.*;
import com.rational.test.ft.script.*;
import com.rational.test.ft.value.*;
import com.rational.test.ft.vp.*;
public class TestCaseOne extends TestCaseOneHelper
{
public void testMain(Object[] args)
{
String action = "";
//Run until the datapool is empty
while(!dpDone())
{
action = dpString("Action");
if(action.equalsIgnoreCase("Start"))
{
startApp("www.BookPool.com");
}
else if(action.equalsIgnoreCase("Search"))
{
text_qs().click(atPoint(77,11));
browser_htmlBrowser(document_bookpoolDiscountCompu(),
DEFAULT_FLAGS).inputChars(
dpString("Arguments"));
button_search_btnGif().click();
}
else if(action.equalsIgnoreCase("Select Result"))
{
//NOTE: this always selects the third result
button_addToCart2Gif2().click();
}
else if(action.equalsIgnoreCase("Check Out"))
{
button_proceedToCheckOutsubmit().click();
}
else if(action.equalsIgnoreCase("...etc..."))
{
//Do something else...
}
else if(action.equalsIgnoreCase("Close"))
{
browser_htmlBrowser(
document_bookpoolDiscountCompu2(),
MAY_EXIT).close();
}
else
{
logError("Invalid action passed: = " + action);
}
// Increment the datapool
dpNext();
}
}
}
As you can see in Listing 5, you simply copy and paste the
code from the original test script. Running the script
should produce results like those shown in Figure 4.
Figure 4. Test case results for the script so far
That's great. You're on your way, but this code still has a
problem. Take your keyword table and compare it to the
nested if-statements to ensure that you've accounted for
every keyword and argument. You'll quickly see that you have
accounted for all the keywords. If you look closer, though,
you'll see that you have not accounted for all the
variables. For the variable on the Select Result keyword,
the test always selects the third result, regardless of what
gets passed in. You need to fix that, as shown in Listing 6.
Listing 6. Final script code
package Scripts;
import resources.Scripts.TestCaseOneHelper;
import com.rational.test.ft.*;
import com.rational.test.ft.object.interfaces.*;
import com.rational.test.ft.script.*;
import com.rational.test.ft.value.*;
import com.rational.test.ft.vp.*;
public class TestCaseOne extends TestCaseOneHelper
{
public void testMain(Object[] args)
{
String action = "";
//Run until the datapool is empty
while(!dpDone())
{
action = dpString("Action");
if(action.equalsIgnoreCase("Start"))
startApp("www.BookPool.com");
else if(action.equalsIgnoreCase("Search"))
{
text_qs().click(atPoint(77,11));
browser_htmlBrowser(document_bookpoolDiscountCompu(),
DEFAULT_FLAGS).inputChars(
dpString("Arguments"));
button_search_btnGif().click();
}
else if(action.equalsIgnoreCase("Select Result"))
{
if(dpString("Arguments").equals("1"))
button_addToCart2Gif().click();
else if(dpString("Arguments").equals("2"))
button_addToCart2Gif2().click();
else if(dpString("Arguments").equals("3"))
button_addToCart2Gif3().click();
else if(dpString("Arguments").equals("4"))
button_addToCart2Gif4().click();
else if(dpString("Arguments").equals("5"))
button_addToCart2Gif5().click();
else if(dpString("Arguments").equals("6"))
button_addToCart2Gif6().click();
else if(dpString("Arguments").equals("7"))
button_addToCart2Gif7().click();
else if(dpString("Arguments").equals("8"))
button_addToCart2Gif8().click();
else logError("Invalid argument for Select Result.");
}
else if(action.equalsIgnoreCase("Check Out"))
button_proceedToCheckOutsubmit().click();
else if(action.equalsIgnoreCase("...etc..."))
{
//Do something else...
}
else if(action.equalsIgnoreCase("Close"))
browser_htmlBrowser(
document_bookpoolDiscountCompu2(),
MAY_EXIT).close();
else logError("Invalid action passed: = " + action);
// Increment the datapool
dpNext();
}
}
}
The most robust way to write the code would be to allow
users to pass in the name of the book. That's quite a bit of
code and doesn't help illustrate how to build a keyword
framework, so I took the easy way out and kept selecting the
Add to Basket button from the Search Results screen. You
also might want to add code that can handle the possibility
of a Preorder Now button instead of an Add to Basket button.
If you go ahead and run the script again to make sure you
didn't break anything, you should get the results shown in
Figure 5.
Figure 5. Keyword-driven test case results
From this point, you can add any verification points you
might want. Moving forward, if you want more test cases, you
could simply add them to the same datapool by adding more
rows, or you could move this code to a class and have
different test scripts call that code using their private
datapools. Keep in mind that you can import datapool data
from a .csv file. That means your users (those who create
the keyword tables for testing) don't ever have to see Java
code or Rational Functional Tester. They can work in
Microsoft Excel® and never know that they are automating
test cases.
Advantages and disadvantages of a keyword-driven framework
Of the three types of frameworks covered in this series
(modularity, data-driven, and keyword-driven),
keyword-driven is the most difficult to implement because it
has lots of loops, conditionals, classes, helper scripts,
and so on. On the flip side, keyword-driven frameworks are
very powerful, and they make it super easy for nontesters
and nontechnical testers to contribute to the test project.
They lead to truly self-documenting automated test cases and
reduce the overall amount of code required to create large
numbers of test cases.
Danny accurately summarizes the main advantages and
disadvantages in his article:
"If you're not a programmer, you can still write automated
tests if your organization supports keyword-driven test
automation. I've talked to some people who have used
keyword-driven testing who say that it's not practical to
expect non-technical staff to write keyword scripts, but
others have said it works fine. The jury is still out on
that point. Managers need to understand that the team still
needs programmers dedicated to implementing and maintaining
the framework.
The real benefit is the improved maintainability of the test
scripts. The keyword approach follows in a longstanding
tradition of modularizing the test automation so that it's
easier to recover from user interface changes in the system
under test. That benefit alone is sufficient to justify
keyword-driven testing, even if all of the test designers
are expert programmers."
|
| Vijay
|
Site Map
