Automation Framework Workflow

If you want to earn from reading mails, Click Here

Basic information on Automation Frame Work Click HERE

Key word Driven Frame work ClickHERE

Data Driven Frame Work Click HERE

We have seen the primary features of our automation framework and now we want to put it to the test. This section provides a test workflow model that works very well with this framework. Essentially, we start by defining our high level Cycle Tables and provide more and more detail down to our Application Map and low-level Step Tables.

For this workflow example, we are going to show the hypothetical test design that might make up security authentication tests for a web site. The order in which we present the information and construct the tables is an ideal workflow for our automation framework. It will also illustrate how we do not even need a functioning application in order to start producing these tests.

1.4.1 High-Level Tests -- Cycle Table

We will start out by defining our high-level Cycle Table in Table 16. This will list tests that verify user authentication functionality. There is no application yet, but we do know that we will authenticate users with a user ID and password via some type of Login page. With this much information we can start designing some tests.

At this level, our tables are merely keywords or actions words of what we propose to do. The keywords represent the names of Suites we expect to implement when we are ready or when more is known about the application.

We will call this particular high-level test, VerifyAuthenticationFunction (Table 16). It will not show all the tests that should be performed to verify the user authentication features of a web site. Just enough to illustrate our test design process.

Cycle Table: VerifyAuthenticationFunction
KEYWORDS (Suite Tables)	TABLE PURPOSE
VerifyInvalidLogin	Tests with Invalid UserID and/or Password
VerifyBlankLogin	Tests with Missing UserID and/or Password
VerifyValidLogin	Tests with Valid UserID and Password

Table 16

The preceding table illustrates that we plan to run three tests to verify the authentication functionality of our application. We may add or delete tests from this list in the future, but this is how we currently believe this functionality can be adequately verified.

1.4.2 Intermediate-Level Tests -- Suite Tables

When we are ready to provide more detail for our authentication test effort we can begin to flesh-out the intermediate-level Suite Tables. Our high-level VerifyAuthenticationFunction Cycle Table defined three Suites that we must now expand upon.

First, we plan to verify invalid login attempts--those with an invalid user ID, invalid password, or both. This first Suite was aptly dubbed, VerifyInvalidLogin (Table 17).

Second, we want to run a simple test without any user ID or password whatsoever. This second Suite was called, VerifyBlankLogin (Table 18).

The last Suite we must implement is the one that insures we can successfully login when we provide a valid user ID and password. This one was named, VerifyValidLogin (Table 19).

Suite Table: VerifyInvalidLogin
KEYWORDS (Step Tables)	USERID	PASSWORD
LaunchSite
Login	BadUser	GoodPassword
VerifyLoginError
Login	GoodUser	BadPassword
VerifyLoginError
Login	BadUser	BadPassword
VerifyLoginError
ExitLogin

Table 17

Suite Table: VerifyBlankLogin
KEYWORDS (Step Tables)	USERID	PASSWORD
LaunchSite
Login	""	""
VerifyLoginError
ExitLogin

Table 18

Suite Table: VerifyValidLogin
KEYWORDS (Step Tables)	USERID	PASSWORD
LaunchSite
Login	GoodUser	GoodPassword
VerifyLoginSuccess
ShutdownSite

Table 19

Notice that we start to see a large amount of test table reuse here. While not yet defined, we can see that we will be using the LaunchSite, Login, VerifyLoginError, and ExitLogin tables quite frequently. As you can probably imagine, VerifyLoginSuccess and ShutdownSite would also be used extensively throughout all the tests for this web site, but we are focusing just on user authentication here.

You should also notice that our Login keyword is describing a Step Table that will accept up to two arguments--the user ID and password we want to supply. We have not gone into the specific design details of how the automation framework should be implemented, but tests accepting variable data like this provide significant reuse advantages. This same table is used to login with different user ID and password combinations.

1.4.3 Application Maps

Up to this point there was little need for an Application Map. Nearly all the high-level and intermediate-level test designs are abstract enough to forego references to specific window components. But as we move towards our final level of test design we indeed must reference specific elements of the application. Thus, for test automation, we need to make our Application Maps.

Naming the Components:
Unless the application has formal design documentation that explicitly provides meaningful names for the windows and components, the test team is going to have to develop this themselves.

The team will discuss and agree upon the names for each window and component. Sometimes we can use the names developers have already given the components in the code. But the team can often come up with more meaningful and identifiable names. Regardless, once defined, the names chosen should not change. However, if the need arises, an item can always have more than one name.

Test Design Application Map:
For low-level test design and manual testing, an Application Map suitable for human consumption should be created. This is often done with screen captures of the various application windows. These screen captures are then clearly annotated with the names given to each component. This may also indicate the Type for each component.

While many components may be exactly what they look like, this is not always the case. Many controls may simply be windowless, painted elements that the underlying environment knows nothing about. They can also be custom components or COM objects that are functionally different from the controls they appear to emulate. So care must be taken when identifying the Type of a component.

Automation Framework Application Map:
For automated testing, an Application Map suitable for the automation framework will be created. We will use the Test Design Application Map as our reference for the names used to identify the components. The identification syntax suitable for the automation tool will then be mapped to each name as discussed in Section 1.3.4.

Our test design so far has indirectly identified three documents or windows that we will interact with: (1) the Login page, (2) an Error dialog or message displayed during our failed Login attempts, and (3) the Home page of our web site displayed following a successful login attempt. We will also need to reference the browser window itself throughout these tests.

We must now create an Application Map that will properly reference these items and their components for the automation framework. For the sake of simplicity, our Login page in Figure 7 will contain only the two textbox components for user ID and password, and two buttons; Submit and Cancel.

Login Page

Figure 7

The Error dialog in Figure 8 is actually a system alert dialog. It has a Label containing the error message and an OK button to dismiss the dialog.

Error Dialog

Figure 8

The Home Page of the application may contain many different elements. However, for the purpose of these tests and this example, we only need to identify the document itself. We will not interact with any of the components in the Home Page, so we will not define them in this sample Application Map.

Table 20 shows the Application Map we will use to identify the browser window and the three documents in a manner suitable for the automation framework.

Site Application Map
OBJECTNAME	IDENTIFICATION METHOD
Browser:
Browser	WindowID=WebBrowser
LoginPage:
LoginPage	FrameID=content;\;DocumentTitle=Login
UserID	FrameID=content;\;DocumentTitle=Login;\;TextBoxIndex=1
Password	FrameID=content;\;DocumentTitle=Login;\;TextBoxIndex=2
SubmitButton	FrameID=content;\;DocumentTitle=Login;\;ButtonText=Submit
CancelButton	FrameID=content;\;DocumentTitle=Login;\;ButtonText=Cancel
ErrorWin:
ErrorWin	WindowCaption=Error
ErrorMessage	LabelIndex=1
OKButton	ButtonText=OK
HomePage:
HomePage	FrameID=content;\;DocumentTitle=Home

Table 20

The physical format of the Application Map will be determined by how the automation framework will access it. It may be delimited files, database tables, spreadsheets, or any number of other possibilities.

Notice that we have identified each document with its own section in the Application Map of Table 20. We do this to avoid naming conflicts among the many components of our application. For instance, there may be many forms that contain a Submit button. Each form can name this button the SubmitButton because the names only have to be unique within each individual section of the Application Map.

With this we have now uniquely identified our components for the framework. The OBJECTNAME is how we reference each component in our low-level Step Tables. The IDENTIFICATION METHOD is the information the automation tool needs to identify the component within the computer operating system environment.

The syntax for these component identifications is different depending upon the automation tool deployed. The example syntax we used is not for any particular automation tool, but is representative of what is needed for automation tools in general. The Application Map designer will need a thorough understanding of the automation tool’s component identification syntax in order to create effective Application Maps.

New Component Types:
As we examine the application and develop an Automation Framework Application Map we may find new component types not handled by existing Component Functions. To deal with this, we need to either implement a new Component Function module for each new component type, or determine how best they can be handled by existing Component Functions. This, of course, is a job for the automation technical staff.

1.4.4 Low-Level Tests -- Steps Tables

Now that the design of our application is known and our Application Map has been developed, we can get down to the step-by-step detail of our tests. We can still develop Step Tables without an Application Map. But without the map, we cannot automatically execute the tests.

Looking back at all the Suites that detailed our high-level VerifyAuthenticationFunction Cycle Table, we see they invoke six reusable, low-level Step Tables that we are now ready to implement. These are listed in Table 21.

Step Tables Invoked by Suites
STEP TABLE	TABLE PURPOSE
LaunchSite	Launch the web app for testing
Login	Login with a User ID and Password
VerifyLoginError	Verify an Error Dialog on Invalid Login
ExitLogin	Exit the Login Page via Cancel
VerifyLoginSuccess	Verify Home Page up after Successful Login
ShutdownSite	Logoff and close the browser

Table 21

We develop these Step Tables using the component names defined in our Application Map and the keyword vocabulary defined by our Component Functions. Where test designers find missing yet needed keywords, actions, or commands in our Component Functions; the technical staff will need to expand the appropriate Component Function module, Driver Commands, Support Libraries, or find some other solution to provide the needed functionality if possible.

Now we can provide the detailed test steps that make-up our six Step Tables.

Step Table: LaunchSite
COMMAND/DOCUMENT	ARG/COMPONENT	ACTION	EXPECTED RESULTS
LaunchBrowser	Default.htm
Browser		VerifyCaption	"Login"

Table 22

Step Table: Login &user &password
DOCUMENT	COMPONENT	ACTION	INPUT DATA
LoginPage	UserID	InputText	&user
LoginPage	Password	InputText	&password
LoginPage	SubmitButton	Click

Table 23

Step Table: VerifyLoginError
DOCUMENT	COMPONENT	ACTION	EXPECTED RESULT
ErrorWin		VerifyCaption	"Error"
ErrorWin	ErrorMessage	VerifyText	"The User ID or Password was invalid."
ErrorWin	OKButton	Click

Table 24

Step Table: ExitLogin
DOCUMENT	COMPONENT	ACTION
LoginPage	CancelButton	Click

Table 25

Step Table: VerifyLoginSuccess
DOCUMENT	COMPONENT	ACTION	EXPECTED RESULT
HomePage		VerifyTitle	"Home"
Browser		VerifyCaption	"Welcome"

Table 26

Step Table: ShutdownSite
DOCUMENT	COMPONENT	ACTION	INPUT DATA
Browser		SelectMenuItem	File>Exit

Table 27

As you can see, our high-level test VerifyAuthenticationFunction has been detailed down to a very sparse set of highly reusable test steps. Our test designs have taken a high-level test specification, given that some abstract scenarios to perform, and finally provided the step-by-step instructions to execute those scenarios. We did all of this without resorting to difficult scripting languages, or complex software development environments.

It may seem simple enough, but that is because we have the underlying automation framework doing so much of the work for us.

Now that we have all of this defined, what does it actually do?

Remember, the automation framework is invoked with one or more high-level Cycle Tables. These tables call the test Suites that, in turn, call the Step Tables to perform the low-level test steps that drive the application.

We developed our own Cycle Table called VerifyAuthenticationFunction. This calls our own collection of Suites, and ultimately the Step Tables we just finished developing. If you look over all of these tables, you can see they form a relatively small set of reusable instructions that make up this test. But they generate a large instruction set at time of execution.

The following section will actually illustrate the automated execution flow of the test we have just implemented.

Table 28

Execution Flow of VerifyAuthenticationFunction Test
Cycle Table Suite Table Step Table	DOCUMENT Or COMMAND	COMPONENT Or COMMAND ARG	ACTION	INPUT DATA Or EXPECTED RESULTS
VerifyAuthenticationFunction
VerifyInvalidLogin
LaunchSite
	LaunchBrowser	Default.htm
	Browser		VerifyCaption	"Login"
Login	&user=BadUser	&password=GoodPassword
	LoginPage	UserID	InputText	&user
	LoginPage	Password	InputText	&password
	LoginPage	SubmitButton	Click
VerifyLoginError
	ErrorWin		VerifyCaption	"Error"
	ErrorWin	ErrorMessage	VerifyText	"The User ID or Password was Invalid."
	ErrorWin	OKButton	Click
Login	&user=GoodUser	&password=BadPassword
	LoginPage	UserID	InputText	&user
	LoginPage	Password	InputText	&password
	LoginPage	SubmitButton	Click
VerifyLoginError
	ErrorWin		VerifyCaption	"Error"
	ErrorWin	ErrorMessage	VerifyText	"The User ID or Password was Invalid."
	ErrorWin	OKButton	Click
Login	&user=BadUser	&password=BadPassword
	LoginPage	UserID	InputText	&user
	LoginPage	Password	InputText	&password
	LoginPage	SubmitButton	Click
VerifyLoginError
	ErrorWin		VerifyCaption	"Error"
	ErrorWin	ErrorMessage	VerifyText	"The User ID or Password was Invalid."
	ErrorWin	OKButton	Click
ExitLogin
	LoginPage	CancelButton	Click
VerifyBlankLogin
LaunchSite
	LaunchBrowser	Default.htm
	Browser		VerifyCaption	"Login"
Login	&user=""	&password=""
	LoginPage	UserID	InputText	&user
	LoginPage	Password	InputText	&password
	LoginPage	SubmitButton	Click
VerifyLoginError
	ErrorWin		VerifyCaption	"Error"
	ErrorWin	ErrorMessage	VerifyText	"The User ID or Password was Invalid."
	ErrorWin	OKButton	Click
ExitLogin
	LoginPage	CancelButton	Click
VerifyValidLogin
LaunchSite
	LaunchBrowser	Default.htm
	Browser		VerifyCaption	"Login"
Login	&user=GoodUser	&password=GoodPassword
	LoginPage	UserID	InputText	&user
	LoginPage	Password	InputText	&password
	LoginPage	SubmitButton	Click
VerifyLoginSuccess
	HomePage		VerifyTitle	"Home"
	Browser		VerifyCaption	"Welcome"
ShutdownSite
	Browser		SelectMenuItem	File>Exit

Table 28