Difference between Client server and webserver applications

Taking a big step back, a Web server serves pages for viewing in a Web browser, while an application server provides methods that client applications can call. A little more precisely, you can say that:

A Web server exclusively handles HTTP requests, whereas an application server serves business logic to application programs through any number of protocols.


Let's examine each in more detail.

The Web server
A Web server handles the HTTP protocol. When the Web server receives an HTTP request, it responds with an HTTP response, such as sending back an HTML page. To process a request, a Web server may respond with a static HTML page or image, send a redirect, or delegate the dynamic response generation to some other program such as CGI scripts, JSPs (JavaServer Pages), servlets, ASPs (Active Server Pages), server-side JavaScripts, or some other server-side technology. Whatever their purpose, such server-side programs generate a response, most often in HTML, for viewing in a Web browser.

Understand that a Web server's delegation model is fairly simple. When a request comes into the Web server, the Web server simply passes the request to the program best able to handle it. The Web server doesn't provide any functionality beyond simply providing an environment in which the server-side program can execute and pass back the generated responses. The server-side program usually provides for itself such functions as transaction processing, database connectivity, and messaging.

While a Web server may not itself support transactions or database connection pooling, it may employ various strategies for fault tolerance and scalability such as load balancing, caching, and clustering—features oftentimes erroneously assigned as features reserved only for application servers.

The application server
As for the application server, according to our definition, an application server exposes business logic to client applications through various protocols, possibly including HTTP. While a Web server mainly deals with sending HTML for display in a Web browser, an application server provides access to business logic for use by client application programs. The application program can use this logic just as it would call a method on an object (or a function in the procedural world).

Such application server clients can include GUIs (graphical user interface) running on a PC, a Web server, or even other application servers. The information traveling back and forth between an application server and its client is not restricted to simple display markup. Instead, the information is program logic. Since the logic takes the form of data and method calls and not static HTML, the client can employ the exposed business logic however it wants.

In most cases, the server exposes this business logic through a component API, such as the EJB (Enterprise JavaBean) component model found on J2EE (Java 2 Platform, Enterprise Edition) application servers. Moreover, the application server manages its own resources. Such gate-keeping duties include security, transaction processing, resource pooling, and messaging. Like a Web server, an application server may also employ various scalability and fault-tolerance techniques.

QA Methodologies followed in any QA

QA Methodologies and Test Management Systems
For the smooth progress of the software development process what is most important is the availability of advanced test management systems. If that is available then it can be said that half the job of the testers is done. But when selecting a test management system make sure that it fits well with the different QA methodologies. Now the most commonly used QA methodologies are Waterfall, Hybrid, V Model, and Agile. Now let’s look at the important aspects of the most common methodologies and how a good test management system should behave in the various methodologies.

Waterfall model, being structured, well documented and a disciplined process is mainly used for stable projects. Since it is non-iterative it allows multiple projects to run at a time. A good test management system will allow multiple releases within each project so that the QA team can handle major, minor and patch releases without changing processes.

V Model is a rigorous and measurable process and therefore is mostly used for complex projects. A suitable test management system helps in maintaining rigor in the testing process. It will also handle multiple testing cycles for System, Integration, and final Acceptance.

Agile, as the name suggests, is a flexible and fast process and its most important aspect is that it minimizes risk by developing software in a short period of time. A good test management system not only organizes and tracks testing by iterations but also handles multiple iterations within each release. It also facilitates information sharing and instant communication among the onsite and offshore testers.

Hybrid model is a combination of two or more QA methods. It makes transition from one model to another throughout the lifecycle of the project. A suitable test management system allows different models to co-exist peacefully by facilitating easy transition between various modes in the same system. It also allows reusability of test assets between different modes and lifecycles.

Assignment on Quality Metrics-1

EXERCISE A– Testing Project

Post delivery defect leakage goal: This is the % of the testing defects identified in our organization(excludes client found defects)/total defects. It should be greater than 99%.

Effort and Schedule goals & formula:


Effort PCB Schedule PCB
Goal(Upper Limit) 15% Goal(Upper Limit) 5%
Goal(Lower Limit) 0% Goal(Lower Limit) -5%

Effort : (Actual Hours-Estimated Hours)* 100 / (Estimated Hours)

Schedule: ((Actual End Date-Actual Start Date)-(Estimated End Date- Estimated Start Date))*100 / (Estimated End Date-Estimated Start Date+1)


Phase Organization Goal Formula
RDD Completion of test preparation >=0.75 Review defects/Size
TDD Completion of test execution >= 1.25 Review defects/Size


Defects distribution across different testing cycles:

Usually 60% defects are found in the 1st test cycle, 90% of defects found by the 2nd test cycle, remaining in the last test cycle.

Note: Goals are usually tighter by 25% of the mean. While totaling defects use the weighted defects i.e. Severity 1 = 10, Severity 2 = 5, Severity 1 =1

Case Study 1:

Setting Goals

You are an independent testing team and the scope of work is system testing. There is another team Denver which has done similar projects within the same financial area. Their data is as follows;

Denver Data
Post delivery defect leakage 95%
Effort Variance UCL 50, LCL -40
Schedule Variance UCL 3, LCL 0
RDD .25
TDD .50
Size 600 Test Case Points

However this project is more complex and has many more interfaces to test. This would be data driven test where many SQL queries would have to be written to extract and validate the data. The size is 1000 Test Case points. You have planned 3 test cycles. Within each test cycle you have planned for 3 test rounds.

Q.1. Set the post delivery defect leakage, effort/schedule variance, RDD and TDD goals. How many defects would you expect at end of test preparation and test execution.
How many defects you would expect at end of each test cycle? Make your assumptions and identify the risks.






A.1.
Project Data Goals
Post delivery defect leakage 95%
Effort Variance UCL 50, LCL -40
Schedule Variance UCL 3, LCL 0
RDD .75, 750 defects
TDD 1.25,1250 defects, 1st Cycle – 750, 2nd cycle -375
Size 600 Test Case Points 1000


At end of test Planning

At end of test preparation the effort, schedule data looks like this;

Planned Actual
Effort 1200 man hrs 1500 man hrs
Schedule 10th June 2004 12th June 2004
Schedule Variance UCL 3, LCL 0
Review defects 450 defects

During your test planning phase some testers resigned and you had to quickly recruit new associates who joined towards the end of test planning phase.

The main business functionalities are adjustments, auto login, customer service transactions, BT related functionality, credit card payment and maintaining batch items. Many queries on the functionality were raised to the client. Mean time to resolve 70% of the queries was 2 days. 30% of the queries took more than 6 days to resolve.

The distribution of queries, test cases and reviews across functionality is;

Function/Module No of Queries Review defects Test Cases
Adjustments 18 65 245
Maintaining Batch Transactions & Create new batch 73
40 136
BT adjustments 27 9 234
Convenience Cheques 23 38 23
Customer Service Transactions 6 13 129
Letters & Rates 40 24 68
Auto Login 9 34 56
Credit card payments 23 55 143
Unpaid Tem Fee 45 19 111
Fee Tallies 12 34 98
Miscellaneous Adjustments 8 38 189
Refund Credit Balance 10 34 245
Other 47 136







Distribution of defects category wise across testware is as follows;

Function/Logic (incorrect testware, not as per functional requirements) ;187 defects
Suggestions ; 50
Test Coverage ; 213 defects

Q 2 What analysis would you do and what actions would you take? What goals would you re-visit? Make your assumptions and identify the risks.


End of Test Execution

At end of cycle 1 - The effort variance is 60%. It took a long time for the client to decide whether the defect raised was really a defect. The client had included some new functionality to test. During test execution the link to the client test environment was down.

Defect Trend Test Cases Defects – Severity
Stages Planned Executed Severity 1 Severity 2 Severity 3 Suggestion Total
Smoke Test 50 40 18 37 5 2
Cycle 2 1500 592 23 76 13 5
Cycle 3 1500 789 15 49 12 2
Cycle 4 1500 1979 25 38 11 4
Cycle 5 1979 1979 7 23 9 9
Cycle 6 1979 1979 2 7 0 0
Staging 2746 2768 39 99 31 5
Total 129 329 81 27
Rejected 34 22 35 5




Defect Reported Status Open Fixed Closed Deferred Reject Later Duplicate Total
Severity 1 2 9 34 32 34 2 16
Severity 2 20 20 170 52 22 11 34
Severity 3 3 6 13 4 35 8 12
Suggestion 5 5 10 5 2
Total




Q3. Plot the chart on test cases planned vs executed, % executed and defect trend for all severity levels? Plot the chart of the defect status, find out the % distribution of defect status? What analysis would you do? What actions you should take? What goals would you re-visit?





Defect Trend Test Cases Defects – Severity
Stages Planned Executed Severity 1 Severity 2 Severity 3 Suggestion Total
Smoke Test 50 40 18 37 5 2 370
Cycle 2 1500 592 23 76 13 5 623
Cycle 3 1500 789 15 49 12 2 407
Cycle 4 1500 1979 25 38 11 4 451
Cycle 5 1979 1979 7 23 9 9 194
Cycle 6 1979 1979 2 7 0 0 55
Staging 2746 2768 39 99 31 5 916
Total 129 329 81 27 3016
Rejected 34 22 35 5 485
Rejected % 26% 7% 43% 19% 16%



Defect Reported Status Open Fixed Closed Deferred Reject Later Duplicate Total
Severity 1 2 9 34 32 34 2 16 129
Severity 2 20 20 170 52 22 11 34 329
Severity 3 3 6 13 4 35 8 12 81
Suggestion 5 5 10 5 2 27
Total 25 40 222 98 96 23 62 665









A3 Find out why the defects were rejected, plot defects across test cases and functionality & re-estimate. Find out how many defects did not have corresponding test cases. Set up the defect resolution time with the client. In fact this should have been done during the planning phase. Do a causal analysis to find out the reasons for the rejected defects. For the client added functionality, re-estimate the testing effort and also find out if the test cases have been added for the newly added functionality





































EXERCISE B–Testing Project

You are executing an independent system testing project. For system test you have planned 4 cycle of 2 weeks each:

Week Week 1 Week 3 Week 5 Week 7
Cycle Cycle 1 Cycle 2 Cycle 3 Cycle 4
Build Build 1 Build 2 Build 3 Build 4
Functionality being added 2 5 3 0

Build_1 was delivered on time i.e. first day of week 1. This was tested till end of second week. Defects found in Cycle 1 were 50. But Build 2 was delivered in week 4 instead of week 3 and that too was a broken build. After discussing this problem with the client, they have decided to give a working build which will have the functionality of Build 2and 3 together in week 5.


Q1. How will you plan testing of build which will have functionality of Build 2 and 3?
A1. In this scenario you will have 3 Cycles instead of 4 and you will add another cycle in Week 9.

Assignment on Quality Metrics-1

EXERCISE A– Testing Project

Post delivery defect leakage goal: This is the % of the testing defects identified in our organization(excludes client found defects)/total defects. It should be greater than 99%.

Effort and Schedule goals & formula:


Effort PCB Schedule PCB
Goal(Upper Limit) 15% Goal(Upper Limit) 5%
Goal(Lower Limit) 0% Goal(Lower Limit) -5%

Effort : (Actual Hours-Estimated Hours)* 100 / (Estimated Hours)

Schedule: ((Actual End Date-Actual Start Date)-(Estimated End Date- Estimated Start Date))*100 / (Estimated End Date-Estimated Start Date+1)


Phase Organization Goal Formula
RDD Completion of test preparation >=0.75 Review defects/Size
TDD Completion of test execution >= 1.25 Review defects/Size


Defects distribution across different testing cycles:

Usually 60% defects are found in the 1st test cycle, 90% of defects found by the 2nd test cycle, remaining in the last test cycle.

Note: Goals are usually tighter by 25% of the mean. While totaling defects use the weighted defects i.e. Severity 1 = 10, Severity 2 = 5, Severity 1 =1

Case Study 1:

Setting Goals

You are an independent testing team and the scope of work is system testing. There is another team Denver which has done similar projects within the same financial area. Their data is as follows;

Denver Data
Post delivery defect leakage 95%
Effort Variance UCL 50, LCL -40
Schedule Variance UCL 3, LCL 0
RDD .25
TDD .50
Size 600 Test Case Points

However this project is more complex and has many more interfaces to test. This would be data driven test where many SQL queries would have to be written to extract and validate the data. The size is 1000 Test Case points. You have planned 3 test cycles. Within each test cycle you have planned for 3 test rounds.

Q.1. Set the post delivery defect leakage, effort/schedule variance, RDD and TDD goals. How many defects would you expect at end of test preparation and test execution.
How many defects you would expect at end of each test cycle? Make your assumptions and identify the risks.






A.1.
Project Data Goals
Post delivery defect leakage 95%
Effort Variance UCL 50, LCL -40
Schedule Variance UCL 3, LCL 0
RDD .75, 750 defects
TDD 1.25,1250 defects, 1st Cycle – 750, 2nd cycle -375
Size 600 Test Case Points 1000


At end of test Planning

At end of test preparation the effort, schedule data looks like this;

Planned Actual
Effort 1200 man hrs 1500 man hrs
Schedule 10th June 2004 12th June 2004
Schedule Variance UCL 3, LCL 0
Review defects 450 defects

During your test planning phase some testers resigned and you had to quickly recruit new associates who joined towards the end of test planning phase.

The main business functionalities are adjustments, auto login, customer service transactions, BT related functionality, credit card payment and maintaining batch items. Many queries on the functionality were raised to the client. Mean time to resolve 70% of the queries was 2 days. 30% of the queries took more than 6 days to resolve.

The distribution of queries, test cases and reviews across functionality is;

Function/Module No of Queries Review defects Test Cases
Adjustments 18 65 245
Maintaining Batch Transactions & Create new batch 73
40 136
BT adjustments 27 9 234
Convenience Cheques 23 38 23
Customer Service Transactions 6 13 129
Letters & Rates 40 24 68
Auto Login 9 34 56
Credit card payments 23 55 143
Unpaid Tem Fee 45 19 111
Fee Tallies 12 34 98
Miscellaneous Adjustments 8 38 189
Refund Credit Balance 10 34 245
Other 47 136







Distribution of defects category wise across testware is as follows;

Function/Logic (incorrect testware, not as per functional requirements) ;187 defects
Suggestions ; 50
Test Coverage ; 213 defects

Q 2 What analysis would you do and what actions would you take? What goals would you re-visit? Make your assumptions and identify the risks.


End of Test Execution

At end of cycle 1 - The effort variance is 60%. It took a long time for the client to decide whether the defect raised was really a defect. The client had included some new functionality to test. During test execution the link to the client test environment was down.

Defect Trend Test Cases Defects – Severity
Stages Planned Executed Severity 1 Severity 2 Severity 3 Suggestion Total
Smoke Test 50 40 18 37 5 2
Cycle 2 1500 592 23 76 13 5
Cycle 3 1500 789 15 49 12 2
Cycle 4 1500 1979 25 38 11 4
Cycle 5 1979 1979 7 23 9 9
Cycle 6 1979 1979 2 7 0 0
Staging 2746 2768 39 99 31 5
Total 129 329 81 27
Rejected 34 22 35 5




Defect Reported Status Open Fixed Closed Deferred Reject Later Duplicate Total
Severity 1 2 9 34 32 34 2 16
Severity 2 20 20 170 52 22 11 34
Severity 3 3 6 13 4 35 8 12
Suggestion 5 5 10 5 2
Total




Q3. Plot the chart on test cases planned vs executed, % executed and defect trend for all severity levels? Plot the chart of the defect status, find out the % distribution of defect status? What analysis would you do? What actions you should take? What goals would you re-visit?





Defect Trend Test Cases Defects – Severity
Stages Planned Executed Severity 1 Severity 2 Severity 3 Suggestion Total
Smoke Test 50 40 18 37 5 2 370
Cycle 2 1500 592 23 76 13 5 623
Cycle 3 1500 789 15 49 12 2 407
Cycle 4 1500 1979 25 38 11 4 451
Cycle 5 1979 1979 7 23 9 9 194
Cycle 6 1979 1979 2 7 0 0 55
Staging 2746 2768 39 99 31 5 916
Total 129 329 81 27 3016
Rejected 34 22 35 5 485
Rejected % 26% 7% 43% 19% 16%



Defect Reported Status Open Fixed Closed Deferred Reject Later Duplicate Total
Severity 1 2 9 34 32 34 2 16 129
Severity 2 20 20 170 52 22 11 34 329
Severity 3 3 6 13 4 35 8 12 81
Suggestion 5 5 10 5 2 27
Total 25 40 222 98 96 23 62 665









A3 Find out why the defects were rejected, plot defects across test cases and functionality & re-estimate. Find out how many defects did not have corresponding test cases. Set up the defect resolution time with the client. In fact this should have been done during the planning phase. Do a causal analysis to find out the reasons for the rejected defects. For the client added functionality, re-estimate the testing effort and also find out if the test cases have been added for the newly added functionality





































EXERCISE B–Testing Project

You are executing an independent system testing project. For system test you have planned 4 cycle of 2 weeks each:

Week Week 1 Week 3 Week 5 Week 7
Cycle Cycle 1 Cycle 2 Cycle 3 Cycle 4
Build Build 1 Build 2 Build 3 Build 4
Functionality being added 2 5 3 0

Build_1 was delivered on time i.e. first day of week 1. This was tested till end of second week. Defects found in Cycle 1 were 50. But Build 2 was delivered in week 4 instead of week 3 and that too was a broken build. After discussing this problem with the client, they have decided to give a working build which will have the functionality of Build 2and 3 together in week 5.


Q1. How will you plan testing of build which will have functionality of Build 2 and 3?
A1. In this scenario you will have 3 Cycles instead of 4 and you will add another cycle in Week 9.

important Basics of Performance Testing-3

Basics: Underlying Operating System and Network Improvements
If you control the OS and hardware where the application will be deployed, there are a number of changes you can make to improve performance. Some changes are generic and affect most applications, while some are application-specific. This article applies to most server systems running Java application, including servlets, where you usually specify (or have specified to you) the underlying system, and where have some control over tuning the system. Client and standalone Java programs are likely to benefit from this chapter only if you have some degree of control over the target system, but some tips in the chapter apply to all Java programs.
It is usually best to target the OS and hardware as a last tuning choice. Tuning the application itself generally provides far more significant speedups than tuning the systems on which the application is running. Application tuning also tends to be easier (though buying more powerful hardware components is easier still and a valid choice for tuning). However, application and system tuning are actually complementary activities, so you can get speedups from tuning both the system and the application if you have the skills and resources. Here are some general tips for tuning systems:

• Constantly monitor the entire system with any monitoring tools available and keep records. This allows you to get a background usage pattern and also lets you compare the current situation with situations previously considered stable.
• You should run offline work during off-hours only. This ensure that there is no extra load on the system when the users are executing online tasks and enhance performance of both online and offline activities.
• If you need to run extra tasks during the day, try to slot them into times with low user activity. Office activity usually peaks at 9am and 2:30pm and has a load between noon and 1pm or at shift changeovers. You should be able to determine the user-activity cycles appropriate to your system by examining the results of normal monitoring. The reduced conflict for system resources during periods of low activity improves performance.
• You should specify timeouts for all process under the control of your application (and others on the system, if possible) and terminate processes that have passed their timeout value.
• Apply any partitioning available from the system to allocate determinate resources to your application. For example, you can specify disk partitions, memory segments, and even CPUs to be allocated to particular process.

As the entire chapter is lengthy, I've split them into various sections namely the links that followed. Click on the links to access the article.
CPU
Disk
Memory
Network

The above is taken from the publication, "Java Performance Tuning" written by Jack Shirazi. I would recommend to read this book as it provides not just tuning and bottleneck concepts bounded by Java. A simplified version (which is the summary of the chapter can be found here [Comming soon]).
Basics: CPU Bottlenecks
Java provides a virtual machine runtime system that is just that: an abstraction of a CPU that runs in software. (Note that this chapter is taken from the "Java Performance Tuning" written by Jack Shirazi and therefore alot of discussions circled around Java technologies.) These virtual machines run on a real CPU, and in this section the book discuss the performance characteristics of those real CPUs.

CPU Load

The CPU and many other parts of the system can be monitored using system-level utilities. On Windows, the task manager and performance monitor can be used for monitoring. On UNIX, a performance monitor (such as perfmeter) is usually available, as well as utilities such as vmstat. Two aspects of the CPU are worth watching as primary performance points. These are the CPU utilization (usually expressed in percentage terms) and the run-able queue of processes and threads (often called the load or the task queue). The first indictor is simply the percentage of the CPU (Or CPUs) being used by all the various threads. If this is up to 100% for significant periods of time, you may have a problem. On the other hand, if it isn’t, the CPU is under-utilized, but that is usually preferable. Low CPU usage can indicate that your application may be blocked for significant periods on disk or network I/O. High CPU usage can indicate thrashing (lack of RAM) or CPU contention (indicating that you need to tune the code and reduce the number of instructions being processed to reduce the impact on the CPU).

A reasonable target is 75% CPU utilization (which from what I read from different authors varies from 75% till 85%). This means that the system is being worked toward its optimum, but that you have left some slacks for spikes due to other system or application requirements. However, note that if more than 50% of the CPU is used by system processes (i.e. administrative and IS process), your CPU is probably under-powered. This can be identified by looking at the load of the system over some period when you are not running any applications.

The second performance indicator, the run-able queue, indicates the average number of processes or threads waiting to be scheduled for the CPU by the OS. They are run-able processes, but the CPY has no time to run them and is keeping them waiting for some significant amount of time. As soon as the run queue goes above zero, the system may display contention for resources, nut there usually some value above zero that still gives acceptable performance for any particular system. You need to determine what that value is in order to use this statistics as a useful warning indicator. A simplistic way to do this is to create a short program that repeatedly does some simple activity. You can then time each run of that activity. You can run copies of this process one after the other so that more and ore copies are simultaneously running. Keep increasing the number of copies being run until the run queue starts increasing. By watching the times recorded for the activity, you can graph that time against the run queue. This should give you some indication of when the run-able queue becomes too large for useful responses on your system, administrator if the threshold is exceeded. A guideline by Adrian Cockcroft is that performance starts to degrade if the run queue grows bigger than four times the number of CPUs.

If you can upgrade the CPU of the target environment, doubling the CPU speed is usually better than doubling the number of CPUs. And remember that parallelism in an application doesn’t necessarily need multiple CPUs. If I/O is significant, the CPU will have plenty of time for many threads.

Process Priorities

The OS also has the ability to prioritize the processes in terms of providing CPU time by allocating process priority levels. CPU priorities provide a way to throttle high-demand CPU processes, thus giving other processes a greater share of the CPU. If there are other processes that need to run on the same machine but it doesn’t matter if they were run slowly, you can give your application processes a (much) higher priority than those other processes, thus allowing your application the lion’s share of CPU time on a congested system. This is worth keeping in mind if your application consists of multiple processes, you should also consider the possibility of giving your various processes different levels of priority.

Being tempted to adjust the priority levels of processes, however, is often a sign that the CPU is underpowered for the tasks you have given it.



The above is taken from the publication, "Java Performance Tuning" written by Jack Shirazi. I would recommend to read this book as it provides not just tuning and bottleneck concepts bounded by Java.

Basics: Disks Bottlenecks
In most cases, applications can be tuned so that disk I/O does not cause any serous performance problems. But if, application tuning, you find that disk I/O s still causing a performance problem; your best bet may be to upgrade the system disks. Identifying whether the system has a problem with disk utilization is the first step. Each system provides its own tools to identify disk usage (Windows has a performance monitor, and UNIX has the sar, vmstat, iostat utilities.) At minimum, you need to identify whether the paging is an issue (look at disk-scan rates) and assess the overall utilization of your disks (e.g. performance monitor on Windows, output from iostat –D on UNIX). It may be that the system has a problem independent of your application (e.g. unbalanced disks), and correcting this problem may resolve the problem issue.

If the disk analysis does not identify an obvious system problem that is causing the I/O overhead, you could try making a disk upgrade or a reconfiguration. This type of tuning can consist of any of the following:
• Upgrading to faster disks
• Adding more swap space to handle larger buffers
• Changing the disk to be striped (where files are striped across several disks, thus providing parallel I/O. e.g. with a RAID system)
• Running the data on raw partitions when this is shown to be faster.
• Distributing simultaneously accessed files across multiple disks to gain parallel I/O
• Using memory-mapped disks or files

If you have applications that run on many systems and you do not know the specification of the target system, bear in mind that you can never be sure that ant particular disk is local to the user. There is a significant possibility that the disk being used by the application is a network-mounted disk. This doubles the variability in response times and throughput. The weakest link will probably not even be constant. A network disk is a shared resource, as is the network itself, so performance is hugely and unpredictably affected by other users and network load.


Disk I/O
Do not underestimate the impact of disk writes on the system as a whole. For example, all database vendors strongly recommend that the system swap files be placed on a separate disk from their databases. The impact of if not doing so can decrease database throughput (and system activity) but an order of magnitude. This performance decreases come from not splitting I/O of two disk-intensive applications (in this case, OS paging and database I/O).

Identifying that there is an I/O problem is usually fairly easy. The most basic symptom is that things take longer than expected, while at the same time the CPU is not at all heavily worked. The disk-monitoring utilities will also tell you that there is a lot of work being done to the disks. At the system level, you should determine the average peak requirements on the disks. Your disks will have some statistics that are supplied by the vendor, including:

The average and peak transfer rates, normally in megabytes (MB) per seconds, e.g. 5MB/sec. Form this, you can calculate how long an 8K page takes to be transferred from disk, and for example, 5MB/sec is about 5K/ms, so an 8K page takes just under 2ms to transfer.

Average seek time, normally in milliseconds (ms). This is the time required for the disk head to move radially to the correct location on the disk.

Rotational speed, normally in revolutions per minutes (rpm), e.g. 7200rpm. From this, you can calculate the average rotational delay in moving the disk under the disk-head reader, i.e., the time taken for half a revolution. For example, for 7200rpm, one revolution takes 60,000ms (60 seconds) divided by 7200rpm, which is about 8.3 ms. So half a revolution takes just over 4ms, which is consequently the average rotational delay.

This list allows you to calculate the actual time it takes to load a random 8K page from the disk, this being seek time + rotational delay + transfer time. Using the examples given in the list, you have 10 + 4 + 2 = 16 ms to load a random 8K page (almost an order of magnitude slower than the raw disk throughput). This calculation gives you a worst –case scenario for the disk-transfer rates for your application, allowing you to determine if the system is up to the required performance. Note that if you are reading data stored sequentially in disk (as when reading a large file), the seek time and rotational delay are incurred less than once per 8K page loaded. Basically, these two times are incurred only at the beginning of opening the file and whenever the file is fragmented. But this calculation is confounded by other processes also executing I/O to the disk at the same time. This overhead is part of the reason why swap and other intensive I/O files should not be put on the same disk.

One mechanism for speeding up disk I/O is to stripe disks. Disk striping allows data from a particular file to be spread over several disks. Striping allows reads and writes to be performed in parallel across the disks without requiring any application changes. This can speed up disk I/O quite effectively. However, be aware that the seek and rotational overhead previously listed still applies, and if you are making many small random reads, there may be no performance gain from striping disks.

Finally, note again that using remote disks adversely affects I/O performance. You should not be using remote disks mounted from the network with any I/O-intensive operations if you need good performance.


Clustering Files

Reading many files sequentially is faster if the files are clustered together on the disk, allowing the disk-head reader to flow from one file to the next. This clustering is best done in conjunction with defragmenting the disks. The overhead in finding the location of a file on the disk (detailed in the previous section) is also minimized for sequential reads if the files are clustered.

If you cannot specify clustering files at the disk level, you can still provide similar functionality by putting all the files together into one large file (as is done with the ZIP file systems). This is fine if all the files are read-only files or if there is just one file that is writable (you place that at the end). However, when there is more than one writable file, you need to manage the location of the internal files in your system as one or more grow. This becomes a problem and is not usually worth the effort. (If the files have a known bounded size, you can pad the files internally, thus regaining the single file efficiency.)
Cached File Systems (RAM Disks, tmpfs, cachefs)

Most OS provide the ability to map a file system into the system memory . This ability can speed up reads and writes to certain files in which you control your target environment. Typically, this technique has been used to speed up the reading and writing of temporary files. For example, some compilers (of languages in general, not specifically Java) generate many temporary files during compilation. If these files are created and written directly to the system memory, the speed of compilation is greatly increased. Similarly, if you have the a set of external files that are needed by your application, it is possible to map these directly into the system memory, thus allowing their reads and writes to be speeded up greatly.

But note that these types of file systems are not persistent. In the same way the system memory of the machine gets cleared when it is rebooted, so these file systems are removed on reboot. If the system crashes, anything in a memory-mapped file system is lost. For this reason, these types of file systems are usually suitable only for temporary files or read-only versions of disk-based files (such as mapping a CD-ROM into a memory-resident file system).

Remember that you do not have the same degree of fine control over these file systems that you have over your application. A memory-mapped file system does not use memory resources as efficiently as working directly from your application. If you have direct control over the files you are reading and writing, it is usually better to optimize this within your application rather than outside it. A memory-mapped file system takes space directly from system memory. You should consider whether it would be better to let your application grow in memory instead of letting the file system take up that system memory. For multi-user applications, it is usually more efficient for the system to map shared files directly into memory, as a particular fule then takes up just one memory location rather than duplicate in each process. Note that from SDK 1.4, memory-mapped files are directly supported from the java.nio package. Memory-mapped files are slightly different from memory-mapped file systems. A memory-mapped file uses system resources to read the file into system memory, and that data can then be accessed form Java through the appropriate java.nio buffer. A memory-mapped file system does not require the java.nio package and, as far as Java is concerned, files in that file system are simply files like any others. The OS transparently handles the memory mapping.

The creation of memory-mapped file systems is completely system-dependent, and there is no guarantee that it is available on any particular system (though most modern OS do support this feature). On UNIX system, the administrator needs to look at the documentation of the mount command and its subsections on cachefs and tmpfs. Under Windows, you should find details by looking at the documentation on how to setup a RAM disk, a portion of memory mapped to a logical disk drive.

In a similar way, there are products available that pre-cache shared libraries (DLL) and even executables in memory. This usually means only that an application starts quicker or loads the quicker, and so may not be much help in speeding up a running system.

But you can apply the technique of memory-mapping file systems directly and quite usefully for applications in which processes are frequently started. Copy the Java distribution and all class files (all JDK, application, and third-party class files) onto a memory-mapped file system and ensure that all executions and classloads take place from the file system. Since everything (executables, DLLs, class files, resources, etc.) is already in memory, the startup time is much faster. Because only the startup (and class loading) time is affected, this technique gives only a small boost to applications that are not frequently starting processes, but can be usefully applied if startup time is a problem.

Disk Fragmentation

When files are stored on disk, the bytes in the files are note necessarily stored contiguously: their storage depends on file size and contiguous space available on the disk. This non-contiguous storage is called fragmentation. Any particular file may have some chunks in one place, and a pointer to the next chunk that may be quite a distance away on the disk.

Hard disks tend to get fragmented over time. This fragmentation delays both reads from files (including loading applications into computer memory on startup) and writes to files. This delay occurs because the disk header must wind on to the next chunk with each fragmentation, and this takes time.

For optimum performance on any system, it is a good idea to periodically defragment the disk. This reunites files that have been split up so that disk heads do not spend so much time searching for data once the file-header locations have been identified, thus speeding up data access. Defragmenting may not be effective on all systems, however.
Disk Sweet Spots

Most disks have a location from which data is transferred faster than from other locations. Usually, the closer the data is to the outside edge of the disk, the faster it can be read from the disk. Most hard disks rotate at constant angular speed. This means that the linear speed of the disk under a point is faster the farther away the point is from the center of the disk. Thus, data at the edge of the disk can be read from (and written to) at the faster possible rate commensurate with the maximum density of data storable on disk.

This location with faster transfer rates usually termed the disk sweet spot. Some
(Commercial) utilities provide mapped access to the underlying disk and allow you to reorganize files to optimize access. On most server systems, the administrator has control over how logical partitions of the disk apply to the physical layout, and how to position files to the disk sweet spots. Experts for high-performance database system sometimes try to position the index tables of the database as close as possible to the disk sweet spot. These tables consist of relatively small amounts of data that affect the performance of the system in a disproportionately large way, so that any speed improvement in manipulating these tables is significant.

Note that some of the latest OS are beginning to include “awareness” of disk sweet spots, and attempt to move executables to sweet spots when defragmenting the disk. You may need to ensure that the defragmentation procedure does not disrupt your own use of the disk sweet spot.
The above is taken from the publication, "Java Performance Tuning" written by Jack Shirazi. I would recommend to read this book as it provides not just tuning and bottleneck concepts bounded by Java.
Basics: Disks Bottlenecks

In most cases, applications can be tuned so that disk I/O does not cause any serous performance problems. But if, application tuning, you find that disk I/O s still causing a performance problem; your best bet may be to upgrade the system disks. Identifying whether the system has a problem with disk utilization is the first step. Each system provides its own tools to identify disk usage (Windows has a performance monitor, and UNIX has the sar, vmstat, iostat utilities.) At minimum, you need to identify whether the paging is an issue (look at disk-scan rates) and assess the overall utilization of your disks (e.g. performance monitor on Windows, output from iostat –D on UNIX). It may be that the system has a problem independent of your application (e.g. unbalanced disks), and correcting this problem may resolve the problem issue.

If the disk analysis does not identify an obvious system problem that is causing the I/O overhead, you could try making a disk upgrade or a reconfiguration. This type of tuning can consist of any of the following:
• Upgrading to faster disks
• Adding more swap space to handle larger buffers
• Changing the disk to be striped (where files are striped across several disks, thus providing parallel I/O. e.g. with a RAID system)
• Running the data on raw partitions when this is shown to be faster.
• Distributing simultaneously accessed files across multiple disks to gain parallel I/O
• Using memory-mapped disks or files

If you have applications that run on many systems and you do not know the specification of the target system, bear in mind that you can never be sure that ant particular disk is local to the user. There is a significant possibility that the disk being used by the application is a network-mounted disk. This doubles the variability in response times and throughput. The weakest link will probably not even be constant. A network disk is a shared resource, as is the network itself, so performance is hugely and unpredictably affected by other users and network load.


Disk I/O
Do not underestimate the impact of disk writes on the system as a whole. For example, all database vendors strongly recommend that the system swap files be placed on a separate disk from their databases. The impact of if not doing so can decrease database throughput (and system activity) but an order of magnitude. This performance decreases come from not splitting I/O of two disk-intensive applications (in this case, OS paging and database I/O).

Identifying that there is an I/O problem is usually fairly easy. The most basic symptom is that things take longer than expected, while at the same time the CPU is not at all heavily worked. The disk-monitoring utilities will also tell you that there is a lot of work being done to the disks. At the system level, you should determine the average peak requirements on the disks. Your disks will have some statistics that are supplied by the vendor, including:

The average and peak transfer rates, normally in megabytes (MB) per seconds, e.g. 5MB/sec. Form this, you can calculate how long an 8K page takes to be transferred from disk, and for example, 5MB/sec is about 5K/ms, so an 8K page takes just under 2ms to transfer.

Average seek time, normally in milliseconds (ms). This is the time required for the disk head to move radially to the correct location on the disk.

Rotational speed, normally in revolutions per minutes (rpm), e.g. 7200rpm. From this, you can calculate the average rotational delay in moving the disk under the disk-head reader, i.e., the time taken for half a revolution. For example, for 7200rpm, one revolution takes 60,000ms (60 seconds) divided by 7200rpm, which is about 8.3 ms. So half a revolution takes just over 4ms, which is consequently the average rotational delay.

This list allows you to calculate the actual time it takes to load a random 8K page from the disk, this being seek time + rotational delay + transfer time. Using the examples given in the list, you have 10 + 4 + 2 = 16 ms to load a random 8K page (almost an order of magnitude slower than the raw disk throughput). This calculation gives you a worst –case scenario for the disk-transfer rates for your application, allowing you to determine if the system is up to the required performance. Note that if you are reading data stored sequentially in disk (as when reading a large file), the seek time and rotational delay are incurred less than once per 8K page loaded. Basically, these two times are incurred only at the beginning of opening the file and whenever the file is fragmented. But this calculation is confounded by other processes also executing I/O to the disk at the same time. This overhead is part of the reason why swap and other intensive I/O files should not be put on the same disk.

One mechanism for speeding up disk I/O is to stripe disks. Disk striping allows data from a particular file to be spread over several disks. Striping allows reads and writes to be performed in parallel across the disks without requiring any application changes. This can speed up disk I/O quite effectively. However, be aware that the seek and rotational overhead previously listed still applies, and if you are making many small random reads, there may be no performance gain from striping disks.

Finally, note again that using remote disks adversely affects I/O performance. You should not be using remote disks mounted from the network with any I/O-intensive operations if you need good performance.


Clustering Files

Reading many files sequentially is faster if the files are clustered together on the disk, allowing the disk-head reader to flow from one file to the next. This clustering is best done in conjunction with defragmenting the disks. The overhead in finding the location of a file on the disk (detailed in the previous section) is also minimized for sequential reads if the files are clustered.

If you cannot specify clustering files at the disk level, you can still provide similar functionality by putting all the files together into one large file (as is done with the ZIP file systems). This is fine if all the files are read-only files or if there is just one file that is writable (you place that at the end). However, when there is more than one writable file, you need to manage the location of the internal files in your system as one or more grow. This becomes a problem and is not usually worth the effort. (If the files have a known bounded size, you can pad the files internally, thus regaining the single file efficiency.)
Cached File Systems (RAM Disks, tmpfs, cachefs)

Most OS provide the ability to map a file system into the system memory . This ability can speed up reads and writes to certain files in which you control your target environment. Typically, this technique has been used to speed up the reading and writing of temporary files. For example, some compilers (of languages in general, not specifically Java) generate many temporary files during compilation. If these files are created and written directly to the system memory, the speed of compilation is greatly increased. Similarly, if you have the a set of external files that are needed by your application, it is possible to map these directly into the system memory, thus allowing their reads and writes to be speeded up greatly.

But note that these types of file systems are not persistent. In the same way the system memory of the machine gets cleared when it is rebooted, so these file systems are removed on reboot. If the system crashes, anything in a memory-mapped file system is lost. For this reason, these types of file systems are usually suitable only for temporary files or read-only versions of disk-based files (such as mapping a CD-ROM into a memory-resident file system).

Remember that you do not have the same degree of fine control over these file systems that you have over your application. A memory-mapped file system does not use memory resources as efficiently as working directly from your application. If you have direct control over the files you are reading and writing, it is usually better to optimize this within your application rather than outside it. A memory-mapped file system takes space directly from system memory. You should consider whether it would be better to let your application grow in memory instead of letting the file system take up that system memory. For multi-user applications, it is usually more efficient for the system to map shared files directly into memory, as a particular fule then takes up just one memory location rather than duplicate in each process. Note that from SDK 1.4, memory-mapped files are directly supported from the java.nio package. Memory-mapped files are slightly different from memory-mapped file systems. A memory-mapped file uses system resources to read the file into system memory, and that data can then be accessed form Java through the appropriate java.nio buffer. A memory-mapped file system does not require the java.nio package and, as far as Java is concerned, files in that file system are simply files like any others. The OS transparently handles the memory mapping.

The creation of memory-mapped file systems is completely system-dependent, and there is no guarantee that it is available on any particular system (though most modern OS do support this feature). On UNIX system, the administrator needs to look at the documentation of the mount command and its subsections on cachefs and tmpfs. Under Windows, you should find details by looking at the documentation on how to setup a RAM disk, a portion of memory mapped to a logical disk drive.

In a similar way, there are products available that pre-cache shared libraries (DLL) and even executables in memory. This usually means only that an application starts quicker or loads the quicker, and so may not be much help in speeding up a running system.

But you can apply the technique of memory-mapping file systems directly and quite usefully for applications in which processes are frequently started. Copy the Java distribution and all class files (all JDK, application, and third-party class files) onto a memory-mapped file system and ensure that all executions and classloads take place from the file system. Since everything (executables, DLLs, class files, resources, etc.) is already in memory, the startup time is much faster. Because only the startup (and class loading) time is affected, this technique gives only a small boost to applications that are not frequently starting processes, but can be usefully applied if startup time is a problem.

Disk Fragmentation

When files are stored on disk, the bytes in the files are note necessarily stored contiguously: their storage depends on file size and contiguous space available on the disk. This non-contiguous storage is called fragmentation. Any particular file may have some chunks in one place, and a pointer to the next chunk that may be quite a distance away on the disk.

Hard disks tend to get fragmented over time. This fragmentation delays both reads from files (including loading applications into computer memory on startup) and writes to files. This delay occurs because the disk header must wind on to the next chunk with each fragmentation, and this takes time.

For optimum performance on any system, it is a good idea to periodically defragment the disk. This reunites files that have been split up so that disk heads do not spend so much time searching for data once the file-header locations have been identified, thus speeding up data access. Defragmenting may not be effective on all systems, however.
Disk Sweet Spots

Most disks have a location from which data is transferred faster than from other locations. Usually, the closer the data is to the outside edge of the disk, the faster it can be read from the disk. Most hard disks rotate at constant angular speed. This means that the linear speed of the disk under a point is faster the farther away the point is from the center of the disk. Thus, data at the edge of the disk can be read from (and written to) at the faster possible rate commensurate with the maximum density of data storable on disk.

This location with faster transfer rates usually termed the disk sweet spot. Some
(Commercial) utilities provide mapped access to the underlying disk and allow you to reorganize files to optimize access. On most server systems, the administrator has control over how logical partitions of the disk apply to the physical layout, and how to position files to the disk sweet spots. Experts for high-performance database system sometimes try to position the index tables of the database as close as possible to the disk sweet spot. These tables consist of relatively small amounts of data that affect the performance of the system in a disproportionately large way, so that any speed improvement in manipulating these tables is significant.

Note that some of the latest OS are beginning to include “awareness” of disk sweet spots, and attempt to move executables to sweet spots when defragmenting the disk. You may need to ensure that the defragmentation procedure does not disrupt your own use of the disk sweet spot.
The above is taken from the publication, "Java Performance Tuning" written by Jack Shirazi. I would recommend to read this book as it provides not just tuning and bottleneck concepts bounded by Java.

I would also recommend visiting the site author by Jack himself and a couple of his mates. It is a very resourceful site for Java performance-related information. Click here to access it.
Basics: Memory Bottlenecks
Maintaining watch directly on the system memory (RAM) is not usually that helpful in identifying performance problems. A better indication that memory might be affecting performance can be gained by watching for paging of data from memory to the swap files. Most current OS have a virtual memory that is made up of the actual (real) system memory using RAM chips, and one or more swap files on the system disks. Processes that are currently running are operating in real memory. The OS can take pages from any of the processes currently in real memory and swap them out to disk. This is known as paging. Paging leaves free space in real memory to allocate to other processes that need to bring in a page from disk.

Obviously, if all the processes currently running can fit into real memory, there is no need for the system to swap out any pages. However, if there are too many processes to fit into real memory, paging allows the system to free up system memory to run more processes. Paging affects system performance in many ways. One obvious way is that if a process has had some pages moved to disk and the process becomes run-able, the OS has to pull back the pages from dusk before that process can be run. This leads to delays in performance. In addition, both CPU and the disk I/O spend time doing the paging, reducing available processing power and increasing the load on the disks. This cascading effect involving both the CPU and I/O can degrade the performance of the whole system in such a way that it maybe difficult to even recognize that paging is the problem. The extreme version of too much paging is thrashing, in which the system is spending so much time moving pages around that it fails to perform any other significant work. (The next step is likely to be a system crash).

As with run-able queues (see CPU section), a little paging of the system does not affect the performance enough to cause concern. In fact, some paging can be considered good. It indicated that the system’s memory resources are fully utilized. But at the point where paging becomes a significant overhead, the system is overloaded.

Monitoring paging is relatively easy. On UNIX, the utilities vmstat and iostat provide details as to the level of paging, disk activity and memory levels. On Windows, the performance monitor has categories to show these details, as well as being able to monitor the system swap files.

If there is more paging than is optimal, the system’s RAM is insufficient or processes are too big. To improve this situation, you need to reduce the memory being used by reducing the number of processes or the memory utilization of some processes. Alternatively, you can add RAM. Assuming that it is your application that is causing the paging (Otherwise, either the system needs an upgrade, or someone else’s processes may also have to be tuned), you need to reduce the memory resources you are using.

When the problem is caused by a combination of your application and others, you can partially address the situation by using process priorities (see the CPU “section”). The equivalent to priority levels for memory usage is an all-or-nothing option, where you can lock process in memory. This option is not available on all systems and is more often applied to shared memory than to processes, but nevertheless, it is useful to know. If this option is applied, the process is locked into real memory and is not paged out at all. You need to be aware that using this option reduces the amount of RAM available to all other processes, which can make overall system performance worse. Any deterioration in system performance is likely occurring at heavy system load, so make sure you extrapolate the effect of reducing the system memory in this way.

Basics: Network Bottlenecks
At the network level, many things can affect performance. The bandwidth (the amount of data that can be carried by the network) tends to be the first culprit checked. Assuming you have determined that bad performance is attributable to the network component of an application, there is more likely cause of bad network performance than network bandwidth. The most likely cause of bad network performance is the application itself and how it is handling distributed data and functionality.
The overall speed of a particular network connection is limited by the slowest link in the connection chain and the length of the chain. Identifying the slowest link is difficult and may not even be consistent: it can vary at different times of the day or for different communication paths. A network communication path lead from an application through a TCP/IP stack (which adds various layers of headers, possibly encrypting and compressing data as well), then through the hardware interface, through a modem, over a phone line, through another modem, over to a service provider’s router, through many heavily congested data lines of various carrying capacities and multiple routers with different maximum throughputs and configurations, to a machine at the other end with its own hard interface, TCP/IP stack and application. A typical web download route is just like this. In addition, there are dropped packets, acknowledgements, retries, bus contention, and so on.
Because so many possibilities causes of bad network performance are external to an application, one option you can consider including in an application is a network speed testing facility that reports to the user. This should test the speed of data transfer from the machine to various destinations: to itself, to another machine on the local network, to the Internet Service Provider, to the target server across the network, and to any other destinations appropriate. This type of diagnostics report can tell users that they are obtaining bad performance from something other than your application. If you feel that the performance of your application is limited by the actual network communication speed, and not by other (application) factors, this facility will report the maximum possible speeds to youruser.


Latency
Latency is different from the load-carrying capacity (bandwidth) of a network. Bandwidth refers to how much data can be sent down the communication channel for a given period of time and is limited by the link in the communication chain that has the lowest bandwidth. The latency is the amount of time a particular data packet takes to get from one end of the communication channel to the other. Bandwidth tells you the limits within which your application can operate before the performance become affected by the volume of data being transmitted. Latency often affects the user’s view of the performance even when bandwidth isn’t a problem.
In most cases, especially Internet traffic, latency is an important concern. You can determine the basic round-trip time for a data packets from any two machines using the ping utility. This utility provides a measure of the time it takes a packet of data to reach another machine and be returned. However, the time measure is for a basic underlying protocol (ICMP packet) to travel between the machines. If the communication channel is congested and the overlying protocol requires re-transmissions (often the case for Internet traffic), one transmission at the application level can actually be equivalent to many round trips.
It is important to be aware of these limitations. It is often possible to tune the application to minimize the number of transfers by packing data together, caching and redesigning the distributed application protocol to aim for a less conversational mode of operation. At the network level, you need to monitor the transmission statistics (using the ping and netstat utilities and packet sniffers) and consider tuning any network parameters that you have access to in order to reduce re-transmissions.
TCP/IP Stacks
The TCP/IP stack is the section of code that is responsible for translating each application-level network request (send, receive, connect, etc.) through the transport layers down to the wire and back up to the application at the other end of the connection. Because the stacks are usually delivered with the operation system and performance-tested before delivery (since a slow network connection on an otherwise fast machine and fast network is pretty obvious), it is unlikely that the TCP/IP stack itself is a performance problem.
In addition to the stack itself, stacks include several tunable parameters. Most of these parameters deal with transmission details beyond the scope of the book. One parameter worth mentioning is the maximum packet size. When your application sends data, the underlying protocol breaks the data into packets that are transmitted. There is an optimal size for packets transmitted over a particular communication channel, and the packet size actually used by the stack is compromise. Smaller packets are less likely to be dropped, but they introduced more overhead, as data probably has to be broken up into more packets with more header overhead.
If your communication takes place over a particular set of endpoints, you may want to alter the packet sizes. For a LAN segment with no router involved, the packets can be big (e.g. 8KB). For a LAN with routers, you probably want to set the maximum packet size to the size the routers allow to pass unbroken. (Routers can break up the packets into smaller ones; 1500 bytes is the typical maximum packet size and the standard for the Ethernet. The maximum packet size is configurable by the router’s network administrator.) If your application is likely to be sending data over the Internet and you cannot guarantee the route and quality of routers it will pass through, 500 bytes per packet is likely to be optimal.
Network Bottlenecks
Other causes of slow network I/O can be attributed directly to the load or configuration of the network. For example, a LAN may become congested when many machines are simultaneously trying to communicate over the network. The potential throughput of the network could handle the load, but the algorithms to provide communication channels slow the network, resulting in a lower maximum throughput. A congested Ethernet network has an average throughput approximately one third the potential maximum throughputs. Congested networks have other problems, such as dropped network packets. If you are using TCP, the communication rate on a congested network is much slower as the protocol automatically resends the dropped packets. If you are using UDP, your application must resend multiple copies for each transfer. Dropping packets in this way is common for the Internet. For LANs, you need to coordinate closely with network administrators to alert them to the problem. For single machines connected by a service provider, suggesting improvements. The phone line to the service provider may be noisier than expected: if so, you also need to speak to the phone line provider. It is also worth checking with the service provider, who should have optimal configurations they can demonstrate.
Dropped packets and re-transmissions are a good indication of network congestion problems, and you should be on constant lookup for them. Dropped packets often occur when routers are overloaded and find it necessary to drop some of the packets being transmitted as the router’s buffer overflow. This means that the overlying protocol will request the packets to be resent. The netstat utility lists re-transmission and other statistics that can identify these sorts of problems. Re-transmissions may indicate that the maximum packet size is too large.
DNS Lookup
Looking up network address is an often-overlooked cause of bad network performance. When your application tries to connect to a network address such as foo.bar.somthing.org (e.g. downloading a webpage from http://foo.bar.something.org), your application first translates foo.bar.somthing.org into a four-byte network IP address such as 10.33.6.45. This is the actual address that the network understands and uses for routing network packets. The is this translation works is that your system is configured with some seldom-used files that can specify this translation, and a more frequently used Domain Name System (DNS) server that can dynamically provide you with the address from the given string. DBS translation works as follows:
1. The machine running the application sends the text string of the hostname (e.g. foo.bar.something.org) to the DNS server.
2. The DNS server checks its cache to find an IP address corresponding to that hostname. If the server does not find an entry in the cache, it asks its own DNS server (usually further up the Internet domain-name hierarchy) until ultimately the name is resolved. (This may be by components of the name being resolved, e.g. first .org, then something.org, etc, each time asking another machine as the search request is successively resolved.) This resolved IP address is added to the DBS server’s cache.
3. The IP address s returned to the original machine running the application.
4. The application uses the IP address to connect to the desired destination.
The address lookup does not need to be repeated once a connection is established, but any other connections (within the same session of the application or in other session s at the same time and later) need to repeat the lookup procedure to start another connection.
You can improve this situation by running a DNS server locally on the machine, or on a local server if the application uses a LAN. A DNS server can be run as a “caching-only” server that resets its cache each time the machine is rebooted. There would be little point in doing this if the machine used only one or two connections per hostname between successive reboots. For more frequent connections, a local DNS server can provide a noticeable speedup to connections. Nslookup is useful for investigating how a particular system does translations.

Important Baiscs in performance Testing -2

Basics: Max. Running Vusers in Analysis is different from Vuser Quantity in Controller?
There are couple of times when an individual starts to pick up LoadRunner ask this question. I've written the explanation here for the convenience of the newbies which I had previously responded in the Yahoo! group discussion.


Max. Running Vuser

It defines at any point of point of time, the maximum number of vusers running together concurrently (in Run state). This is the "state" or usually the requirement of a load test to reach "X" number of concurrent users. If a load test were to required to run 100 concurrent users, then the Max. Running Vuser must be 100. This is different from Vuser Quantity explained in the following.


Vuser Quantity

In the Controller, the Vuser Quantity is the total number of participating in the load test but it is different from Max. Running Vuser which is explained above. To view the total number of Vuser participating the load test (Vuser Quantity), open the Vuser Summary graph. Please take note that the Summary Report in the Analysis session displays the maximum number of vusers running in the scenario and not the total vuser particiapting.

Importanat Basics of Performance Testing-1

Basics: Transaction Response TIme
What is Transaction Response Time?

Transaction Response Time represents the time taken for the application to complete a defined transaction or business process.


Why is important to measure Transaction Response Time?

The objective of a performance test is to ensure that the application is working perfectly under load. However, the definition of “perfectly” under load may vary with different systems.
By defining an initial acceptable response time, we can benchmark the application if it is performing as anticipated.

The importance of Transaction Response Time is that it gives the project team/ application team an idea of how the application is performing in the measurement of time. With this information, they can relate to the users/customers on the expected time when processing request or understanding how their application performed.


What does Transaction Response Time encompass?

The Transaction Response Time encompasses the time taken for the request made to the web server, there after being process by the Web Server and sent to the Application Server. Which in most instances will make a request to the Database Server. All this will then be repeated again backward from the Database Server, Application Server, Web Server and back to the user. Take note that the time taken for the request or data in the network transmission is also factored in.

To simplify, the Transaction Response Time comprises of the following:
1. Processing time on Web Server
2. Processing time on Application Server
3. Processing time on Database Server
4. Network latency between the servers, and the client
The following diagram illustrates Transaction Response Time.



Figure 1 Transaction Response Time = (t1 + t2 + t3 + t4 + t5 + t6 + t7 + t8 + t9) X 2

Note:
Factoring the time taken for the data to return to the client.


How do we measure?
Measuring of the Transaction Response Time begins when the defined transaction makes a request to the application. From here, till the transaction completes before proceeding with the next subsequent request (in terms of transaction), the time is been measured and will stop when the transaction completes.

Differences with Hits Per Seconds
Hits per Seconds measures the number of “hits” made to a web server. These “hits” could be a request made to the web server for data or graphics. However, this counter does not represent well to users on how well their application is performing as it measures the number of times the web server is being accessed.
How can we use Transaction Response Time to analyze performance issue?
Transaction Response Time allows us to identify abnormalities when performance issues surface. This will be represented as slow response of the transaction, which differs significantly (or slightly) from the average of the Transaction Response Time.
With this, we can further drill down by correlation using other measurements such as the number of virtual users that is accessing the application at the point of time and the system-related metrics (e.g. CPU Utilization) to identify the root cause.
Bringing all the data that have been collected during the load test, we can correlate the measurements to find trends and bottlenecks between the response time, the amount of load that was generated and the payload of all the components of the application.

How is it beneficial to the Project Team?
Using Transaction Response Time, Project Team can better relate to their users using transactions as a form of language protocol that their users can comprehend. Users will be able to know that transactions (or business processes) are performing at an acceptable level in terms of time.
Users may be unable to understand the meaning of CPU utilization or Memory usage and thus using a common language of time is ideal to convey performance-related issues.

My Favourite QTP Blog

Yes, It's absolutely my favourite blog on QTP.

Visit here, You may be interested this one

Quality Center Interview Questions and Answers

1. Why is TestDirector used for?

TestDirector is a test management tool. The completely web-enabled TestDirector supports high level of communication and association among various testing teams, driving a more effective and efficient global application-testing process. One can also create reports and graphs to help review the progress of planning tests, executing tests, and tracking defects before a software release.

2. Why are the requirements linked to the test cases?

TestDirector connects requirements directly to test cases, ensuring that all the requirements have been covered by the test cases.

3. What are the benefits and features of TestDirector?

TestDirector incorporates all aspects of the testing process i.e. requirement management, test planning, test case management, scheduling, executing tests and defect management into a single browser-based application. It maps requirements directly to the test cases ensuring that all the requirements have been covered by the test cases. It can import requirements and test plans from excel sheet accelerating the testing process. It executes both manual and automated tests.

4. What is the use of filters in TD?

Filters in TestDirector are mainly used to filter out for the required results. It helps to customize and categorize the results. For eg: to quickly view the passed and failed tests separately filters are used.

5. What is Test Lab?

In the Test Lab the test cases are executed. Test Lab will always be linked to the test plan. Usually both are given the same name for easy recognition.

6. How to customize the defect management cycle in Quality Center?

Firstly one should collect all the attributes that has to be part of the defect management like version, defect origin, defect details, etc. Later using the modify options in QC one can change the defect module accordingly.

7. What is the advantage of writing test cases in Quality Center than writing in excel sheet?

Although creating test cases in excel sheet will be faster than doing it in QC as excel is more user friendly when compared to QC one require to upload them to QC and this process may cause some delay due to various reasons. Also QC provides link to other tests which in turn is mapped to the requirements.

8. What is the difference between TestDirector and Quality Center?

The main difference is QC is more secured than TestDirector. In Quality Center the login page shows projects associated only to the logged in user unlike in Test Director where one can see all the available projects. Also test management is much more improved in QC than TD. Defect linkage functionality in QC is more flexible when compared to TD.

9. What is meant by Instance?

Test instance is an instance of test case in Test Lab. Basically it means the test case which you have imported in Test lab for execution.

10. What is the use of requirement option in TestDirector?

Requirement module in TD is used for writing the requirements and preparing the traceability matrix.

11. Is it possible to maintain test data in TestDirector?

Yes one can attach the test data to the corresponding test cases or create a separate folder in test plan to store them.

12. If one tries to upgrade from TestDirector 7.2 to QC 8.0 then is there risk of losing any data?

No there is no risk of losing the data during the migration process. One has to follow proper steps for successful migration.

13. How is a bug closed in TestDirector?

Once the test cases are executed in the Test Lab and bugs are detected, it is logged as a defect using the Defect Report Tab and sent to the developer. The bug will have 5 different status namely New, Open, Rejected, Deferred and Closed. Once the bug has been fixed and verified its status is changed to closed. This way the bug lifecycle ends.

14. In TD how are the test cases divided into different groups?

In the test plan of TestDirector one can create separate folder for various modules depending on the project. A main module in the test plan can be created and then sub modules be added to that.

15. What is the difference between TD and Bugzilla?

TestDirector is a test management tool. In TD one can write manual and automated test cases, add requirements, map requirements to the test cases and log defects. Bugzilla is used only for logging and tracking the defects.

16. Are TestDirector and QC one and the same?

Yes TestDirector and Quality Center are same. Version of TD 8.2 onwards was known as Quality Center. The latest version of Quality Center is 9.2. QC is much more advanced when compared to TD.

17. What is the instance of the test case inside the Test Set?

Test set is a place containing sets of test cases. We can store many test cases inside test set. Now instance of test case is the test case which you have imported in the test tab. If an another test case has the same steps as this test case till half way then you can create the instance of this test case.

18. What are the various types of reports in TestDirector?

In TD reports are available for requirements, test cases, test execution, defects, etc. The reports give various details like summary, progress, coverage, etc. Reports can be generated from each TestDirector module using the default settings or it can be customized. When customizing a report, filters and sort conditions can be applied and the required layout of the fields in the report can be specified. Sub-reports can also be added to the main report. The settings of the reports can be saved as favorite views and reloaded as required.

19. How can one map a single defect to more than one test script?

Using the 'associate defect' option in TestDirector one can map the same defect to a number of test cases.

20. Is it possible to create custom defect template in TestDirector?

It is not possible to create ones own template for defect reporting in TestDirector but one can customize the template that is already available in TestDirector as required.

21. Can a script in TD be created before recording script in Winrunner or QTP?

Any automation script can be created directly in TD. You need to open the tool (Winrunner or QTP) and then connect to TD by specifying the url, project, domain, userid and password. And then you just record the script like you always do. When you save the script, you can save it in TD instead of your local system.

22. How to ensure that there is no duplication of bugs in TestDirector?

In the defect tracking window of TD there is a “find similar defect” icon. When this icon is clicked after writing the defect, if anybody else has entered the same defect then it points it out.

23. How is the Defect ID generated in TestDirector?

The Defect ID is automatically generated once the defect is submitted in TD.

24. What does the test grid contain?

The test grid displays all the relevant tests related to a project in TD. It contains the some key elements like test grid toolbar with various buttons for commands which are commonly used when creating and modifying the tests, grid filter which displays the filter that is currently applied to a column, description tab which displays a description of the selected test in the test grid and history tab that displays the changes made to a test.

25. What are the 3 views in TD?

The three views in TD are Plan Test which is used to prepare a set of test cases as per the requirements, Run Test which is used for executing the prepared test scripts with respect to the test cases and finally Track Defects which is used by the test engineers for logging the defects.

26. How to upload data from an excel sheet to TestDirector?

In order to upload data from excel sheet to TD firstly excel addin has to be installed, then the rows in the excel sheet which has to be imported to TD should be selected, and then finally the Export to TD option in the tools menu of TestDirector should be selected.

27. How many types of tabs are available in TestDirector?

There are 4 types of tabs available in TestDirector. They are Requirement, Test Plan, Test Lab and Defect. It is possible to customize the names of these tabs as desired.

28. Is 'Not covered' and 'Not run' status the same?

Not Covered status means all those requirements for which the test cases are not written whereas Not Run status means all those requirements for which test cases are written but are not run.

29. How does TestDirector store data?

In TD data is stored on the server.

30. Why should we create an Instance?

Test Instance is used to run the test case in the test lab. It is the test instance that you can run since you can't run the test case in test set.


Set 2 :

1. What is meant by test lab in Quality Centre?
Test lab is a part of Quality Centre where we can execute our test on different cycles creating test tree for each one of them. We need to add test to these test trees from the tests, which are placed under test plan in the project. Internally Quality Centre will refer to this test while running then in the test lab.

2. Can you map the defects directly to the requirements(Not through the test cases) in the Quality Centre?
In the following methods is most likely to used in this case:
Create your Req.Structure
Create the test case structure and the test cases
Map the test cases to the App.Req
Run and report bugs from your test cases in the test lab module.

The database structure in Quality Centre is mapping test cases to defects, only if you have created the bug from Application. test case may be we can update the mapping by using some code in the bug script module(from the customize project function), as per as i know, it is not possible to map defects directly to an requirements.

3. how do you run reports from Quality Centre. Does any one have good white paper or articles?
This is how you do it
1. Open the Quality Centre project
2. Displays the requirements modules
3. Choose report
Analysis > reports > standard requirements report

4. Can we upload test cases from an excel sheet into Quality Centre?
Yes go to Add-In menu Quality Centre, find the excel add-In, and install it in your machine.
Now open excel, you can find the new menu option export to Quality Centre. Rest of the procedure is self explanatory.

5. Can we export the file from Quality Centre to excel sheet. If yes then how?
Requirement tab– Right click on main req/click on export/save as word, excel or other template. This would save all the child requirements

Test plan tab: Only individual test can be exported. no parent child export is possible. Select a test script, click on the design steps tab, right click anywhere on the open window. Click on export and save as.

Test lab tab: Select a child group. Click on execution grid if it is not selected. Right click anywhere. Default save option is excel. But can be saved in documents and other formats. Select all or selected option

Defects Tab: Right click anywhere on the window, export all or selected defects and save excel sheet or document.

6. How many types of tabs are there in Quality Centre. Explain?
There are four types of tabs are available

1. Requirement : To track the customer requirements
2. Testplan : To design the test cases and to store the test scripts
3. test lab : To execute the test cases and track the results.
4. Defect : To log a defect and to track the logged defects.

7. How to map the requirements with test cases in Quality Centre?
1. In requirements tab select coverage view
2. Select requirement by clicking on parent/child or grandchild
3. On right hand side(In coverage view window) another window will appear. It has two tabs
a) Tests coverage
b) Details
Test coverage tab will be selected by default or you click on it.
4. Click on select tests button a new window will appear on right hand side and you will see a list of all tests. You cans elect any test case you want to map with your requirements.

8. How to use Quality Centre in real time project?
Once completed the preparing of test cases
1. Export the test cases into Quality Centre( It will contained total 8 steps)
2. The test cases will be loaded in the test plan module
3. Once the execution is started. We move the test cases from test plan tab to the test lab module.
4. In test lab, we execute the test cases and put as pass or fail or incomplete. We generate the graph in the test lab for daily report and sent to the on site (where ever you want to deliver)
5. If we got any defects and raise the defects in the defect module. when raising the defects, attach the defects with the screen shot.

9. Difference between Web Inspect-QA Inspect?
QA Inspect finds and prioritizes security vulnerabilities in an entire web application or in specific usage scenarios during testing and presents detail information and remediation advise about each vulnerability.
Web Inspect ensures the security of your most critical information by identifying known and unknown vulnerabilities within the web application. With web Inspect, auditors, compliance officers and security experts can perform security assessments on a web enabled application. Web inspect enables users to perform security assessments for any web application or web service, including the industry leading application platforms.

10. How can w add requirements to test cases in Quality Centre?
Just you can use the option of add requirements.
Two kinds of requirements are available in TD.
1. Parent Requirement
2. Child requirements.

Parent Requirements nothing but title of the requirements, it covers high level functions of the requirements

Child requirement nothing but sub title of requirements, it covers low level functions of the requirements.

Advanced Quality Center 9.2 course

Advanced Quality Center 9.2
INTENDED AUDIENCE
�� Quality Center Administrator
�� Quality Center Project managers
�� Quality Assurance Leads
�� Other Quality Center users responsible for managing projects, users, and workflow customizations
DURATION: 2 DAYS
OVERVIEW
This course covers the tasks that are performed by site administrators, project managers, and Quality Assurance (QA) leads to set up Quality Center (QC) projects before users can work with requirements, test cases, and defects. This hands-on course focuses on planning, creating, and customizing Quality Center projects. It is an important course for people involved in creating projects, modifying existing fields, creating user- defined fields, creating users, setting user permissions, and defining basic workflow.
COURSE OBJECTIVES
At the end of the course, you will be able to:
�� Identify the organization level specifications that need to be customized in Quality Center
�� Configure organization-wide Quality Center parameters
�� Customize project-specific Quality Center parameters •Maintain Quality Center projects
PREREQUISITES
Working knowledge of:
�� Windows
�� Testing concepts
INTRODUCTION TO QUALITY CENTER 9.2
• Identify the Quality Center modules.
PLANNING THE QUALITY CENTER IMPLEMENTATION
• Planning: considerations and key roles.
INTRODUCTION TO SITE ADMINISTRATION
• Logging on to Site Administration.
• Site Administrator components.
• Monitoring user connections and licenses.
CUSTOMIZING PROJECTS AT A GLOBAL LEVEL
• Project structure in Quality Center.
• Creating domains and projects.
MAINTAINING QUALITY CENTER PROJECTS
• Querying project tables.
• Copying, renaming, removing, backing up, restoring , and upgrading projects.
• Deleting Projects and Domains.
CUSTOMIZING USERS AT A GLOBAL LEVEL
• Creating and managing users.
• Configuring user authentication and LDAP import settings.
CUSTOMIZING OTHER QUALITY CENTER MODULES AT A GLOBAL LEVEL
• Configure mail parameters.
• Configure connection parameters.
CUSTOMIZING PROJECT-SPECIFIC ENTITIES
• An overview of Project Customization.
• An overview of customizing project-specific.
• Adding user-defined fields.
• Managing system or user-defined fields.
CUSTOMIZING LOOKUP LISTS
• Creating project-specific lists.
• Associating a list to a project entity.
• Managing project-specific lists.
MANAGING PROJECT-SPECIFIC USERS AND USER GROUPS
• Adding users to a project.
• Creating user groups.
• Customizing module access for user groups.
CUSTOMIZING MAILING OPTIONS AND ALERT RULES
• Designating mail fields.
• Defining mailing conditions.
• Configuring alert rules.
• Customizing Alert Rules.
• Configuring traceability notification rules.
CUSTOMIZING WORKFLOW
• Customizing lists.
• Customizing fields by user groups.
IMPLEMENTING WORKFLOW USING THE SCRIPT EDITOR
• Using the Script Editor.
• Adding a command button by using the Script Editor.