Bug Defect tracking

Sample bug report

Below sample bug/defect report will give you exact idea of how to report a bug in bug tracking tool.

Here is the example scenario that caused a bug:

Lets assume in your application under test you want to create a new user with user information, for that you need to logon into the application and navigate to USERS menu > New User, then enter all the details in the ‘User form’ like, First Name, Last Name, Age, Address, Phone etc. Once you enter all these information, you need to click on ‘SAVE’ button in order to save the user. Now you can see a success message saying, “New User has been created successfully”.

But when you entered into your application by logging in and navigated to USERS menu > New user, entered all the required information to create new user and clicked on SAVE button. BANG! The application crashed and you got one error page on screen. (Capture this error message window and save as a Microsoft paint file)

Now this is the bug scenario and you would like to report this as a BUG in your bug-tracking tool.

How will you report this bug effectively?

Here is the sample bug report for above mentioned example:
(Note that some ‘bug report’ fields might differ depending on your bug tracking system)

SAMPLE BUG REPORT:

Bug Name: Application crash on clicking the SAVE button while creating a new user.
Bug ID: (It will be automatically created by the BUG Tracking tool once you save this bug)
Area Path: USERS menu > New Users
Build Number: Version Number 5.0.1
Severity: HIGH (High/Medium/Low) or 1
Priority: HIGH (High/Medium/Low) or 1
Assigned to: Developer-X
Reported By: Your Name
Reported On: Date
Reason: Defect
Status: New/Open/Active (Depends on the Tool you are using)
Environment: Windows 2003/SQL Server 2005

Description:
Application crash on clicking the SAVE button while creating a new
user, hence unable to create a new user in the application.

Steps To Reproduce:
1) Logon into the application
2) Navigate to the Users Menu > New User
3) Filled all the user information fields
4) Clicked on ‘Save’ button
5) Seen an error page “ORA1090 Exception: Insert values Error…”
6) See the attached logs for more information (Attach more logs related to bug..IF any)
7) And also see the attached screenshot of the error page.

Expected result: On clicking SAVE button, should be prompted to a success message “New User has been created successfully”.

(Attach ‘application crash’ screen shot.. IF any)

Save the defect/bug in the BUG TRACKING TOOL.  You will get a bug id, which you can use for further bug reference.
Default ‘New bug’ mail will go to respective developer and the default module owner (Team leader or manager) for further action.

How to write a good bug report? Tips and Tricks

Why good Bug report?
If your bug report is effective, chances are higher that it will get fixed. So fixing a bug depends on how effectively you report it. Reporting a bug is nothing but a skill and I will tell you how to achieve this skill.

“The point of writing problem report(bug report) is to get bugs fixed” – By Cem Kaner. If tester is not reporting bug correctly, programmer will most likely reject this bug stating as irreproducible. This can hurt testers moral and some time ego also. (I suggest do not keep any type of ego. Ego’s like “I have reported bug correctly”, “I can reproduce it”, “Why he/she has rejected the bug?”, “It’s not my fault” etc etc..)

What are the qualities of a good software bug report?
Anyone can write a bug report. But not everyone can write a effective bug report. You should be able to distinguish between average bug report and a good bug report. How to distinguish a good or bad bug report? It’s simple, apply following characteristics and techniques to report a bug.

1) Having clearly specified bug number:
Always assign a unique number to each bug report. This will help to identify the bug record. If you are using any automated bug-reporting tool then this unique number will be generated automatically each time you report the bug. Note the number and brief description of each bug you reported.

2) Reproducible:
If your bug is not reproducible it will never get fixed. You should clearly mention the steps to reproduce the bug. Do not assume or skip any reproducing step. Step by step described bug problem is easy to reproduce and fix.

3) Be Specific:
Do not write a essay about the problem. Be Specific and to the point. Try to summarize the problem in minimum words yet in effective way. Do not combine multiple problems even they seem to be similar. Write different reports for each problem.

How to Report a Bug?

Use following simple Bug report template:
This is a simple bug report format. It may vary on the bug report tool you are using. If you are writing bug report manually then some fields need to specifically mention like Bug number which should be assigned manually.

Reporter: Your name and email address.

Product: In which product you found this bug.

Version: The product version if any.

Component: These are the major sub modules of the product.

Platform: Mention the hardware platform where you found this bug. The various platforms like ‘PC’, ‘MAC’, ‘HP’, ‘Sun’ etc.

Operating system: Mention all operating systems where you found the bug. Operating systems like Windows, Linux, Unix, SunOS, Mac OS. Mention the different OS versions also if applicable like Windows NT, Windows 2000, Windows XP etc.

Priority:
When bug should be fixed? Priority is generally set from P1 to P5. P1 as “fix the bug with highest priority” and P5 as ” Fix when time permits”.

Severity:
This describes the impact of the bug.
Types of Severity:

• Blocker: No further testing work can be done.
• Critical: Application crash, Loss of data.
• Major: Major loss of function.
• Minor: minor loss of function.
• Trivial: Some UI enhancements.
• Enhancement: Request for new feature or some enhancement in existing one.

Status:
When you are logging the bug in any bug tracking system then by default the bug status is ‘New’.
Later on bug goes through various stages like Fixed, Verified, Reopen, Won’t Fix etc.

Assign To: 
If you know which developer is responsible for that particular module in which bug occurred, then you can specify email address of that developer. Else keep it blank this will assign bug to module owner or Manger will assign bug to developer. Possibly add the manager email address in CC list.

URL:
The page url on which bug occurred.

Summary:
A brief summary of the bug mostly in 60 or below words. Make sure your summary is reflecting what the problem is and where it is.

Description:
A detailed description of bug. Use following fields for description field:

• Reproduce steps: Clearly mention the steps to reproduce the bug.
• Expected result: How application should behave on above mentioned steps.
• Actual result: What is the actual result on running above steps i.e. the bug behavior.

These are the important steps in bug report. You can also add the “Report type” as one more field which will describe the bug type.

The report types are typically:
1) Coding error
2) Design error
3) New suggestion
4) Documentation issue
5) Hardware problem

Some Bonus tips to write a good bug report:

1) Report the problem immediately:If you found any bug while testing, do not wait to write detail bug report later. Instead write the bug report immediately. This will ensure a good and reproducible bug report. If you decide to write the bug report later on then chances are high to miss the important steps in your report.

2) Reproduce the bug three times before writing bug report:Your bug should be reproducible. Make sure your steps are robust enough to reproduce the bug without any ambiguity. If your bug is not reproducible every time you can still file a bug mentioning the periodic nature of the bug.

3) Test the same bug occurrence on other similar module:
Sometimes developer use same code for different similar modules. So chances are high that bug in one module can occur in other similar modules as well. Even you can try to find more severe version of the bug you found.

4) Write a good bug summary:
Bug summary will help developers to quickly analyze the bug nature. Poor quality report will unnecessarily increase the development and testing time. Communicate well through your bug report summary. Keep in mind bug summary is used as a reference to search the bug in bug inventory.

5) Read bug report before hitting Submit button:
Read all sentences, wording, steps used in bug report. See if any sentence is creating ambiguity that can lead to misinterpretation. Misleading words or sentences should be avoided in order to have a clear bug report.

6) Do not use Abusive language:
It’s nice that you did a good work and found a bug but do not use this credit for criticizing developer or to attack any individual.

Conclusion:
No doubt that your bug report should be a high quality document. Focus on writing good bug reports, spend some time on this task because this is main communication point between tester, developer and manager. Mangers should make aware to their team that writing a good bug report is primary responsibility of any tester. Your efforts towards writing good bug report will not only save company resources but also create a good relationship between you and developers.

For better productivity write a better bug report.

How to get your all bugs resolved without any ‘Invalid bug’ label?

I hate “Invalid bug” label from developers for the bugs reported by me, do you? I think every tester should try to get his/her 100% bugs resolved. This requires bug reporting skill. See my previous post on “How to write a good bug report? Tips and Tricks” to report bugs professionally and without any ambiguity.

The main reason for bug being marked as invalid is “Insufficient troubleshooting” by tester before reporting the bug. In this post I will focus only on troubleshooting to find main cause of the bug. Troubleshooting will help you to decide whether the ambiguity you found in your application under test is really a bug or any test setup mistake.

Yes, 50% bugs get marked as “Invalid bugs” only due to testers incomplete testing setup. Let’s say you found an ambiguity in application under test. You are now preparing the steps to report this ambiguity as a bug. But wait! Have you done enough troubleshooting before reporting this bug? Or have you confirmed if it is really a bug?

What troubleshooting you need to perform before reporting any bug?

Troubleshooting of:

• What’s not working?
• Why it’s not working?
• How can you make it work?
• What are the possible reasons for the failure?

Answer for the first question “what’s not working?” is sufficient for you to report the bug steps in bug tracking system. Then why to answer remaining three questions? Think beyond your responsibilities. Act smarter, don’t be a dumb person who only follow his routine steps and don’t even think outside of that. You should be able to suggest all possible solutions to resolve the bug and efficiency as well as drawbacks of each solution. This will increase your respect in your team and will also reduce the possibility of getting your bugs rejected, not due to this respect but due to your troubleshooting skill.

Before reporting any bug, make sure it isn’t your mistake while testing, you have missed any important flag to set or you might have not configured your test setup properly.

Troubleshoot the reasons for the failure in application. On proper troubleshooting report the bug. I have complied a troubleshooting list. Check it out – what can be different reasons for failure.

Reasons of failure:
1) If you are using any configuration file for testing your application then make sure this file is upto date as per the application requirements: Many times some global configuration file is used to pick or set some application flags. Failure to maintain this file as per your software requirements will lead to malfunctioning of your application under test. You can’t report it as bug.

2) Check if your database is proper: Missing table is main reason that your application will not work properly.
I have a classic example for this: One of my projects was querying many monthly user database tables for showing the user reports. First table existence was checked in master table (This table was maintaining only monthly table names) and then data was queried from different individual monthly tables. Many testers were selecting big date range to see the user reports. But many times it was crashing the application as those tables were not present in database of test machine server, giving SQL query error and they were reporting it as bug which subsequently was getting marked as invalid by developers.

3) If you are working on automation testing project then debug your script twice before coming to conclusion that the application failure is a bug.

4) Check if you are not using invalid access credentials for authentication.

5) Check if software versions are compatible.

6) Check if there is any other hardware issue that is not related to your application.

7) Make sure your application hardware and software prerequisites are correct.

8 ) Check if all software components are installed properly on your test machine. Check whether registry entries are valid.

9) For any failure look into ‘system event viewer’ for details. You can trace out many failure reasons from system event log file.

10) Before starting to test make sure you have uploaded all latest version files to your test environment.

These are all small and common mistakes but can mostly impact on your relations and credibility in your team. When you will find that your bug is marked as invalid and the invalid bug reason is from above mentioned list – it will be a silly mistake and it will definitely hurt you. (At least to me!)

Share mistakes done by you while reporting any bug. This will help other readers to learn from your experience!

Should companies charge the clients on the basis of number of bugs?

I came across an interesting software testing outsourcing technique here based on pay per bug approach. This testing consultancy is offering testing services and clients will be charged on the basis of number of bugs found in the application.Why outsourcing?

Many companies are outsourcing software testing work to concentrate on their core business competencies. The company will save time and money on testing processes that is tedious to perform in house.

Current offshore teams are working on per project deal means cost of software testing is decided according to the:

• Project size
• Time for testing
• Number of resources
• or Deliverables.

What if offshore teams start charging the clients on the basis of number of bugs?

Many companies don’t want to pay for documentation, as they might not require it in future. So such clients are more interested in getting the testing work done in less cost and within their budget. Pay per bug approach can work greatly for small projects where requirements are straightforward and clear. In this approach client have freedom to select the testing area as only UI or only Functional or only Security testing as per their requirements.

Will Pay per bug outsourcing technique work?

But I see many complexities in this model. It might not work for all kinds of projects. What if the tester is working to find only smaller and easy to find bugs to increase the bug count and not concentrating on the complex modules? One solution for this could be – Charge the client depending on fixed price per bug or can you can vary this amount according to the bug complexity and the type of testing.

Another problem with this module is – How to decide the severity of the bug? Tester will log bugs with high severity if they are earning high rate on the basis of bug complexity. Client will try to reduce the bug severity as to reduce the testing cost. It’s also difficult to decide whether it is a bug or not Or it is a feature itself? And what about the rejected or ‘won’t fix’ type bugs?

Worst case is if you are getting paid on the basis of number of bugs and you spent enough time to find the bugs, but if application is robust enough that you end up with very few or no bugs then how would you cover the testing recourses cost in this case?

There are lots many such areas of controversies but the concept is good and can be a good outsourcing solution if handled effectively.

I am interested to know what our readers think about this pay per bug approach. Is it a good model or service to outsource the testing services on pay per bug basis?

Bug life cycle

What is Bug/Defect?

Simple Wikipedia definition of Bug is: “A computer bug is an error, flaw, mistake, failure, or fault in a computer program that prevents it from working correctly or produces an incorrect result. Bugs arise from mistakes and errors, made by people, in either a program’s source code or its design.”

Other definitions can be:
An unwanted and unintended property of a program or piece of hardware, especially one that causes it to malfunction.

or
A fault in a program, which causes the program to perform in an unintended or unanticipated manner.

Lastly the general definition of bug is: “failure to conform to specifications”.

If you want to detect and resolve the defect in early development stage, defect tracking and software development phases should start simultaneously.

We will discuss more on Writing effective bug report in another article. Let’s concentrate here on bug/defect life cycle.

Life cycle of Bug:

1) Log new defect
When tester logs any new bug the mandatory fields are:
Build version, Submit On, Product, Module, Severity, Synopsis and Description to Reproduce

In above list you can add some optional fields if you are using manual Bug submission template:
These Optional Fields are: Customer name, Browser, Operating system, File Attachments or screenshots.

The following fields remain either specified or blank:
If you have authority to add bug Status, Priority and ‘Assigned to’ fields them you can specify these fields. Otherwise Test manager will set status, Bug priority and assign the bug to respective module owner.

Look at the following Bug life cycle:

[Click on the image to view full size] Ref: Bugzilla bug life cycle

The figure is quite complicated but when you consider the significant steps in bug life cycle you will get quick idea of bug life.

On successful logging the bug is reviewed by Development or Test manager. Test manager can set the bug status as Open, can Assign the bug to developer or bug may be deferred until next release.

When bug gets assigned to developer and can start working on it. Developer can set bug status as won’t fix, Couldn’t reproduce, Need more information or ‘Fixed’.

If the bug status set by developer is either ‘Need more info’ or Fixed then QA responds with specific action. If bug is fixed then QA verifies the bug and can set the bug status as verified closed or Reopen.

Bug status description:
These are various stages of bug life cycle. The status caption may vary depending on the bug tracking system you are using.

1) New: When QA files new bug.

2) Deferred: If the bug is not related to current build or can not be fixed in this release or bug is not important to fix immediately then the project manager can set the bug status as deferred.

3) Assigned: ‘Assigned to’ field is set by project lead or manager and assigns bug to developer.

4) Resolved/Fixed: When developer makes necessary code changes and verifies the changes then he/she can make bug status as ‘Fixed’ and the bug is passed to testing team.

5) Could not reproduce: If developer is not able to reproduce the bug by the steps given in bug report by QA then developer can mark the bug as ‘CNR’. QA needs action to check if bug is reproduced and can assign to developer with detailed reproducing steps.

6) Need more information: If developer is not clear about the bug reproduce steps provided by QA to reproduce the bug, then he/she can mark it as “Need more information’. In this case QA needs to add detailed reproducing steps and assign bug back to dev for fix.

7) Reopen: If QA is not satisfy with the fix and if bug is still reproducible even after fix then QA can mark it as ‘Reopen’ so that developer can take appropriate action.

8 ) Closed: If bug is verified by the QA team and if the fix is ok and problem is solved then QA can mark bug as ‘Closed’.

9) Rejected/Invalid: Some times developer or team lead can mark the bug as Rejected or invalid if the system is working according to specifications and bug is just due to some misinterpretation.

How to write software Testing Weekly Status Report

Writing effective status report is as important as the actual work you did! How to write a effective status report of your weekly workat the end of each week?

Here I am going to give some tips. Weekly report is important to track the important project issues, accomplishments of the projects, pending work and milestone analysis. Even using these reports you can track the team performance to some extent. From this report prepare future actionables items according to the priorities and make the list of next weeks actionable.

So how to write weekly status report?

Follow the below template:
Prepared By:
Project:
Date of preparation:
Status:
A) Issues:
Issues holding the QA team from delivering on schedule:
Project:
Issue description:
Possible solution:
Issue resolution date:

You can mark these issues in red colour. These are the issues that requires managements help in resolving them.

Issues that management should be aware:

These are the issues that not hold the QA team from delivering on time but management should be aware of them. Mark these issues inYellow colour. You can use above same template to report them.

Project accomplishments:
Mark them in Green colour. Use below template.
Project:
Accomplishment:
Accomplishment date:

B) Next week Priorities:
Actionable items next week list them in two categories:

1) Pending deliverables: Mark them in blue colour: These are previous weeks deliverables which should get released as soon as possible in this week.
Project:
Work update:
Scheduled date:
Reason for extending:

2) New tasks:
List all next weeks new task here. You can use black colour for this.
Project:
Scheduled Task:
Date of release:

C) Defect status:

Active defects:
List all active defects here with Reporter, Module, Severity, priority, assigned to.

Closed Defects:
List all closed defects with Reporter, Module, Severity, priority, assigned to.

Test cases:
List total number of test cases wrote, test cases passed, test cases failed, test cases to be executed.

This template should give you the overall idea of the status report. Don’t ignore the status report. Even if your managers are not forcing you to write these reports they are most important for your work assessment in future.

Try to follow report writing routine. Use this template or at least try to report it in your own words about the overall work of which you can keep some track.

Do you have any better idea for this routine work? Comment it out!