How to Use Keyword Driven Testing (KDT) to Overcome Barriers of Automation Testing
This article covers following points:
1) What Is KDT (Keyword Driven Testing)?
2) Limitations of Automation testing – Why do Test Automation projects fails?
3) How to use Keyword Driven Testing method to overcome Automation testing barriers?
4) What is Test Language?
5) How to use Test Language for Keyword Driven Automation Testing?
6) Responsibilities of Functional Testers and Test Automation experts in KDT.
2) Limitations of Automation testing – Why do Test Automation projects fails?
3) How to use Keyword Driven Testing method to overcome Automation testing barriers?
4) What is Test Language?
5) How to use Test Language for Keyword Driven Automation Testing?
6) Responsibilities of Functional Testers and Test Automation experts in KDT.
Please follow below link to download this free article in PDF file format:
Automated Regression Testing Challenges in Agile Environment
In this article Author J.B. Rajkumar is sharing his experience in Implementing Automated Regression Testing in the Agile Environment.
Abstract
Recently, when I wanted to start my new Automated Testing Project with four resources, I thought of applying any one of the Agile methodologies. But I was not able to continue because, a series of questions were raised inside my mind. The questions are like “Is it possible to use Agile methodologies in Automated Testing?”, “Can I use traditional tools”, “Should I have to go for open-source tools”, “What are the challenges I have to face if I am implementing automation in Agile Environment”. In this article let us analyze some of challenges we face while implementing Automation with Agile methodologies. Automated testing in the Agile environment stands a risk of becoming chaotic, unstructured and uncontrolled.
Recently, when I wanted to start my new Automated Testing Project with four resources, I thought of applying any one of the Agile methodologies. But I was not able to continue because, a series of questions were raised inside my mind. The questions are like “Is it possible to use Agile methodologies in Automated Testing?”, “Can I use traditional tools”, “Should I have to go for open-source tools”, “What are the challenges I have to face if I am implementing automation in Agile Environment”. In this article let us analyze some of challenges we face while implementing Automation with Agile methodologies. Automated testing in the Agile environment stands a risk of becoming chaotic, unstructured and uncontrolled.
Agile Projects present their own challenges to the Automation team; Unclear project scope, Multiple iterations, Minimal documentation, early and frequent Automation needs and active stakeholder involvement all demand lot of challenges from the Automation Team.Some of these challenges are:
Challenge 1: Requirement Phase
Test Automation developer captures requirements in the form of “user stories”, which are brief descriptions of customer-relevant functionality.
Test Automation developer captures requirements in the form of “user stories”, which are brief descriptions of customer-relevant functionality.
Each requirement has to be prioritized as follows:
High: These are mission critical requirements that absolutely have to be done in the first release
Medium: These are requirements that are important but can be worked around until implemented.
Low: These are requirements that are nice-to-have but not critical to the operation of the software.
Medium: These are requirements that are important but can be worked around until implemented.
Low: These are requirements that are nice-to-have but not critical to the operation of the software.
Once priories are established, the release “iterations” are planned. Normally, each Agile release iteration takes between 1 to 3 months to deliver. Customers/software folks take liberty to make too many changes to the requirements. Sometimes, these changes are so volatile that the iterations are bumped off. These changes are greater challenges in implementing Agile Automation testing process.
Challenge 2: Selecting the Right Tools
Traditional, test-last tools with record-and-playback-features force teams to wait until after the software is done. More over, traditional test automation tools don’t work for an Agile context because they solve traditional problems, and those are different from the challenges facing Agile Automation teams. Automation in the early stages of an agile project is usually very tough, but as the system grows and evolves, some aspects settle and it becomes appropriate to deploy automation. So the choice of testing tools becomes critical for reaping the efficiency and quality benefits of agile.
Challenge 3: Script Development Phase
The Automation testers, developers, business analysts and project stakeholders all contribute to kick-off meetings where “user-stories” are selected to next sprint. Once the “user-stories” are selected for the sprint, they are used as the basis for a set of tests.
As functionality grows with each iteration, regression testing must be performed to ensure that existing functionality has not been impacted by the introduction of new functionality in each iteration cycle. The scale of the regression testing grows with each sprint and ensures that this remains a manageable task the test team use the test automation for the regression suite.
Challenge 4: Resource Management
The Agile approach requires a mixture of testing skills, that is, test resource will be required to define unclear scenarios and test cases, conduct manual testing alongside developers, write automated regression tests and execute the automated regression packages. As the project progresses, specialist skills will also be required to cover further test areas that might include integration and performance testing. There should be an appropriate mix of domain specialist who plan and gather requirements. The challenging part in the Resource management is to find out test resources with multiple skills and
allocate them.
allocate them.
Challenge 5: Communication
Good communication must exist among Automation testing team, developers, business analysts and stake holders. There must be highly collaborative interaction between client and the delivery teams. More client involvement implies more suggestions or changes from the client. It implies more bandwidth for communication. The key challenge is that the process should be able to capture and effectively implement all the changes and data integrity needs to be retained. In traditional testing, developers and testers are like oil and water, but in agile environment, the challenging task is that they both must work together to achieve the target.
Challenge 6: Daily Scrum Meeting
Daily Scrum Meeting is one of the key activities in Agile Process. Teams do meet for 15 minutes stand up sessions. What is the effectiveness of these meetings? How far these meetings help Automation practice Developers?
Challenge 7: Release Phase
The aim of Agile project is to deliver a basic working product as quickly as possible and then to go through a process of continual improvement. This means that there is no single release phase for a product. The challenging part lies in integration testing and acceptance testing of the product.
If we can meet these challenges in a well optimized manner, then Automated Regression Testing in Agile environment is an excellent opportunity for QA to take leadership of the agile processes. It is better placed to bridge the gap between users and developers, understand both what is required, how it can be achieved and how it can be assured prior to deployment. Automation practice should have a vested interest in both the how and the result, as well as continuing to assure that the whole evolving system meets business objectives and is fit for purpose.
About the Author:
J.B.Rajkumar has more than 15 years of experience in both Academics and Software Testing. He has worked as Corporate Trainer, Test Lead, QA Manager and QC Manager. He is a frequent speaker for International Conferences, Colleges, Universities and Software Industries. Presently he is with Automation Practice, in one of the top MNCs
J.B.Rajkumar has more than 15 years of experience in both Academics and Software Testing. He has worked as Corporate Trainer, Test Lead, QA Manager and QC Manager. He is a frequent speaker for International Conferences, Colleges, Universities and Software Industries. Presently he is with Automation Practice, in one of the top MNCs
Learning Basics of Rational Robot – IBM Test automation tool
This is a guest article from J.B.Rajkumar. Author is having good experience on automation tools and is currently working with a top MNC.
Learning Basics of Rational Robot (7.0)
1. Features of Rational Robot
Rational Robot is an automated functional, regression testing tool for automating Windows, Java, IE and ERP applications under windows platform. Rational Robot provides test cases for common objects such as menus, lists, bitmaps and specialized test cases for objects specific to the development environment. It integrates with tools like Rational Test Manager, Rational Clearquest and Requisite Pro in the Rational Unified Processor for Defect Tracking, Change Management and Requirement Traceability. It also supports UI technologies like Java, the Web, all VS.NET controls, Oracle Forms, Borland Delphi and Sybase Power Builder applications.
Rational Robot is an automated functional, regression testing tool for automating Windows, Java, IE and ERP applications under windows platform. Rational Robot provides test cases for common objects such as menus, lists, bitmaps and specialized test cases for objects specific to the development environment. It integrates with tools like Rational Test Manager, Rational Clearquest and Requisite Pro in the Rational Unified Processor for Defect Tracking, Change Management and Requirement Traceability. It also supports UI technologies like Java, the Web, all VS.NET controls, Oracle Forms, Borland Delphi and Sybase Power Builder applications.
2. Rational Administrator
It is a tool for managing associations between Rational artifacts such as Test Datastores, Requisite Pro projects and Rose models.
It is a tool for managing associations between Rational artifacts such as Test Datastores, Requisite Pro projects and Rose models.
It is a tool for managing associations between Rational artifacts such as Test Datastores, Requisite Pro projects and Rose models.
- Rational Projects are created using Rational Administrator
- Users and Groups can be maintained
- Project assets can be upgraded
3. Recording Options
Using Object oriented technology, Robot identifies an object by its name property not by its location coordinates. There are two different options
Using Object oriented technology, Robot identifies an object by its name property not by its location coordinates. There are two different options
- GUI - Functional Testing
- VU - Performance Testing
4. SQABasic language
SQABasic is similar to Microsoft Visual Basic. All the scripts will be in scriptname.rec format. When you playback the script, Robot automatically compiles and runs the script, which repeats your actions and executes the verification points.
SQABasic is similar to Microsoft Visual Basic. All the scripts will be in scriptname.rec format. When you playback the script, Robot automatically compiles and runs the script, which repeats your actions and executes the verification points.
5 Shell Scripts
It is a master script that calls other automated scripts and plays them back in sequence. “callscript test1” is a command to call script named test1. Combined into a single shell script, scripts can run in unattended mode and perform comprehensive test coverage. It centralizes test results into one test log.
It is a master script that calls other automated scripts and plays them back in sequence. “callscript test1” is a command to call script named test1. Combined into a single shell script, scripts can run in unattended mode and perform comprehensive test coverage. It centralizes test results into one test log.
6 Low level Recording
Turn “Low Level Recording On” in Robot during recording, mouse and keyboard actions are automatically stored in an external file.
Turn “Low Level Recording On” in Robot during recording, mouse and keyboard actions are automatically stored in an external file.
7 Verification Points
Verification points verify that a certain action has taken place, or verify the state of an object. There are 11 Verification points in Robot
Verification points verify that a certain action has taken place, or verify the state of an object. There are 11 Verification points in Robot
- Alpha-numeric : Verifies alphanumeric data. Used for edit boxes, pushbuttons, labels, text fields, etc.,
- Object Properties: Tests object attributes such as color, font and position.
- Menu: Verifies the menu values and optionally their state (enabled or disabled) of a window
- Clip Board: Verifies the contents of the windows clipboard
- Window Existence: Tests to see if a particular window does or does not exist on the screen.
- Region Image: Graphically compares an area of the screen you specify
- Window Image: Graphically compares an entire window such as a window box.
- Object Data: Test data contents of objects(eg. Dropdown)
- File Comparison: Compares the contents of the two files (size and the contents)
- File Existence: Checks for the existence of a specified file
- Module Existence: Used to verify whether a specified module is loaded into a specified context, or loaded anywhere in memory.
When you are creating verification points, there will be two options – Wait State and expected Results.
Wait states are useful when AUT requires an unknown amount of time to complete a task. Using a wait state keeps the verification point form failing if the task is not completed immediately or if the data is not accessible immediately.
Expected Results – Click Pass or Fail in the Verification Point Name dialog box.
Wait states are useful when AUT requires an unknown amount of time to complete a task. Using a wait state keeps the verification point form failing if the task is not completed immediately or if the data is not accessible immediately.
Expected Results – Click Pass or Fail in the Verification Point Name dialog box.
8 Variable Window
During debugging, if you want to examine variable and constant values, you can variables window. View->Variables.
During debugging, if you want to examine variable and constant values, you can variables window. View->Variables.
9. Object Mapping
If AUT contains a custom object or any object that Robot does not recognize, you
can create a custom object mapping before start recording. By adding the object’s
class to the list of classes that Robot recognizes, and then associating the class to a
standard object type. Robot saves this custom class/object type mapping in the
project and uses it to identify the custom object during playback.
If AUT contains a custom object or any object that Robot does not recognize, you
can create a custom object mapping before start recording. By adding the object’s
class to the list of classes that Robot recognizes, and then associating the class to a
standard object type. Robot saves this custom class/object type mapping in the
project and uses it to identify the custom object during playback.
10 Debug Tools
Animate(F11) – Animation mode allows you to see each line of script as it executes.
Step Over(F10) – Use to execute a single command line within a script
Step Into(F8) – Use to being single step execution
Step Out(F7) – Use to step out of the called script and return to the calling script.
Go Until Cursor(F6) – Use to play back the active GUI script, stopping at the text cursor location.
Step Over(F10) – Use to execute a single command line within a script
Step Into(F8) – Use to being single step execution
Step Out(F7) – Use to step out of the called script and return to the calling script.
Go Until Cursor(F6) – Use to play back the active GUI script, stopping at the text cursor location.
11 Library Files and Header Files
Header files have .sbh extensions and contain the procedure declarations and global variables referred to in your script files. There are two types of library files. Those with .sbl extensions can’t have verification points. Those with .rec extensions are stored in the project and can have verification points. Both Header and library are in \SQABAS32 in the project directory.
Header files have .sbh extensions and contain the procedure declarations and global variables referred to in your script files. There are two types of library files. Those with .sbl extensions can’t have verification points. Those with .rec extensions are stored in the project and can have verification points. Both Header and library are in \SQABAS32 in the project directory.
12 Image Masks used for dynamic objects
Image masks are used to hide an area of the screen. When you play back a script that contains an Image VP and a mask, Robot ignores the masked area when comparing actual results to the recorded baseline.
Image masks are used to hide an area of the screen. When you play back a script that contains an Image VP and a mask, Robot ignores the masked area when comparing actual results to the recorded baseline.
13 Data Pool
A Datapool is a test dataset that supplies data variables in a test script during playback. Using datapools allows you to run multiple iterations of a script using different data each time. It can be created and managed using Test Manager for data driven tests.
A Datapool is a test dataset that supplies data variables in a test script during playback. Using datapools allows you to run multiple iterations of a script using different data each time. It can be created and managed using Test Manager for data driven tests.
14. Important Web Site for Rational Robot trial version download and Rational Robot tutorial:
http://www.ibm.com/developerworks/rational/downloads/
http://www.ibm.com/developerworks/rational/downloads/
Hope this is a good start for those who want to learn Rational Robot. If you have any questions please feel free to comment below.
Manual and Automation testing Challenges
Software Testing has lot of challenges both in manual as well as in automation. Generally in manual testing scenario developers through the build to test team assuming the responsible test team or tester will pick the build and will come to ask what the build is about? This is the case in organizations not following so-called ‘processes’. Tester is the middleman between developing team and the customers, handling the pressure from both the sides. And I assume most of our readers are smart enough to handle this pressure. Aren’t you?
This is not the case always. Some times testers may add complications in testing process due to their unskilled way of working.In this post I have added most of the testing challenges created due to testing staff, developing staff, testing processes and wrong management decisions.
So here we go with the top challenges:
So here we go with the top challenges:
1) Testing the complete application:
Is it possible? I think impossible. There are millions of test combinations. It’s not possible to test each and every combination both in manual as well as in automation testing. If you try all these combinations you will never ship the product
Is it possible? I think impossible. There are millions of test combinations. It’s not possible to test each and every combination both in manual as well as in automation testing. If you try all these combinations you will never ship the product
2) Misunderstanding of company processes:
Some times you just don’t pay proper attention what the company-defined processes are and these are for what purposes. There are some myths in testers that they should only go with company processes even these processes are not applicable for their current testing scenario. This results in incomplete and inappropriate application testing.
Some times you just don’t pay proper attention what the company-defined processes are and these are for what purposes. There are some myths in testers that they should only go with company processes even these processes are not applicable for their current testing scenario. This results in incomplete and inappropriate application testing.
3) Relationship with developers:
Big challenge. Requires very skilled tester to handle this relation positively and even by completing the work in testers way. There are simply hundreds of excuses developers or testers can make when they are not agree with some points. For this tester also requiresgood communication, troubleshooting and analyzing skill.
Big challenge. Requires very skilled tester to handle this relation positively and even by completing the work in testers way. There are simply hundreds of excuses developers or testers can make when they are not agree with some points. For this tester also requiresgood communication, troubleshooting and analyzing skill.
4) Regression testing:
When project goes on expanding the regression testing work simply becomes uncontrolled. Pressure to handle the current functionality changes, previous working functionality checks and bug tracking.
When project goes on expanding the regression testing work simply becomes uncontrolled. Pressure to handle the current functionality changes, previous working functionality checks and bug tracking.
5) Lack of skilled testers:
I will call this as ‘wrong management decision’ while selecting or training testers for their project task in hand. These unskilled fellows may add more chaos than simplifying the testing work. This results into incomplete, insufficient and ad-hoc testing throughout thetesting life cycle.
I will call this as ‘wrong management decision’ while selecting or training testers for their project task in hand. These unskilled fellows may add more chaos than simplifying the testing work. This results into incomplete, insufficient and ad-hoc testing throughout thetesting life cycle.
6) Testing always under time constraint:
Hey tester, we want to ship this product by this weekend, are you ready for completion? When this order comes from boss, tester simply focuses on task completion and not on the test coverage and quality of work. There is huge list of tasks that you need to complete within specified time. This includes writing, executing, automating and reviewing the test cases.
Hey tester, we want to ship this product by this weekend, are you ready for completion? When this order comes from boss, tester simply focuses on task completion and not on the test coverage and quality of work. There is huge list of tasks that you need to complete within specified time. This includes writing, executing, automating and reviewing the test cases.
7) Which tests to execute first?
If you are facing the challenge stated in point no 6, then how will you take decision which test cases should be executed and with what priority? Which tests are important over others? This requires good experience to work under pressure.
If you are facing the challenge stated in point no 6, then how will you take decision which test cases should be executed and with what priority? Which tests are important over others? This requires good experience to work under pressure.
8 ) Understanding the requirements:
Some times testers are responsible for communicating with customers for understanding the requirements. What if tester fails to understand the requirements? Will he be able to test the application properly? Definitely No! Testers require good listening and understanding capabilities.
Some times testers are responsible for communicating with customers for understanding the requirements. What if tester fails to understand the requirements? Will he be able to test the application properly? Definitely No! Testers require good listening and understanding capabilities.
9) Automation testing:
Many sub challenges – Should automate the testing work? Till what level automation should be done? Do you have sufficient and skilled resources for automation? Is time permissible for automating the test cases? Decision of automation or manual testing will need to address the pros and cons of each process.
Many sub challenges – Should automate the testing work? Till what level automation should be done? Do you have sufficient and skilled resources for automation? Is time permissible for automating the test cases? Decision of automation or manual testing will need to address the pros and cons of each process.
10) Decision to stop the testing:
When to stop testing? Very difficult decision. Requires core judgment of testing processes and importance of each process. Also requires ‘on the fly’ decision ability.
When to stop testing? Very difficult decision. Requires core judgment of testing processes and importance of each process. Also requires ‘on the fly’ decision ability.
11) One test team under multiple projects:
Challenging to keep track of each task. Communication challenges. Many times results in failure of one or both the projects.
Challenging to keep track of each task. Communication challenges. Many times results in failure of one or both the projects.
12) Reuse of Test scripts:
Application development methods are changing rapidly, making it difficult to manage the test tools and test scripts. Test script migration or reuse is very essential but difficult task.
Application development methods are changing rapidly, making it difficult to manage the test tools and test scripts. Test script migration or reuse is very essential but difficult task.
13) Testers focusing on finding easy bugs:
If organization is rewarding testers based on number of bugs (very bad approach to judge testers performance) then some testers only concentrate on finding easy bugs those don’t require deep understanding and testing. A hard or subtle bug remains unnoticed in such testing approach.
If organization is rewarding testers based on number of bugs (very bad approach to judge testers performance) then some testers only concentrate on finding easy bugs those don’t require deep understanding and testing. A hard or subtle bug remains unnoticed in such testing approach.
14) To cope with attrition:
Increasing salaries and benefits making many employees leave the company at very short career intervals. Managements are facing hard problems to cope with attrition rate. Challenges – New testers require project training from the beginning, complex projects are difficult to understand, delay in shipping date!
Increasing salaries and benefits making many employees leave the company at very short career intervals. Managements are facing hard problems to cope with attrition rate. Challenges – New testers require project training from the beginning, complex projects are difficult to understand, delay in shipping date!
These are some top software testing challenges we face daily. Project success or failure depends largely on how you address these basic issues.
For further reference and detailed solutions on these challenges refer book “Surviving the Top Ten challenges of Software Testing” written by William E. Perry and Randall W. Rice.
Over to you:
Many of you are working in manual and/or automation testing field. Though I have addressed many of above challenges in our previous articles, I want your views on handling these software testing challenges. Feel free to express your views in comment section below.
Many of you are working in manual and/or automation testing field. Though I have addressed many of above challenges in our previous articles, I want your views on handling these software testing challenges. Feel free to express your views in comment section below.
What you need to know about BVT (Build Verification Testing)
What is BVT?
Build Verification test is a set of tests run on every new build to verify that build is testable before it is released to test team for further testing. These test cases are core functionality test cases that ensure application is stable and can be tested thoroughly. Typically BVT process is automated. If BVT fails that build is again get assigned to developer for fix.
BVT is also called smoke testing or build acceptance testing (BAT)
New Build is checked mainly for two things:
New Build is checked mainly for two things:
- Build validation
- Build acceptance
Some BVT basics:
- It is a subset of tests that verify main functionalities.
- The BVT’s are typically run on daily builds and if the BVT fails the build is rejected and a new build is released after the fixes are done.
- The advantage of BVT is it saves the efforts of a test team to setup and test a build when major functionality is broken.
- Design BVTs carefully enough to cover basic functionality.
- Typically BVT should not run more than 30 minutes.
- BVT is a type of regression testing, done on each and every new build.
BVT primarily checks for the project integrity and checks whether all the modules are integrated properly or not. Module integration testing is very important when different teams develop project modules. I heard many cases of application failure due to improper module integration. Even in worst cases complete project gets scraped due to failure in module integration.
What is the main task in build release? Obviously file ‘check in’ i.e. to include all the new and modified project files associated with respective builds. BVT was primarily introduced to check initial build health i.e. to check whether – all the new and modified files are included in release, all file formats are correct, every file version and language, flags associated with each file.
These basic checks are worth before build release to test team for testing. You will save time and money by discovering the build flaws at the very beginning using BVT.
These basic checks are worth before build release to test team for testing. You will save time and money by discovering the build flaws at the very beginning using BVT.
Which test cases should be included in BVT?
This is very tricky decision to take before automating the BVT task. Keep in mind that success of BVT depends on which test cases you include in BVT.
Here are some simple tips to include test cases in your BVT automation suite:
- Include only critical test cases in BVT.
- All test cases included in BVT should be stable.
- All the test cases should have known expected result.
- Make sure all included critical functionality test cases are sufficient for application test coverage.
Also do not includes modules in BVT, which are not yet stable. For some under-development features you can’t predict expected behavior as these modules are unstable and you might know some known failures before testing for these incomplete modules. There is no point using such modules or test cases in BVT.
You can make this critical functionality test cases inclusion task simple by communicating with all those involved in project development and testing life cycle. Such process should negotiate BVT test cases, which ultimately ensure BVT success. Set some BVT quality standards and these standards can be met only by analyzing major project features and scenarios.
Example: Test cases to be included in BVT for Text editor application (Some sample tests only):
1) Test case for creating text file.
2) Test cases for writing something into text editor
3) Test case for copy, cut, paste functionality of text editor
4) Test case for opening, saving, deleting text file.
1) Test case for creating text file.
2) Test cases for writing something into text editor
3) Test case for copy, cut, paste functionality of text editor
4) Test case for opening, saving, deleting text file.
These are some sample test cases, which can be marked as ‘critical’ and for every minor or major changes in application these basic critical test cases should be executed. This task can be easily accomplished by BVT.
BVT automation suits needs to be maintained and modified time-to-time. E.g. include test cases in BVT when there are new stable project modules available.
What happens when BVT suite run:
Say Build verification automation test suite executed after any new build.
1) The result of BVT execution is sent to all the email ID’s associated with that project.
2) The BVT owner (person executing and maintaining the BVT suite) inspects the result of BVT.
3) If BVT fails then BVT owner diagnose the cause of failure.
4) If the failure cause is defect in build, all the relevant information with failure logs is sent to respective developers.
5) Developer on his initial diagnostic replies to team about the failure cause. Whether this is really a bug? And if it’s a bug then what will be his bug-fixing scenario.
6) On bug fix once again BVT test suite is executed and if build passes BVT, the build is passed to test team for further detail functionality, performance and other testes.
Say Build verification automation test suite executed after any new build.
1) The result of BVT execution is sent to all the email ID’s associated with that project.
2) The BVT owner (person executing and maintaining the BVT suite) inspects the result of BVT.
3) If BVT fails then BVT owner diagnose the cause of failure.
4) If the failure cause is defect in build, all the relevant information with failure logs is sent to respective developers.
5) Developer on his initial diagnostic replies to team about the failure cause. Whether this is really a bug? And if it’s a bug then what will be his bug-fixing scenario.
6) On bug fix once again BVT test suite is executed and if build passes BVT, the build is passed to test team for further detail functionality, performance and other testes.
This process gets repeated for every new build.
Why BVT or build fails?
BVT breaks sometimes. This doesn’t mean that there is always bug in the build. There are some other reasons to build fail like test case coding error, automation suite error, infrastructure error, hardware failures etc.
You need to troubleshoot the cause for the BVT break and need to take proper action after diagnosis.
BVT breaks sometimes. This doesn’t mean that there is always bug in the build. There are some other reasons to build fail like test case coding error, automation suite error, infrastructure error, hardware failures etc.
You need to troubleshoot the cause for the BVT break and need to take proper action after diagnosis.
Tips for BVT success:
1) Spend considerable time writing BVT test cases scripts.
2) Log as much detailed info as possible to diagnose the BVT pass or fail result. This will help developer team to debug and quickly know the failure cause.
3) Select stable test cases to include in BVT. For new features if new critical test case passes consistently on different configuration then promote this test case in your BVT suite. This will reduce the probability of frequent build failure due to new unstable modules and test cases.
4) Automate BVT process as much as possible. Right from build release process to BVT result – automate everything.
5) Have some penalties for breaking the build Some chocolates or team coffee party from developer who breaks the build will do.
1) Spend considerable time writing BVT test cases scripts.
2) Log as much detailed info as possible to diagnose the BVT pass or fail result. This will help developer team to debug and quickly know the failure cause.
3) Select stable test cases to include in BVT. For new features if new critical test case passes consistently on different configuration then promote this test case in your BVT suite. This will reduce the probability of frequent build failure due to new unstable modules and test cases.
4) Automate BVT process as much as possible. Right from build release process to BVT result – automate everything.
5) Have some penalties for breaking the build
Some chocolates or team coffee party from developer who breaks the build will do.Conclusion:
BVT is nothing but a set of regression test cases that are executed each time for new build. This is also called as smoke test. Build is not assigned to test team unless and until the BVT passes. BVT can be run by developer or tester and BVT result is communicated throughout the team and immediate action is taken to fix the bug if BVT fails. BVT process is typically automated by writing scripts for test cases. Only critical test cases are included in BVT. These test cases should ensure application test coverage. BVT is very effective for daily as well as long term builds. This saves significant time, cost, resources and after all no frustration of test team for incomplete build.
BVT is nothing but a set of regression test cases that are executed each time for new build. This is also called as smoke test. Build is not assigned to test team unless and until the BVT passes. BVT can be run by developer or tester and BVT result is communicated throughout the team and immediate action is taken to fix the bug if BVT fails. BVT process is typically automated by writing scripts for test cases. Only critical test cases are included in BVT. These test cases should ensure application test coverage. BVT is very effective for daily as well as long term builds. This saves significant time, cost, resources and after all no frustration of test team for incomplete build.
If you have some experience in BVT process then please share it with our readers in comments below.
Learning basics of QTP automation tool and preparation of QTP interview questions
This post is in continuation with QTP interview questions series.Following questions will help for preparing interview as well as learning the QTP basics.
Quick Test Professional: Interview Questions and answers.
1. What are the features and benefits of Quick Test Pro(QTP)?
1. Key word driven testing
2. Suitable for both client server and web based application
3. VB script as the script language
4. Better error handling mechanism
5. Excellent data driven testing features
2. Suitable for both client server and web based application
3. VB script as the script language
4. Better error handling mechanism
5. Excellent data driven testing features
Learning basics of QTP automation tool and preparation of QTP interview questions
This post is in continuation with QTP interview questions series. Following questions will help for preparing interview as well as learning the QTP basics.
Quick Test Professional: Interview Questions and answers.
1. What are the features and benefits of Quick Test Pro(QTP)?
1. Key word driven testing
2. Suitable for both client server and web based application
3. VB script as the script language
4. Better error handling mechanism
5. Excellent data driven testing features
2. Suitable for both client server and web based application
3. VB script as the script language
4. Better error handling mechanism
5. Excellent data driven testing features
2. How to handle the exceptions using recovery scenario manager in QTP?
You can instruct QTP to recover unexpected events or errors that occurred in your testing environment during test run. Recovery scenario manager provides a wizard that guides you through the defining recovery scenario. Recovery scenario has three steps
1. Triggered Events
2. Recovery steps
3. Post Recovery Test-Run
1. Triggered Events
2. Recovery steps
3. Post Recovery Test-Run
3. What is the use of Text output value in QTP?
Output values enable to view the values that the application talks during run time. When parameterized, the values change for each iteration. Thus by creating output values, we can capture the values that the application takes for each run and output them to the data table.
4. How to use the Object spy in QTP 8.0 version?
There are two ways to Spy the objects in QTP
1) Thru file toolbar: In the File ToolBar click on the last toolbar button (an icon showing a person with hat).
2) Thru Object repository Dialog: In Objectrepository dialog click on the button “object spy…” In the Object spy Dialog click on the button showing hand symbol. The pointer now changes in to a hand symbol and we have to point out the object to spy the state of the object. If at all the object is not visible or window is minimized then hold the Ctrl button and activate the required window to and release the Ctrl button.
1) Thru file toolbar: In the File ToolBar click on the last toolbar button (an icon showing a person with hat).
2) Thru Object repository Dialog: In Objectrepository dialog click on the button “object spy…” In the Object spy Dialog click on the button showing hand symbol. The pointer now changes in to a hand symbol and we have to point out the object to spy the state of the object. If at all the object is not visible or window is minimized then hold the Ctrl button and activate the required window to and release the Ctrl button.
5. What is the file extension of the code file and object repository file in QTP?
File extension of
Per test object rep: filename.mtr
Shared Object rep: filename.tsr
Code file extension id: script.mts
File extension of
Per test object rep: filename.mtr
Shared Object rep: filename.tsr
Code file extension id: script.mts
6. Explain the concept of object repository and how QTP recognizes objects?
Object Repository: displays a tree of all objects in the current component or in the current action or entire test( depending on the object repository mode you selected).
we can view or modify the test object description of any test object in the repository or to add new objects to the repository.
Quicktest learns the default property values and determines in which test object class it fits. If it is not enough it adds assistive properties, one by one to the description until it has compiled the unique description. If no assistive properties are available, then it adds a special Ordianl identifier such as objects location on the page or in the source code.
we can view or modify the test object description of any test object in the repository or to add new objects to the repository.
Quicktest learns the default property values and determines in which test object class it fits. If it is not enough it adds assistive properties, one by one to the description until it has compiled the unique description. If no assistive properties are available, then it adds a special Ordianl identifier such as objects location on the page or in the source code.
7. What are the properties you would use for identifying a browser and page when using descriptive programming?
“name” would be another property apart from “title” that we can use. OR
We can also use the property “micClass”.
ex: Browser(“micClass:=browser”).page(“micClass:=page”)
We can also use the property “micClass”.
ex: Browser(“micClass:=browser”).page(“micClass:=page”)
8. What are the different scripting languages you could use when working with QTP?
You can write scripts using following languages:
Visual Basic (VB), XML, JavaScript, Java, HTML
Visual Basic (VB), XML, JavaScript, Java, HTML
9. Tell some commonly used Excel VBA functions.
Common functions are:
Coloring the cell, Auto fit cell, setting navigation from link in one cell to other saving
Coloring the cell, Auto fit cell, setting navigation from link in one cell to other saving
10. Explain the keyword createobject with an example.
Creates and returns a reference to an Automation object
syntax: CreateObject(servername.typename [, location])
Arguments
servername:Required. The name of the application providing the object.
typename : Required. The type or class of the object to create.
location : Optional. The name of the network server where the object is to be created.
syntax: CreateObject(servername.typename [, location])
Arguments
servername:Required. The name of the application providing the object.
typename : Required. The type or class of the object to create.
location : Optional. The name of the network server where the object is to be created.
11. Explain in brief about the QTP Automation Object Model.
Essentially all configuration and run functionality provided via the QuickTest interface is in some way represented in the QuickTest automation object model via objects, methods, and properties. Although a one-on-one comparison cannot always be made, most dialog boxes in QuickTest have a corresponding automation object, most options in dialog boxes can be set and/or retrieved using the corresponding object property, and most menu commands and other operations have corresponding automation methods. You can use the objects, methods, and properties exposed by the QuickTest automation object model, along with standard programming elements such as loops and conditional statements to design your program.
12. How to handle dynamic objects in QTP?
QTP has a unique feature called Smart Object Identification/recognition. QTP generally identifies an object by matching its test object and run time object properties. QTP may fail to recognize the dynamic objects whose properties change during run time. Hence it has an option of enabling Smart Identification, wherein it can identify the objects even if their properties changes during run time.
Check out this:
If QuickTest is unable to find any object that matches the recorded object description, or if it finds more than one object that fits the description, then QuickTest ignores the recorded description, and uses the Smart Identification mechanism to try to identify the object.
While the Smart Identification mechanism is more complex, it is more flexible, and thus, if configured logically, a Smart Identification definition can probably help QuickTest identify an object, if it is present, even when the recorded description fails.
Check out this:
If QuickTest is unable to find any object that matches the recorded object description, or if it finds more than one object that fits the description, then QuickTest ignores the recorded description, and uses the Smart Identification mechanism to try to identify the object.
While the Smart Identification mechanism is more complex, it is more flexible, and thus, if configured logically, a Smart Identification definition can probably help QuickTest identify an object, if it is present, even when the recorded description fails.
The Smart Identification mechanism uses two types of properties:
Base filter properties – The most fundamental properties of a particular test object class; those whose values cannot be changed without changing the essence of the original object. For example, if a Web link’s tag was changed from to any other value, you could no longer call it the same object. Optional filter properties – Other properties that can help identify objects of a particular class as they are unlikely to change on a regular basis, but which can be ignored if they are no longer applicable.
Base filter properties – The most fundamental properties of a particular test object class; those whose values cannot be changed without changing the essence of the original object. For example, if a Web link’s tag was changed from to any other value, you could no longer call it the same object. Optional filter properties – Other properties that can help identify objects of a particular class as they are unlikely to change on a regular basis, but which can be ignored if they are no longer applicable.
13. What is a Run-Time Data Table? Where can I find and view this table?
In QTP, there is data table used, which is used at runtime.
-In QTP, select the option View->Data table.
-This is basically an excel file, which is stored in the folder of the test created, its name is Default.xls by default.
-In QTP, select the option View->Data table.
-This is basically an excel file, which is stored in the folder of the test created, its name is Default.xls by default.
14. How does Parameterization and Data-Driving relate to each other in QTP?
To data driven we have to parameterize. i.e. we have to make the constant value as parameter, so that in each interaction(cycle) it takes a value that is supplied in run-time data table. Through parameterization only we can drive a transaction (action) with different sets of data. You know running the script with the same set of data several times is not suggested, and it’s also of no use.
15. What is the difference between Call to Action and Copy Action.?
Call to Action: The changes made in Call to Action, will be reflected in the original action (from where the script is called). But where as in Copy Action , the changes made in the script ,will not effect the original script(Action)
16. Explain the concept of how QTP identifies object.
During recording qtp looks at the object and stores it as test object. For each test object QT learns a set of default properties called mandatory properties, and look at the rest of the objects to check whether this properties are enough to uniquely identify the object. During test run, QTP searches for the run time objects that matches with the test object it learned while recording.
17. Differentiate the two Object Repository Types of QTP.
Object repository is used to store all the objects in the application being tested.
Types of object repository: Per action and shared repository.
In shared repository only one centralized repository for all the tests. where as in per action for each test a separate per action repository is created.
Types of object repository: Per action and shared repository.
In shared repository only one centralized repository for all the tests. where as in per action for each test a separate per action repository is created.
18. What the differences are and best practical application of Object Repository?
Per Action: For Each Action, one Object Repository is created.
Shared: One Object Repository is used by entire application
Shared: One Object Repository is used by entire application
19. Explain what the difference between Shared Repository and Per Action Repository
Shared Repository: Entire application uses one Object Repository , that similar to Global GUI Map file in WinRunner
Per Action: For each Action, one Object Repository is created, like GUI map file per test in WinRunner
Per Action: For each Action, one Object Repository is created, like GUI map file per test in WinRunner
20. Have you ever written a compiled module? If yes tell me about some of the functions that you wrote.
Sample answer (You can tell about modules you worked on. If your answer is Yes then You should expect more questions and should be able to explain those modules in later questions): I Used the functions for Capturing the dynamic data during runtime. Function used for Capturing Desktop, browser and pages.
21. Can you do more than just capture and playback?
Sample answer (Say Yes only if you worked on): I have done Dynamically capturing the objects during runtime in which no recording, no playback and no use of repository is done AT ALL.
-It was done by the windows scripting using the DOM(Document Object Model) of the windows.
-It was done by the windows scripting using the DOM(Document Object Model) of the windows.
22. How to do the scripting. Are there any inbuilt functions in QTP? What is the difference between them? How to handle script issues?
Yes, there’s an in-built functionality called “Step Generator” in Insert->Step->Step Generator -F7, which will generate the scripts as you enter the appropriate steps.
23. What is the difference between check point and output value?
An output value is a value captured during the test run and entered in the run-time but to a specified location.
EX:-Location in Data Table[Global sheet / local sheet]
EX:-Location in Data Table[Global sheet / local sheet]
24. How many types of Actions are there in QTP?
There are three kinds of actions:
Non-reusable action – An action that can be called only in the test with which it is stored, and can be called only once.
Reusable action – An action that can be called multiple times by the test with which it is stored (the local test) as well as by other tests.
External action – A reusable action stored with another test. External actions are read-only in the calling test, but you can choose to use a local, editable copy of the Data Table information for the external action.
Non-reusable action – An action that can be called only in the test with which it is stored, and can be called only once.
Reusable action – An action that can be called multiple times by the test with which it is stored (the local test) as well as by other tests.
External action – A reusable action stored with another test. External actions are read-only in the calling test, but you can choose to use a local, editable copy of the Data Table information for the external action.
25. I want to open a Notepad window without recording a test and I do not want to use System utility Run command as well. How do I do this?
You can still make the notepad open without using the record or System utility script, just by mentioning the path of the notepad “( i.e. where the notepad.exe is stored in the system) in the “Windows Applications Tab” of the “Record and Run Settings window.
No comments:
Post a Comment