The main considerations in selecting a tool for an organization include:

  • Assessment of organizational maturity, strengths and weaknesses and identification of opportunities for an improved test process supported by tools
  • Evaluation against clear requirements and objective criteria
  • A proof-of-concept, by using a test tool during the evaluation phase to establish whether it performs effectively with the software under test and within the current infrastructure or to identify changes needed to that infrastructure to effectively use the tool
  • Evaluation of the vendor (including training, support and commercial aspects) or service support suppliers in case of non-commercial tools
  • Identification of internal requirements for coaching and mentoring in the use of the tool
    • Evaluation of training needs considering the current test team’s test automation skills
    • Estimation of a cost-benefit ratio based on a concrete business case

Introducing the selected tool into an organization starts with a pilot project, which has the following objectives:

  • Learn more detail about the tool
  • Evaluate how the tool fits with existing processes and practices, and determine what would need to change
  • Decide on standard ways of using, managing, storing and maintaining the tool and the test assets (e.g., deciding on naming conventions for files and tests, creating libraries and defining the modularity of test suites)
  • Assess whether the benefits will be achieved at reasonable cost

Success factors for the deployment of the tool within an organization include:

  • Rolling out the tool to the rest of the organization incrementally
    • Adapting and improving processes to fit with the use of the tool
    • Providing training and coaching/mentoring for new users
  • Defining usage guidelines
  • Implementing a way to gather usage information from the actual use
  • Monitoring tool use and benefits
  • Providing support for the test team for a given tool
  • Gathering lessons learned from all teams