Category Archives: Test

Quilting and hacking

Two blogs to which I subscribe had new posts this week, and they just fitted together:

Now I know sometimes I can be a little obsessive about the analogy between sewing and IT; the need for a workbox, the need for new ways of using old methods, the way the projects are similar… but I urge you to read them both and compare.

A project works when you understand the materials you are working with, how they fit together, and how you can modify them. Repurposing happens with fabric and with code. Hacking can be a good thing – with code and with fabric. But you need to understand the pillars for success. The Googletesting blog suggests to me:

  • refactoring old code so you can reuse it
  • consistent formatting giving patterns that you can follow and
  • use of structure and architecture to hold everything together

Knitnkwilt suggests to me

  • working with what you have got by reusing/repurposing old things
  • using a consistent pattern framework to hold the design together and
  • enhancing the old with new structure/fabric.

 

 

 

Quality in Use tutorial: SEETEST Bucharest

Quality in use is a really important concept for IT teams to understand. We get very focused on building, testing and delivering functionality. We also put some emphasis on what testers call “non functional testing” in other words testing quality attributes other than functionality. In particular, performance and security testing are a focus for many teams. However, customers choosing a product will be looking at whether it is usable, flexible and safe – they assume it’ll work functionally. They will also be drawn to products that provide the right experiences – we use words like excitement, flow and trust to describe how someone experiences a product or service. Attributes like functionality, security and performance are important because they are the foundations that underpin great quality in use, and a beautiful user experience.

I’m really happy to be going to Bucharest in September for SEETEST, where I will present a half day tutorial on Quality in Use and the user experience. It’s my first time in Romania, so I’m taking a few extra days to look around Bucharest. Looking forward to that, and to the conference. The tutorial summary is below.

Quality in use

… the beating heart of the user experience

Public presentation

  • Half day tutorial for SEETEST 15 September 2016
  • Also available as an in-house 1 day tutorial on request

Audience

The class is principally aimed at practicing testers and test managers who want to improve the alignment of their testing to the business and user needs.  This includes people in roles such as system testers, test analysts, test engineers, test consultants, test managers, user acceptance testers and software developers. No prior knowledge is assumed and attendees do NOT need a PC for this course.

Concepts

As testers, should we focus on details or on the big picture? Our skill set and comfort zone is often in the detail; we find excitingly obscure bugs and report these problems in detail. But does our zeal sometimes cause us to miss the big picture of what the customer and the business really needs? Do we focus on software defects at the expense of human and commercial factors? In today’s business environment, the user experience and the commercial imperatives have become overwhelmingly important. As testers it is vital that we understand quality in use and the user experience, in order that we focus our tests correctly.

“Quality in use” measures products and services looking at human, business and societal impacts; this includes usability, accessibility, and flexibility for the customer and business, as well as commercial, human and environmental safety. These are underpinned by technical and engineering attributes of a product, and build together into a “User Experience”. For the people selling, supporting or using the products, this is the beating heart of the customer experience. How well are people supported to effectively and efficiently carry out their tasks? Is the product accessible to all the people who want to use it? Does the experience of using the product generate human reactions of trust, excitement or other reactions that will encourage users to continue using and to recommend the product? Do we reach the customers’ hearts as well as their purses?

Without these “big picture” attributes, delivered software will not be acceptable, will not keep our organizations in profit, and may not be legal. The benefits of designing in and testing these attributes will make the users more effective, more efficient, and increase the marketplace for our products and services.

In the tutorial, Isabel will use examples from ISO25000/ISO25022 and from real projects to discuss how testers design tests derived from the user personas, contexts of use, and acceptance criteria.  This requires testing during early testing of concepts and designs and later testing on built products.

Standard ISO25000/ISO25022 defines a group of attributes and metrics that build from the internal engineering qualities and attributes, such as the functional attributes, performance measures and security, to the Quality in Use attributes of usability, context coverage and freedom from risk, with the user experience attributes such as trust, excitement and flow are at the top of the pyramid. Trust, excitement and flow in a task all affect the human heartbeat – user experience is the beating heart of the user experience.

Participants will learn:

  • To distinguish the layers of quality that must be designed and built into products, and tested;
  • How to understand and meet the context of use for each customer persona, from the internal quality through quality in use, to the user experience;
  • How to focus testing on customers, end users and the business;
  • How to select attributes from each layer of the user experience pyramid to track and measure during testing;
  • How to agree acceptance criteria for testing internal quality, quality in use and the user experience.

UX? What about TX for Test Automation?

Test automation is intended to increase speed and accuracy of information about the SUT partly by allowing engineers to improve the speed and usefulness of their communications.

The best possible interfaces and user experience for the person testing are required to support this otherwise the use of automation will decrease rather than increase velocity of projects.

If the speed and accuracy of test information provided to teams is lowered, with poor test reporting and inaccurate decision making, engineers and managers will become frustrated. It may even lead to disaster.

Good automation tools will help us make good decisions about the SUT and maximise the value of the limited time we have to deliver software products to market. Poor automation tools will delay decision making, increase the likelihood of errors of judgement, and they frustrate both engineers and managers.

Current practices (agile, build pipelines, devops) arise from a need to address delivery speed and accuracy as well as engineering quality. But automating the tests and then forcing people to spend time inaccurately and slowly interpreting the outcomes simply is not cost effective or helpful.

There are many examples of poor interfaces and tools contributing to, or even forcing, humans to make bad, even fatal decisions. Examples such as the London Ambulance Dispatch system (http://bit.ly/1tr2TJZ), and the EU Farm Payments online application system (http://bbc.in/1xEoUE3) show us that poor interfaces can be time wasting, expensive and dangerous.

This matters for test automation because, although automation tools are written and serviced by engineers, the people who use the automation can be non-technical for example user acceptance representative, product owners, business sponsors, managers or end users. Does the information they get from the automation and provide to engineers improve in speed, accuracy and usefulness as a result of the automation or not? What will maximise our ability to get the most from the test automation? What will maximise the accuracy and usefulness of the information provided to engineers, managers and others?

There is a need for TX for Test Automation: that is, the Test Experience for those people who will request, design, and review the results of the automated tests and monitoring. This requires information design and delivery (arguably the purpose of our industry).

Attention to detail “front of house” for the UX for customers and end users can be extended behind the scenes to TX for the engineers, benefiting all. TX can be improved, by consideration of the UX for the automation tool and the tests so that methods and lessons from User Experience Design (UXD) and User Experience Testing (UXT) may be applied to test automation.

We need to consider human factors (as the engineers are human too) as well as the support of improved decision making around quality, speed and accuracy of responses to issues.

My three key points?
– Test automation requires consideration of the UX for the tool and the tests;
– People who use automation might not always be technical but they are always human;
– UXD and UXT for test automation supports improved decision making and quality.

(This post also appears on my “Isabel Evans Writing” blog.)