It’s been a busy year for the Scott Logic testing team since we last visited Software Testing Conference North. This year offered the opportunity to catch up with other members of the testing community, and to hear how companies such as EE Digital, MoneySupermarket, Lloyds Banking Group are tackling challenges and opportunities.

This is a round up of some of the themes from two days in York.

The role of the tester

Scott Logic’s Charlotte Roberts recently rounded up overlooked aspects of testing, such as patience and the ability to “hear” rather than just listen. But what do other organisations value in testers?

The BBC’s senior test manager David Shannon said he looked for testers that “could look at anything and think about how they’d want to test it, rather than what it functionally does”. For Jo Kitchen of EE Digital, it was adaptability, as well as “an eye for detail beyond belief, and the ability to think of the most obscure thing a user could do and test for it”.

Al Lines, QA manager for Moodys noted the growing respect for testers inside companies.

“Testing used to be seen as a cost”, he said. “No one asks me about that any more.”

Al approached the changing landscape of work and how to adapt to it. He observed that large, dedicated, office-based testing teams were shifting to small, blended teams with “developers in test”, based in disperse locations.

Independent head of QA Stephen Moss said that testing came down to a “willingness to understand.”

“A good tester is a craftsman”, he said. “They’re always seeking to evolve what they know and learn more about what they do.”

Stephen’s talk at the end of day one focused on the importance of a “quality roadmap” inside teams. He quipped that the role of a quality champion was like being the navigator in a rally car, ensuring the driver (or developer, in this case) avoids obstacles, keeps to the right pace, and makes it over the finishing line in one piece.

He stressed the idea of embracing a “test-first” mindset, and getting as many people involved and enthused about the testing process as possible. His approach was to assign a score of 1 to 5 to each area of performance, and to test progression month on month.

We caught up with Stephen afterwards to discuss his thoughts further.

Balancing automated and manual

EE Digital’s senior portfolio test manager Jo Kitchen gave a frank talk with attendees about the company’s move from a manual to automated test suite, in which she explained the company’s thinking as it made the shift.

“It’s not a holy grail”, she said. “It’s good, but it has to be used for the right thing.”

Among the factors that EE Digital considered in its process were whether to look to recruit staff with a rare blend of skills permanently or opt for hiring on a contract basis, estimating the costs of automation, and setting the timescales for delivery and benefit realisation.

Most importantly, Jo stressed the importance of maintaining morale amongst the team’s manual testers. This included assigning them to where they added most value, such as working with architecture teams and on the user story definition.

We’ve worked on a number of projects advising companies on how to make that change in a way that worked best for them, so this was a theme that interested us. We caught up with Jo Kitchen during the conference to learn more about her experience.

Moving Faster

The programme featured several sessions from companies who were adjusting to nimbler movement.

Chris Thacker of MoneySupermarket has been working on agile projects since 2007, and talked about leading a team in a culture in a regulated environment in which experimentation was more difficult.

His approach involved “four amigos” reviews, in which the product owners, developers and testers were joined by representatives from risk/compliance and anyone else who could add value. Chris said that “testing is not just a tester’s problem. It’s everyone’s problem.”

We found out more about his approach after his session.

Allan Woodcock, head of central QA for Lloyds Banking Group, described how his company was moving from traditional delivery to a quality engineering model. Despite the size of the organisation, Lloyds discussed a willingness to adapt to change, and embrace more modern agile processes.

“What’s driving us is the demand from the customer”, Allan said. “We have 8 million mobile customers. If they want the interface changed, they need it changed tomorrow.”

Allan added that – in a regulated environment such as banking. – there were some features and services in which “there was no grey area on quality”, but that there were some areas in which customers would accept “good enough” if it saved them from inconvenience, such as “having to traipse into a branch”.

Adopting that mindset probably won’t happen overnight.

“As a bank we had a culture of ‘zero experimentation’ for a long time, and people really struggled with that concept of failure. So I’d say: ‘Don’t be afraid to fail, but fail fast’.”

The Jenga tower of Agile

Scott Logic’s Paul Hands recently wrote a blog post on what can arise when businesses adopt the Scaled Agile Framework, and how to resolve or mitigate issues. That was the subject of Testagility test coach Duncan Nisbet’s talk on day one.

Duncan recently spent 18 months working in an organisation that was adopting the Scaled Agile Framework. The experience gave him a very clear idea of what’s needed to make things work, and what factors can make them “unstable”.

“I’ve seen a few instances of people using agile terminology just to get through a meeting, but really translating it into the processes they already use in their own heads.”

He’s constructed a “Jenga tower” of concepts that teams need to grasp to deliver stable scaled agile projects. Among his advice is to understand the vision and quantify it where possible, rather than progress with several different interpretations of a vague one. He also advocates taking the time to identify stakeholders and value streams, make sure you have at least a basic knowledge of systems thinking, and to visualise the work and challenges ahead.

We talked to Duncan about his thoughts on managing SAFe projects after the session.

Pick the right tool for the job

Great testers love to learn and explore problems, so new thinking and technologies are always intriguing. But it’s important not to get hooked on a shiny new approach when another might be more appropriate.

The role of Artificial Intelligence on testing is a fascinating topic. In fact, Scott Logic’s Laveena Ramchandani recently explored the potential of “cognitive testing” on our blog. But what effect will it really have?

Software Testing Conference North opened with a talk from Antony Edwards of Eggplant. Antony discussed the potential of Artificial Intelligence, and how it might impact on testing over the next couple of years. Antony saw opportunities for AI to assist in auto-generating test scripts, particularly when it comes to validation points. He also discussed its potential in test optimisation, release analytics, and in delivering customer service insights.

However, he stressed that it was important to avoid getting hooked on one approach when another might be more appropriate.

“We need to make sure that we don’t get into a polarised conversation about AI and testing, where it’s either AI or nothing.”

That tone was echoed the following day by Richard J Self, a senior lecturer in governance of advanced and emerging technologies at the University of Derby.

Richard’s talk focused on the potential of blockchain, and what it might mean for testing. The thrust of his argument was that – while there were “a few valid and useful use cases for blockchain that will deliver business benefit, there are an enormous number of ideas…that will fail”.

One area that particularly concerned him was how difficult it was to recover the 64 hexadecimal code that proves ownership of an asset. He worries that losing a key could be particularly disastrous when it comes to large assets such as housing.

He believes that software testing experts can help to “make sure that some of the critical issues are thought of very early”, especially when it comes to security and performance.

We spoke to Richard after his talk to get a clearer idea of his thoughts.

Making the most of tools

Day two’s keynote speaker was Robert Lynch, global manager of Splunk at Murex. Robert focused on how his team tackled what he called “the trillion dollar problem”.

Murex was working with a clearing house in Europe. This client clears a trillion dollars notional daily, and needed to perform 60 position updates a second. Using Splunk – a big data tool designed to produce insights from a variety of data inputs – Robert’s team were able to analyse and present 100 million lines of data in around 10 seconds so that it was usable for testers and developers. The approach let the team overlap data sets so that bottlenecks were more obvious, and helped users get data into Splunk more quickly. He also discussed the creation of a Splunk ID, which enabled users to save a particular point in an investigation and share it with colleagues.

The talk demonstrated the benefit of using logs for more than just debugging issues. Robert’s team use Splunk in a much more pre-emptive way, debugging code from a non-functional perspective.

“Splunk won’t fix your issue on its own”, Robert said. “It’s only a tool. It’s about bringing the right information to your development team, so they can say if there’s a problem or not there.”

If you want to find out more about how to reduce bottlenecks in your testing operations, we’ve created a guide that will give you advice and guidance on your journey. Download the guide, and get in touch to find out how our testing experts can improve performance and protect against threats.