System Thinking and UX part 2: System Traps and advice on how to avoid them

In the first part of this article on System Thinking we started with a hypothesis that many projects become ‘addicted’ to ‘Surface Thinking’. The interface instead as the solution, instead of exploring the ‘messy middle’ of a problem and understanding what the eventual interface design needs to do. We then looked beyond the shores of User Experience to explore the realm of Systems in general and saw how the studies of Systems can be applied to designing experiences. In part two, we are going to look closer at why so many systems become badly designed despite being created by sane rational people. We also start to look at how System Thinking is an alternative way to look at the systems we design.

So what is System Thinking, in detail, and how does it relate to UX? To give a complete detailed answer would take a book, one with a lot more examples of real world UX projects comparing results of System and Surface approaches. The book ‘Thinking in Systems’ by Donella Meadows, on which much of the article is based, includes a lot of very useful lessons in systems that can be applied to UX. I’ll shine a light on two areas of the book. The concept of System Traps and some maxims of advice Meadows provides for those designing systems.

System Traps

The concept of system traps are especially useful when looking at how Surface Thinking comes to be. Meadows explains in the book why rational people end up creating bad systems unintentionally. These ‘System Traps’ are described in the book as ‘Archetypal problem-generating structures’. The traps can result in things not going as planned, which in turn can result in a culture of blame in a company and innocent people getting fired. The effects are often far from trivial. It can lead to needless spending by government departments as things fail to behave as expected leading to more changes being implemented.

In the talk I gave at Euro IA in 2014 that this article is based upon, I focused on two of the eight traps listed in ‘Thinking in Systems’.

Seeking the Wrong Goal

This is a core system trap and it is worth repeating part of the definition of a system.

"The least obvious part of a system, it’s function and purpose, is often the most critical determinant of the systems behaviour."

The idea project will start off with clear documentation of how the system behaves and how goal of the future state of the project. It will include details of customer behavour, how the company is organised to achieve the desired result and all the nuance needed to fine tune the design of a system. This doesn’t happen in reality.

As UX professionals, our job is to go beyond the brief and requirements, (even when not asked to) to attempt to uncover what the real aims and goals of a project should be. This is not out of a sense of contrarianism, but in order to better understand what we’re being asked to do. By doing this we can often discover that one goal hides another, if a project goal is even stated.

Just as a well constructed brief leads to better creative work so an honest and actionable set of goals on a project completely changes the direction of a project. All too often the concept of ‘Do then ask for forgiveness later’ becomes the main way things get done or the goal of a project is something that does not lead the system in the right place. One early company had a web team where the ‘goal’ was to publish as many web pages as possible and for them to be on brand. This was a large technology company known for it’s semiconductors. The result, when looking at the site statistics, was that the site was a sprawling mess with thousands of pages that were hard to distinguish for the user. This lead to many sections having next to no traffic despite having a significant budget spent on them. It would have been cheaper to just give away products to a handful of people and get them to tell a friend or two about them.

Meadows also names a few wider examples of incorrect goal setting. For example Gross National Product (GNP) as a goal does not lead to a growing economy. The amount spent on the military does not equate to national security. In fact. security may be undermined if that money is taken from other key parts of a nation's budget. A good education system is shown to lead to less crime which increases national security, for example. Imagine if the desired system state is to provide good education. It may be decided that best way to do this is to spend more on each student. The result is a system state where more money is spent per student, not better education. If the goal is a high pass on a standardised test then the system will produce students that are good at standardised tests. The goal changes the way the system works, so measuring the system in the right way is important. Or, to put it another way, we should be careful what we wish for - we might just get it!

Sometimes goal chasing can lead to short term actions to reach a goal. This leads on to the next System Trap I explored.

Shifting the Burden to the Intervenor - Addiction

This trap is about the reliance on interventions in order to make a system reach its desired state as measured by the system goals. An intervention is a short term fix and is, in my view, equitable to Surface Thinking. In System Thinking there is another word associated with the over reliance on interventions to reach goals, and that is Addiction.

Addiction is most easily understood when we talk about drug addiction or addiction to gambling, computer games or Facebook. The concept of addiction also applies to systems large and small. For example an industry may become addicted to government subsidy or farmers may become reliant upon fertilizers to make their land fertile instead of using crop rotation or other methods. Many Western economies have an addiction to cheap oil to drive an economy or weapons manufacture as a way to bring in money.

This trap is known as addiction, dependence and, in terms of systems, shifting the burden to the ‘intervenor’. The Intervenor is an act of intervention that makes everything appear to be working, on a short term basis. If the desired system state is to feel less depressed, then drugs will do this for a short while. However afterwards it is likely the state will not only revert back to where it was but the world of the user may be worse than before. Likewise if the aim of a business is to increase sales of TVs then a large spend on advertising and pay per click search results will do this without needing to improve the website. But as soon as the spending stops sales will slump again.

Another example of an addiction to intervention is the overuse of street signs and street furniture. It shows that each step is logical by itself and it leads to a system that works but is expensive, complex and needs constant maintenance.

Kew Bridge Junction

This is the junction a few miles away from where I live near Kew Bridge in west London. It has a dizzying amount of road markings, signs, barriers and traffic lights and is a major bottleneck. It illustrates a systematic approach to a problem where each element is there for a reason yet, together, the overall system is close to chaos. No one was fired for putting this in place. But there are alternatives that take a more of high level system approach to traffic control.

Exhibition Road This is Exhibition Road, a few miles away from the Kew Junction in Kensington, home of the Victoria and Albert, Science and Natural History Museums. Here division between road and passenger space has been removed to create a shared space. This project was led by architect Ben Hamilton-Bailee who is an advocate of removing as much clutter from our streets as possible. On the surface this approach is highly counterintuitive and may appear to lead to a higher risk to pedestrians.

In practice what happens is that the governing of the system is put back into the hands of the vehicles and pedestrians involved. Because there are less clear boundaries it adds a degree of uncertainty drivers, and this then leads to lower speeds. This in turn increases the safety of areas like this. The perception of risk leads to more careful users which then reduces the overall risk. This has been criticised as being hostile to those with sight and hearing difficulties. However if the goal of the system is to allow access to cars and pedestrians, and reduce the risk to both, then this overall approach works well. It creates a working system by removing the interventions of behavour on its users and instead relies more upon their individual intelligence and behaviour to create a better system. It decentralises the control by creating an environment that stimulates careful use.

Is Surface Thinking all bad? When are Interventions needed?

At the Euro IA conference where I originally talked about this hypothesis there was a wonderful presentation by Jane Austin of the Telegraph newspaper. She recounted with great humour and insight the struggle they had had to drag a very conservative (with a small c) newspaper into the future. As part of that effort she required a ‘laxative’ project, in this case a redesign of the mobile app. It was done with minimal research and in time to avoid a key bit of middleware hitting its end of life. The team used many approaches that can be described as interventions. The result was an app that some loved and some hated. This is not surprising given the ‘build it and hope’ approach taken. The key thing here is it got things going. A project that is far from perfect but is released is better than a project that is nearly perfect but never sees the light of day!

No organisation or project situation is ever perfect. A project may be without meaningful user testing. A UXer may be dropped into an agile project that is several sprints in and is having difficulties. A UX team may try to drive change in an organisation where some areas are ignorant / resistant to the efforts of the team. In all these situations the key thing is to be aware of an intervention, to know when it is being used and to work towards a situation where the focus is on long term structure; the conversion of interventions into sustainable solutions.

Interventions in User Experience

So what interventions are we addicted to in user experience? Here are a few examples of old and new User Experience outputs that can be considered interventions and, when heavily used, are addictions.

Microsites

For the world of advertising and marketing the allure of building small, self contained campaign sites is often too much to resist or simply the standard practice. Microsites get rid of a lot of complications by being stand alone. The budget is self contained, the focus is obvious and it can be tightly targeted to particular campaign. In addition creatives can go wild with no constraints on CMS limitations and expect a good chance of winning an award at the end of it. Awesome.

Except once the campaign is over often the microsites disappear, with all the work put into them evaporating, or they are left to gather dust and get out of date. Meanwhile the main site for that brand is left to customer support purposes and is generally unloved.

In short if the project is a campaign, integrate it with all the goodies that the brand has and make it a set of landing pages that are plumbed into the brand - not a stand alone entity. It’s not as easy but it’s better for the brand in the long term!

Home Pages

Many experienced UXers working on websites know the problems with home pages and how they can become a lightening rod for views within an organisation. For this reason many now leave the home page to last or avoid discussing it as much as possible. This is a smooth move as home pages are like the bikeshed in the nuclear power station example given in part 1; everyone has a view and but feels it is easier to ignore the overall situation and focus on what can easily be understood.

Yet there are still many who feel creating an impressive home page is the way to go to the extent that some sites try to be all home page (one page sites). These will only work for very simple propositions that don’t require a site to grow over time.

Radical Redesigns - Big Bangs!

If a website is not working, is looking terrible and in general everyone is fed up with it then often the most obvious solution is to knock it down and start again. It works for buildings so it should work for websites. Except this often ignores the huge amount of lessons learnt from having a website live.

Spending time working out why a website is not working is a hard sell to many companies, especially when there are sexy new sites out there that are responsive on newer shinier platforms. It ignores the content that has been successful in the past and can so easy throw the good out with the old.

A refresh of a large website should be done by first understanding the old site in detail, seeing what worked, why it worked and what needs to be migrated to any new site. If this is not done then it can lead to a series of radical redesigns happening to the same site within the same company every couple of years as each site fails to satisfy the business.

If content and what works are thrown away every two years the site never matures and wisdom is wasted.

Visual / Trend Rehashes

Updating the look and feel of an app or website can be a good thing to do. But what can happen is either that updating is done with no real change to the overall experience or it is an excuse to throw away a lot content that does’t fit with the design.

Concept Projects

These are short projects that think outside of the box and throw together some wicked visual concepts that can be presented to the board / stakeholders. They are about driving change and are often several degrees removed from reality. They may even have different iterations. This is an addiction for many large companies where people want to show that things are happening.

I have seen these projects whistle by me as I get stuck in the innards of the real world system. Agencies are great at doing this kind of projects, especially in video form, and can often come unstuck when told ‘great - can we have that in three months’. I have been once asked to help work out the details of the UX for a large ecommerce site based upon a very slick and impressive video presentation with the time budget of about four days. In their mind and the client’s mind all the major thinking had been done. In reality they had a bunch of barely connected ideas with 90% of the experience missing.

If it appears such a project is happening then the solution is to bring down the walls between the concept project and those solving real world problems and let that inform what comes out of the project. Concepts are vital to a project, but isolating them from the how the existing system works will lead to concepts that are bright and shiney but ultimately disruptive in a bad way.

A/B Testing

The idea is simple. A few variations are made to a live experience, usually interface changes, with different users seeing the different variations. The effectiveness of these variations are measured and the successful variations are used. This process is repeated and the interface is evolved over time. The company is happy because they have hard evidence of improvement on the site. Solid figures are hard to argue against.

Yet this approach to improving an experience hides several bigger issues. System theory, for example, shows us that changing something here will alter something over there whilst we’re not looking. So for A/B testing to really work it would require measuring the effect on the whole of the site, not just the page being used. That is very hard to do given the amount of other variables flying around.

In addition evolution by gradual steps will hit local maxima. To understand this picture possible solutions to a problem being represented by a flat landscape. Next use how well a solution solves performs to set it’s height. You end up with a fitness landscape, with multiple peaks. A local maxima is a point near where you started and you started to climb. It is unlikely to the highest point on the landscape as other might be much higher. If the drive is always to a better solution by gradual steps the solution is always forced up the nearest hill. Also the landscape will shift over time, so what worked today may not work well in a month.

A/B testing appears to give solid answers when in reality it is open to the unpredictability of any other approach. It is a good tool for fine tuning an interface, but ultimately a bad tool for designing a user experience. I have seen a creator of collaboration software addicted to A/B testing to the point where user research was not seen as needed.

MVPs

Minimal Viable Products are a way to get things out there quicker and come from the set of ideas that is Lean Startup (and it’s sibling Lean UX). By creating something that does the minimum needed in a viable way, it means that users can start giving feedback earlier and the company has something real to build upon.

In theory it’s a vast improvement on feature rich projects taking many months or years to see the light of day only to fail. ‘Fail quick’ is the idea here.

The issue with MVPs lies in the definition of what is viable. As it is now used so often in big and small companies the concept of what is viable has been stretched in many different directions or simply left out. A technology demo is not an MVP if it is unusable by the end user. Likewise, if key parts of the user experience are missing then it is also not viable.

There is also the danger that once a minimal site or app goes live the pressure to get something released disappears. In a side issue the concept of relying upon customer feedback is also a deeply flawed idea compared to well constructed testing and user interviews (see my post on evidence based design for more on this).

So in summary not going mad with features for launch is great, but missing the point of what is viable is not. What is released has to work as a system, not as a series of bits of functionality.

Some advice for dealing with systems

System Thinking has a habit of denying our hopes of absolute control over what we create. It makes the idea of the Rockstar User Experience Designer that much more preposterous as, beyond microsites and small peer pleasing projects, designing a system is a constant process of discovery.

Key points to remember about Systems are:

  • We can never have complete control over a system.

  • Any mildly complex system will be unpredictable. A series of simple interactions lead to complex results.

  • We will never completely understand a system as systems change over time.

There is a reason why we will never completely understand a system and that is because of human perception. We only see and experience part of any given system, even if we are the ones creating it. The perception of a system can be broken down into three states.

Dean’s Rules of System Perception*

Ideal - An understanding of how the system should work and what is should do. This is never how it actually works.

Perceived - How the system is taught and understood by users. This is how they understand the ideal. It is a perception step away from the ideal and only contains a subset of the information of how a system actually works.

Actual - How users actually use a system. Users will often do things that get good results without really understanding why, providing it works, and also bend rules to work for them. As their goals change so the system changes and will be different to the ideal (for better or worse) and beyond what is perceived.

*Oh sweet vanity.

These states also tell us that no users or creators of a system can claim to be total experts and everyone is a different level of intermediate user. No one has a complete picture of how the system workings.

Some Helpful Advice

It may appear that any attempt to gain knowledge and control of a system is pointless, so why attempt it? The simple answer is that attempting to understand and influence a system is much better than ignoring the mess and simply focusing on the, easy neat and quantifiable things; the apparent ‘low hanging fruit’.

So what tools and thinking can we use to combat this mess, this messy middle? Turning back to the book, Meadows offers many useful methods for systems in general that apply to User Experience. Here are seven out of the thirteen bits of Thinking System advice given in the book.

1. Get the Beat of the System

Whilst we may never know every corner of a system we can get the overall rhythms and beats of a system. This is done by watching its behaviours from different points of views and understanding it’s cycles. John Lewis is very good at understanding the rhythm of sales patterns and use it to drive online and offline sales activity through accurate forecasting. It is good business sense to be beyond the point of reacting to events. Expanding timescales and watching behaviour over events are vital parts of System Thinking.

2. Expose Your Mental Models

John Lewis Content Model Any understanding of a system is a model. Models can be communicated in different ways. Often these are just verbal and in User Experience we are very used to creating diagrams to show what our models are. This bit of advice advocates creating these models and sharing them so others can add their perspective. In another John Lewis example this is a model I created to explain how content related within the overall experience of John Lewis online. It was a model that was used to discuss the overall content strategy for the site and shows existing areas of content.

By creating a model, be it a persona of a user or a experience map showing how users think at different steps of a process, it allows us to show how the system works in a way that others can feedback on. The messy middle becomes something that team members can see, understand and add their perception to.

3. Honor, Respect and Distribute Information

The lesson from System Thinking is that the better the information, the better the system. Systems rely upon feedback to self maintain and if the information is incomplete, delayed or incorrect then the feedback can send a system crazy and generally things break down.

Information is power. Attempting to control and limit information in order to make a system work better is an intervention and should only be done with a deep understanding and intent. As understanding is rarely complete controlling and so information is always going to have an unpredictable outcome.

With User Experience good information gathering leads to better experiences. One key example is information about users. Personas are often used to represent the end users and, if done well, these take a project to a new level. For personas to work they need to be based upon real information. Personas are models, they need to stand up to scrutiny. If a new team member can go and talk to key users and find the personas don’t touch upon their goals, scenarios and behaviour then the personas are invalid. A persona that is based upon assumptions can lead a project down completely the wrong path and end up creating an experience that is only useful to a mythical user.

4. Use Language With Care

The full title of this advice is ‘Use Language with Care and Enrich it with System Concepts’. The later part needs some explaining. I have used many system concepts in this article, ideas such as emergence, decentralisation and feedback. These mean concrete things and relate to how systems work. Many of these words are relatively new or used in ways that don’t relate to the way systems work. To successfully communicate the ideas behind system thinking requires enlarging the language to include these terms and for them to be used in a consistent way.

As UX people we are very used to new terms appearing on a regular basis and, if we are unlucky, these terms will be adopted without an understanding of what lies behind them. We hear people use Agile to mean Scrum or use Lean to mean something done in a hurry without the correct steps. Terminology is only valid if it is understood in the same way by everyone using a system. It is very easy to assume all users are familiar with a term that is used only be a selected few.

It is important to take care over the language we use both when we communicate in a team and when we communicate to our users. In a system clear and concrete language is how information is passed around. If we use language only known to a few without consideration then we fail to communicate and there will be a break in the information flow.

5. Pay Attention To What Is Important

There is a big temptation to find things that are easy to measure to demonstrate the effectiveness of a system. Hard figures are seductive and give the illusion of certainty and so it is easy to pay attention to what is easiest to quantify rather than things that are important.

What we are attempting is to discover the quality of the system, not just the quantity of a few variables. Just as understanding a system is about behaviour, not events, so what we pay attention to needs to be broader and more inline with how a system works. We need to model not only the easy things to measure but the apparently subjective and opaque.

How ugly something a on screen is or how the text on a page comes across are quality issues that are hard to measure. Those who have experienced user research will know that the nuances count, things best expressed as a story or anecdote. These shape the user experience just as much, if not more, than number of page views on a product landing page. Users are our measuring tools for those things that are hard to quantify and we need to resist the urge to boil the complex down to numbers. A representative quote is better than a three out of five score on an arbitrary attribute.

6. Celebrate Complexity!

To provide insight to this advice I’ll quote directly from Meadows.

"Let’s face it, the universe is messy. It is nonlinear, turbulent, and dynamic. It spends its time in transient behaviour on its way to somewhere else, not in mathematically neat equilibria. It self- organises and evolves. It creates diversity and uniformity. That’s what makes the world interesting, that’s what makes it beautiful, and that’s what makes it work."

If we want to understand the systems we work with, and everything is a system, then we need to enjoy the messy middle and embrace it. That way we can design great experiences that answer deep requirements. We are attracted to easy numbers, to short term solutions that raise our stature amongst our peers and to the simple over the complex. These approaches are like fast food, they are popular but not good for us.

We do have a counter intuition that we can tap into, and it’s tied into the idea of what is beautiful. Nature is rarely simple and straightforward but most of us prefer a view of trees over a brutalist collection of hard concrete lines. Yet the complexity of nature is well understood in terms of the maths of fractals and in the study of life like systems in the field of artificial life. Nature is a non linear system that is both complex and understandable at the same time.

If we approach our projects as if we’re creating an ecosystem then we can get closer to embracing the branch of complex systems we have a connection with already.

7. Learn to Dance

This is the most enigmatic but important bit of advice Meadows gives. When we dance we move according to something else and often with someone else. We become part of a system in which we have no total control yet it is a very freeing feeling.

System Thinking tells us that complete top down control is an illusion. Some Information Architects I talk to cling to the promise of complete control over a taxonomy and therefore the system. Yet as soon as others become involved they may not understand things in the same way. When creating a taxonomy for a online department store, it is possible to shape the top most layers yet no one person will be able to say exactly where each item should go because what a shop sells over time changes. It is the individual members of the merchandising team who show a deeper understanding of this. Letting go of absolute control is vital in this situation.

No system lives in isolation, is static or without other minds shaping it. The best thing to do is to embrace that idea and move with the music. We should remain open minded, constantly learning and playful. We feel the rhythm of the system and, over time, we get to understand what we can influence in order to design an amazing experience.

Just like a white water rafter uses the rapids to move down a river or a single musician will interact with other band members to make a sound that moves an audience or a single ant plays its part in creating a thriving ants nest so we, as designers of Interactive Systems, should adapt to each new project that faces us. We should get above the trends, the easy numbers and the absolute practices and try to understand the whole universe of our problem before we delve back in and guide it into a better place.

About this Article

This article is based upon, and expanded, from a talk I gave in September at the Euro IA conference in Brussels. It was the first IA/UX conference I have attended and the first I have presented at. Special thanks to Margaret Hanley for letting me use her Laptop to present from as we couldn’t work out how to get the iPad I had carried around Europe on my honeymoon to connect to the big screens.

This article is a statement of work in progress and there is a books worth of additional material and real world experiences that could be added, even with much better editing. I had to split this version of my thoughts into two to make it more digestible to others. I hope it adds to your thoughts on User Experience.

You can follow me on twitter at @stewdean. Or you can email me at stew [at] stewdean.com.


See Also:

System Thinking and UX part 1

More UX Stew Posts

Comments