Aviva Rosenstein – Working with UX in an Agile Environment

Podcasts – UIE Brain Sparks

[ Transcript Available ]

Integrating UX into an Agile workflow has historically been a bit of a challenge. This could be due to a general lack of communication with the development team, or not feeling like the proper time or value is given to UX within the organization. Through her research, Aviva Rosenstein discovered that many problems people were having are commonplace. Additionally, she found that others had actually already worked out solutions to some of these.

In her virtual seminar, Making UX Work with Agile Scrum Teams, Aviva discusses the position of UX on Agile teams and some of the problems they face. There were a bunch of great questions from the audience during the live seminar and Aviva joins Adam Churchill to answer some of those in this podcast.

• How do you manage the change from Waterfall to Agile?
• Are requirements fairly well defined before the Agile process?
• If the designers are working sprints ahead, then how much time are they also spending on the current sprint?
• Where do research and testing fit into the Agile process?
• Can you give some examples of UX tasks that are estimated?
• What are some best practices for documenting design in this process?
• What’s the development team’s role in UX design?
• How do you handle technology limits in UX design?
• Are there UX success measures for new products?
• Can a dedicated UX design team work successfully with product development teams in this scrum environment?

Recorded: September, 2014
[ Subscribe to our podcast via ?This link will launch the iTunes application.]
[ Subscribe with other podcast applications.]

Full Transcript.

---

Adam Churchill: Welcome, everyone, to this episode of the SpoolCast. Recently, Aviva Rosenstein presented a virtual seminar for a super-engaged audience.

It’s called “Making UX Work with Agile Scrum Teams.” The recording of this seminar, along with 180 other UX seminars, are now part of UIE’s All You Can Learn.

Where does design fit into sprints? How do companies let go of waterfall methodology? If you’re struggling to confidently and clearly answer either of those questions, then, this seminar is one you should certainly check out.

In it, Aviva shares how to clarify roles and responsibilities and more effectively track and estimate UX work. There are case studies of companies that brought teams together to work more collaboratively, iteratively, and harmoniously in an Agile process.

We had lots of questions from a super audience and just couldn’t get to them all. In today’s podcast, Aviva was able to join us, and we’re going to discuss some of those questions that came in from the audience during the seminar.

Hey, Aviva. Thanks for joining us.

Aviva Rosenstein: Hi, Adam. It’s great to be here.

Adam: Those that maybe weren’t with us for your seminar that are listening now, can you give us an overview of what you talked about that day?

Aviva: Sure. The presentation was based on some research that I did a while back on what challenges people were facing working with agile teams.

I sent a survey around to several of my peers and colleagues to find out what was working well for them in their relationship with Agile teams and what wasn’t working out so well. What was amazing is that I found that for every problem that one person or two people or three people raised, some other person had an idea that they had found that worked well to solve that problem.

It was clear that there were some common problems that plagued UX in Agile teams, and there were also a number of solutions that people had found that were effective for addressing that. The presentation covers both those patterns, where things get a little wonky, as well as some of the ideas that people had come up to address them.

Those ranged from people saying that they weren’t communicating well with their development team, or they didn’t feel valued in the organization, or they didn’t feel like they were doing things at the right time, or they weren’t really sure what practices worked in the scrum timeframe. If you listen to the seminar, you’ll get to hear some of the ideas that people had recommended that had worked for them.

Adam: Let’s get to some of those questions that were leftover. The UA Bank team, the UX team there, had this question.

There are many teams who are simply comfortable with a Waterfall process. How do you get those same teams comfortable with Agile and Scrum?

Aviva: Change is hard. It doesn’t matter what you’re changing from or to. The best way to get people comfortable is to make sure that they have appropriate expectations and that they feel like they have some skills that they’re going to need.

Part of it is helping them clarify what everybody’s roles are, what they’re expected to do, what’s going to happen next. Some of it may be training them on specific techniques that will work in that Agile context. I think what’s scary is when you’re making these big changes and people have all this fear and uncertainty and doubt about what’s going to happen and will things work well?

They may have heard some stories from other people that were negative. But you can defuse that with good training and addressing things head on.

One of the things, I think, that was really apparent from the interviews that I conducted was that designers sometimes weren’t comfortable talking to developers and vice versa.

They were using similar terms for completely different processes and would miscommunicate a lot. Just by helping them get to know each other and clarifying their roles to each other, that made everybody a lot more comfortable with doing Scrum.

Just because they knew what to expect for themselves and from their peers. It also helped them understand that developers and designers were really all on the same team. They weren’t competing with each other. One wasn’t in service of the other. They were really teammates. That kind of an understanding really helped people feel comfortable.

Adam: Kerry made a comment. She said it sounds as if requirements are pretty well defined before this Agile process begins. Is that accurate?

Aviva: I wish it was. In an ideal world, that would be accurate. But I don’t think any of us work in that ideal world.

There is this concept in Agile called “The Definition of Ready,” which is that before a development team accepts work into a sprint, the work meets certain criteria for readiness. It needs to be estimable. You need to be able to understand how long something is going to take. You have to understand what it fits in. It has to be clear to the development team what it is that they’re doing.

A lot of times, development teams will not realize that they can push back on user stories that aren’t really ready for development. The UX vision — understand the users and their needs, understanding what it is exactly that you’re building — that needs to be clarified in order for the teams to be taking on work that has a UI component.

Ideally, the requirements are pretty clearcut before the scrum begins. That’s not the whole agile process. There are certain kinds of requirements, like who are the users and what are we doing, what are we building, what’s our strategy. Those kinds of things should be fairly well-designed before the development team starts its iterations.

Adam: We had a number of attendees ask about what I’ll call the overlapping nature of the sprints.

If the designers are working sprints ahead, then how much time are they also spending on the current sprint? In other words, are they working with the team or just evaluating the team’s output when they get to what was designed multiple sprints back in the process? Does that make sense?

Aviva: It does. It really varies. There’s no easy answer. It depends a lot on the composition of the team. It depends a lot on the nature of the project that’s being worked on or where it is in its project life cycle.

I didn’t get a chance to talk about this during the seminar, but Desiree Sy, in her book, she has this great visual model of the flow between design and development. She separates it out as there’s a developer track and an interaction-designer track. That’s one way to do it.

It’s not necessarily the only way to do it, but she points out that there’s this staggering that happens, where, in any given cycle, the designers are probably designing a sprint ahead of the developers, in terms of getting the wire-frames ready or the flow ready, so that the developers can implement that.

At the same time, they may also be reviewing code from the last cycle or testing code that was developed in the last cycle.

There’s these little hand-offs that happen. It’s less of a waterfall and more of a cataract, if you can imagine little small streams crossing. In terms of how much time they’re spending on each component, that really depends on where they are in the process and the nature of the design.

It also depends on the composition of the team. If you have a dedicated user researcher on the team, they may be spending most of their time doing RITE testing — we’ll get to that later — doing iterative testing on prototypes or wireframes, before they get into development.

Or, they may be testing code that actually was created in the last cycle. There’s always these hand-offs. If there’s an interaction designer and there’s no dedicated user researcher, then, the amount of time that they’re spending on things might differ.

Adam: Zeke wants to know where you’ve seen research and usability testing fit into the agile process.

Aviva: It’s a great question. As a user researcher, that one’s really close to my heart.

The thing about doing usability testing in RITE, or any kind of user research in RITE, is that you want to be doing it at the right time.

If you’re trying to understand customer needs so that you can set your UX strategy, you want to get that done before development starts. Some people use a sprint zero for that. Some people will use something called a spike, which is a concept from XP that’s outside the context of the sprint cycle.

The idea is that you’re understanding your user needs, you’re developing your personas, you’re clarifying all that stuff, outside the context of the sprint.

The next thing is that you want to be testing things before they’re actually coded and built, because it’s really expensive to re-factor things that don’t work. How do you do that? You do that through doing iterative testing on low-fidelity wireframes to high-fidelity prototypes, depending on what skill set you have on your team.

You might have somebody who’s really good at Axure, who can build a pretty high-fidelity, interactive prototype, outside of the context of the development sprint, and iterate on that quickly with user feedback. There’s a technique called the RITE method that came out of Microsoft’s Game Studio. It stands for Rapid Iterative Testing and Evaluation.

It’s basically a way of getting things in front of users for feedback and observing very closely where they have problems and making changes very quickly in response to those problems, so that you’re not conducting a whole usability test with five to eight users, and then, making changes, but you’re making changes right away and then validating those changes with the next set of testers.

You can do that a sprint ahead of the developers, so that when you’re handing off the designs for implementation, it’s already been vetted by representatives of your user base.

That’s one place to do it. There are other places that you can do user research along the way. You can do quick experiments. If there are different ideas for design and the design team is trying to figure out, “Do we go in this direction or in that direction?”

You might want to do a quick click-test experiment, or even do card sorts before you get ready for fixing a design, an information architecture. That can all happen a sprint or two ahead of the developer implementing those designs.

Adam: The GrubHub Chicago team had this question. Can you give some examples of UX tasks that are estimated?

Aviva: Yeah. I’m mostly going to talk about user research ones, because those are the ones I know the best, but I’ve found it’s really useful to use T-shirt size estimation sizing.

For me, if I know something is going to take half a day, that’s small. If I know something is going to take maybe two weeks, that’s large. If I know something might actually take a month to get done or longer, that might be extra-large.

Some examples of that. Your standard usability test usually takes a week of effort, although it takes two weeks start to finish, from the time where you know who you’re going to recruit to the time where you can deliver some findings. That’s basically a large study, for me. If I’m doing an experiment that might take more like four days, that might be a medium estimate.

The reason that I like T-shirt-size sizing is it’s a little bit different than what the developer teams are using for estimation, you’re not confusing your sizing with their sizing. It helps set expectations for how much work you can take on in a particular sprint.

If you know that you’re going to be dedicated to something over two weeks, but it’s only going to take a week’s worth of effort, then you can fit in some other exercises as well that might be small to medium in that two-week cycle.

Adam: Mr. Montoya wants to know some best practices for documenting design in this process.

Aviva: The best practices are very dependent on your circumstance. If you are sitting with your developers, if you’re all in the same row, in the same floor, the best practices are going to be very different from if you’re working with an offshore team.

If you’re working with an offshore team, if your hand-offs are to somebody that’s working in a completely different track, you’re going to be needing to do more documentation at a higher level of fidelity than if you’re right next to somebody and you can just show them or talk something out or work at a white board together.

The idea is right-sizing the amount of documentation to your circumstance, there’s no easy answers here. There’s no best practices that work across the board.

I think using your common sense and trying to find ways to eliminate unnecessary documentation — but if you need documentation, because you’re working with someone offshore or in a different part of the country, whatever it is, by all means, Agile doesn’t mean don’t document. It means that working code is more effective than the documentation.

From a design perspective, what that often means for my teams is that a working prototype that’s been commented out somehow is often the most effective kind of deliverable. Somebody will build a prototype, and then, accompany the prototype with a list of bullet points with more details about things that aren’t spec’ed out in the prototype.

That way people can actually play with the prototype and see how things are supposed to work and have their attention called to what’s important in the design that they might not think of just from looking at it.

That seems to be a really effective way of doing this work, but that only works, again, if you’ve got somebody on your team who’s good at prototyping. If you can learn how to prototype, if you can get up to speed on prototyping, it will solve so many problems for you on your team.

Adam: Chuck asks, what’s the development team’s role in UX design? He followed that up with, how do you handle technology limits in UX design?

Aviva: It’s a really interesting couple of questions, because the answer to the second half of that question, how do you handle technology limits, begs the first question, what’s their role?

Developers and designers are on the same team. Even if they’re working in slightly different tracks, they’re all working together to develop and ship product. Everybody’s opinion is relevant.

Developers often don’t have a good sense of what the designers’ rules are, and so making sure everybody understands each other’s roles is important, but even more important is understanding that anybody on the team may have a good design idea.

It’s not limited to interaction designers or visual designers. The developers may have great ideas about how the technology could be used that didn’t even occur to design, because they hadn’t worked with that technology before, weren’t as familiar with it.

Their role is they’re recommenders. They get to make suggestions. They may come up with ideas for improvements as the team decides which direction to go.

Now, where I’ve seen things go wonky is when the design team hands off some kind of specification, and the developers then, on their own initiative, make changes to it without discussing it with the designers first. Sometimes, they’re great ideas, and sometimes, they miss the mark because they’re not thinking of users in the same way.

It’s important for the developers and the designers to have open channels of communication, they can check on each other’s work and make sure that any suggestions for changes are workable ones.

There’s this idea of peer programming in extreme programming. Which, I like to steal the idea of a designer and a developer sitting next to each other, sharing a screen and working things out together. The iterations, then, are very short, because you’re next to that person.

The developer says, “What will happen if we did this instead?” The designer can say, “That’s an interesting idea except then this other thing won’t work so well or users may not see that particular affordance.”

The developer might make a suggestion and the designer will say, “Oh wow, I never thought of that. That would be awesome, let’s do that.”

The roles, everybody’s a colleague. Everybody shares ideas. The end goal is to support the business and to support the user.

Adam: Aviva, the team at CapTech Consulting makes this comment that success metrics are more easily defined for existing products. I’m wondering if there are UX success measures for new products?

Aviva: There are a set of measures that I like to use. At the end of the day, success is what the business is defined what success is.

Often, if they’re doing something a net promoter score or customer satisfaction, that’s not going to work so well when you’re at the prototype stage when things don’t really work out that well. There are some metrics that are really useful.

Basically, they’re based on the work that was done on the AINS on the system usability scale, is it effective? Is it usable? Those kinds of questions can be asked even when something’s in development.

I often use a scale with a Lilkert scale one to five agreement scale. This was easy to learn or I had to learn a lot of things to get going with this design or this design integrates well with the way I ordinarily work or this is something that I expect to use frequently. Those kinds of statements are very useful in the context of evaluating and design in testing.

I actually use those scales not as hard and fast metrics but as a ways of prompting people to talk about their reaction. I’ll ask the question, “On a scale of one to five, how well would you agree with the statement, I had to learn a lot of things before I could get going with this system.”

Then, based on their answer, I’ll asked them, “Why did you give it that score?” I’ll get more information about their explanation than I am for the actual metric. At the end of the day, could they finish the task? Did they like the experience? If they didn’t, those are probably the most useful metrics of success.

Adam: Let’s wrap up with talking a bit more about everybody being on the team. Mikolas asks, “Can a dedicated UX design team work successfully with product development teams in this scrum environment?”

Aviva: I’m not entirely sure what the person asked that question meant by a dedicated UX team. There are different models for structure of the UX in an organization.

Sometimes, it’s to have design integrated with their development teams, and so, that ends up being a metric, because they work with a dedicated to a particular development team, but they still have a relationship with the other designers on the team and they still maybe managed by a design manager. They just spend more time with the people that are working on the product they’re supporting.

It could be arranged where there is some kind of centralized theme that gets loaned to a particular products. Both of them can work. I actually think that the design is most successful when it’s, to some degree, embedded in the scrum theme as a dedicated resource for the period of the release. It can work both ways.

The point is to make sure that you’ve got really clear understanding of what people’s responsibilities are and that they’re not trying to support multiple teams at the same time that are going to need the same level of support at the same time because then they run into conflicts.

You want the designers to be able to attend some of the stand up meetings. You want them to be able to attend the different retrospectives. You want them to be considered part of the scrum team.

However, you can set that up, whether it’s a dedicated team that occasionally hangs out with other designers or whether it’s a bunch of designers that occasionally hang up with developers. However, you structure that as long as people are communicating effectively then, it’s all good.

Adam: Can you say a bit more about that as it pertains to global UX teams?

Aviva: I think what you gain from a global UX team is you gain the ability to look across the designs and patterns for the entire organization.

You help each other by creating some consistent design patterns, by making it possible for people to avoid reworking things that there’s already good solutions to. There’s somebody who needs to be managing that process and sharing that information.

Where the UX team is going to be most effective is working closely with design. They need to have a foot in both worlds. They need to have a foot in what’s happening with their particular product and getting ready for their particular ship date.

Maybe occasionally looking ahead or forward or behind to how can we take the patterns that we have created for this project and share them with our colleagues globally. What problems do we know we’re going to have solve that we’re going to get input from other people on the team from.

It’s a challenge. It’s really helpful to have somebody at the management level who can look across and share information between teams.

Adam: Very good. Aviva, thanks for spending some more time with us.

Aviva: It was great. It’s been my pleasure.

Adam: To everyone listening in, thanks for doing so and for your support of the UIE Virtual Seminar Program. Goodbye for now.