16. Marianne Berkovich of Glooko

Dollars to Donuts

In this episode of Dollars to Donuts, I speak with Marianne Berkovich, Head of User Research & Consumer Insights at Glooko. We talk about doing research through leadership changes, setting up opportunities for self-critique, and how to build empathy, especially in health technology, by experiencing some aspect of the condition and treatment yourself.

It really bothers me when smart people go out and build things and spend a lot of time and energy to build things that are not for humans. And I’m like, erh, why didn’t they do that? For me it’s more about empowering people who have that energy and who have that entrepreneurial spirit to make the things that are right. Not just make stuff, but actually, make the right thing. It’s a set of skills that I think maybe everybody should have and maybe once everybody has those skills and can do it well, maybe the role of researcher doesn’t need to exist. Until then, I feel like it’s my duty to go out and spread the gospel, as it were, of this is how you talk to users. – Marianne Berkovich

Show Links

• World IA Day San Francisco
• Marianne on LinkedIn
• Marianne on Twitter
• Glooko
• Google
• Adobe
• Blue Ridge Labs
• The Human-Computer Interaction Institute at Carnegie Mellon University
• Lockheed Martin
• Bonnie John
• Honing your user research craft: Doing a critique through a “Don’t Leave Data on the Table” class (Medium)
• UX Interviewing: Personalized Coaching to Avoid Leaving Data on the Table (from From CHI EA ’17)
• Interviewing Workshop: Don’t Leave Data on the Table (from Mixed Methods podcast)
• Net Promoter Score
• Three Ways to Build Empathy when working in Health Tech (Medium)
• Rufus the Bear at the JDRF
• 9Round Fitness
• More human: designing a world where people come first
• Venture for America
• Nasdaq Entrepreneurial Center

Follow Dollars to Donuts on Twitter and help other listeners find the podcast by leaving a review on iTunes.

Transcript

Steve Portigal: Well, hi, and welcome to Dollars to Donuts, the podcast where I talk to people who lead user research in their organization.

I was reading the New York Times today and I noticed something unusual, although I’m seeing this sort of thing more and more. After the byline, but before the article starts, is some italicized text in square brackets. It reads “What you need to know to start the day: Get New York Today in your inbox.” This is copy that only belongs online, in the app or the website or an email newsletter. Presumably you would click on something. But I’m looking at the newspaper – I feel like I’m starring in one of those YouTube videos where a toddler is trying to swipe a magazine and can’t figure out why it’s not a touch screen. I see the New York Times making this kind of error in their print edition every few weeks, and it’s kind of appalling, because it suggests a lack of detail that I don’t expect from a high quality product like the Times. When you get an email that has the wrong name in the salutation, even though we’ve all done it ourselves, it brings your appreciation down a notch or two. They clearly aren’t taking the care that they used to, and that we would hope for.

And it’s especially interesting because I remember when the opposite used to be true; when the experience we had online, with news especially, was a not-quite-there translation of a print experience. And now it’s flopped. The print edition readers are not the primary customers. The organization has identified a different key user and it’s not us.

I can’t help but wonder about any of the users we learn about, are they in a less desirable category or perceived that way because of changes in internal processes or organizational structure? Do we care about their experience, or are we making them into what we call “edge cases” which is a fancy way of dismissing them. It’s hard to imagine the print reader of the New York Times as an edge case, but hey, that’s where we are.

I want to remind you that I’m looking for ways to be able to keep making this podcast for you. Here’s how you can help. You can hire me! I plan and lead user research projects, I coach teams who are working to learn from their customers, and I run training workshops to teach people how to be better at research and analysis. I’ve got two books you can buy- the classic Interviewing Users and Doorbells, Danger, and Dead Batteries, a book of stories from other researchers about the kinds of things that happen when they go out into the field. You rating review this podcast on iTunes, and you can review both books on Amazon. With your support, I can keep doing this podcast for you.

Let’s get to my interview with Marianne Berkovich. We did this a little differently – as part of a live event. We closed out San Francisco’s local edition of “World Information Architecture Day” with our interview, live on stage, in front of an audience. Then when we got off stage, we sat down in a somewhat noisy room and talked a little more to cover some of the things we didn’t have time for. We’ve cleaned it up as best we can but the audio may be a little bit different from how things normally sound. It was really fun for the two of us to speak on stage, and I hope to have more opportunities to record episodes of the podcast in similar settings.

Marianne is the Head of User Research + Customer Insights at Glooko. She’s worked as a consultant and for Google and Adobe.

Thanks for agreeing to talk with me, Marianne. Why don’t we just start by having you introduce yourself. What do you do? Where do you work? Tell us about that.

Marianne Berkovich: My name is Marianne Berkovich and I am Head of User Research & Consumer Insights at Glooko and we’re an online diabetes management platform.

Steve: What is an online diabetes management platform?

Marianne: That’s a great question. So, diabetes is a condition that’s a lot to do with numbers. You want to keep your blood sugar not too low, not too high. So, it’s really conducive and lends itself well to technology and checking your numbers. We have an app for the person with diabetes. They can track all sorts of things there that could affect their blood sugar and then they can also send that information to their clinician who can further see patterns of you’re high in the mornings and what are we going to do about that? So, it’s a platform that both the clinician can access and look at those things as well as for the patient themselves.

Steve: Where in the history of this company and its product did you come in, to bring in user research?

Marianne: So, I am the first user researcher. The company was founded in 2010. I’ve been there for about a year and a half and I am learning all sorts of things of what it’s like to be, first in a healthcare company, or a health tech company, and also being the first researcher.

Steve: Okay, let’s see how easy this is. What are some of the things you’re learning about being the first researcher and working in a health tech company?

Marianne: First of all, as a researcher, it just feels very weird to be answering the questions and not asking the questions. Maybe I answer questions in a way that makes it easier for the researcher to ask the next follow-up question. So, but you know what a great researcher Steve is, so this is showcasing his talent too, right! So I think some of the things that I’m learning is the role of advocacy. That even though there was a need for hey it’s time for a researcher, we need someone full-time – before they were hiring vendors or kind of doing it ad hoc. And so, it was time to have somebody who could be dedicated and who’s trained in this. But at the same time there’s a lot of – I don’t want to say – in some cases there was resistance, but I think in a lot of cases it’s more of just not knowing what good research looks like, or part of the reason my title is so long is that people had an impression that oh, user research is just usability studies. And so, I talked with my manager and we put in the “& Consumer Insights.” Head of User Research & Consumer Insights – so it’s like it’s everything. It’s going to be going out and doing field visits. It might be surveys. It might be a lot of different things. So, to kind of help with that advocacy. And I feel like I do spend – I mean, I think as researchers we always spend a lot of our time in advocacy mode, but I’m not surprised – or maybe surprised – it’s just taking a lot more of my effort to do that advocacy work of what research is and how can it be used.

Steve: I just want to clarify. You’re advocating for user research?

Marianne: Yeah. So, I think even though the company talks about being patient centered and user centered, that what does that really mean? I think one of the things that I’m finding out – so, there’s a lot of people who have been in the field for a long time and they’re like we understand diabetes. We’ve been in this for many, many years. And to have somebody come in – I don’t have any experience with diabetes; I don’t have diabetes myself – to come in with these, you know, different insights or different ways of doing things and people are like, we’ve been there, we know how to do this. And so, to advocate for, hey, we went out and talked to some people. We learned this thing that’s different and knew and it’s maybe against the conventional wisdom, how might we use this? And maybe some of that resistance of like, well we’ve always done it this way. Or conventional wisdom says this other thing.

Steve: So, the other part that you were learning was working in healthcare tech. It’s a new industry for you. So, talk about maybe what that’s been like.

Marianne: Yeah, so – and before that I was – I did a fellowship and we were working in the legal space. And the legal space, the health tech space, are slower moving than just straight up consumer products. It’s like hey let’s build something and we’ll launch an app and do these things. You know regulated spaces of like what is the FDA like? I had a little bit of background in human factors testing, but we have a product that actually got FDA approved. But what does human factors testing look like? How is that different from just regular usability testing? What does it mean to understand the whole ecosystem of payers and employers and insurance companies and PDMs and all that type of thing of moving that needle and kind of inching things forward when you’re in this space that can’t be disrupted so easily because there’s good reasons why these constraints exist.

Steve: So, the company was about 8 years in existence when you joined. So, how much of that were you building from scratch? Describe a little bit about what you encountered and what you had to build around sort of the tech and regulatory aspects maybe.

Marianne: So, actually, luckily the regulatory aspect is just one part of what we offer. It’s the Mobile Insulin Dosing System. So, when you start on insulin, how do you come up to taking the right dose of a certain type of insulin. So, the product does a lot more than that. So, this was just one small part of it. And it was actually – that was already in the works when I joined. So, the app was already out. The website was already out. MIDS was already sort of in progress. So, I think for me it’s been a role of a little bit of back to basics of like let’s learn about our users. How are people with Type 1 diabetes different from Type 2 diabetes. So, in fact, I was telling Steve before this, I’m doing an ethnography right now and so I was in Fresno this week talking to folks with Type 1 diabetes. I’m in Phoenix on Monday, but kind of building that basic understanding and actually building personas and really having a deeper understanding rather than like, yeah well, we have people now who are Type 1s, they know that they need. To move beyond that, to really have some foundational knowledge, and it’s causing us to go back and be like okay, well, is this – is there things in the app that are working the best way. Maybe we can make the clinician experience more efficient. Maybe there’s things that we learn when we go to visit a clinician – be like oh, that’s not how we thought that worked, can we rethink that. So, I think it’s a little bit of back to basics and rethinking certain things rather than to build anything from scratch.

Steve: Before you talked about advocacy which was kind of saying hey, we need to learn these kinds of things, but in some ways doing that reveals the most scary thing of all – oh we need to rethink assumptions and yeah, a 10-year-old company, it’s entrenched. So, what happens when the things that you’re uncovering are inviting, challenging – they’re inviting the challenging of established belief structures around the product and what it looks like and everything. How are you doing that?

Marianne: I think it leads to another kind of wrinkle of working in a startup is that we’ve had a lot of turnover at the leadership level, actually. So, the CEO who was the CEO when I joined was not the founder. He was the second CEO. And he reached a point where he was like, “you know what I really like getting the company to where it is now, but now that we’re really – it’s time to scale and grow things, it’s not what I want to do.” So, we got a new CEO. We got a new head of product. We got a new commercialization officer. So, all that advocacy I had done and sort of like when we did they Type 2 ethnography when I first started, like all that – you know taking people on the road and showing them and doing these empathy workshops and all that – whoosh, out the window because we’ve got a whole new cast of characters now. So, I think that was part of it too. It was a little bit humbling to have to re-sort of create my credibility again with – it’s a whole new cast of characters and a whole new set of people to influence. Luckily my manager has been a great champion of that. And so, I think it’s finding ways that we can leverage and sort of maybe it’s not the right time for things. So, we’ll go out and do this research and we have this foundational stuff. We sort of did a big aha, then those people went away. Can we save that and find another time to sort of bring that to the fore when it might be a little bit more accepted? So, I think it’s both sort of pushing for it and finding the right time to sort of introduce those things.

Steve: So, what does a manager do? How does that championing work?

Marianne: I think, – and I don’t know if this is true, but I feel like as a woman you know I can say a little of things if she’s also a woman, but like women in general, like it’s much more effective if instead of me saying it, I say like, “yeah, what Steve said.” Or, you know, what somebody else said. So, I think because of that, I think it becomes more effective. So, she’s not tooting her own horn. I’m not tooting my own horn, but she is sort of pointing to the work that I’m doing. Plus I think we have different kind of communication styles, and sometimes she can sort of, – I’ll say a bunch of stuff and she can sort of synthesize it in a really nice way and it sort of becomes a little bit more impactful that way, that like I can spend all my time sort of yelling and screaming – metaphorically speaking – and then she can sort of bring it home in a very succinct way.

Steve: So, part of what she’s doing then is also – you talk about finding the right moments. Or, what should we be doing to have impact on the organization as the layers above us are changing. I don’t know. I’m wondering about when new people come in, and this is going to vary on a case by case basis, but you’ve gone through this whole series of trying to open people’s minds up a little bit and say hey things are different. Those people leave. New people come in, but is there an opportunity there? Do those people – what baggage do they have, or do they bring in? I’m not asking you to like slag anybody individually, but as you see these kinds of changes happen in an organization and you’re trying to craft a story that’s about how the world really is vs. maybe what we hope or assume – how does the – what changes when new people come in, with or without baggage, around what the truth is?

Marianne: I think part of it is understanding where people are coming from. Because I think in my last role – I was at Google for a long time and I think like certain assumptions I made that like everybody knew what user research was and everybody knew this and I kind of got used to that, that that was the norm. Like product managers of course know what I do, and everybody knows how this works. And so, I had to figure out like, oh, okay, I can’t assume that people know what these things are, or that when I say user research and what you think in your experience with user research, like maybe that was focus groups. And they’re like yeah, we did those and it was great and I was like okay, great that’s a start. So, I kind of know where I’m starting from with folks. So, I think that’s part of it. And I think also, because I am the only researcher and things move a little bit more slowly, like we did the stuff around Type 2s and now we’re doing the Type 1s. And so now I’m using all that stuff that I learned last time of like what was effective in terms of running a workshop and getting people to come on visits with me. And it’s like it’s a little bit smoother again this time, but we’re doing that again this time around. So, it seems that having that opportunity to redo that foundational research has resurfaced itself. And so, to take advantage of that opportunity, that it’s like well, let’s look at Type 1s now.

Steve: There’s something here about you’ve had lots of experience. This is not your first job. You’ve been at lots of different organizations, you’ve done lots of research and influenced stakeholders and product teams and so on, but it sounds like that there’s a good measure of learning in this job which is about how do I do the thing that I know how to do to be effective in this context. Does that ever go away for researchers, do you think?

Marianne: I hope not. I think it’s – and I think that that’s part of the reason that like, you know – one of the reasons I left Google is like it took me a while to realize the thing that I was doing there and the things that – that, kind of really I’m passionate about are not things that I could do at Google. But like who leaves Google, right. So, I think finding ways to find my path and things that are not sort of the traditional way of moving up in, you know, you’re a junior research, then you’re a senior researcher and then you’re a manager and all these things. To find my own path and sort of be okay with that and find different ways to learn in different contexts. Like the fellowship that I did with Blue Ridge Labs which is a social impact incubator. And I was again, the only researcher, and so I got to do more mentoring. And so that was interesting. So, I think finding ways to both follow my passion and find ways that like what does this organization look like? And I think we’ve probably heard this all before in terms of doing sort of user research or bringing that lens to our stakeholders or our people that we’re working with as well. I think that part never goes away and it’s always changing because it’s always a different set of people.

Steve: Right, more so – is the landscape changing more in a place like Glooko than in a place like Google?

Marianne: That’s a good question. I think it’s very different. Google’s obviously a very – there’s just many people. And so, I think the dynamics of what it’s like to have an organization that big and when I joined I knew all the researchers and now the scale – each product’s team is much larger than that. So, I think it’s a different type of – it’s a different set of issues. So, I think wherever you go – like I’m from the East Coast and people ask – they’re like well what’s better, east coast or west coast, or all this stuff? And I’m like they’re different, right. I don’t think there are things that you can really compare. So, for it’s, – and this is my first time working in a startup and so I don’t have another startup to compare it to. So, maybe if I went to a different startup I could do more of that comparing and contrasting, but in some ways, it feels like apples and oranges of a large organization that’s established and, you know, is well funded and all that vs. a smaller one that is working in a very different space.

Steve: Just hypothetical – I know we’re not supposed to ask – apparently you aren’t supposed to ask hypothetical projection questions in user research, so good thing this is not user research. If you were to look for a job at another startup, I’m just thinking about you joining an 8-year-old company, is there a maturity that you would look for, or a timespan that you would think differently about in the next stages of your career?

Marianne: I think it’s less for me about that. It’s more about really getting jazzed about the problem that I’m solving. So, for me diabetes is a very big issue. Thirty million people in the U.S. and growing. And so that feels like a real meaty issue and that gets me excited. So, I’m like even if some days it feels like I’m pushing a boulder uphill, it seems like a worthwhile thing and that’s what gets me up in the morning. So, for me it’s much more about that and going in with eyes wide open of like well what kind of organization in it – is it, and like okay do I want to take that on. And if the answer is yes then working around those constraints because that’s just the nature of the beast.

Steve: You talked before about figuring out the right job title that would describe, to the rest of the organization, the way that you were going to work. As you – can you say more about sort of what that process was, what those conversations were that identified an opportunity, that helped get you excited that this was something you wanted to do?

Marianne: Well honestly, I think part of it was naivety, that like this was – I think it was actually the way it was posted or sort of written up was Manager of Consumer Insights and I’m like okay, whatever. You know, I’m not on a ladder. I’m the only one, so it doesn’t really matter. So, for me, I was so excited – in fact I was consulting before and so Glooko was one of my clients and I got hired. So, we crafted the role a little bit based on what I was doing. And so for me it was less about negotiating the right title and in fact I think only after I joined – because I was like, sounds great, like it sounds like this is a good fit, sign me up, and then only when I started and I was like hey, I want to make business cards, but I don’t really like this Manager of Consumer Insights – we had that conversation after I joined. And so that’s when my manager started telling me about like hey, this is some of the perceptions because I had gone in very naively also of like everybody knows what user research is. Like, we’re in Silicon Valley, like everybody knows what this is. And so, it didn’t even really occur to me that that advocacy, or the extent of that advocacy that would need to be done.

Steve: So, what – you worked with the organization as a consultant and then came in-house to kind of lead the effort. What was similar and different about – you know, the before and after that transition?

Marianne: So I think one thing is, it was very intimidating. I didn’t know anything about diabetes and so I ran my first study and like everybody showed up and so they were listening in on the call and I’m like how’s this going to go? Like I hope I don’t say anything stupid. And so, you know I think as a consultant, coming up to speed on a different domain very quickly, and I think to me it’s about asking those open-ended questions and asking questions in a way that it doesn’t really matter if you know the domain. I mean it’s certainly much better if you do, but there are ways that you can sort of cover that up, and especially if you’re going in with an apprentice mindset and all those types of things, it kind of helps the situation along a little bit. So, I think for me that was definitely intimidating to not know the domain and have everybody show up.

I think another difference is as a consultant, and I wasn’t a consultant for very long. It was about a year or so before I decided to take the role with Glooko , but I was very cognizant of how I was spending every single hour and whether it was going to lead to making money or not, because is this going to generate a lead? Is it going to generate a sale? All those types of questions were very much top of mind whereas when you’re in-house I think you can spend the sort of hours – like you’re spending your capital in a different way. You’re building relationships and that’s kind of how you’re sort of making money. So, to me it’s sort of a different way that you don’t have to be so aware of every hour leading to money.

Steve: I’m going to switch gears a little bit and maybe we could just rewind. I’d love to hear you describe maybe your path. How did you get into this stuff? What brought you to where we are sitting today?

Marianne: So, I was an English major for undergrad and after I graduated I didn’t know what I wanted to do. So, I spent a couple of years – I actually at that point did work. I was in D.C. and I worked at the National Museum of American Art. We were digitizing the collections. It was a very long time ago and so I was actually cleaning up all the photos and I’m like eh, I’m not really into this technology thing. My Dad also was a computer science professor. I’m like definitely not into that technology thing. Like that’s stuff that my Dad does. Boring. So, I spent a couple of years kind of bumming around a little bit, decided to move out to Denver and I started working as a technical writer. And I was working at a financial services company and so my role was to write the help text for this complex financial software. So, I would sit with the developers. They would explain to me how the complex software worked, and I would write it up. And then I had a thought. I’m like if we just made the software easier to use I wouldn’t have to write anything. So, that led me to first get a certificate in technical communication. And in that I started kind of looking around a little bit more and found the whole field of human centered design and human computer interaction. I looked around and I found a program at Carnegie Mellon. And at that point I had actually switched over to Lockheed Martin, which is a great role – you can ask me about that in a second. Sorry, I just keep getting the questions to ask. And, but I – you know I decided that I really needed a degree in this thing. That just kind of reading books about it or whatever wasn’t enough. So, I decided to go to grad school and I remember just seeing that description of the master’s program and I was like that’s exactly what I want to do and like all my life had been leading to that point to be like that’s exactly what I want to do and having that feeling of like, yup, this is the direction I want to head in.

Steve: Which master’s program? And then keep going. Yes, please.

Marianne: It was a master’s in Human-Computer Interaction at Carnegie Mellon University. So, the role at Lockheed Martin. So, I’m old. I’m definitely over 35. And this was sort of in the olden days and what we were doing is taking – so, you know how we have wildfires and all that type of stuff? So, the resources to manage all those things, like sending the air tankers and the trucks and the crews and all that – that was being done by hand. So, Lockheed Martin got a contact to turn that into – it wasn’t even online. It was just kind of a digital program to do that. So, we would interview subject matter experts and I was like a requirements analyst. Like we didn’t have roles of like designer researcher. Like that wasn’t a thing yet. And so, for me it was like super exciting to solve a real problem. We got to actually go to Boulder and like see the command center and see some of those things. I mean I really felt like I was making a difference and I knew what this was about. But I felt very much at a disadvantage of like, I don’t know what should the icons be? I don’t know. Nobody has best practices around icons yet. And so, it felt like early days and that’s why I really wanted to go back to school and like learn some stuff because I knew there was a lot that had been done already.

Steve: So, you come out of that program and where do you go?

Marianne: Consulting. Yeah, I actually stuck around Pittsburgh for a while and I again ran my own consulting thing. I do that to sort of – when I’m in exploration phase. Or, when I’m dating somebody who just started a PhD program and I can’t leave Pittsburgh. One or the other. So, yeah, and the guy I was dating, he was very entrepreneurial, and he was like such a great cheerleader. He was like yeah, we can figure this out. He had been running his own startup before he started the PhD program and he was like, yeah you can do this. I’m like, yeah, I can do this. So, he did help me a lot and so I was finding projects and was kind of getting my feet wet with that. But then I found that it was time to leave Pittsburgh. That it wasn’t what I wanted to be doing. I really wanted to learn from other people and so I started looking for other opportunities.

Steve: What were those opportunities that you found?

Marianne: You can take it in a different direction. So, I actually interviewed at Google at that point and they didn’t want me, I didn’t want them. It was a very different company at that point. But I also interviewed at Adobe. There’s actually a pipeline from Carnegie Mellon straight here to Silicon Valley. I mean seriously, just pull up a bus, just put the graduates on it and bus us all out here. So, there were a bunch of people who I had gone to grad school with who were at Adobe and I went to interview there, and it felt like a really nice fit. I remember actually – after the Google interview I came home to my hotel and I just turned on the Simpsons and like all I could do was just sit there and like not move. But after the Abode interview, I actually – it was one of my first times out here in Silicon Valley and like I went hiking. I was like I was that inspired and that energized that I went hiking. I’m like this is a sign that maybe I should take this role, that this is a good role for me.

Steve: We can cue up some Simpsons for you after this conversation. You know, just put you in a dark room. Um- so, when you talk about the work that you’re doing now, you’re doing field work, you’re doing ethnography, you’re taking people out. When did you learn how to do that?

Marianne: That’s a good question. I feel like I should credit CMU with setting me on the path and teaching me about contextual inquiry – thank you Bonnie John. And I think for me one of the things that they told us at Carnegie Mellon is – it’s a one-year program, so if you didn’t come in as a computer scientist you’re not leaving as a computer scientist. If you didn’t come in as a designer, you’re not leaving a designer. But we’re going to give you enough exposure to coding and designing and having your stuff on a wall and getting critiqued and to do some social science stuff. And so, for me, having been an English major and I minored in art history and theology, I mean I was as liberal arts as you could get. I wasn’t going to come out as any of those things and I didn’t gravitate towards it. I was like I’m not a designer. So, I think I just gravitated towards research. And I think maybe the English degree kind of prepped me for that in terms of asking good questions and thinking about things and looking for patterns. So, I think just naturally I’m a pattern seeker and I think honing my ability to sort of ask questions just came with time. So, I think it was having a good solid foundation and then just kind of having a natural affinity for it. And I read your book – much, much later, but I did read your book.

Steve: Right. By the time my book came out you’d been doing this for a while. Can you talk about the environment at Adobe? What was there there to develop the practice of user research? In the practitioners like yourself that are coming in with X amount of experience, what did they do well that let you get to the next level of your craft?

Marianne: I think actually I was really lucky at both Adobe and Google, being surrounded by really smart people. So, that made it okay of like I don’t know how to do this, and can someone help me? Like I learned how to do surveys through some of the really awesome people at Google. So, I think for me it was just being surrounded by really supportive people and I think that was one of the things that was part of the culture at Abode. Like we would go to my manager’s house. She would have like all the researchers at her house and we would do these offsite – and it really felt like family. I mean it sounds a little cheesy, but – and so it felt like it was a place to grow and learn. So, I think for me that was a great stepping stone as my first sort of real job after graduate school, to be like oh, okay, this is what being a researcher is like. Oh, this is how you interact with product teams. So, I think it was just learning all those basics. But I think one of the things that – kind of coming back to the question of learning to do ethnographic work and all that is once you’re out – and especially you’re probably one researcher. You’re rarely lucky enough to work with another researcher who can sort of observe you can say like hey, why are you asking that question? Or, maybe we could do it differently. One of the things that I found, when I did have some opportunities at Google to work with other researchers, I’m like we don’t all have the same skills. Like what happens between you and a participant is different. And so, one of the things that having noticed that is I started a class at Google to critique ourselves. So, give researchers an opportunity to observe each other moderating and to give feedback to each other. And then I read a Medium article about it. It’s called Don’t Leave Data on the Table, if you want to look it up. I think that’s one of the things too is that we don’t get that critical eye anymore and we assume that like every researcher is the same and it’s like well actually, you know – and those things can also creep up on us. Like I see myself all the time, especially now that I’m sort of teaching other designers at my company more of like, ah, leading question. Yes or no question. You know. So, I find myself still doing that, even if I’m aware of it. But I think having those things pointed out to us is also really helpful too.

Steve: What’s the structure of the critiquing class?

Marianne: So, it was slightly different when I taught it externally at CHI, and internally. So, internally all our videos are available to each other. So, basically people would send in clips from research that they did. We would make small groups, so it lets you work in a group of five, and we would watch a segment of somebody conducting a study. And at any point people could stop the video and be like hey, I noticed this. It was like when we were actually practicing to teach it, one of my co-teachers, she actually had her pen and she would do a lot of like pointing and sort of like do a lot of things that indicated high status. And she was like, I never noticed that – until she saw herself, until we talked about it, she wasn’t aware of that body language and what it was conveying. And so basically we’d spend 5/10 minutes, or however the time divides up, to watch that video and stop if we see anything. And we asked people to rent that idea that like, again, we don’t know the context of what came before. We don’t know the context after. But this is what I’m noticing. And that’s what we do as researchers is we notice. So, it could be a pattern, it could not, but at least invite people to consider something that they’re doing. And also, by seeing other people, just having that conversation, seeing what other people – and be like okay, I didn’t show that, I didn’t do that in my clip, but like I’ve definitely done that before too. So, learning from each other as well has been really valuable.

Steve: It sounds like a way to create kind of a safe space where it’s okay to critique is that everybody is kind of up for that critique and we’re all being exposed together. Is that kind of the way to make it safe? Because this is our workplace and now we’re talking about how we’re not living up to sort of the high standards of our profession.

Marianne: Yeah. And I think that’s one of the things that we tried really hard to make it a safe space. And one of the things that we did was actually we had – spoiler alert – we’d give out like balloons, which are also like these cords that you pull to sort of stop the production line. But just having a room full of balloons made it a little bit more fun. When I taught at CHI we had to create the videos and so I had these little stuffed animal guys that people could put their phones on when they were doing the videos. So, then we had little stuffed animals on all the tables. And so, – and also modeling it ourselves that – I don’t remember if we actually showed some of the videos, but we definitely told some of the stories of like when we did this ourselves – it’s like I’ve been a practitioner for how many years? Like I’m still learning. I still screw this stuff up. So, making ourselves vulnerable was part of making that safe space too. And also, I think what we tried to do was, when we did it within Google, was not have everybody who was on the same team be together. So, it wasn’t like anybody who you worked with directly.

Steve: You used the phrase “sort of screw this up” and I think like the pen example and I’m sure there’s others that are clearly things we shouldn’t do as researchers. To me it seems like there’s a certain amount of stuff that’s subjective that you and I would do it differently because we’re different and we have different personalities and different energies. For me, critique about that would be really interesting because I like to see how other people handle things. My question here I guess is, is there – does the critique approach look at alternative approaches? How do you talk about that vs. best and worst practices?

Marianne: I think we also gave an opportunity for people to stop the cord on themselves. To be like I didn’t know what to do here. Like help me brainstorm. So, there’s that opportunity too of like for the person themselves to be like how will we handle this? And to hear different perspectives on – there’s, you know, I could have done this, or I could have done this, or in situations in the past I’ve handled it differently. So, I think having just that variety of there’s not one right way to do that, that it depends on the context. And maybe sometimes somebody explains like well this is what happened earlier in the session and so it led me to do it in this way and we’re like, oh, okay, well that makes a lot of sense. And so, I think having a little bit more of that context helps too.

Steve: Are you able to bring any of this into the environment you’re in now where, if anything, you’re teaching people and leading them.

Marianne: Yeah. And I think I’m learning a lot more about screwing it up also. While I’m out in the field doing these Type 1 ethnography, like I’m trying to help designers to be able to do their own research. And so, one of our more junior researchers is actually doing one on one interviews. So, I helped, and we talked about it and I wrote the discussion guide and the study plan. We talked about it and she was all on board and off to the races we went. And then I actually had a cancellation, so I was able to watch one of the sessions and I was like oh no, I didn’t – I screwed up. I didn’t prepare her well enough. It was a little bit more than she could handle and that was squarely on me, you know. That because I’m so used to doing it and it comes so naturally to me that I didn’t recognize that she was in a different space and she would need a different level of support and that I just hadn’t prepared her. And so, we role played a little bit and we practiced a little bit. But I still feel like – I feel guilty. I’m like she’s got more sessions this week and like I don’t know if she’s really prepared. So, I think that part is on me as well to make sure that the things that I’m modeling and the things that I’m sort of teaching are doable for where the designers are and not just hey it’s easy, anybody can do this.

Steve: In my opinion there are some things in life that we do that we maybe learn more from screwing up than we do by succeeding. I don’t know what falls into that category and what doesn’t. I wonder if, as much as you feel for this person and feel responsible for them, I wonder just ultimate perspective, did you set them up for a lot of learning they might not have otherwise got to by – of seeing the edges of what they’re able to control or what they’re able to execute on?

Marianne: I can’t speak for her.

Steve: Good. Alright. Well we hit a dead end. Let’s see what other questions I have written for you here. If we were to have this conversation again in a year or two years, what kinds of things would you want to talk about that you would have achieved in this organization?

Marianne: I hope that there’s less sort of friction. I think we’re already starting to see the seeds of it, that you know product managers who have been at the company for a long time are saying like oh, I’m doing this new thing, you know I need user research. They don’t know what it is. They don’t know why exactly, but they’re sort of like it’s a good thing generally. So, I think to have more of that. Honestly, I think like some of the things that are more – a little boring, kind of more infrastructure stuff – like I hate recruiting. I’m a terrible recruiter. I’m terrible with like time zones and all those details, that type of stuff. So, like I’d love to have a recruiter. I’d love to have another researcher on board. And some of the things that I feel like I have a great success under my belt, that it looks like we’re moving away from NPS (Net Promoter Score), hooray, thank you, thank you, and moving towards satisfaction. You know asking satisfaction question and how often we’re going to do that. You know, that’s a really interesting thing that I’ve been thinking about is can we pop that up in an app. I mean this is an app that people use for health and like they might be having a low blood sugar. You know this is a stressful time that people are using the app. Like nobody is excited about managing their diabetes. So, to pop up something in the app, to be like how are we doing, that’s not the right thing. So, I haven’t figured out like how we might introduce those satisfaction questions and when is the right time to do those things. But at least moving away from net promoter and to asking the satisfaction questions and asking that on a regular cadence so that we can see over time like are we getting better and kind of what the trajectory is, I think that would be a huge thing to have accomplished as well

Steve: That’s a great note to end it on. So, thank you very much. It was a great conversation. Thank you. Thanks, everybody.

Okay that was the end of the interview on stage, and now here’s the remainder of our conversation. So, welcome back Marianne.

Marianne: Thank you. It’s the bonus track, huh?

Steve: Right, this is the director’s cut version of the podcast. So, one of the questions I wanted to follow-up with you on is you talked about the work you’re doing on the Type 2 and the Type 1 and really trying to drive change in the organization. I’m wondering, sort of tactically or practically, what are ways to bring those experiences that patients have into the organization itself?

Marianne: I think actually one of the most fundamental ways that it started is using our app itself and so – I don’t have diabetes myself, but for the first 2 weeks that I was – or first couple of weeks I was at Glooko, I sat down with our certified diabetes educator and we pretended that I had just been diagnosed. And so, she talked me through what somebody would hear. How they would be introduced to pricking their finger and measuring their blood glucose. And so, I spent 2 weeks and I logged all the food that I ate, all the exercise that I did. I measured my blood glucose twice a day just to get that experience of what it was like and like yeah, it was pretty crappy, even after a couple of days, to prick your finger. And to imagine that’s for the rest of your life. Like, I had the luxury of stopping. And so, for me it was a really small way to understand what it was like for somebody who was newly diagnosed with diabetes to do that. And I also presented that back to the team and I sort of, encouraged other people to try. So, we instituted a program. So, anybody who starts at Glooko now can get a blood glucose meter and to try it out themselves and to see what that experience is like. So, it’s one small way to at least start building that empathy.

Steve: Are there others that you want to describe?

Marianne: Yeah, actually, so I wrote a Medium article about different ways to build empathy when working in health tech. So, one of them was experience it yourself. So, actually to try the app and try measuring your blood glucose all the time. See for yourself. So, when we did the ethnographic visits, to bring people out so they could actually talk to and see people. And listen to others. So, some people that I work with actually do have diabetes and so I wanted to hear their stories. So, we started something called Life with Diabetes Storytelling. So, just a couple of folks each time and they’ll either tell their own story of how they were diagnosed and what it was like. Or some people have children who have diabetes and that whole experience. Or some people have parents who have diabetes. So, it humanizes it even further because these are people that we actually work with. So, it’s one thing to visit somebody’s home and you never see that stranger again, but these people that we see all the time and we think nothing of somebody at the lunch counter, or lunch table, pricking their finger and taking their blood glucose, or pulling up their shirt and dosing insulin right at the table. Like that’s just kind of what happens. And to ask those questions and making it okay to ask like well when did you tell your wife? And what is it like to do sports and kind of do those things that you used to do? So, it’s been a really great way to create empathy and even bond as a team, to hear each other’s stories. We put them up all on the website so people who start new can watch the videos themselves and learn about people stories.

Steve: And what’s – the format is videos. Are there are materials that are – I guess describe what this looks like?

Marianne: So, I was a little ambitious at first. I was like we’re going to have three people who speak once a month and I realized the company’s not that big and a lot of people actually don’t want to talk. And even I was encouraging people to – even if you don’t have diabetes, you don’t know anyone with diabetes, like there’s lot of this stuff on the Internet. Like watch some videos. Read some people’s blogs. Like bring that in. Haven’t had any takers on that yet. So, now kind of what we’re doing is we just have two speakers and we do it about every other month. And I just give people the floor. I say you’ve got 15-20 minutes, tell your story. Some people take less time. Some people are a little bit more hands – they take a little bit more time. But all I’m doing is giving them the floor, just creating that space. And it’s been wildly popular. Like engineers will come. Like people – we actually fill the room where it is. So, it’s just an opportunity for people to tell their story. And I also wanted to make it a little bit special, so the JDRF has these little bears called Rufus the Bear that teach kids who are diagnosed like how to do their insulin and all that. So, I’m riffing off that. So, I have these little bears that I give people after they speak, and it says, dear so and so, thanks for sharing a diabetes story. So, it’s personalized and people sit them on their desk so that when you walk around you can see who shared a story. So, I wanted to make it a little special of like, yeah, thanks for sharing and we know whose stories are out there too.

Steve: That’s great. So maybe a different question, following up on some of the things we talked about earlier. You know thinking – we talked about how you learn research and kind of your process through these different environments that you were in. You know at this point, do you have a super power?

Marianne: I think my super power is complexity busting. So, I think that based on what I’m hearing – and I think qualitative research gets poo-pooed a little bit, but I found that being able to suss out those patterns from just a few interviews and turn that into a robust framework of – you know I did a thing when I worked at Google around ads and what are the characteristics in ads and that framework is still being used. So, some of the things that I’m doing now in terms of persona research – or developing personas and also the Bingo card framework of the needs of people with diabetes. So, there’s a couple of columns, there’s a couple of rows and people have different needs and sort of fill that down individually. And those types of things that lead to action and sort of like synthesizing, encapsulating like here’s the things you need to know. Like, I’ve thought about it and I’ve taken all this chaos and I’ve put it into something that’s usable and actionable that the teams can use. And I think that’s one of the things that really makes me happy when I’ve figured something out and then turned it into a product that the team is like aha, we can run with it. This is actually useful for making decisions.

Steve: Right. I think those two pieces are really interesting because I’m with you on the finding patterns rapidly and being able to tell a new story about them, but I think you’re productizing them and communicating some kind of communication design around them so that other people can understand that, I think that’s two superpowers in one, maybe the way you’re describing it. Yeah. Do you have a way of thinking about your own brand, your own identify as someone that – as a researcher, or someone that works in tech, works in these kinds of spaces? How do you think about yourself that way?

Marianne: I think for a long time I thought I’m sort of like everybody else and like we’re all sort of doing the same things. And I have an opportunity now – I’m part of this incubator for women leaders and it’s having me sort of rethink what is my brand? And that one needs to have a brand. I think I also maybe started thinking about it when I started having my own consulting practice and I’m like how do I stand out and what do I offer and how do I package these super powers, and kind of bring that to the fore? So, – and part of this incubator that I’m in, it just started somewhat recently – but the women who are in it, like it’s not just people who are in tech. I think a lot of us are in tech, but we have like doctor and we have some lawyers and we have some other people who are doing other things. And so, it’s great to be with that different community. And so, one of the things that we did in our first session was really think about ourselves as a company. It’s like we all work for companies. We’re all like what’s the vision statement? What’s the mission statement? Every company, you’ve done SWOT analysis. So, we’re turning that lens on ourselves. And so, part of our homework for this month is to figure that out. And I think one of the things that I was thinking about is my mission is really user centered everything. That user centered stuff can be applied to any domain and having worked in finance and wildland firefighting and consumer stuff and with creative professionals, I think I’ve seen it there. But that’s all been within tech. But really, applied to things outside of tech. So, it can be applied to parenting. It can be applied to government. It can be applied to just a lot of different things. Like in fact I joined a gym recently. It’s called 9Round Kickboxing. And I realize how user centered they are. So, I’ve got to riff on this for a little bit. So, some insight somebody had along the way was people don’t want to spend a lot of time at the gym. They want to go at any time they want to. They don’t want to wait for a particular class and they like the special attention from a trainer. So, what does 9Round offer? You can literally go at any time. There’s 9 rounds that you do in order, but you can start as soon as you get there. Each round is 3 minutes. So, you’re done in 30 minutes and it’s one of these high intensity workouts. And there’s a trainer there who can like help you and spar with you at a particular moment. I’m like this is a fabulous example of being user centered and really understanding what your customers need. And I was like I signed up for the year. I’m like done. This is exactly – I’m your target user. You’ve figured out my needs and this is fantastic. So, thinking a lot more about human centered everything and how can we apply that to domains and sort of just everyday life because when things are human centered it’s more humane.

Steve: Is there a source or a reference or an inspiration for that human label, kind of in what you’re looking at?

Marianne: I think it came from you know having worked in a lot of different domains. I think what really crystalized it for me was I read a book called More Human by Steve Hilton and he’s married to a Silicon Valley exec, but he was in the UK and then he moved here and he was like – went to – started hanging out at Stanford and came upon design thinking and he’s like aha, all these things I’ve been doing and thinking about policy and applying it to business and all these things. And so, he really crystalized it and I’m like, yeah, I believe that. Like what if our businesses were more focused that way and what if like the policy of how do we get rid of homelessness actually like prototyped for us, rather than just spending tons of money and doing a particular project for 5 years. So, I think he really captured that human centered everything for me.

Steve: So, as you think about that as your brand, how are you going to operationalize that brand – to use a terrible phrase. But how does this go beyond a concept for you? What do you think you’re going to be doing?

Marianne: So, I think it’s something that I’ve been doing in terms of mentoring, like working with entrepreneurs – so when I left Google, I was sort of exploring two paths. One was more mission oriented something or other and the entrepreneur side of everything. So, right now my full-time job is much more in that mission driven kind of aspect of it, but I’ve also been doing mentoring for entrepreneurs. I’m mentoring a woman in Venture for America right now. I’ve done kind of office hours for a bunch of things. I’ve taught at Nasdaq Entrepreneurial Center. And so, I think those are ways where I can find different ways to apply it and find ways that – how do you find intrapreneurs and entrepreneurs to give them the tools to be able to ask questions and go out and talk to users to apply it in a lot of different ways? That is a way to make that happen. And so, I think that for me, whether it’s my full-time job, or kind of like the balance of those things may change in the future. So, maybe I’ll be spending more time and I’ll find a role that lets me do more of that working with entrepreneurs and helping them develop that and maybe less in this oriented stuff, or maybe I’ll be able to find ways to combine the two. But I think for me, in sort of the next phase, it’s kind of how do I find the overlap, or sort of finding the balance of those two things?

Steve: You know sort of the thesis of this podcast is around – I don’t know thesis, but just the people we want to have as guests are people who are user research types and I’m wondering, as you describe this sort of human centered everything, and these mentorship roles, are you being a researcher, as a way you would define it in doing that work? We have labels and so I want to ask you to kind of unpack the labels and see what you’re sort of framing on this work you’re exploring vs. maybe the work that connects you and I and why we’re having this conversation?

Marianne: Yeah, that’s a really interesting question. I think for me, it’s – I think once I get sort of hot under the collar about something, like everything else goes out the window. You know I was talking earlier about once I found Glooko and I’m like wow, they’re working on a really big, juicy problem. This is something I wanted to do, who cares what the actual label is, or what the title is. So, I think for me, like it really bothers me when people – smart people – go out and build things and spend a lot of time and energy to build things that are not for humans. And I’m like, erh, why didn’t they do that? So, I think for me it’s more about empowering people who have that energy and who have that entrepreneurial spirit to make the things that are right. So, not just make stuff,
but actually, make the right thing. So, for me it’s a set of skills that I think maybe everybody should have and maybe once everybody has those skills and can do it well, maybe the role of researcher doesn’t need to exist. But until then, I feel like it’s my duty to go out and spread the gospel, as it were, of this is how you talk to users. This is how the type of information you can get. These are types of things that you can’t get and the limitations of the things that we can do as well.

Steve: So, it’s a really articulate lovely reminder of why we do research? It’s what we’re trying to have happen and you’re looking at achieving that goal through research, or through mentorship, or through advocacy and all the things that you’re doing are putting stuff out in the world that helps people – I don’t want to put – now I’m putting a lot of words in your mouth, but there’s kind of a producing of things ideal that you are a champion of. So, I see research – as you explain it, I can see research as a part of that, but not the only one.

Marianne: I think you summed that up really nicely. So, thank you for making me sound more articulate.

Steve: That’s the reflecting back technique. Okay, so now maybe wrapping up our epilogue, anything to add in this? Anything I should have asked you about?

Marianne: I was just thinking another user centered everything. I’m doing some volunteering and working with a foster child and I’ve been able to turn that into a user centered thing too. Because she doesn’t live with me, I just see her for a few hours each week, so I think very intently about the activities I’m going to do and being a crafty person also, I have spent a lot of time on YouTube videos and Pinterest and whatnot and making activities that I’m like well we’ve got to work on like fine motor skills. We’ve got to work on like counting or matching and stuff like that. And I’m having such a great time designing these little activities for her and also seeing how she responds and like, oh, she doesn’t like those types of things. Or, she needs more of these types of things. So, I feel like bringing it to – like I think I just naturally bring it to all aspects of my life and I see the power that it has, and I just feel I’m like – I just need to be a champion for those things because I see the value of it and I just want the world to know.

Steve: That’s a lovely note to wrap up on. So, thank you so much, Marianne.

Marianne: Thank you so much.

Steve: Well! That wraps up this episode of Dollars to Donuts. Go to portigal.com/podcast for the transcript as well as links for this episode. You can follow us on Twitter, and subscribe to the podcast at portigal.com, or iTunes, or Spotify, or Stitcher, or anyplace excellent podcasts are distributed. You can buy my books are available at Amazon and rosenfeldmedia.com. The amazing theme music was written and performed by Bruce Todd.

The post 16. Marianne Berkovich of Glooko first appeared on Portigal Consulting.