Artwork

Content provided by Steve Portigal. All podcast content including episodes, graphics, and podcast descriptions are uploaded and provided directly by Steve Portigal or their podcast platform partner. If you believe someone is using your copyrighted work without your permission, you can follow the process outlined here https://player.fm/legal.
Player FM - Podcast App
Go offline with the Player FM app!

49. Sarah Gregory of Coinbase

1:11:08
 
Share
 

Manage episode 424477991 series 62327
Content provided by Steve Portigal. All podcast content including episodes, graphics, and podcast descriptions are uploaded and provided directly by Steve Portigal or their podcast platform partner. If you believe someone is using your copyrighted work without your permission, you can follow the process outlined here https://player.fm/legal.

This episode of Dollars to Donuts features my conversation with Sarah Gregory, Director of User Research for Consumer at Coinbase. We talk about research comms, archiving user research, and doing research that no one is yet asking for.

Our email is designed for one very specific leadership stakeholder, and it is tailored to how that person likes to consume information. There’s a different stakeholder that hates email. That person, I use Slack. Another stakeholder tends to listen very well when they’re live in a regularly recurring monthly meeting. And so I make sure that research always has one or two slides in that meeting. You have to know exactly who you want to be listening, and you have to change your techniques depending on who that is. Which is really just understanding your user, right? – Sarah Gregory

Show Notes

Help other people find Dollars to Donuts by leaving a review on Apple Podcasts.

Transcript

Steve Portigal: Welcome to Dollars to Donuts, the podcast where I talk with the people who lead user research in their organization. I’m Steve Portigal.

I was recently spoke with Rally’s Lauren Gibson for an Ask Me Anything session about user research maturity. I’ll link to the detailed writeup and the recording of the full conversation, but here’s a short clip.

Lauren Gibson: I’d really quick, like to go back to the skills point you were mentioning. Are there any group or team skills, like say a team is like, we’re trying to upscale or like hiring staffing. What skills would you say would be good ground zero ones to add to your team or have your team focus on to kind of increase that maturity?

Steve: I have a bias here. My bias is about the work that I do and the stuff that I write about and that I teach. So I’ll own that, but I wouldn’t say interviewing skills, just like I wouldn’t say survey writing skills or Qualtrics skills. I think there’s that piece above it that makes us better as researchers kind of method aside, domain aside. And those are things like, and these are big words, which you have to chew on a little bit to get to where they’re meaningful. But, you know, we talk about words like empathy and we talk about words like curiosity and listening. I think there’s a big skill around sort of not just knowing yourself, but kind of hearing yourself. We work in fast paced environments. We are asked to be experts. And so developing comfort with not knowing and being able to be confident and curious to be able to honestly ask a question that we think we’re supposed to know the answers to. Like these are, these are about knowing ourselves and kind of hearing ourselves and being that person in the meeting that asks the question that no one’s willing to ask, being able to say something. So this is maybe about storytelling, but there’s an empathy and compassion aspect to it as well. Being able to do some research and bring it back in a way. And I don’t mean what is your deliverable look like? I mean, how do you kind of set yourself and how do you talk to a person that you have a relationship with so that you help them hear something that’s new or that’s slightly new, but that’s impactful and significant and might suck for them to hear, right. We’ve learned that the thing that we’re doing is not going to work. And there’s another new problem to solve. Like that’s a great thing to find out, but presented without compassion and without some nuance can be seen as harmful.

And so these are like emotional maturity, hearing your own discomfort, being sensitive to other people’s discomfort, being a good storyteller. These are all in service of relationships. I guess that’s maybe what it kind of gets down to the relationships that we can build with people. And sometimes they’re like at a distance relationships, kind of like we’re having with everybody today. We don’t all know each other and we haven’t spent weeks and weeks in the same room and kind of shared ownership, but we’re all trying to connect and share information and help each other learn and draw from this. So we’re in a lot of environments and we work in different ways where we want to use these kinds of self-knowledge and emotional maturity to kind of build relationships because that’s how these things that we’re trying to accomplish. And change. So, yeah, if I want to talk about upskilling, I think those are the things I would kind of work on because they pay off across the board. And you can do that explicitly. Like I have taught a storytelling workshop. Like you can say, lean into that, but these are also side effects from practicing any of the more technical skills of research. You want to practice survey writing and trying to go to survey. You’re going to learn humility. You’re going to learn empathy, right? It is baked into everything, I think that we do, if we are reflective about our own learning.

Check out the whole session, if you like. And why not buy your nail salon worker and your tax preparation specialist their very own copies of the second edition of Interviewing Users. If you really wanna help me out, write a very short review of Interviewing Users on Amazon.
As always, I’d love to talk with you about the challenges your team is facing and how I can help.

Okay, let’s get to today’s episode with Sarah Gregory. She’s the Director of User Research for Consumer at Coinbase. Sarah, it’s great to have you on the podcast. Thanks for being on Dollars to Donuts.

Sarah Gregory: Thank you, it’s an honor to be here.

Steve: It’s an honor to have you. Can we start off with an intro from you?

Sarah: Yeah, absolutely.

Steve: Are there other directors of research?

Sarah: My name is Sarah Gregory and I am a director of user research at Coinbase, specifically working on our consumer and retail products.

Steve: There are currently no other directors.

Sarah: I am the only one, but we do have different product groups that could have a director at some point. Right now, I’m technically the only one.

Steve: And what is Coinbase?

Sarah: So we are a cryptocurrency company. So it can be anything from where you first bought your first Bitcoin. A lot of people tell us that their first Bitcoin ever they bought on Coinbase. But since then, we also have institutional products for institutions that want to custody cryptocurrency. We have developer products that for people who are actually building with blockchains. So we have all of those, which is the product groups that I mentioned. So I specifically focus on our retail and consumer presence, which is Coinbase.com, the big blue website, the big blue app, our retail customer.

Steve: If that’s retail, does that mean that’s consumers?

Sarah: Yes, consumers.

Steve: I hesitate to sort of open the lid on Pandora’s box here, but for the context of this conversation, what’s the minimum viable explanation of crypto Bitcoin blockchain that you can provide to people like me that know what you’re talking about, but don’t know what you’re talking about? Sorry.

Sarah: Oh, my goodness. I wondered if this one was coming. I hope I do it justice. No, it’s OK. I’ll do my best. So it’s the most famous cryptocurrency is Bitcoin. But there are others. And you can think of it as programmable money. So there are some cryptocurrencies that are just trying to be money, new money. You could use them for paying people. You can trade with them. You can do all kinds of money things with them. So you can have a programmable, borderless money that is not controlled by a nation state. Or there are different other things you can build with some cryptos. You can actually create like different kinds of economic situations. You can do things that are unrelated to money. Not censorable, so you can build something that allows you to ostensibly tweet on a blockchain that could never be removed because it is on blockchain. So there’s all kinds of different things that you can do. So yeah, that’s what I got.

Steve: And how long have you been working at Coinbase?

Sarah: Over six years now, which is crazy. I never would have thought I would last that long. If you had asked me six years ago if I would still be working in crypto in six years, I would have said you’re crazy. You know, it’s just a very fast changing environment. It’s a fast moving industry. I didn’t know very much about it when I started. So I would have thought that there would be some reason to move on and do something different. But it turns out it’s really fun. So there you go. Well, I certainly didn’t know very much when I started.

Steve: Going back over those six years, do you recall the journey, if it was a journey, just to sort of understand the specifics of the landscape and be able to understand it and talk about it?

Sarah: I’d heard of Bitcoin. I had vaguely heard of Ethereum, which is the second most widely used cryptocurrency. But I didn’t know anything beyond that. And they said, don’t worry, you don’t have to know anything. We really just want a researcher to build out a team. And I said, OK. Later wondered if that was a horrible mistake, because that sounded scary and I had no idea what I was getting into. But I knew that I was going to learn along the way. I knew that if I stayed curious and just anthropological, I suppose, in being able to observe and learn about a new emerging technology, then it was going to be OK. And it was. And emerging technology is fascinating. I mean, it’s never the same. You wake up every day, it’s different innovations, different things happening in the greater environment. And so to study that from a research perspective and different communities and cultures that are popping up and different things that are suddenly hitting the scene and trends taking off, there is nothing really like it. Oh, yeah. Yeah, it does. I think that I mean, it’s it’s three times longer than I’ve ever been at any workplace.

Steve: You made that comment about at the beginning of the six years, you wouldn’t have imagined that it would be sustainable. Not your words exactly, but I’m getting that because it has changed, it continues to be, I don’t know, interesting, new.

Sarah: I think that’s probably why I wouldn’t have thought that I would stick around just because every job I’d had prior, it’s like you kind of hit that two year cycle of like, this was fun. I feel that I mastered some things about this. Am I bored now? And sometimes the answer is yes. And in this case, the answer just kept being no. I felt like I did different tours of duty. The role changed as I went and I felt like I was always learning. So focusing on consumers, we’re focused on regular people who are not companies.

Steve: So you’re focused on the retail part of the business. Can you paint a picture of what research looks like? You know, anything about sort of structure, activities, team?

Sarah: They’re not developers building apps. They are just regular people who are interested in buying and owning and using cryptocurrency. So because we’re responsible for Coinbase.com and the app, we’re also responsible for Coinbase Wallet, which is our slightly more intermediate crypto app. I’m happy to explain that if that’s interesting, but just, you know, people who are interested in engaging in the crypto space, sometimes as an investment. Sometimes they’re doing other interesting things with it. That is our retail customer. So it’s everything from the super early funnel of, okay, how many people own crypto? Who’s interested and what do they know about it? Where are they starting from both in the US and internationally? And then all the way down to, hey, we’re about to release a staking feature. Okay. Like what features should it have? Who’s going to use this and what are the requirements that they’re interested in for this feature? So it can get pretty deep. We have a pretty diverse user base. You can imagine that there’s a really big difference between the really advanced crypto users who are super into this stuff from like a hobbyist perspective, all the way to people who are just hearing about Bitcoin. They got interested in it. They figured they wanted to own or buy some. And what does the experience look like for them and everybody in between? There’s a large range. So we’re doing some foundational and strategic. We’re doing some long term stuff. We’re doing a lot of tactical stuff. We are doing a lot of concept testing.

Steve: What does your team look like?

Sarah: Working on retail and consumer right now, there’s six, seven, I suppose, including myself, eight, if you’re counting our lovely intern. And then there are a handful of other researchers working in the different product groups, not on retail and consumer. And so I believe it’s 10 or 11.

Steve: Before you said, oh, those groups could have directors of research but don’t.

Sarah: Right, exactly.

Steve: That’s those other researchers.

Sarah: It sort of gets into how we’re organized, which I know different research teams may or may not report into design or product or, you know, are you centralized? Where are you located in the business? We are in product reporting through whatever is the specific area of product managers that are our stakeholders. So we have our consumer product group, which is focused on retail and consumer. So that is headed up by a head of product who then has a senior director of design and research in our case, his name is Jeff. And then I report to him. So we have, we report into design, I guess, if you wanted to put it that way, who reports into product for retail and consumer. And that would be true of institutional, that’s true of developer. And so it’s interesting because, yes, we report into design, but we have a very close relationship with product as a stakeholder. We used to be centralized. There were previous iterations of the company where we had a VP of design and then, you know, we had a director of research who was across all the different product groups. And that came with its own challenges, as I’m sure anyone who’s worked on a centralized team would tell you. You’re kind of expected to cover everything and everything is your responsibility. Whereas when you are reporting into the product group that is also your stakeholders, obviously that comes with its own limitations and challenges. But at the very least, you’re close to where the decisions are being made. You’re close to the people that are, you know, working on the thing. And so that’s true of, we’re very close to our design stakeholders. We’re very close to our product stakeholders. And so we have a friendly and happy communal relationship with the other researchers and the other product groups. But fundamentally they are studying different audiences. They are working on different products. And so we have like a shared tool stack. We have a shared budget and we engage in some practices together like research crit and things like that. But oftentimes we’ll be working on different things.

Steve: For your folks, what’s the mechanism for determining what you’re going to work on?

Sarah: So obviously we’re very close to whatever are going to be the priorities of the product group that we’re in. They have OKRs, very common setup, but we’re not necessarily married to exactly what’s on their roadmap. I tend to think that a healthy diet for a researcher is some stuff supporting their roadmap where either they’ve specifically requested it or we have gone in as experts in subject matter experts and user research and said, “Hey, we think you should really do this research.” And then also having our own proactive ideas about maybe the things that nobody’s asking for. So we own our roadmaps. Product does not write them for us. We generally say based on what we’re observing of your priorities and what you’re working on, here’s our quarterly roadmap and what we’d like to do. And maybe they have some feedback or suggestions, but generally they trust us to say, “Here’s the research we think you should have.”

Steve: Yeah, can we talk a little about the research that you’re proposing that’s not directly tied to what the product’s roadmap is?

Sarah: So I think it’s a little bit of keeping your ear to the ground of what are the inklings that you’re starting to hear. They’re not asking for a project, but maybe there’s a particular topic that seems to be coming up a lot or a question that you hear leadership kind of grappling with. And you’re like, “I’m not sure if they know that research can help with this, but I know that research can help with this.” And so we’re going to proactively reserve some room on our roadmap to really give this a shot. And oftentimes, they’re grateful that we did it, even though they didn’t ask for it because you never know when they’re going to react really positively. It’s like, “Oh, I didn’t know you were working on that project, but I’m so grateful that you did. That’s super interesting.” Or maybe they do ask for it six months later. You’re like, “Wow, I’m really glad that six months ago I thought to invest in this research because you correctly anticipated the thing that they would be interested in.”

Steve: We could probably draw some lovely two by two because who doesn’t love a lovely two by two? But if there’s these categories of, you know, research that’s directly responding to things that are being done like OKRs and things that are about keeping your ear to the ground on one axis, I’m thinking about the other axis because you talked about foundational or tactical and sort of other ways of dividing up the research. I’m curious kind of how that maps out and maybe that’s not a fair question but to me it’s very different, I would guess, to say, oh, we’re going to, you know, on our own go evaluate some design directions or some interaction mechanism versus we’re going to go understand some motivations or some set of behaviors. OK, so I drew with my hands the two by two but I’m not even sure that’s the right way to think about it. Can you say a little more about, you know, how you might characterize or break down these studies that you’re identifying the need for?

Sarah: It’s definitely in the latter that you mentioned, which is better understanding a particularly emergent behavior or a motivation. Maybe there’s a competitor name or a particular space that we continue to hear coming up a lot and we’re like, hey, we don’t really have a lot of information about that or I can tell that there are some strategic discussions or decisions that it sounds like they’re kind of getting stopped up on. So it’s very much in the foundational category that we are proactively suggesting things. There are times that we go out there and we say, hey, this is a designer, a feature that hasn’t been evaluated in a while and we think that it’s important to go look at this. But I would say more often it’s in the foundational strategic. For example, one that we’re considering right now is there’s this particular trend that we’re seeing happen and we’re trying to figure out whether or not that’s a space that Coinbase wants to play in. This is something new that we’re seeing people do with crypto. We’d like to better understand exactly who’s doing this, why are they doing this? Is this an area where we think we have a competitive strategic angle or is this something that we could support in some different way? Or do we just really want to better understand this and now six months from now when a stakeholder comes and asks me, hey, what do we know about this? And I can say, oh, we looked into it.

Steve: Well, that leads me to a follow-up in terms of what happens with this research. It’s not responding to a request or it’s not tied to the — I’m sort of floundering even, like, what’s the label for the kind of research we’re talking about?

Sarah: I’ve heard it labeled many things because I have heard other research leaders talk about this often f like, what do you call it? Is it forward looking pathfinder, horizontal? I mean, it’s like there’s so many different buzzwords you could throw around. Like we all deal with this, right? We all deal with what do you do when you’re trying to get out in front of a question? So I’m not even really sure we know what we call it, but we certainly know it when we see it. And you can also see different research leaders have different strategies for how do you do that? Do you have a team that is specifically not dedicated to a certain group of stakeholders? They’re kind of like a centralized team. Do you get contractors for that? Do you pay vendors for that? Like how generally do you deal with it? And what I will say is that even though it can be hard to know what are the shots to take, like when do you take your roadmap and reserve a big part of it for this thing that you’re not even really sure if it’s going to land impact or lead to something? Every time we’ve done it, I’ve been so grateful that we really like scraped by and managed to find the space and time to do it in whatever way, because inevitably what happens. So here’s something about crypto. Crypto is cyclical. So the price of Bitcoin goes up. You get a whole bunch of new people who are very interested. They’re curious about this space. They’re hearing it in the news. They may not know a lot, but they’re excited to engage. And so then your doors are flooded with all of these people who are in this like beginner mindset. Then the price of Bitcoin goes down, as it always does.

And then suddenly you still have very active users, but it’s the people who are really hardcore in the space. They’re very hobbyist crypto people who never… They came a long time ago. They’re never going to leave. Like they’re just really interested in this space and they stuck around. But there’s fewer of them. They definitely engage with the product in a very different way. They have a very different set of needs. But inevitably, because, you know, last I checked, crypto has not died yet, even though they like to say Bitcoin is dead or whatever they say, it has not died. So inevitably the price goes back up again. Well, now, I mean, all the research that you’ve been doing, that user that you’ve been looking into and serving for the past year or two or however long it takes, now you’re flooded with the new users again. And so you actually, we invest a lot in lit reviews and summarizing past research, often as like the first step to a research study of, you know, like what have we looked into before? What is relevant? What is evergreen from the last time that we entered the cycle? And the time that we entered the cycle before, you can start to see longitudinal patterns. And that can be a very helpful way to guide where to go next, because you can’t always predict exactly how the market is going to evolve and what’s going to change. But you can say, we’ve been here before and here’s what we think might become relevant later. So you can try to call your shots. Sometimes, yeah, I would say we tend to do them in one of two cases.

Steve: Are you doing lit reviews to look for what those shots might be or what those speculative projects might be?

Sarah: The first is when a stakeholder comes to us and says, hey, what, you know, we want to do a research project on X. What do we know about X? It’s like, well, let’s have the very step number one be to check the archive and say, what do we already know? And sometimes we know a lot, sometimes we know a little, but the very least we can check the box and say, hey, maybe we don’t even need to do research. We actually already have a perspective on this. We’re able to save ourselves a lot of time and have a much more comprehensive strategic point of view for you. And 10 out of 10 times, the stakeholder is like, great, awesome. Like, I’m so glad that’s even better than the thing that I asked for. And we’re able to just machete that request away and move on. And then there’s the kind of more long-term cadence where maybe once or twice a year, I’d say, we’re doing it, where we’re genuinely looking across all the research that we did over the last six months and sometimes even prior and saying, what have we been learning? What do we know? What are the themes that are coming out? Let’s get together as a team. Let’s run that workshop. And we take a different approach every time. It doesn’t always look the same. We kind of experiment with different brainstorm formats or different structures for it, I guess you could say. But then the outcome is, here’s the trends we’re seeing. Here’s what we’ve been learning, big themes. And sometimes it identifies, here’s a big area that we don’t know about. It actually resulted in this recommendation for ourselves of this trend that now we want to go follow with our next six months. We revealed a knowledge gap for ourselves. So that, I’d say, we do maybe once or twice a year.

Steve: I want to ask just a very tactical facet of that because, I mean, this is a challenge for so many teams. What information do we have? Where is it? Who knows about it? How do we access it? I think you called it the archive.

Sarah: Yes.

Steve: How have you organized your information such that you can do this?

Sarah: I feel very passionately on this topic. I think that my team, if you were to ask them, what are the things I’m best known for? One of them is aggressively making people document in the archive their studies, because I’m such a big believer in pulling out past studies to not only save time for yourself, but also invest in those longitudinal data points and perspectives. It’s literally just a spreadsheet. I think a lot of people have this misconception that it needs to be some fancy tool. We’ve tried that. It didn’t work out because then we lost budget for fancy tool. Fancy tool went away. Now you’re scrambling to try to migrate the entirety of your archive, which in my case, goes back over six years, right, that we’ve been working in this space. And so we literally just use a glorified spreadsheet. And you’d actually be amazed at how many other research leaders I’ve met who also say, yeah, it’s basically just a glorified spreadsheet. It’s like all these tools that have their merits for sure, and yet for what you need, how much better is the glorified spreadsheet, right, Google Docs? And so we do have our glorified spreadsheet.

And I think that no matter what tool or what format you’re using, the most important thing is regular accountability for updating it. I think that that’s where a lot of archives fail is you have a nice one, but is it consistent with making sure that you’re putting new stuff in it and then tagging that stuff and making it easy for researchers to go back and look through? And so at the end of every month, we look across what are all the studies that happened this month? And everybody has to update the archive. And there are various ways that we keep ourselves accountable for that. And then we hold each other accountable for when there’s a new study request. Did you check the archive? Have we looked at this before? I think that it’s more of a cultural practice than it is any kind of magic bullet tool. Oh, my gosh, let’s pull it up right now, shall we?

Steve: What are some of the columns in the glorified spreadsheet?

Sarah: So I think that we’ve had a number of org changes, so I actually don’t pay a lot of attention to like what team or, you know, because team names may change, teams may change. I think what’s more important that we have is what was the top key insight? The single top key insight, not a list, the single top key insight and recommendation. And that can give you like your two to three sentence summary of what we found in this study. And if you’re interested, then you can click the link to learn more. Usually that contains all the keywords that you need. There is a spot to add additional keywords if your two to three sentences of your top key insight did not include those keywords. But also the DRIs, sorry, excuse me, that’s such a corporate term, the directly responsible individual who did this study, who was involved. Literally, those are the only things that you need. Yes. Yes, that is the researcher that did the study.

Steve: It sounds like very few columns. The DRI is the person that led the study, that did the research?

Sarah: Sometimes there’s multiple names listed. We also have the month and the year and the title of the study with the link to the report. And that’s it.

Steve: Is that archive just for researchers?

Sarah: The primary audience for the archive is researchers, primarily because I like it best when a stakeholder is coming to us saying, what do we think about X? Right? I mean, sure, we could give them the link to the archive and they could search it themselves. But that’s not the same thing as coming up with a research informed point of view, which I find is actually the primary value that we provide and the primary thing that they actually would like from us. Sure, they can go and search for past studies, but oftentimes they want to know, what do we think? Do we think we should move forward with this? What do we think the opportunity is? Which comes from a human researcher who has access to the archive more so than a PM or a designer running around looking for their own studies.

Steve: I was in a conversation recently and talking to people, they were running basically a mentorship matching organization. And so we were trying to understand how did that work? And they described so much about what kind of data was captured, all these different fields that they had, but then they said that most often their ability to match was that so-and-so who did intake for somebody remembered that they had just talked to somebody else. Like there was institutional knowledge or knowledge held among the people as opposed to the data itself. I’m thinking about them as you’re talking about this, like these tools invite a certain usage like you’ve captured everything so you can dive into it, but people work on stuff and they tell stories like culture kind of carries a lot of this. What kind of things happen when a stakeholder comes and says, “Oh, I have this,” is as I’m kind of surmising recall and interactive social, cultural stuff. Does that impact the sense of what we know versus maybe a more cold-blooded look at data and fields and reports and so on?

Sarah: The short answer is yes, I do think it does. I think there’s no replacement for the people who remember the last time and the last time that we did this study and the people who were on the receiving end of that request. And one nice thing, this is just luck, I suppose, is our research team is still very small. And it’s also a lot of people who have worked a long time in this space and at the company. And so they do remember the last time. And I think in those situations, having as a practice to rely on the fellow researchers and have a strong community of researchers who will phone a friend and say, hey, has anyone ever looked into X? Or does anyone remember a time when a stakeholder asked about X? And in addition to also combing the archive and seeing what exists, I think both are equally powerful and important in their own ways. And that a good comprehensive look at what we know is probably going to involve both. And not just researchers, right? I mean, what about also talking to the other important stakeholders and functions, PMs, designers, data people in our lives? And so I absolutely am a believer in that institutional knowledge. And I’ve seen what it looks like when you lose that institutional knowledge.

So I mentioned that our industry is very cyclical. Price goes up, price goes down. The user base changes. Well, also the size of the company has changed and the people working at the company have changed. I am absolutely a dinosaur by Coinbase and crypto standards. And so I remember the last time, am I the only person in the room that remembers when we discussed this, right? And we were in a period of hypergrowth where the company’s strategy at the time to address the growth was to just hire, just relentlessly hire. And it was like every time you talk to a person, that person was new. Their manager was new. Their team was new. They didn’t know anything. And it feels like Groundhog Day. You know, like I’m snapping here. It’s like Groundhog Day. Like you end up just rinsing and repeating the same things. And in those moments, I think that you’re not doing your job if you’re not looking at what we might have looked at before. You’d end up executing the same study every three months because every three months, you know, a stakeholder shows up and asks the same question. Where are your personas? That’s a common one. So I think that that’s where we really learned how much time we could save by keeping track of the last time that we did it. And fortunately, we are — I mean, fortunately, not fortunately, I don’t know. Non-normatively, we are not in a state of hypergrowth at this time. So the nice thing is there’s just less turnover with human beings. And so there’s less thrash with like the same types of questions and, you know, a lot of that chaos.

Steve: You said a bunch of interesting things, but one reaction is that phoning a friend, I tend to think of that, my own bias there is that’s kind of a failure, but you’re characterizing it as a success. That’s a valid way to tap into what we as a collective group of people who work together know.

Sarah: Yeah, absolutely. Because I think that a lot of knowledge is still in human beings. And we can try to automate away. We can try to make it factoids and bullets. But I think that our knowledge is cumulative. Like as researchers, we are anthropologists in this space. We are deeply putting together everything in our heads and having a point of view. And that point of view is what we offer. I mean, in 10 years, when all of our jobs have been automated away with all these AI tools or whatever, I mean, the last thing I think to go will be. But you, as a researcher, yes, I know all the findings, all the data points, all the studies. But what do you think we should do? Like what is your strategic recommendation? That is the thing that is hardest to automate away. And I think that that is the value that we most provide to the company and to our stakeholders and to each other.

Steve: I wonder if the discipline that you’ve created around updating the spreadsheet, it’s like why we take notes about things and then we remember them better. You know, as you’re talking about these two approaches, I think you’re saying they’re interrelated in an interesting way.

Sarah: Yeah.

Steve: So, yes, if you have a discipline of updating the database, the glorified spreadsheet, then that might make you better at recall when someone says, hey, what do we know about X?

Sarah: Absolutely. I mean, even if I were the person who conducted all of the studies for the last few years on a particular topic and I put them in the archive, I still might be going through and using the archive to bring all of those studies that I have done before to kind of bring them all out, look at them, think about them, come up with a take, maybe do some affinity mapping if you’re like a Miro enthusiast or something like that. And so it is helping me put together the data points that I have collected, regardless of who collected them. You’re kind of engaging in this coming up with the perspective. Okay.

Steve: Maybe we’ll go back to the kind of research that has no name. You gave the scenario a couple of times of, you know, someone, a stakeholder approaches you and says, you know, you’re able to say, like, well, we’ve actually done that research and here it is. And so that makes me think about, oh, you’ve got sort of the stuff kind of held back. And I was wondering when you do the research that I’m not effectively categorizing, yes,

Sarah: The research that shall not be named.

Steve: when you’re doing the research that shall not be named and you get to some point, you’re, quote, done with it. And you’re not really done, but, you know, wrapping that up, what kind of things happen with it?

Sarah: I think that in the ideal perfect world, somebody says this is revelatory and life changing and we’re going to go do everything you said we should do. And I think we all know that that doesn’t always happen. Sometimes it’s the opposite where it kind of is the tree that falls in the forest and there were people around to hear it, but they were fairly unmoved and it just wasn’t the right time, you know, or maybe there were some unanswered questions. It informed a V2, but it appeared to just stop where it was and it went into the archive and nothing happened. And I think that’s the thing that you kind of have to do is kind of put a flavor in second lives and third lives and fourth lives of research that you were kind of bummed because maybe it didn’t have the revelatory life changing impact that you wanted it to have at the time. And then you bring it out again and you say, actually, this is exactly what I needed. And maybe it just was a little bit ahead of its time. We’ve seen that happen a lot. Big believer in evergreen research. A lot of jobs to be done research is like this, I find. And that’s like a whole category because then you also have to help stakeholders understand exactly what to do with that. But a lot of that stuff, I mean, I cannot tell you how many times we have pulled it. We all have our favorite studies, right, where we looked at behaviors in banking. I mean, completely outside of crypto, just unrelated. How do people deal with their regular money? What’s important to them in banking? At the time, it was like, how relevant is this? I mean, most people aren’t seeing crypto as their bank. It’s very different. But the number of times that we’ve gone back and said, here are the canonical behaviors that we observe with people and money has been, I would do that study all over again. It was a great investment. And we just didn’t realize it necessarily one month, two months after we finished it.

Steve: You’re painting a picture a little bit of, you know, we talked about the second lives, then that first life, the research is completed and doesn’t get the uptake that it might that first go around. How does it get published or shared or socialized when it’s something that is on your roadmap and not on somebody else’s?

Sarah: We share it in largely the same way. We have certain established channels, cadences, ways that we summarize recent research. And it doesn’t really matter where it originated or even sometimes who on the team did it. We’re still going to do the roll up of like, here’s the biggest insights that came out this month. We have a channel where we share our completed work. And we would share it the same way as anything else. And at that point, it might be like, oh, my gosh, this is amazing. Or at that point, it might be crickets. And sometimes you have to go further beyond that. It’s like, what live channels do we have access to? Is there like a regular leadership monthly where we can present one or two insights from this? Or are we going to signal boost in different channel? I mean, obviously, the way that we work digitally and remotely impacts this a lot. We’ve had to think a lot about research communications in a variety of different ways. We’re never not thinking about research comms, actually, if I’m honest. And so you really try to make sure that you’re signal boosting it to the right audiences in the channels that they pay attention to in a variety of different formats in every flavor. And then if it still feels like it’s not generating conversation, then you file it away. And it’s amazing how suddenly in six to 12 months, they start asking again, and you’re really grateful you did it.

Steve: This to me gets at a complaint or an anxiety that researchers have. And I don’t know if this correlates to sort of the overall research maturity. You know, I can imagine someone listening to this and saying like, wow, you guys are going off and doing research that you think is important. And you’re like, I wish I had time for that. But let’s say I did have time for it. I did that.

Sarah: Right.

Steve: Well, no one wants it and no one cares about it. And so it would just be crickets. But you’re describing a larger strategy. As you said, you never are not thinking about research comms. So assuming relevance, assuming that, you know, everybody can understand the relevant research, you have a built out structure for comms, that kind of a baseline. And then you’re also thinking about on a piece of material or a set of insights specifically, how to go about that. And then I think, as you said, you know, you bide your time as well, because you have a handle on what the information is. It doesn’t have to sync. It can hang around because you have a way of retrieving that and saying, oh, yes, we do know something about X. But it’s something about Y.

Sarah: For sure. And I think you have to know the shots to take. I’m not saying people should just go do all this research and be OK if it doesn’t go anywhere. I’m a big believer in trying to make sure that you’re taking the most impactful shots that you can take based on your knowledge about the organization that you’re working in, like the space. And if putting together all the information, the best information you have at the time and saying, look, I really think this is a knowledge gap that strategically matters to our business and to what we’re doing. And if you’re right about that, then I do think it’s just a matter of time that somebody comes back and is glad that you did the thing. And there is no magic formula. I mean, it really is like you have to sometimes beg, borrow, and steal your time of like, yes, we all have a million requests that are coming from stakeholders all the time. And how do you still reserve the space? It’s hard. It’s hard for everybody. I wouldn’t say we’ve found a magic formula. Although, if I did have one tip of something that has worked, if there is something that you really want to prioritize that is proactively initiated and not a request from somewhere outside of your research team, what are you going to deprioritize? Are you good at deprioritizing something else? And that is a muscle that we’ve been able to work over time of like, okay, I really want to make sure that we have room for this foundational research this quarter. Can I make sure that my requests from stakeholders are not eating up my entire roadmap? Are there things that I feel that they don’t need as much as they’re saying or that aren’t going to be as impactful as they’re making it sound or that I think can be answered in other more lightweight ways? And if you’re good at doing that, then you’ll always be able to reserve your time for the other things. But that is hard to do. That’s hard for everyone. I do think that the best no’s sound like yes’s.

Steve: And you didn’t use the phrase saying no to things. That is often the label that’s put on that. What are best practices or guidance you might have in the deprioritizing and telling someone that whatever you have to tell them as a result of that?

Sarah: Because at the end of the day, I think it’s important to remember why your stakeholder is coming to you. They’re coming to you because they have a question or there’s a decision that they need to make. And there are all kinds of different ways to help them with that goal that don’t necessarily involve dropping everything you’re doing and running a study. It can be, well, can we run an A/B test? Isn’t that the better way to handle this? Or here’s an educated guess we can make based on this other study that finished recently. Or we have a process where certain very tactical, not super high stakes research, it’s a form of democratization we call partner-led research. So it’s not them doing it themselves. They are leading it as the partner. The partner is PM or design. And we are helping them, but we’re making sure that it’s like the proper method and tool for the job. Like we have a way of helping them execute it themselves when it’s something that’s fairly well scoped to just their area. And so there are all kinds of different tricks of like if they’re coming to you because they have a question or they have a decision they need to make, how do you help them answer that and make the decision in ways that don’t necessarily involve a study? There’s all kinds. And then they’re grateful because they get what they need. And it usually doesn’t sound like a no, we’re not going to do that. It’s like, thank you so much for coming to me. What an interesting question. Let me try to find the best way to answer that for you. And the best way could look a bunch of different ways.

Steve: So you’re helping. It’s not “Go away.”

Sarah: Yeah, I think it’s a solutions-orientedness in terms of they’re coming to you because they want a solution. And so whatever you do, it doesn’t necessarily have to fall within the bounds of research. Sometimes they really just need help reframing their question in the right way and actually realizing that it’s a data question, like a data science question. Or it’s talking through the options and realizing that actually we can probably just make a guess about this, like use our design instinct. It’s actually like just a design — how should I say this? We should use our instincts as professional designers to make a call. Like are we really going to run a study about whether the button should be red or green? Probably not. But they may not know that. And I can’t necessarily expect the stakeholder to always know when is the right moment to do research. That’s my job to know when is the right moment to do research. So if they’re asking me something, that is always a conversation that I’m ready to have. And it doesn’t matter whether it results in a study or not. I think in terms of roadmap, I think that they are the experts in their own domain, right?

Steve: How does it work with the researchers that are elsewhere in the organization that don’t have the same kind of leadership structure in terms of managing priorities and managing responsibilities?

Sarah: And so I trust that they are running their roadmap in a way that works for them. I’m not necessarily going to go try to micromanage that for them. But, of course, we are here if they ever want to bounce ideas or get a take. We also are very transparent with what we’re working on. They’re transparent with what they’re working on. Like we have a very collegial relationship. So I would say that that works with smaller teams because they can always just say, “Hey, has anyone looked at this?” But we also do certain shared practices. Like I mentioned, we have a shared crit where twice a week we get together. And any researcher from any part of the organization can bring a challenge or a topic or something they’d like some feedback on. We have a shared channel. And so if they’re really like, “Hey, I’m not sure whether or not I should prioritize this. A stakeholder is coming to me asking this, and I’m not really sure how I should think about it.” There is a community of researchers, myself included, who are really open to that. And so we have a shared community of researchers, myself included, who can always help bounce ideas.

Steve: So, crit might include what approach to take about a question.

Sarah: Yeah, absolutely. It can be anything. I mean, crit, I suppose, makes it sound like we’re always critiquing research plans or reports of findings. But actually it’s mostly just like a shared discussion forum where sometimes people come and they have a very challenging stakeholder management question. Like, “How should I deal with this?” Or, “What’s the deal with this tool?” Or, “How do I do this thing?” It’s really just an open forum for researchers supporting researchers.

Steve: Do you have any other research comms?

Sarah: Yes, always. Always research comms.

Steve: Do you have some examples of things that you’ve tried that didn’t work and things that you’ve tried that did work?

Sarah: I think you have to be very careful with regular newsletters. We’ve tried a couple of different iterations. I think we still have one monthly, but we really had to iterate many times to get it to be read by the right people. Something that’s very top of mind is having strong, crisp product recommendations, which I’m told is actually different than a lot of environments. I mean, I guess it’s been over six years since I worked in any other environment. But researchers are saying, “Hey, at other companies, oftentimes they want the insights. They don’t want me to tell them what they should do. They don’t want me to be specific about what products should do and what design should do. And I think you should build this, and I think you should make X into Y.” That is something we absolutely do. And not only do we do it, we’re held responsible for doing it. Like, we’re perceived as not doing our jobs if we are not having a strong point of view. And it doesn’t always come naturally to a researcher. We’re data people. We like hanging out at insights. We like the “how might we” statements. We kind of like the collaborative energy. And definitely there’s a way to do that. I wouldn’t throw it out. But at the end of the day, especially for a certain level of leadership, what they’re looking for is, “But what do you recommend?” And I want it to be three bullets, and I don’t want to have to read it very much. And I want you to be very specific, not just make X easier. You know, how should I be making it easier? What is the specific change? And that means there’s a lot of trust that our stakeholders have in us. Like, I don’t get in a lot of debates about methodology, for example. I don’t have a lot of people who want to sit through an hour-long presentation just so they can question whether I talked to the right audience or not. You know, like, they trust that we did our jobs right. What they want is to dedicate five seconds to reading my bulleted list of three things that I think product or design should do. And the more that we have put that at the front and center, the TL;DR is what we call it. It’s very internet lingo. The TL;DR at the top of the research newsletter, which I mentioned we still have. And really the three main TL;DRs from that month. Not everything, not extensive, very short, easy to consume, and it took a while to find the right recipe. And to identify who is the right person to read that, I don’t necessarily expect that everybody responds to email. Actually, our email is designed for one very specific leadership stakeholder, and it is tailored to how that person likes to consume information, just based on what we’ve observed about how that person requests information to be delivered by other teams, like how they like to receive updates. There’s a different stakeholder where we actually don’t expect that that person is going to read. That person hates email. That person, I use Slack. Another leadership stakeholder I want to make sure is listening, tends to listen very well when they’re live in a regularly recurring monthly meeting. And so I make sure that research always has one or two slides in that meeting. So I think you have to know exactly who you want to be listening, and you have to change your techniques depending on who that is. Which ultimately at the end of the day is really just understanding your user, right? Like except that in this case, the product is us as a team and what we provide, and the user is a specific stakeholder.

Steve: I want to follow up a bit on the specific recommendations part. I am always anxious about finding a balance because I think here’s the thing that we learned is no one does anything with it, but if you go too far without the context in here’s how we think you should address it or how we think we should address it. I want to set up a dynamic where someone can say yes and to that because they know the engineering constraints, they know some piece of software they’ve bought or they know what rolls out uphill versus downhill or they know all kinds of stuff. There’s other decisions being made and other expertise that they’re bringing. When I say we learned X and we think we should do Y, what I don’t want is to get shot down, no Y wouldn’t work and then we’re at the end of it. If I have to defend sort of how might we questions, I think it’s to try to lead to the generation of the best solution with what everybody knows. I totally get the more you can bring people closer to taking action on the research, the more impact you’re going to have.

Sarah: No, I completely understand where you’re coming from, because I think if you talk to every person on our research team, they’d say the exact same thing. They don’t want to get shot down because they don’t want it to be the end of the conversation. We had to completely change how we think about that. We had to become okay with getting shut down and to realize that that actually was not necessarily a worst case scenario. I’ll give you an anecdote that jumps to mind. We just did one of those regular roll-ups that I was talking about where we kind of look across the last six months of research and say, “These are big themes, and this is what we think you should do,” and we presented it to a very large leadership audience in a large forum. And we knew we were taking a big swing where we were being very opinionated with, “Here are the three things we think you should do.” And we get into that leadership. It was a presentation. We did this one synchronously live. And I’m like, “Okay, here’s the three major themes. Let’s talk about this first theme. What are people’s thoughts?” And literally a person raises their hand and says, “I think this is a terrible idea. I do not think that we should do this.” And then the next person said, “You know, I mean, maybe we shouldn’t do it in that way, but I actually think we could do it if we did it in this different way.” And then the next person chimes in and says, “You know, actually, I’m somewhere in between you two. Like, I see the value in this, but maybe…”

And it is so funny because we have been beating our heads against the wall for years because we felt like our stakeholders were constantly being like, “You need to be more specific with your recommendations. You need more of a TL;DR. You need to have more of a point of view.” And we were like, “We don’t know what you mean. Like, how are we not being specific?” And we realized that it’s because we were afraid to get shot down. And what we didn’t realize is that they will perceive us to be not doing our jobs if we don’t come out with a spicy opinion. They want the spicy opinion. And if we are disagreed with, then that actually is going to lead them to other different ways of coming up with different solutions way faster than a “how might we” statement would. It was crazy. And I think that this could be a way in which the Coinbase environment is different than other environments. Like, it turns out that after all that time, the best way to get them to ideate different solutions was to throw out an idea and a recommendation that they hated. And that did more than a “how might we” statement ever did. It was unbelievable. And so now that’s been really encouraging the whole team to be like, “It’s okay to get shot down. It’s okay to get disagreed with. It’s okay for people to think that that isn’t what we should do.” Because to express a point of view, a strongly held opinion, is better than not expressing one at all. Because if we don’t express one at all, they’re going to be like, “Oh, I just wish our research team had stronger recommendations.”

Steve: You reminded me of this article which I’ll link to in the show notes. This guy is basically describing, I think it’s the McDonald’s theory. And he’s working on site with people in their pre-times and having this conversation every day at lunch. And he says, “Do you want to go for lunch?” “I don’t know, I don’t know, I don’t know.” And he decides to facilitate the resolution of that by proposing McDonald’s. And then everyone says, “No, let’s go…” They come up with specific choices which they weren’t doing beforehand. And he says basically the same thing that you do. People will come up with good ideas to fend off bad ideas. And so I don’t think you’re proposing recommendations that you think are bad ideas. But you’re proposing a solution to a problem. And so that’s what the audience does in that example. And they are working to resolve them.

Sarah: Yeah, that absolutely resonates where they want to know what we really think. And sometimes they’ll find a report and the report has a very clear, you know, like, here’s what we found and, you know, therefore here’s our recommendations and whatever. And that stakeholder will actually DM me and say, what do you think? Like, even if what I think is actually some version of what the person said, you know, like what the researcher put, what they want is what I think. That is the value that I provide, even if I’m really just kind of like restating what they said. And so I think it’s a good reminder of like, they don’t have to agree with me, but they do need to know what I think. And if I think something that they don’t agree with, at the very least, they know what I think.

Steve: What does shot down mean? That they disagreed with the recommendation in that meeting. But there was a robust conversation about what happens next.

Sarah: Totally. I mean, that, that to me is a success. I would say the failure state is where they just don’t trust that the research is valid. And fortunately, that’s not a situation we ever, I mean, I’m not saying we never find ourselves in that situation, but it is, it is rare. They trust that our research is valid and now it’s an equally valid discussion about what to do with it. And there could be a variety of different opinions and a variety of different inputs, right? We’re not always just going to go do what users said we should do. Right. I think that’s where it starts to become what’s the strategic opportunity. What’s the business opportunity? What’s, you know, what are the other data sources that we have? What about instinct? What about all these other factors that ultimately lead us to a good decision as a company with research and user insights just being one component of that?

Steve: I’m gonna go back to something you said early on that you’re talking about these six years. And you’ve described changes in crypto and then the changes in the company as a response to that. And you said you’ve done these different tours of duty that your role has changed over those six years. What are some moments or eras that are different than the one that we’ve been talking about?

Sarah: Well, there was the era of Sian Townsend. Hi, Sian. I hope you’re listening. We love you. There was a time when I mentioned that we had a centralized research team and there was a leader of that centralized team back when the team was like closer to 30. I lost count. I don’t even know. It’s much larger than it is now. And the leader of that research team was named Sian Townsend. And that was the first and only time that I have ever reported to another research person. I’d only ever reported to design managers, people in products. And that’s great. But ultimately, you’re not talking to somebody who deeply understands your practice and your function. And so for me, I got so much out of that couple of years where I learned how to be a research leader in ways because I’ve only ever worked at small companies. I will admit I’m a little bit of a bootstrapped case where I’ve worked at smaller companies and/or on smaller research teams that just don’t have the level of structure and process. Like I’ve never worked at Facebook or Google or some of these larger research teams where you just learn the gold standard of how a lot of these things are done. And a lot of these problems that sometimes small company people encounter every day, those problems have actually been solved in a different way. But that’s why I think sometimes you go to these research meetups and you see a bunch of research teams of one or people who maybe just have one or two other researchers they work with. And they’re like, “Hi, how do you address this problem? Please help me.” And there’s such a community amongst researchers where we struggle with these challenging problems.

And to see the way that she had the tools in her arsenal to be able to handle these conversations at a level that I had not previously been exposed to was the best mentorship that, I mean money can’t buy that kind of mentorship. So this is like a big love letter Sian, but that is an era, and that was also an era when the team and by the team I mean the research team, but I also mean Coinbase as a company, we were scaling enormously. I mean we were in that period of hyper growth I mentioned where it was like we were three-xing by the end of the year and it was just constant hiring and constant new people. And that was intentional on the part of the company to really try to scale hugely to meet the opportunity, but that comes with its own drawbacks and challenges and it was just a very different time where we were a much bigger team, we dealt with much bigger team issues. Right now we’re back to being fairly small, which is nice, it’s like comfort zone for me, but I learned a lot in that period. What is the difference between a job to be done and a job to be done?

Steve: Is there anything you’ve done recently as a research leader that you can trace back to something you learned from working with Sian?

Sarah: I had been familiar with Jobs To Be Done, but I had not actually lived it and breathed it and we still reference that stuff today. I think that we no longer necessarily do the roadshow of like really actively trying to educate our stakeholders on jobs to be done, we tend to bring it up more in context of like, “Hey, this actually sounds like let’s talk about this in terms of jobs to be done.” So we tend to talk about it more at a small scale, but that’s because we’re no longer educating the company, the company now kind of understands and now we’re just like teaching them like ways and times to like bring it out and use it and how to apply it in various situations. So that was like the first thing she did, she showed up and she was like, “You all need to read the jobs to be done playbook and I’m buying it for you all.” So we literally did a book club and read it, but we still use a lot of that stuff today. So that’s definitely one that jumps to mind.

Steve: You just made note of having worked primarily in small organizations before Coinbase. And maybe just to riff on that a bit, could you talk about your path? How did you find user research and what did that look like to bring you up to Coinbase?

Sarah: It’s always fun to relive this. I majored in cognitive science with a concentration in linguistics in college and undergrad. And I had a fellowship as a lab assistant in a speech production lab and great people, but I just really did not enjoy the academic environment. So that led me to believe that grad school was probably not the right path. I just wasn’t really sure it was out there. I didn’t really know what I wanted to do. I just knew that it was a major that I had enjoyed, but I set it aside for a while and said, “I’ll just go out there and get a jobby job because I don’t really know.” And then I spent the first two years of my career being a data analyst and wow, I have so much respect for data analysts, people who are in there doing the work of pulling the SQL, understanding the table structures, creating the dashboards, the charts. That was me and that is a rough job. I have so much respect for those people. And I was like, “Oh, it’s definitely not that. I definitely do not want to do this anymore.” So I quit that and I was like, “What do I want to do? Do I want to do HR, PR? What do I want to do?” And I thought back to undergrad where it was not like it is now where you have master’s programs for UX design. I mean, it was around, but I wasn’t exposed to it. It didn’t have such a strong presence in undergrad, but I’d taken one class in human-computer interaction. It’s just like an elective. I didn’t even know what it was. And I actually loved it. We worked on a project. We actually won best project. And the award, this is so ironic in retrospect, the award for best project was you get a paid internship doing essentially user research. And I had declined it because I had already committed to join this data analyst full-time role after graduation. And so I was like, “Maybe I should have done that paid internship. Maybe I got it all wrong.” And so that’s when I started thinking maybe this human-computer interaction thing, maybe this is a job that you can have. Maybe this is a career. And I just was finding anyone who would take a chance on me. I was in Chicago at the time, so a smaller, more nascent startup scene. And so I was just knocking on doors of smaller companies. I think I was even trying to get hired as a project manager just because then I could do an internal hop or something. And I found a very small agency that had a research contract, a research project from one of their regular clients that they didn’t know how they were going to fulfill. And they were like, “Oh, you want to do this? We’ll hire you to do this contract.” And I was like, “Yes, please.” And I’m so eternally grateful to that agency for giving me that chance because I’ve been a researcher ever since and I love it. I’m so glad that I am lucky enough to have found a job that I love.

Steve: I also came through a different era, but human-computer interaction was my toe in the water or peek through the keyhole if there was this thing.

Sarah: Yeah. It’s funny to watch how the terms evolve over time. Human-computer interaction, HCI, I still see classes that are called that. You still have academic texts that are called that, but no one would say that they would work in HCI anymore or anything like that. But I think a lot of us, that was one of the first words we heard to describe this thing we thought was addictive and wonderful.

Steve: Is there anything that you’d want to share about yourself maybe that’s outside the professional realm that would give us a little picture of you?

Sarah: Yeah. Well, we just bought a house. So first time homeowners. I’m based in Seattle and it’s fun to do all this stuff. I waited a while to own a house, I guess. I was very cool being a renter, but getting to know all kinds of things like researching lawnmowers and things, although mostly that’s my fiancé. I have to credit him. He’s doing most of the researching on lawnmowers. Well, here’s the thing. We are level one sommeliers, actually. We have done the training in the court of master sommeliers. We’ve done the wine testing, taken the tests and we love wine. And so that’s a hobby we have on the side. He works in food and beverage. And so that’s a thing we really like to do. We like to go to wine country and we like to do wine travel in other countries. So yeah, I have my little certificate right here, my sommelier certificate. I mean, short of becoming a professional sommelier, mostly just like, I don’t know, impress people with it. I don’t know. He might study for his level too, because I think at a certain point it does start to matter. But I think being a professional sommelier is not necessarily the most common job in the world. I think you use it to plan trips and be a nerd and I don’t know, impress your friends. That’s what I do with it.

Steve: And you did that certification process together.

Sarah: Yeah, we did. It was actually fairly early in our relationship a couple years ago. And I had mentioned that it was something I was always interested in. Having lived in San Francisco prior, I spent a lot of time in wine country and got curious about the words that I was hearing and what I was drinking. And so when I met him, I had already had an interest in doing it. And he was like, “Oh my gosh, what a coincidence. Me too. Let’s do it together.” And I was like, “Well, I guess we only just started dating, but okay.” And actually next year when we get married, we’re getting married in wine country at a winery. So I guess it’s just what we do now. Yes, we love it. I love that show. I would

Steve: There’s this television show, and I wonder if you’ve seen it, if you have an opinion about it, called “Drops of God.”

Sarah: Yes, we love it. I love that show.

Steve: Yeah, how would you describe that show?

Sarah: describe it as very well researched. I mean, I’m not like the world’s foremost wine expert or anything, but I really enjoyed it. And I found that the wine details were dead on. It’s also just, it has that melodramatic flair that is a guilty pleasure for a lot of people. We really enjoyed it. We’re super excited for season two.

Steve: To me, and I think that show was about incredible expertise, right? It’s about a woman whose strange father dies and is some kind of just global expert in wine with this amazing book. And he’s left France. She’s French. And he’s been in Japan, and he’s mentored this Japanese man. And when he dies, he leaves everything to one of the two of them, and there’s this competition that they have to go through. And the competition hinges on this incredible level of expertise that they have in tasting, identifying, and there the drama unfolds.

Sarah: Oh yeah. I love that philosophical question of like, she was like born into it, right? Like she has this instinct of she hasn’t looked at wine in a while, but she just has this great palate. And then he is the trained, the mentored, he’s very well studied. Like he worked really hard to get where he is. And so it’s a little bit like, is it innate? Can you learn it? Like it’s sort of asking these questions about the knowledge that we have is one more legitimate, is one better. So I won’t give away the twist, but people should watch it.

Steve: That’s a great framing on it, because I don’t personally care about anything about wine, but I loved that show because it was about knowledge and passion for a topic.

Sarah: Yeah, absolutely. It’s like, what can you own and claim as like, I have the right to this. I deserve to win this competition. Particularly with something that a lot of people don’t know a lot about wine. A lot of people are kind of intimidated by wine. That’s a reaction that I often get is like, oh, well, I don’t know anything. I really, I don’t. And it’s like, I mean, you know, like you just drink what you like. I’m not going to judge anyone’s wine taste. Like I love stuff you can get at a grocery store or whatever. It doesn’t need to be fancy. And I think particularly because people feel that they don’t know a lot about it. It was like a fascinating topic area to choose for that question.

Steve: Is there anything you think we should have talked about today that we didn’t get to?

Sarah: I mean, you didn’t ask me to sell you on cryptocurrency. My explanation was horribly basic. And you’re not just going to say, oh my gosh, please explain to me, extol the virtues of blockchain.

Steve: I actually can’t tell if you’re being sarcastic or serious right now.

Sarah: No, no, no. I mean, I think that people, I love talking to people about crypto and blockchain because I realized that it can be kind of like a weirdo topic for some folks. Equally intimidating like wine, actually, similar to what we were talking about. But some people are like, you know what, I will ask you if I’m curious about the details of how blockchain works. But maybe not everybody wants to be hearing about it constantly all the time. And I don’t necessarily want to. I don’t know, like if people want to go there, that’s great. But if they don’t, they don’t. I mean, you can imagine at a cocktail party, it used to be easy. I used to work for Ancestry. And people were like, oh, what do you do? I work for Ancestry. Oh, my grandma loves Ancestry. My aunt, you know, like I love it. I did my family tree. Oh, I work for Coinbase. What’s that? A cryptocurrency company. A lot of people are like, oh, like you can tell immediately the people who are like, please explain blockchain to me versus the people who are like, and I’m going to change the topic. I wish you just said you were an accountant. So, you know, I try to just I never know how deep to go, but I do find it a very interesting space.

Steve: I mean, to be transparent, I was, you know, when we talked about it at the beginning, I’m thinking about what would I ask you versus what would you and I ask people listening to this to listen to? So, yes, how much to ask, how far to go.

Sarah: Yeah, I didn’t know what to what to expect. I mean, I didn’t know if this was going to be like, so, you know, like explain blockchain. And it’s great. We actually talked mostly about research stuff, and it’s amazing how industries can be different. But research as a practice stays very much the same. But crypto has a weird brand associated with it. I think that it can there’s just a lot of like weird PR that makes it feel a little like ooky. And I can speak from a first person perspective is very off putting when you’re you’re first in it and you’re like, who are these crazy people that would want to work in this space? Like, is this even legit? And I can understand like, for a lot of folks, they’re just kind of like, yeah, if I really wanted to be like, there are plenty of people running around extolling the virtues of blockchain. And I’d be more than happy to be one of those. But I realize that sometimes it’s like, let’s just talk about the research.

Steve: Well, maybe you can take us out on a nugget or theory or something blockchainy, since you’ve kind of teased us now. Maybe we can wrap on that.

Sarah: I think most people when they first get into crypto, it is an investment. And that’s okay, because investing is not bad. Investing is great. And, you know, that’s very cool when people are interested in investing in it. And we have a lot of like financially related, you know, features and things. But crypto is not just about making money. And those are the parts that are fascinating and kind of change often and are still super early and nascent. And to a lot of people are like, what are you talking about? Like, that’s craziness. Like the idea that I said that it was like programmable money. They’re like, what even is that? But it can be dense, but you don’t have to be an engineer or a very aggressive, like financial person. There are parts of it that are equally interesting. So anyway, but like, I encourage people to be curious about all the different things that can be done with blockchain that are interesting to a lot of different people. No, I don’t. I don’t think so. And like, it doesn’t always evolve in the ways that

Steve: It kind of loops back to your point from much earlier that, you know, there’s a reason that you’ve stuck around for six years, your whole am I bored yet? And you’re describing a technology that you’re involved in that has potential that it hasn’t come anywhere near reaching.

Sarah: you think it’s going to evolve, right? Like, do I think that Bitcoin is going to replace the US dollar tomorrow? No, nor do I necessarily think it should. I don’t know. Maybe that’s a debatable topic. But I mean, different things, like people got really sick of hearing about NFTs. Understandably, they were everywhere. And people are like, why would I even? This is like a JPEG. I do not get this. Like, why is this constantly everywhere? Like, I get it. I get how it feels. But there’s cool things that you can do with NFTs. Like, I won’t bore you, but like, what if there was a way to verify your identity without having to share any personally identifiable information? Like, it’s basically an attestation token of attesting to your identity without having to actually share information. I mean, are a ton of people doing that today? No, some are. But I think that that’s where there’s still a lot of interesting technological development in the space. So is it an empty promise? Is it going to be the thing that we all do tomorrow? Is it somewhere in between? I think that’s where you kind of have to stay tuned to find out.

Steve: Sarah, thanks for a great conversation. We covered a lot of stuff and went into a few different nooks and crannies. It was really great to have you on Dollars to Donuts, so thank you for the time today.

Sarah: Thank you. It was great to be here. Thank you.

Steve: There ya go, whaddya know, that’s the show! Find Dollars to Donuts wherever you get podcasts, and at Portigal dot com slash podcast for all of the episodes with show notes and transcripts. I would love love love for you to rate and review Dollars to Donuts on Apple Podcasts. Our theme music is by Bruce Todd.

The post 49. Sarah Gregory of Coinbase first appeared on Portigal Consulting.
  continue reading

63 episodes

Artwork

49. Sarah Gregory of Coinbase

Dollars to Donuts

208 subscribers

published

iconShare
 
Manage episode 424477991 series 62327
Content provided by Steve Portigal. All podcast content including episodes, graphics, and podcast descriptions are uploaded and provided directly by Steve Portigal or their podcast platform partner. If you believe someone is using your copyrighted work without your permission, you can follow the process outlined here https://player.fm/legal.

This episode of Dollars to Donuts features my conversation with Sarah Gregory, Director of User Research for Consumer at Coinbase. We talk about research comms, archiving user research, and doing research that no one is yet asking for.

Our email is designed for one very specific leadership stakeholder, and it is tailored to how that person likes to consume information. There’s a different stakeholder that hates email. That person, I use Slack. Another stakeholder tends to listen very well when they’re live in a regularly recurring monthly meeting. And so I make sure that research always has one or two slides in that meeting. You have to know exactly who you want to be listening, and you have to change your techniques depending on who that is. Which is really just understanding your user, right? – Sarah Gregory

Show Notes

Help other people find Dollars to Donuts by leaving a review on Apple Podcasts.

Transcript

Steve Portigal: Welcome to Dollars to Donuts, the podcast where I talk with the people who lead user research in their organization. I’m Steve Portigal.

I was recently spoke with Rally’s Lauren Gibson for an Ask Me Anything session about user research maturity. I’ll link to the detailed writeup and the recording of the full conversation, but here’s a short clip.

Lauren Gibson: I’d really quick, like to go back to the skills point you were mentioning. Are there any group or team skills, like say a team is like, we’re trying to upscale or like hiring staffing. What skills would you say would be good ground zero ones to add to your team or have your team focus on to kind of increase that maturity?

Steve: I have a bias here. My bias is about the work that I do and the stuff that I write about and that I teach. So I’ll own that, but I wouldn’t say interviewing skills, just like I wouldn’t say survey writing skills or Qualtrics skills. I think there’s that piece above it that makes us better as researchers kind of method aside, domain aside. And those are things like, and these are big words, which you have to chew on a little bit to get to where they’re meaningful. But, you know, we talk about words like empathy and we talk about words like curiosity and listening. I think there’s a big skill around sort of not just knowing yourself, but kind of hearing yourself. We work in fast paced environments. We are asked to be experts. And so developing comfort with not knowing and being able to be confident and curious to be able to honestly ask a question that we think we’re supposed to know the answers to. Like these are, these are about knowing ourselves and kind of hearing ourselves and being that person in the meeting that asks the question that no one’s willing to ask, being able to say something. So this is maybe about storytelling, but there’s an empathy and compassion aspect to it as well. Being able to do some research and bring it back in a way. And I don’t mean what is your deliverable look like? I mean, how do you kind of set yourself and how do you talk to a person that you have a relationship with so that you help them hear something that’s new or that’s slightly new, but that’s impactful and significant and might suck for them to hear, right. We’ve learned that the thing that we’re doing is not going to work. And there’s another new problem to solve. Like that’s a great thing to find out, but presented without compassion and without some nuance can be seen as harmful.

And so these are like emotional maturity, hearing your own discomfort, being sensitive to other people’s discomfort, being a good storyteller. These are all in service of relationships. I guess that’s maybe what it kind of gets down to the relationships that we can build with people. And sometimes they’re like at a distance relationships, kind of like we’re having with everybody today. We don’t all know each other and we haven’t spent weeks and weeks in the same room and kind of shared ownership, but we’re all trying to connect and share information and help each other learn and draw from this. So we’re in a lot of environments and we work in different ways where we want to use these kinds of self-knowledge and emotional maturity to kind of build relationships because that’s how these things that we’re trying to accomplish. And change. So, yeah, if I want to talk about upskilling, I think those are the things I would kind of work on because they pay off across the board. And you can do that explicitly. Like I have taught a storytelling workshop. Like you can say, lean into that, but these are also side effects from practicing any of the more technical skills of research. You want to practice survey writing and trying to go to survey. You’re going to learn humility. You’re going to learn empathy, right? It is baked into everything, I think that we do, if we are reflective about our own learning.

Check out the whole session, if you like. And why not buy your nail salon worker and your tax preparation specialist their very own copies of the second edition of Interviewing Users. If you really wanna help me out, write a very short review of Interviewing Users on Amazon.
As always, I’d love to talk with you about the challenges your team is facing and how I can help.

Okay, let’s get to today’s episode with Sarah Gregory. She’s the Director of User Research for Consumer at Coinbase. Sarah, it’s great to have you on the podcast. Thanks for being on Dollars to Donuts.

Sarah Gregory: Thank you, it’s an honor to be here.

Steve: It’s an honor to have you. Can we start off with an intro from you?

Sarah: Yeah, absolutely.

Steve: Are there other directors of research?

Sarah: My name is Sarah Gregory and I am a director of user research at Coinbase, specifically working on our consumer and retail products.

Steve: There are currently no other directors.

Sarah: I am the only one, but we do have different product groups that could have a director at some point. Right now, I’m technically the only one.

Steve: And what is Coinbase?

Sarah: So we are a cryptocurrency company. So it can be anything from where you first bought your first Bitcoin. A lot of people tell us that their first Bitcoin ever they bought on Coinbase. But since then, we also have institutional products for institutions that want to custody cryptocurrency. We have developer products that for people who are actually building with blockchains. So we have all of those, which is the product groups that I mentioned. So I specifically focus on our retail and consumer presence, which is Coinbase.com, the big blue website, the big blue app, our retail customer.

Steve: If that’s retail, does that mean that’s consumers?

Sarah: Yes, consumers.

Steve: I hesitate to sort of open the lid on Pandora’s box here, but for the context of this conversation, what’s the minimum viable explanation of crypto Bitcoin blockchain that you can provide to people like me that know what you’re talking about, but don’t know what you’re talking about? Sorry.

Sarah: Oh, my goodness. I wondered if this one was coming. I hope I do it justice. No, it’s OK. I’ll do my best. So it’s the most famous cryptocurrency is Bitcoin. But there are others. And you can think of it as programmable money. So there are some cryptocurrencies that are just trying to be money, new money. You could use them for paying people. You can trade with them. You can do all kinds of money things with them. So you can have a programmable, borderless money that is not controlled by a nation state. Or there are different other things you can build with some cryptos. You can actually create like different kinds of economic situations. You can do things that are unrelated to money. Not censorable, so you can build something that allows you to ostensibly tweet on a blockchain that could never be removed because it is on blockchain. So there’s all kinds of different things that you can do. So yeah, that’s what I got.

Steve: And how long have you been working at Coinbase?

Sarah: Over six years now, which is crazy. I never would have thought I would last that long. If you had asked me six years ago if I would still be working in crypto in six years, I would have said you’re crazy. You know, it’s just a very fast changing environment. It’s a fast moving industry. I didn’t know very much about it when I started. So I would have thought that there would be some reason to move on and do something different. But it turns out it’s really fun. So there you go. Well, I certainly didn’t know very much when I started.

Steve: Going back over those six years, do you recall the journey, if it was a journey, just to sort of understand the specifics of the landscape and be able to understand it and talk about it?

Sarah: I’d heard of Bitcoin. I had vaguely heard of Ethereum, which is the second most widely used cryptocurrency. But I didn’t know anything beyond that. And they said, don’t worry, you don’t have to know anything. We really just want a researcher to build out a team. And I said, OK. Later wondered if that was a horrible mistake, because that sounded scary and I had no idea what I was getting into. But I knew that I was going to learn along the way. I knew that if I stayed curious and just anthropological, I suppose, in being able to observe and learn about a new emerging technology, then it was going to be OK. And it was. And emerging technology is fascinating. I mean, it’s never the same. You wake up every day, it’s different innovations, different things happening in the greater environment. And so to study that from a research perspective and different communities and cultures that are popping up and different things that are suddenly hitting the scene and trends taking off, there is nothing really like it. Oh, yeah. Yeah, it does. I think that I mean, it’s it’s three times longer than I’ve ever been at any workplace.

Steve: You made that comment about at the beginning of the six years, you wouldn’t have imagined that it would be sustainable. Not your words exactly, but I’m getting that because it has changed, it continues to be, I don’t know, interesting, new.

Sarah: I think that’s probably why I wouldn’t have thought that I would stick around just because every job I’d had prior, it’s like you kind of hit that two year cycle of like, this was fun. I feel that I mastered some things about this. Am I bored now? And sometimes the answer is yes. And in this case, the answer just kept being no. I felt like I did different tours of duty. The role changed as I went and I felt like I was always learning. So focusing on consumers, we’re focused on regular people who are not companies.

Steve: So you’re focused on the retail part of the business. Can you paint a picture of what research looks like? You know, anything about sort of structure, activities, team?

Sarah: They’re not developers building apps. They are just regular people who are interested in buying and owning and using cryptocurrency. So because we’re responsible for Coinbase.com and the app, we’re also responsible for Coinbase Wallet, which is our slightly more intermediate crypto app. I’m happy to explain that if that’s interesting, but just, you know, people who are interested in engaging in the crypto space, sometimes as an investment. Sometimes they’re doing other interesting things with it. That is our retail customer. So it’s everything from the super early funnel of, okay, how many people own crypto? Who’s interested and what do they know about it? Where are they starting from both in the US and internationally? And then all the way down to, hey, we’re about to release a staking feature. Okay. Like what features should it have? Who’s going to use this and what are the requirements that they’re interested in for this feature? So it can get pretty deep. We have a pretty diverse user base. You can imagine that there’s a really big difference between the really advanced crypto users who are super into this stuff from like a hobbyist perspective, all the way to people who are just hearing about Bitcoin. They got interested in it. They figured they wanted to own or buy some. And what does the experience look like for them and everybody in between? There’s a large range. So we’re doing some foundational and strategic. We’re doing some long term stuff. We’re doing a lot of tactical stuff. We are doing a lot of concept testing.

Steve: What does your team look like?

Sarah: Working on retail and consumer right now, there’s six, seven, I suppose, including myself, eight, if you’re counting our lovely intern. And then there are a handful of other researchers working in the different product groups, not on retail and consumer. And so I believe it’s 10 or 11.

Steve: Before you said, oh, those groups could have directors of research but don’t.

Sarah: Right, exactly.

Steve: That’s those other researchers.

Sarah: It sort of gets into how we’re organized, which I know different research teams may or may not report into design or product or, you know, are you centralized? Where are you located in the business? We are in product reporting through whatever is the specific area of product managers that are our stakeholders. So we have our consumer product group, which is focused on retail and consumer. So that is headed up by a head of product who then has a senior director of design and research in our case, his name is Jeff. And then I report to him. So we have, we report into design, I guess, if you wanted to put it that way, who reports into product for retail and consumer. And that would be true of institutional, that’s true of developer. And so it’s interesting because, yes, we report into design, but we have a very close relationship with product as a stakeholder. We used to be centralized. There were previous iterations of the company where we had a VP of design and then, you know, we had a director of research who was across all the different product groups. And that came with its own challenges, as I’m sure anyone who’s worked on a centralized team would tell you. You’re kind of expected to cover everything and everything is your responsibility. Whereas when you are reporting into the product group that is also your stakeholders, obviously that comes with its own limitations and challenges. But at the very least, you’re close to where the decisions are being made. You’re close to the people that are, you know, working on the thing. And so that’s true of, we’re very close to our design stakeholders. We’re very close to our product stakeholders. And so we have a friendly and happy communal relationship with the other researchers and the other product groups. But fundamentally they are studying different audiences. They are working on different products. And so we have like a shared tool stack. We have a shared budget and we engage in some practices together like research crit and things like that. But oftentimes we’ll be working on different things.

Steve: For your folks, what’s the mechanism for determining what you’re going to work on?

Sarah: So obviously we’re very close to whatever are going to be the priorities of the product group that we’re in. They have OKRs, very common setup, but we’re not necessarily married to exactly what’s on their roadmap. I tend to think that a healthy diet for a researcher is some stuff supporting their roadmap where either they’ve specifically requested it or we have gone in as experts in subject matter experts and user research and said, “Hey, we think you should really do this research.” And then also having our own proactive ideas about maybe the things that nobody’s asking for. So we own our roadmaps. Product does not write them for us. We generally say based on what we’re observing of your priorities and what you’re working on, here’s our quarterly roadmap and what we’d like to do. And maybe they have some feedback or suggestions, but generally they trust us to say, “Here’s the research we think you should have.”

Steve: Yeah, can we talk a little about the research that you’re proposing that’s not directly tied to what the product’s roadmap is?

Sarah: So I think it’s a little bit of keeping your ear to the ground of what are the inklings that you’re starting to hear. They’re not asking for a project, but maybe there’s a particular topic that seems to be coming up a lot or a question that you hear leadership kind of grappling with. And you’re like, “I’m not sure if they know that research can help with this, but I know that research can help with this.” And so we’re going to proactively reserve some room on our roadmap to really give this a shot. And oftentimes, they’re grateful that we did it, even though they didn’t ask for it because you never know when they’re going to react really positively. It’s like, “Oh, I didn’t know you were working on that project, but I’m so grateful that you did. That’s super interesting.” Or maybe they do ask for it six months later. You’re like, “Wow, I’m really glad that six months ago I thought to invest in this research because you correctly anticipated the thing that they would be interested in.”

Steve: We could probably draw some lovely two by two because who doesn’t love a lovely two by two? But if there’s these categories of, you know, research that’s directly responding to things that are being done like OKRs and things that are about keeping your ear to the ground on one axis, I’m thinking about the other axis because you talked about foundational or tactical and sort of other ways of dividing up the research. I’m curious kind of how that maps out and maybe that’s not a fair question but to me it’s very different, I would guess, to say, oh, we’re going to, you know, on our own go evaluate some design directions or some interaction mechanism versus we’re going to go understand some motivations or some set of behaviors. OK, so I drew with my hands the two by two but I’m not even sure that’s the right way to think about it. Can you say a little more about, you know, how you might characterize or break down these studies that you’re identifying the need for?

Sarah: It’s definitely in the latter that you mentioned, which is better understanding a particularly emergent behavior or a motivation. Maybe there’s a competitor name or a particular space that we continue to hear coming up a lot and we’re like, hey, we don’t really have a lot of information about that or I can tell that there are some strategic discussions or decisions that it sounds like they’re kind of getting stopped up on. So it’s very much in the foundational category that we are proactively suggesting things. There are times that we go out there and we say, hey, this is a designer, a feature that hasn’t been evaluated in a while and we think that it’s important to go look at this. But I would say more often it’s in the foundational strategic. For example, one that we’re considering right now is there’s this particular trend that we’re seeing happen and we’re trying to figure out whether or not that’s a space that Coinbase wants to play in. This is something new that we’re seeing people do with crypto. We’d like to better understand exactly who’s doing this, why are they doing this? Is this an area where we think we have a competitive strategic angle or is this something that we could support in some different way? Or do we just really want to better understand this and now six months from now when a stakeholder comes and asks me, hey, what do we know about this? And I can say, oh, we looked into it.

Steve: Well, that leads me to a follow-up in terms of what happens with this research. It’s not responding to a request or it’s not tied to the — I’m sort of floundering even, like, what’s the label for the kind of research we’re talking about?

Sarah: I’ve heard it labeled many things because I have heard other research leaders talk about this often f like, what do you call it? Is it forward looking pathfinder, horizontal? I mean, it’s like there’s so many different buzzwords you could throw around. Like we all deal with this, right? We all deal with what do you do when you’re trying to get out in front of a question? So I’m not even really sure we know what we call it, but we certainly know it when we see it. And you can also see different research leaders have different strategies for how do you do that? Do you have a team that is specifically not dedicated to a certain group of stakeholders? They’re kind of like a centralized team. Do you get contractors for that? Do you pay vendors for that? Like how generally do you deal with it? And what I will say is that even though it can be hard to know what are the shots to take, like when do you take your roadmap and reserve a big part of it for this thing that you’re not even really sure if it’s going to land impact or lead to something? Every time we’ve done it, I’ve been so grateful that we really like scraped by and managed to find the space and time to do it in whatever way, because inevitably what happens. So here’s something about crypto. Crypto is cyclical. So the price of Bitcoin goes up. You get a whole bunch of new people who are very interested. They’re curious about this space. They’re hearing it in the news. They may not know a lot, but they’re excited to engage. And so then your doors are flooded with all of these people who are in this like beginner mindset. Then the price of Bitcoin goes down, as it always does.

And then suddenly you still have very active users, but it’s the people who are really hardcore in the space. They’re very hobbyist crypto people who never… They came a long time ago. They’re never going to leave. Like they’re just really interested in this space and they stuck around. But there’s fewer of them. They definitely engage with the product in a very different way. They have a very different set of needs. But inevitably, because, you know, last I checked, crypto has not died yet, even though they like to say Bitcoin is dead or whatever they say, it has not died. So inevitably the price goes back up again. Well, now, I mean, all the research that you’ve been doing, that user that you’ve been looking into and serving for the past year or two or however long it takes, now you’re flooded with the new users again. And so you actually, we invest a lot in lit reviews and summarizing past research, often as like the first step to a research study of, you know, like what have we looked into before? What is relevant? What is evergreen from the last time that we entered the cycle? And the time that we entered the cycle before, you can start to see longitudinal patterns. And that can be a very helpful way to guide where to go next, because you can’t always predict exactly how the market is going to evolve and what’s going to change. But you can say, we’ve been here before and here’s what we think might become relevant later. So you can try to call your shots. Sometimes, yeah, I would say we tend to do them in one of two cases.

Steve: Are you doing lit reviews to look for what those shots might be or what those speculative projects might be?

Sarah: The first is when a stakeholder comes to us and says, hey, what, you know, we want to do a research project on X. What do we know about X? It’s like, well, let’s have the very step number one be to check the archive and say, what do we already know? And sometimes we know a lot, sometimes we know a little, but the very least we can check the box and say, hey, maybe we don’t even need to do research. We actually already have a perspective on this. We’re able to save ourselves a lot of time and have a much more comprehensive strategic point of view for you. And 10 out of 10 times, the stakeholder is like, great, awesome. Like, I’m so glad that’s even better than the thing that I asked for. And we’re able to just machete that request away and move on. And then there’s the kind of more long-term cadence where maybe once or twice a year, I’d say, we’re doing it, where we’re genuinely looking across all the research that we did over the last six months and sometimes even prior and saying, what have we been learning? What do we know? What are the themes that are coming out? Let’s get together as a team. Let’s run that workshop. And we take a different approach every time. It doesn’t always look the same. We kind of experiment with different brainstorm formats or different structures for it, I guess you could say. But then the outcome is, here’s the trends we’re seeing. Here’s what we’ve been learning, big themes. And sometimes it identifies, here’s a big area that we don’t know about. It actually resulted in this recommendation for ourselves of this trend that now we want to go follow with our next six months. We revealed a knowledge gap for ourselves. So that, I’d say, we do maybe once or twice a year.

Steve: I want to ask just a very tactical facet of that because, I mean, this is a challenge for so many teams. What information do we have? Where is it? Who knows about it? How do we access it? I think you called it the archive.

Sarah: Yes.

Steve: How have you organized your information such that you can do this?

Sarah: I feel very passionately on this topic. I think that my team, if you were to ask them, what are the things I’m best known for? One of them is aggressively making people document in the archive their studies, because I’m such a big believer in pulling out past studies to not only save time for yourself, but also invest in those longitudinal data points and perspectives. It’s literally just a spreadsheet. I think a lot of people have this misconception that it needs to be some fancy tool. We’ve tried that. It didn’t work out because then we lost budget for fancy tool. Fancy tool went away. Now you’re scrambling to try to migrate the entirety of your archive, which in my case, goes back over six years, right, that we’ve been working in this space. And so we literally just use a glorified spreadsheet. And you’d actually be amazed at how many other research leaders I’ve met who also say, yeah, it’s basically just a glorified spreadsheet. It’s like all these tools that have their merits for sure, and yet for what you need, how much better is the glorified spreadsheet, right, Google Docs? And so we do have our glorified spreadsheet.

And I think that no matter what tool or what format you’re using, the most important thing is regular accountability for updating it. I think that that’s where a lot of archives fail is you have a nice one, but is it consistent with making sure that you’re putting new stuff in it and then tagging that stuff and making it easy for researchers to go back and look through? And so at the end of every month, we look across what are all the studies that happened this month? And everybody has to update the archive. And there are various ways that we keep ourselves accountable for that. And then we hold each other accountable for when there’s a new study request. Did you check the archive? Have we looked at this before? I think that it’s more of a cultural practice than it is any kind of magic bullet tool. Oh, my gosh, let’s pull it up right now, shall we?

Steve: What are some of the columns in the glorified spreadsheet?

Sarah: So I think that we’ve had a number of org changes, so I actually don’t pay a lot of attention to like what team or, you know, because team names may change, teams may change. I think what’s more important that we have is what was the top key insight? The single top key insight, not a list, the single top key insight and recommendation. And that can give you like your two to three sentence summary of what we found in this study. And if you’re interested, then you can click the link to learn more. Usually that contains all the keywords that you need. There is a spot to add additional keywords if your two to three sentences of your top key insight did not include those keywords. But also the DRIs, sorry, excuse me, that’s such a corporate term, the directly responsible individual who did this study, who was involved. Literally, those are the only things that you need. Yes. Yes, that is the researcher that did the study.

Steve: It sounds like very few columns. The DRI is the person that led the study, that did the research?

Sarah: Sometimes there’s multiple names listed. We also have the month and the year and the title of the study with the link to the report. And that’s it.

Steve: Is that archive just for researchers?

Sarah: The primary audience for the archive is researchers, primarily because I like it best when a stakeholder is coming to us saying, what do we think about X? Right? I mean, sure, we could give them the link to the archive and they could search it themselves. But that’s not the same thing as coming up with a research informed point of view, which I find is actually the primary value that we provide and the primary thing that they actually would like from us. Sure, they can go and search for past studies, but oftentimes they want to know, what do we think? Do we think we should move forward with this? What do we think the opportunity is? Which comes from a human researcher who has access to the archive more so than a PM or a designer running around looking for their own studies.

Steve: I was in a conversation recently and talking to people, they were running basically a mentorship matching organization. And so we were trying to understand how did that work? And they described so much about what kind of data was captured, all these different fields that they had, but then they said that most often their ability to match was that so-and-so who did intake for somebody remembered that they had just talked to somebody else. Like there was institutional knowledge or knowledge held among the people as opposed to the data itself. I’m thinking about them as you’re talking about this, like these tools invite a certain usage like you’ve captured everything so you can dive into it, but people work on stuff and they tell stories like culture kind of carries a lot of this. What kind of things happen when a stakeholder comes and says, “Oh, I have this,” is as I’m kind of surmising recall and interactive social, cultural stuff. Does that impact the sense of what we know versus maybe a more cold-blooded look at data and fields and reports and so on?

Sarah: The short answer is yes, I do think it does. I think there’s no replacement for the people who remember the last time and the last time that we did this study and the people who were on the receiving end of that request. And one nice thing, this is just luck, I suppose, is our research team is still very small. And it’s also a lot of people who have worked a long time in this space and at the company. And so they do remember the last time. And I think in those situations, having as a practice to rely on the fellow researchers and have a strong community of researchers who will phone a friend and say, hey, has anyone ever looked into X? Or does anyone remember a time when a stakeholder asked about X? And in addition to also combing the archive and seeing what exists, I think both are equally powerful and important in their own ways. And that a good comprehensive look at what we know is probably going to involve both. And not just researchers, right? I mean, what about also talking to the other important stakeholders and functions, PMs, designers, data people in our lives? And so I absolutely am a believer in that institutional knowledge. And I’ve seen what it looks like when you lose that institutional knowledge.

So I mentioned that our industry is very cyclical. Price goes up, price goes down. The user base changes. Well, also the size of the company has changed and the people working at the company have changed. I am absolutely a dinosaur by Coinbase and crypto standards. And so I remember the last time, am I the only person in the room that remembers when we discussed this, right? And we were in a period of hypergrowth where the company’s strategy at the time to address the growth was to just hire, just relentlessly hire. And it was like every time you talk to a person, that person was new. Their manager was new. Their team was new. They didn’t know anything. And it feels like Groundhog Day. You know, like I’m snapping here. It’s like Groundhog Day. Like you end up just rinsing and repeating the same things. And in those moments, I think that you’re not doing your job if you’re not looking at what we might have looked at before. You’d end up executing the same study every three months because every three months, you know, a stakeholder shows up and asks the same question. Where are your personas? That’s a common one. So I think that that’s where we really learned how much time we could save by keeping track of the last time that we did it. And fortunately, we are — I mean, fortunately, not fortunately, I don’t know. Non-normatively, we are not in a state of hypergrowth at this time. So the nice thing is there’s just less turnover with human beings. And so there’s less thrash with like the same types of questions and, you know, a lot of that chaos.

Steve: You said a bunch of interesting things, but one reaction is that phoning a friend, I tend to think of that, my own bias there is that’s kind of a failure, but you’re characterizing it as a success. That’s a valid way to tap into what we as a collective group of people who work together know.

Sarah: Yeah, absolutely. Because I think that a lot of knowledge is still in human beings. And we can try to automate away. We can try to make it factoids and bullets. But I think that our knowledge is cumulative. Like as researchers, we are anthropologists in this space. We are deeply putting together everything in our heads and having a point of view. And that point of view is what we offer. I mean, in 10 years, when all of our jobs have been automated away with all these AI tools or whatever, I mean, the last thing I think to go will be. But you, as a researcher, yes, I know all the findings, all the data points, all the studies. But what do you think we should do? Like what is your strategic recommendation? That is the thing that is hardest to automate away. And I think that that is the value that we most provide to the company and to our stakeholders and to each other.

Steve: I wonder if the discipline that you’ve created around updating the spreadsheet, it’s like why we take notes about things and then we remember them better. You know, as you’re talking about these two approaches, I think you’re saying they’re interrelated in an interesting way.

Sarah: Yeah.

Steve: So, yes, if you have a discipline of updating the database, the glorified spreadsheet, then that might make you better at recall when someone says, hey, what do we know about X?

Sarah: Absolutely. I mean, even if I were the person who conducted all of the studies for the last few years on a particular topic and I put them in the archive, I still might be going through and using the archive to bring all of those studies that I have done before to kind of bring them all out, look at them, think about them, come up with a take, maybe do some affinity mapping if you’re like a Miro enthusiast or something like that. And so it is helping me put together the data points that I have collected, regardless of who collected them. You’re kind of engaging in this coming up with the perspective. Okay.

Steve: Maybe we’ll go back to the kind of research that has no name. You gave the scenario a couple of times of, you know, someone, a stakeholder approaches you and says, you know, you’re able to say, like, well, we’ve actually done that research and here it is. And so that makes me think about, oh, you’ve got sort of the stuff kind of held back. And I was wondering when you do the research that I’m not effectively categorizing, yes,

Sarah: The research that shall not be named.

Steve: when you’re doing the research that shall not be named and you get to some point, you’re, quote, done with it. And you’re not really done, but, you know, wrapping that up, what kind of things happen with it?

Sarah: I think that in the ideal perfect world, somebody says this is revelatory and life changing and we’re going to go do everything you said we should do. And I think we all know that that doesn’t always happen. Sometimes it’s the opposite where it kind of is the tree that falls in the forest and there were people around to hear it, but they were fairly unmoved and it just wasn’t the right time, you know, or maybe there were some unanswered questions. It informed a V2, but it appeared to just stop where it was and it went into the archive and nothing happened. And I think that’s the thing that you kind of have to do is kind of put a flavor in second lives and third lives and fourth lives of research that you were kind of bummed because maybe it didn’t have the revelatory life changing impact that you wanted it to have at the time. And then you bring it out again and you say, actually, this is exactly what I needed. And maybe it just was a little bit ahead of its time. We’ve seen that happen a lot. Big believer in evergreen research. A lot of jobs to be done research is like this, I find. And that’s like a whole category because then you also have to help stakeholders understand exactly what to do with that. But a lot of that stuff, I mean, I cannot tell you how many times we have pulled it. We all have our favorite studies, right, where we looked at behaviors in banking. I mean, completely outside of crypto, just unrelated. How do people deal with their regular money? What’s important to them in banking? At the time, it was like, how relevant is this? I mean, most people aren’t seeing crypto as their bank. It’s very different. But the number of times that we’ve gone back and said, here are the canonical behaviors that we observe with people and money has been, I would do that study all over again. It was a great investment. And we just didn’t realize it necessarily one month, two months after we finished it.

Steve: You’re painting a picture a little bit of, you know, we talked about the second lives, then that first life, the research is completed and doesn’t get the uptake that it might that first go around. How does it get published or shared or socialized when it’s something that is on your roadmap and not on somebody else’s?

Sarah: We share it in largely the same way. We have certain established channels, cadences, ways that we summarize recent research. And it doesn’t really matter where it originated or even sometimes who on the team did it. We’re still going to do the roll up of like, here’s the biggest insights that came out this month. We have a channel where we share our completed work. And we would share it the same way as anything else. And at that point, it might be like, oh, my gosh, this is amazing. Or at that point, it might be crickets. And sometimes you have to go further beyond that. It’s like, what live channels do we have access to? Is there like a regular leadership monthly where we can present one or two insights from this? Or are we going to signal boost in different channel? I mean, obviously, the way that we work digitally and remotely impacts this a lot. We’ve had to think a lot about research communications in a variety of different ways. We’re never not thinking about research comms, actually, if I’m honest. And so you really try to make sure that you’re signal boosting it to the right audiences in the channels that they pay attention to in a variety of different formats in every flavor. And then if it still feels like it’s not generating conversation, then you file it away. And it’s amazing how suddenly in six to 12 months, they start asking again, and you’re really grateful you did it.

Steve: This to me gets at a complaint or an anxiety that researchers have. And I don’t know if this correlates to sort of the overall research maturity. You know, I can imagine someone listening to this and saying like, wow, you guys are going off and doing research that you think is important. And you’re like, I wish I had time for that. But let’s say I did have time for it. I did that.

Sarah: Right.

Steve: Well, no one wants it and no one cares about it. And so it would just be crickets. But you’re describing a larger strategy. As you said, you never are not thinking about research comms. So assuming relevance, assuming that, you know, everybody can understand the relevant research, you have a built out structure for comms, that kind of a baseline. And then you’re also thinking about on a piece of material or a set of insights specifically, how to go about that. And then I think, as you said, you know, you bide your time as well, because you have a handle on what the information is. It doesn’t have to sync. It can hang around because you have a way of retrieving that and saying, oh, yes, we do know something about X. But it’s something about Y.

Sarah: For sure. And I think you have to know the shots to take. I’m not saying people should just go do all this research and be OK if it doesn’t go anywhere. I’m a big believer in trying to make sure that you’re taking the most impactful shots that you can take based on your knowledge about the organization that you’re working in, like the space. And if putting together all the information, the best information you have at the time and saying, look, I really think this is a knowledge gap that strategically matters to our business and to what we’re doing. And if you’re right about that, then I do think it’s just a matter of time that somebody comes back and is glad that you did the thing. And there is no magic formula. I mean, it really is like you have to sometimes beg, borrow, and steal your time of like, yes, we all have a million requests that are coming from stakeholders all the time. And how do you still reserve the space? It’s hard. It’s hard for everybody. I wouldn’t say we’ve found a magic formula. Although, if I did have one tip of something that has worked, if there is something that you really want to prioritize that is proactively initiated and not a request from somewhere outside of your research team, what are you going to deprioritize? Are you good at deprioritizing something else? And that is a muscle that we’ve been able to work over time of like, okay, I really want to make sure that we have room for this foundational research this quarter. Can I make sure that my requests from stakeholders are not eating up my entire roadmap? Are there things that I feel that they don’t need as much as they’re saying or that aren’t going to be as impactful as they’re making it sound or that I think can be answered in other more lightweight ways? And if you’re good at doing that, then you’ll always be able to reserve your time for the other things. But that is hard to do. That’s hard for everyone. I do think that the best no’s sound like yes’s.

Steve: And you didn’t use the phrase saying no to things. That is often the label that’s put on that. What are best practices or guidance you might have in the deprioritizing and telling someone that whatever you have to tell them as a result of that?

Sarah: Because at the end of the day, I think it’s important to remember why your stakeholder is coming to you. They’re coming to you because they have a question or there’s a decision that they need to make. And there are all kinds of different ways to help them with that goal that don’t necessarily involve dropping everything you’re doing and running a study. It can be, well, can we run an A/B test? Isn’t that the better way to handle this? Or here’s an educated guess we can make based on this other study that finished recently. Or we have a process where certain very tactical, not super high stakes research, it’s a form of democratization we call partner-led research. So it’s not them doing it themselves. They are leading it as the partner. The partner is PM or design. And we are helping them, but we’re making sure that it’s like the proper method and tool for the job. Like we have a way of helping them execute it themselves when it’s something that’s fairly well scoped to just their area. And so there are all kinds of different tricks of like if they’re coming to you because they have a question or they have a decision they need to make, how do you help them answer that and make the decision in ways that don’t necessarily involve a study? There’s all kinds. And then they’re grateful because they get what they need. And it usually doesn’t sound like a no, we’re not going to do that. It’s like, thank you so much for coming to me. What an interesting question. Let me try to find the best way to answer that for you. And the best way could look a bunch of different ways.

Steve: So you’re helping. It’s not “Go away.”

Sarah: Yeah, I think it’s a solutions-orientedness in terms of they’re coming to you because they want a solution. And so whatever you do, it doesn’t necessarily have to fall within the bounds of research. Sometimes they really just need help reframing their question in the right way and actually realizing that it’s a data question, like a data science question. Or it’s talking through the options and realizing that actually we can probably just make a guess about this, like use our design instinct. It’s actually like just a design — how should I say this? We should use our instincts as professional designers to make a call. Like are we really going to run a study about whether the button should be red or green? Probably not. But they may not know that. And I can’t necessarily expect the stakeholder to always know when is the right moment to do research. That’s my job to know when is the right moment to do research. So if they’re asking me something, that is always a conversation that I’m ready to have. And it doesn’t matter whether it results in a study or not. I think in terms of roadmap, I think that they are the experts in their own domain, right?

Steve: How does it work with the researchers that are elsewhere in the organization that don’t have the same kind of leadership structure in terms of managing priorities and managing responsibilities?

Sarah: And so I trust that they are running their roadmap in a way that works for them. I’m not necessarily going to go try to micromanage that for them. But, of course, we are here if they ever want to bounce ideas or get a take. We also are very transparent with what we’re working on. They’re transparent with what they’re working on. Like we have a very collegial relationship. So I would say that that works with smaller teams because they can always just say, “Hey, has anyone looked at this?” But we also do certain shared practices. Like I mentioned, we have a shared crit where twice a week we get together. And any researcher from any part of the organization can bring a challenge or a topic or something they’d like some feedback on. We have a shared channel. And so if they’re really like, “Hey, I’m not sure whether or not I should prioritize this. A stakeholder is coming to me asking this, and I’m not really sure how I should think about it.” There is a community of researchers, myself included, who are really open to that. And so we have a shared community of researchers, myself included, who can always help bounce ideas.

Steve: So, crit might include what approach to take about a question.

Sarah: Yeah, absolutely. It can be anything. I mean, crit, I suppose, makes it sound like we’re always critiquing research plans or reports of findings. But actually it’s mostly just like a shared discussion forum where sometimes people come and they have a very challenging stakeholder management question. Like, “How should I deal with this?” Or, “What’s the deal with this tool?” Or, “How do I do this thing?” It’s really just an open forum for researchers supporting researchers.

Steve: Do you have any other research comms?

Sarah: Yes, always. Always research comms.

Steve: Do you have some examples of things that you’ve tried that didn’t work and things that you’ve tried that did work?

Sarah: I think you have to be very careful with regular newsletters. We’ve tried a couple of different iterations. I think we still have one monthly, but we really had to iterate many times to get it to be read by the right people. Something that’s very top of mind is having strong, crisp product recommendations, which I’m told is actually different than a lot of environments. I mean, I guess it’s been over six years since I worked in any other environment. But researchers are saying, “Hey, at other companies, oftentimes they want the insights. They don’t want me to tell them what they should do. They don’t want me to be specific about what products should do and what design should do. And I think you should build this, and I think you should make X into Y.” That is something we absolutely do. And not only do we do it, we’re held responsible for doing it. Like, we’re perceived as not doing our jobs if we are not having a strong point of view. And it doesn’t always come naturally to a researcher. We’re data people. We like hanging out at insights. We like the “how might we” statements. We kind of like the collaborative energy. And definitely there’s a way to do that. I wouldn’t throw it out. But at the end of the day, especially for a certain level of leadership, what they’re looking for is, “But what do you recommend?” And I want it to be three bullets, and I don’t want to have to read it very much. And I want you to be very specific, not just make X easier. You know, how should I be making it easier? What is the specific change? And that means there’s a lot of trust that our stakeholders have in us. Like, I don’t get in a lot of debates about methodology, for example. I don’t have a lot of people who want to sit through an hour-long presentation just so they can question whether I talked to the right audience or not. You know, like, they trust that we did our jobs right. What they want is to dedicate five seconds to reading my bulleted list of three things that I think product or design should do. And the more that we have put that at the front and center, the TL;DR is what we call it. It’s very internet lingo. The TL;DR at the top of the research newsletter, which I mentioned we still have. And really the three main TL;DRs from that month. Not everything, not extensive, very short, easy to consume, and it took a while to find the right recipe. And to identify who is the right person to read that, I don’t necessarily expect that everybody responds to email. Actually, our email is designed for one very specific leadership stakeholder, and it is tailored to how that person likes to consume information, just based on what we’ve observed about how that person requests information to be delivered by other teams, like how they like to receive updates. There’s a different stakeholder where we actually don’t expect that that person is going to read. That person hates email. That person, I use Slack. Another leadership stakeholder I want to make sure is listening, tends to listen very well when they’re live in a regularly recurring monthly meeting. And so I make sure that research always has one or two slides in that meeting. So I think you have to know exactly who you want to be listening, and you have to change your techniques depending on who that is. Which ultimately at the end of the day is really just understanding your user, right? Like except that in this case, the product is us as a team and what we provide, and the user is a specific stakeholder.

Steve: I want to follow up a bit on the specific recommendations part. I am always anxious about finding a balance because I think here’s the thing that we learned is no one does anything with it, but if you go too far without the context in here’s how we think you should address it or how we think we should address it. I want to set up a dynamic where someone can say yes and to that because they know the engineering constraints, they know some piece of software they’ve bought or they know what rolls out uphill versus downhill or they know all kinds of stuff. There’s other decisions being made and other expertise that they’re bringing. When I say we learned X and we think we should do Y, what I don’t want is to get shot down, no Y wouldn’t work and then we’re at the end of it. If I have to defend sort of how might we questions, I think it’s to try to lead to the generation of the best solution with what everybody knows. I totally get the more you can bring people closer to taking action on the research, the more impact you’re going to have.

Sarah: No, I completely understand where you’re coming from, because I think if you talk to every person on our research team, they’d say the exact same thing. They don’t want to get shot down because they don’t want it to be the end of the conversation. We had to completely change how we think about that. We had to become okay with getting shut down and to realize that that actually was not necessarily a worst case scenario. I’ll give you an anecdote that jumps to mind. We just did one of those regular roll-ups that I was talking about where we kind of look across the last six months of research and say, “These are big themes, and this is what we think you should do,” and we presented it to a very large leadership audience in a large forum. And we knew we were taking a big swing where we were being very opinionated with, “Here are the three things we think you should do.” And we get into that leadership. It was a presentation. We did this one synchronously live. And I’m like, “Okay, here’s the three major themes. Let’s talk about this first theme. What are people’s thoughts?” And literally a person raises their hand and says, “I think this is a terrible idea. I do not think that we should do this.” And then the next person said, “You know, I mean, maybe we shouldn’t do it in that way, but I actually think we could do it if we did it in this different way.” And then the next person chimes in and says, “You know, actually, I’m somewhere in between you two. Like, I see the value in this, but maybe…”

And it is so funny because we have been beating our heads against the wall for years because we felt like our stakeholders were constantly being like, “You need to be more specific with your recommendations. You need more of a TL;DR. You need to have more of a point of view.” And we were like, “We don’t know what you mean. Like, how are we not being specific?” And we realized that it’s because we were afraid to get shot down. And what we didn’t realize is that they will perceive us to be not doing our jobs if we don’t come out with a spicy opinion. They want the spicy opinion. And if we are disagreed with, then that actually is going to lead them to other different ways of coming up with different solutions way faster than a “how might we” statement would. It was crazy. And I think that this could be a way in which the Coinbase environment is different than other environments. Like, it turns out that after all that time, the best way to get them to ideate different solutions was to throw out an idea and a recommendation that they hated. And that did more than a “how might we” statement ever did. It was unbelievable. And so now that’s been really encouraging the whole team to be like, “It’s okay to get shot down. It’s okay to get disagreed with. It’s okay for people to think that that isn’t what we should do.” Because to express a point of view, a strongly held opinion, is better than not expressing one at all. Because if we don’t express one at all, they’re going to be like, “Oh, I just wish our research team had stronger recommendations.”

Steve: You reminded me of this article which I’ll link to in the show notes. This guy is basically describing, I think it’s the McDonald’s theory. And he’s working on site with people in their pre-times and having this conversation every day at lunch. And he says, “Do you want to go for lunch?” “I don’t know, I don’t know, I don’t know.” And he decides to facilitate the resolution of that by proposing McDonald’s. And then everyone says, “No, let’s go…” They come up with specific choices which they weren’t doing beforehand. And he says basically the same thing that you do. People will come up with good ideas to fend off bad ideas. And so I don’t think you’re proposing recommendations that you think are bad ideas. But you’re proposing a solution to a problem. And so that’s what the audience does in that example. And they are working to resolve them.

Sarah: Yeah, that absolutely resonates where they want to know what we really think. And sometimes they’ll find a report and the report has a very clear, you know, like, here’s what we found and, you know, therefore here’s our recommendations and whatever. And that stakeholder will actually DM me and say, what do you think? Like, even if what I think is actually some version of what the person said, you know, like what the researcher put, what they want is what I think. That is the value that I provide, even if I’m really just kind of like restating what they said. And so I think it’s a good reminder of like, they don’t have to agree with me, but they do need to know what I think. And if I think something that they don’t agree with, at the very least, they know what I think.

Steve: What does shot down mean? That they disagreed with the recommendation in that meeting. But there was a robust conversation about what happens next.

Sarah: Totally. I mean, that, that to me is a success. I would say the failure state is where they just don’t trust that the research is valid. And fortunately, that’s not a situation we ever, I mean, I’m not saying we never find ourselves in that situation, but it is, it is rare. They trust that our research is valid and now it’s an equally valid discussion about what to do with it. And there could be a variety of different opinions and a variety of different inputs, right? We’re not always just going to go do what users said we should do. Right. I think that’s where it starts to become what’s the strategic opportunity. What’s the business opportunity? What’s, you know, what are the other data sources that we have? What about instinct? What about all these other factors that ultimately lead us to a good decision as a company with research and user insights just being one component of that?

Steve: I’m gonna go back to something you said early on that you’re talking about these six years. And you’ve described changes in crypto and then the changes in the company as a response to that. And you said you’ve done these different tours of duty that your role has changed over those six years. What are some moments or eras that are different than the one that we’ve been talking about?

Sarah: Well, there was the era of Sian Townsend. Hi, Sian. I hope you’re listening. We love you. There was a time when I mentioned that we had a centralized research team and there was a leader of that centralized team back when the team was like closer to 30. I lost count. I don’t even know. It’s much larger than it is now. And the leader of that research team was named Sian Townsend. And that was the first and only time that I have ever reported to another research person. I’d only ever reported to design managers, people in products. And that’s great. But ultimately, you’re not talking to somebody who deeply understands your practice and your function. And so for me, I got so much out of that couple of years where I learned how to be a research leader in ways because I’ve only ever worked at small companies. I will admit I’m a little bit of a bootstrapped case where I’ve worked at smaller companies and/or on smaller research teams that just don’t have the level of structure and process. Like I’ve never worked at Facebook or Google or some of these larger research teams where you just learn the gold standard of how a lot of these things are done. And a lot of these problems that sometimes small company people encounter every day, those problems have actually been solved in a different way. But that’s why I think sometimes you go to these research meetups and you see a bunch of research teams of one or people who maybe just have one or two other researchers they work with. And they’re like, “Hi, how do you address this problem? Please help me.” And there’s such a community amongst researchers where we struggle with these challenging problems.

And to see the way that she had the tools in her arsenal to be able to handle these conversations at a level that I had not previously been exposed to was the best mentorship that, I mean money can’t buy that kind of mentorship. So this is like a big love letter Sian, but that is an era, and that was also an era when the team and by the team I mean the research team, but I also mean Coinbase as a company, we were scaling enormously. I mean we were in that period of hyper growth I mentioned where it was like we were three-xing by the end of the year and it was just constant hiring and constant new people. And that was intentional on the part of the company to really try to scale hugely to meet the opportunity, but that comes with its own drawbacks and challenges and it was just a very different time where we were a much bigger team, we dealt with much bigger team issues. Right now we’re back to being fairly small, which is nice, it’s like comfort zone for me, but I learned a lot in that period. What is the difference between a job to be done and a job to be done?

Steve: Is there anything you’ve done recently as a research leader that you can trace back to something you learned from working with Sian?

Sarah: I had been familiar with Jobs To Be Done, but I had not actually lived it and breathed it and we still reference that stuff today. I think that we no longer necessarily do the roadshow of like really actively trying to educate our stakeholders on jobs to be done, we tend to bring it up more in context of like, “Hey, this actually sounds like let’s talk about this in terms of jobs to be done.” So we tend to talk about it more at a small scale, but that’s because we’re no longer educating the company, the company now kind of understands and now we’re just like teaching them like ways and times to like bring it out and use it and how to apply it in various situations. So that was like the first thing she did, she showed up and she was like, “You all need to read the jobs to be done playbook and I’m buying it for you all.” So we literally did a book club and read it, but we still use a lot of that stuff today. So that’s definitely one that jumps to mind.

Steve: You just made note of having worked primarily in small organizations before Coinbase. And maybe just to riff on that a bit, could you talk about your path? How did you find user research and what did that look like to bring you up to Coinbase?

Sarah: It’s always fun to relive this. I majored in cognitive science with a concentration in linguistics in college and undergrad. And I had a fellowship as a lab assistant in a speech production lab and great people, but I just really did not enjoy the academic environment. So that led me to believe that grad school was probably not the right path. I just wasn’t really sure it was out there. I didn’t really know what I wanted to do. I just knew that it was a major that I had enjoyed, but I set it aside for a while and said, “I’ll just go out there and get a jobby job because I don’t really know.” And then I spent the first two years of my career being a data analyst and wow, I have so much respect for data analysts, people who are in there doing the work of pulling the SQL, understanding the table structures, creating the dashboards, the charts. That was me and that is a rough job. I have so much respect for those people. And I was like, “Oh, it’s definitely not that. I definitely do not want to do this anymore.” So I quit that and I was like, “What do I want to do? Do I want to do HR, PR? What do I want to do?” And I thought back to undergrad where it was not like it is now where you have master’s programs for UX design. I mean, it was around, but I wasn’t exposed to it. It didn’t have such a strong presence in undergrad, but I’d taken one class in human-computer interaction. It’s just like an elective. I didn’t even know what it was. And I actually loved it. We worked on a project. We actually won best project. And the award, this is so ironic in retrospect, the award for best project was you get a paid internship doing essentially user research. And I had declined it because I had already committed to join this data analyst full-time role after graduation. And so I was like, “Maybe I should have done that paid internship. Maybe I got it all wrong.” And so that’s when I started thinking maybe this human-computer interaction thing, maybe this is a job that you can have. Maybe this is a career. And I just was finding anyone who would take a chance on me. I was in Chicago at the time, so a smaller, more nascent startup scene. And so I was just knocking on doors of smaller companies. I think I was even trying to get hired as a project manager just because then I could do an internal hop or something. And I found a very small agency that had a research contract, a research project from one of their regular clients that they didn’t know how they were going to fulfill. And they were like, “Oh, you want to do this? We’ll hire you to do this contract.” And I was like, “Yes, please.” And I’m so eternally grateful to that agency for giving me that chance because I’ve been a researcher ever since and I love it. I’m so glad that I am lucky enough to have found a job that I love.

Steve: I also came through a different era, but human-computer interaction was my toe in the water or peek through the keyhole if there was this thing.

Sarah: Yeah. It’s funny to watch how the terms evolve over time. Human-computer interaction, HCI, I still see classes that are called that. You still have academic texts that are called that, but no one would say that they would work in HCI anymore or anything like that. But I think a lot of us, that was one of the first words we heard to describe this thing we thought was addictive and wonderful.

Steve: Is there anything that you’d want to share about yourself maybe that’s outside the professional realm that would give us a little picture of you?

Sarah: Yeah. Well, we just bought a house. So first time homeowners. I’m based in Seattle and it’s fun to do all this stuff. I waited a while to own a house, I guess. I was very cool being a renter, but getting to know all kinds of things like researching lawnmowers and things, although mostly that’s my fiancé. I have to credit him. He’s doing most of the researching on lawnmowers. Well, here’s the thing. We are level one sommeliers, actually. We have done the training in the court of master sommeliers. We’ve done the wine testing, taken the tests and we love wine. And so that’s a hobby we have on the side. He works in food and beverage. And so that’s a thing we really like to do. We like to go to wine country and we like to do wine travel in other countries. So yeah, I have my little certificate right here, my sommelier certificate. I mean, short of becoming a professional sommelier, mostly just like, I don’t know, impress people with it. I don’t know. He might study for his level too, because I think at a certain point it does start to matter. But I think being a professional sommelier is not necessarily the most common job in the world. I think you use it to plan trips and be a nerd and I don’t know, impress your friends. That’s what I do with it.

Steve: And you did that certification process together.

Sarah: Yeah, we did. It was actually fairly early in our relationship a couple years ago. And I had mentioned that it was something I was always interested in. Having lived in San Francisco prior, I spent a lot of time in wine country and got curious about the words that I was hearing and what I was drinking. And so when I met him, I had already had an interest in doing it. And he was like, “Oh my gosh, what a coincidence. Me too. Let’s do it together.” And I was like, “Well, I guess we only just started dating, but okay.” And actually next year when we get married, we’re getting married in wine country at a winery. So I guess it’s just what we do now. Yes, we love it. I love that show. I would

Steve: There’s this television show, and I wonder if you’ve seen it, if you have an opinion about it, called “Drops of God.”

Sarah: Yes, we love it. I love that show.

Steve: Yeah, how would you describe that show?

Sarah: describe it as very well researched. I mean, I’m not like the world’s foremost wine expert or anything, but I really enjoyed it. And I found that the wine details were dead on. It’s also just, it has that melodramatic flair that is a guilty pleasure for a lot of people. We really enjoyed it. We’re super excited for season two.

Steve: To me, and I think that show was about incredible expertise, right? It’s about a woman whose strange father dies and is some kind of just global expert in wine with this amazing book. And he’s left France. She’s French. And he’s been in Japan, and he’s mentored this Japanese man. And when he dies, he leaves everything to one of the two of them, and there’s this competition that they have to go through. And the competition hinges on this incredible level of expertise that they have in tasting, identifying, and there the drama unfolds.

Sarah: Oh yeah. I love that philosophical question of like, she was like born into it, right? Like she has this instinct of she hasn’t looked at wine in a while, but she just has this great palate. And then he is the trained, the mentored, he’s very well studied. Like he worked really hard to get where he is. And so it’s a little bit like, is it innate? Can you learn it? Like it’s sort of asking these questions about the knowledge that we have is one more legitimate, is one better. So I won’t give away the twist, but people should watch it.

Steve: That’s a great framing on it, because I don’t personally care about anything about wine, but I loved that show because it was about knowledge and passion for a topic.

Sarah: Yeah, absolutely. It’s like, what can you own and claim as like, I have the right to this. I deserve to win this competition. Particularly with something that a lot of people don’t know a lot about wine. A lot of people are kind of intimidated by wine. That’s a reaction that I often get is like, oh, well, I don’t know anything. I really, I don’t. And it’s like, I mean, you know, like you just drink what you like. I’m not going to judge anyone’s wine taste. Like I love stuff you can get at a grocery store or whatever. It doesn’t need to be fancy. And I think particularly because people feel that they don’t know a lot about it. It was like a fascinating topic area to choose for that question.

Steve: Is there anything you think we should have talked about today that we didn’t get to?

Sarah: I mean, you didn’t ask me to sell you on cryptocurrency. My explanation was horribly basic. And you’re not just going to say, oh my gosh, please explain to me, extol the virtues of blockchain.

Steve: I actually can’t tell if you’re being sarcastic or serious right now.

Sarah: No, no, no. I mean, I think that people, I love talking to people about crypto and blockchain because I realized that it can be kind of like a weirdo topic for some folks. Equally intimidating like wine, actually, similar to what we were talking about. But some people are like, you know what, I will ask you if I’m curious about the details of how blockchain works. But maybe not everybody wants to be hearing about it constantly all the time. And I don’t necessarily want to. I don’t know, like if people want to go there, that’s great. But if they don’t, they don’t. I mean, you can imagine at a cocktail party, it used to be easy. I used to work for Ancestry. And people were like, oh, what do you do? I work for Ancestry. Oh, my grandma loves Ancestry. My aunt, you know, like I love it. I did my family tree. Oh, I work for Coinbase. What’s that? A cryptocurrency company. A lot of people are like, oh, like you can tell immediately the people who are like, please explain blockchain to me versus the people who are like, and I’m going to change the topic. I wish you just said you were an accountant. So, you know, I try to just I never know how deep to go, but I do find it a very interesting space.

Steve: I mean, to be transparent, I was, you know, when we talked about it at the beginning, I’m thinking about what would I ask you versus what would you and I ask people listening to this to listen to? So, yes, how much to ask, how far to go.

Sarah: Yeah, I didn’t know what to what to expect. I mean, I didn’t know if this was going to be like, so, you know, like explain blockchain. And it’s great. We actually talked mostly about research stuff, and it’s amazing how industries can be different. But research as a practice stays very much the same. But crypto has a weird brand associated with it. I think that it can there’s just a lot of like weird PR that makes it feel a little like ooky. And I can speak from a first person perspective is very off putting when you’re you’re first in it and you’re like, who are these crazy people that would want to work in this space? Like, is this even legit? And I can understand like, for a lot of folks, they’re just kind of like, yeah, if I really wanted to be like, there are plenty of people running around extolling the virtues of blockchain. And I’d be more than happy to be one of those. But I realize that sometimes it’s like, let’s just talk about the research.

Steve: Well, maybe you can take us out on a nugget or theory or something blockchainy, since you’ve kind of teased us now. Maybe we can wrap on that.

Sarah: I think most people when they first get into crypto, it is an investment. And that’s okay, because investing is not bad. Investing is great. And, you know, that’s very cool when people are interested in investing in it. And we have a lot of like financially related, you know, features and things. But crypto is not just about making money. And those are the parts that are fascinating and kind of change often and are still super early and nascent. And to a lot of people are like, what are you talking about? Like, that’s craziness. Like the idea that I said that it was like programmable money. They’re like, what even is that? But it can be dense, but you don’t have to be an engineer or a very aggressive, like financial person. There are parts of it that are equally interesting. So anyway, but like, I encourage people to be curious about all the different things that can be done with blockchain that are interesting to a lot of different people. No, I don’t. I don’t think so. And like, it doesn’t always evolve in the ways that

Steve: It kind of loops back to your point from much earlier that, you know, there’s a reason that you’ve stuck around for six years, your whole am I bored yet? And you’re describing a technology that you’re involved in that has potential that it hasn’t come anywhere near reaching.

Sarah: you think it’s going to evolve, right? Like, do I think that Bitcoin is going to replace the US dollar tomorrow? No, nor do I necessarily think it should. I don’t know. Maybe that’s a debatable topic. But I mean, different things, like people got really sick of hearing about NFTs. Understandably, they were everywhere. And people are like, why would I even? This is like a JPEG. I do not get this. Like, why is this constantly everywhere? Like, I get it. I get how it feels. But there’s cool things that you can do with NFTs. Like, I won’t bore you, but like, what if there was a way to verify your identity without having to share any personally identifiable information? Like, it’s basically an attestation token of attesting to your identity without having to actually share information. I mean, are a ton of people doing that today? No, some are. But I think that that’s where there’s still a lot of interesting technological development in the space. So is it an empty promise? Is it going to be the thing that we all do tomorrow? Is it somewhere in between? I think that’s where you kind of have to stay tuned to find out.

Steve: Sarah, thanks for a great conversation. We covered a lot of stuff and went into a few different nooks and crannies. It was really great to have you on Dollars to Donuts, so thank you for the time today.

Sarah: Thank you. It was great to be here. Thank you.

Steve: There ya go, whaddya know, that’s the show! Find Dollars to Donuts wherever you get podcasts, and at Portigal dot com slash podcast for all of the episodes with show notes and transcripts. I would love love love for you to rate and review Dollars to Donuts on Apple Podcasts. Our theme music is by Bruce Todd.

The post 49. Sarah Gregory of Coinbase first appeared on Portigal Consulting.
  continue reading

63 episodes

All episodes

×
 
Loading …

Welcome to Player FM!

Player FM is scanning the web for high-quality podcasts for you to enjoy right now. It's the best podcast app and works on Android, iPhone, and the web. Signup to sync subscriptions across devices.

 

Quick Reference Guide