I am a firm believer that one of the reasons that data science and 
analytics has a high failure rate is a lack of product management and design. To me, product is about a mindset just as much as a job title, and I am repeatedly hearing how more and more voices in the data community are agreeing with me on this (Gartner CDO v4, International Inst. for Analytics, several O’Reilly authors, Karim Lakhani’s new book on AI, and others). This is even more true as more companies begin to leverage AI. So many of these companies fear what startups and software companies are doing, yet they do not copy the way tech companies build software applications and enable specific user experiences that unlock the desired business value.
Integral to building software is the product management function—and when these applications and tools have humans in the loop, the product/UX design function is equally as important to ensure adoption, usability, engagement, and alignment with the business objectives.
In modern tech companies, the overlap between product design and product management can be significant, and frequently, product leaders in tech companies come up through both design and engineering ranks and indeed my own work heavily overlaps with product. What this tells me is that product is a mindset, and it’s a role many can learn if they believe it’s critical.
So why aren’t more data science and analytics leaders forming strong product design and analytics functions? I don’t know, so I decided to bring Carlos onto the show to talk about his company, Product School, which offers product management training and features instructors from many of the big tech companies on how to do it. In this episode, Carlos provides a comprehensive overview of why he launched Product School, what makes an effective product manager, and the importance of having structured vision and alignment when developing products.
This conversation explores:
The State of Product Analytics (Also referred to as The Future of Product Analytics in the audio)
Mixpanel, company that they partnered with to create the above report
Episode 17 of Experiencing Data
“You can become a product manager by building products. You don’t need to be a software engineer. You don’t need to have an MBA. You don’t need to be an incredible, inspiring visionary. This is stuff that you can learn, and the best way to learn it is by doing it.” – Carlos
“A product manager is a generalist. And in order to become a generalist, usually you have to have some sort of [specialty] before. So, we define product management as the intersection in between business, engineering, and design. And you can become a good product manager from either of those options.” – Carlos
“If you have [a power trio of technology, product, and design] and the energy is right, and the relationships are really strong, boy, you can get a lot of stuff done, and you can iterate quickly, and really produce some great stuff.” – Brian
“I think part of the product management mindset… is to realize part of your job now is to be a problem finder, it’s to help set the strategy, it’s to help ensure that a model is not the solution.” – Brian
“I think about a bicycle wheel with the hub in the center and the spokes coming out. Product management is that hub, and it reports up into the business, but you have all these different spokes, QA, and software engineering, maybe data science and analytics, product design, and user experience design. These are all kind of spokes.” – Brian
“These are people who are constantly learning, but not just about their products. They’re constantly learning in general. Reading books, practicing sports, doing whatever it is, but always looking at what’s new and wanting to play around with it, just to be dangerous enough. So, I think those three areas: obsession with a customer based on data; obsession with empathy; and then obsession with learning, or just being curious are really critical.” – Carlos
Brian: Hi everyone, it’s Brian. Welcome back to Experiencing Data. I’m really happy to have the CEO of Product School on the show today. This is Carlos Gonzalez de Villaumbrosia. And, Carlos, bienvenidos a Experiencing Data. How are you? ¿Qué tal?
Carlos: Hola. Great, thanks for having me.
Brian: Yeah, yeah. We’re going to nerd out a little bit to product management, and part of the reason I wanted to have you on the show today was, as some of my listeners and my mailing list subscribers know, I talk a lot about data product management, and the need to think product, even if you’re working inside a company to use data to produce software applications, and models, and data-driven solutions, having a product mindset where you think about, “What if we had to draw revenue on this solution that we’re building? What would the quality need to be of that? What would the user experience need to be if this was a revenue-driving thing?” So, even if you’re not building revenue-generating tools directly, that product mindset and the way of developing software, I think, is a really powerful one to get out of the mindset of someone throws requirements over the wall, and you’re an order taker, and you give them exactly what they want because that’s not really what most business leaders today want.
They don’t want their teams, especially their expensive data science and analytics teams, providing—I mean, there you could say, “Yeah, our job is to give what we were asked to give,” but it’s not. It’s really to give what’s needed. It’s to figure out what’s needed and to provide valuable solutions. So, Carlos, tell me a little bit about your background. You run Product School, which is a training school for product managers. Why that? How did you move out of doing it, to running a school to teach it?
Carlos: Sure. So, as you said, I run Product School, which is the global leader in product management training. I started this company six years ago, and it was just an idea; a solution to my own problem. Because I come from a technical background, I was a software engineer. And back in the day, it wasn’t really clear what the options were, if you are an engineer that doesn’t want to code for the rest of their life. So, my solution at that time was, “Well, let me try the opposite. Let me go to business school.” And that’s what I did.
But that doesn’t really resonate, always. Actually, I met a lot of engineers who were in business school who were having the same issue. They were thinking business and that’s why they thought about business school, but at the same time they didn’t want to lose touch with the customer. They wanted to be able to build stuff and get the best out of both worlds. And that’s exactly what I decided to do.
Product School is a hybrid in between an engineering school and a business school. And I’ve worked as a software engineer for multiple years; I’ve been building products for different companies, but at the end of the day, my passion is in education, and I believe that it’s very important to empower people to do what they want. And hopefully they can do it while they keep their life, and they don’t have to quit their jobs. And I’m very passionate about lifelong learning. And one of the things I’m most proud of with Product School is that this is for professionals who want to learn on the side; you don’t have to quit your job; you don’t have to put your life on hold. You can continue doing what you’re doing, and this is something that you can do on the side to help you grow your career, or break into product management.
Brian: Yeah, I’m curious, I feel like in all the tech companies that I’ve worked with—I’ve worked with a lot of startups in the Boston area particularly—I feel like product management is often this thing that you grow. No one goes directly to school and then goes into product management. It’s like, “I used to be an engineer .” “Oh, I was a designer and I got into this.” “Oh, I was whatever. I was a salesperson.” “I was in marketing.” Does everyone still come into it that way? And are you seeing native product managers? And is there a difference between a native career Product Manager versus someone who’s come up through another channel, another discipline?
Carlos: That was exactly the main reason why I decided to build the first Product School because it seemed that, oh my god, you cannot be a product person, unless you have incredible ideas, or you are almost like Steve Jobs. There were a lot of misconceptions, too. Do I need an MBA? Do I need an engineering degree? Can I just learnt on the go? And the answer is, all of the above. You can become a product manager by building products. You don’t need to be a software engineer. You don’t need to have an MBA. You don’t need to be an incredible, inspiring visionary. This is stuff that you can learn, and the best way to learn it is by doing it.
Brian: I agree. I think a lot of the user experience and product design work, and my own course that I teach on designing human-centered data products is very focused on the act of doing as a means of learning. And I think a lot of adult learning is best done that way, as opposed to sitting in a class and memorizing skills, and then 10 years later you go out and try to apply them. You need to learn this by doing. And part of that is because I feel like, from my clients who are often product managers, it’s a very squishy, gray job where in some contexts, you might have a heavy marketing responsibility; in other ones, you might be heavily involved with engineering, particularly if it’s a very technical type of product.
You can be pulled in these different channels, and at different times you might be doing forecasting of what’s the revenue potential for this thing, or what’s the business plan? And then all of a sudden you’re in the weeds in JIRA, managing backlog and, you know… [laughs]. Do you agree it’s kind of for a generalist person who might have a strong vertical specialization in one area? Is that the personality who works well as a product manager?
Carlos: Yes. a product manager is a generalist. And in order to become a generalist, usually you have to have some sort of specialist before. So, we define product management as the intersection in between business, engineering, and design. And you can become a good product manager from either of those options.
So, the best PMs that we’ve seen are people who have previous experience building something as engineers, designers, data people, whatever it is, but they’re very good at something very specific, and they’re able to become generalists to learn enough about the other areas, just to be dangerous enough. Because as a generalist, you are never going to be the absolute best at everything. You just need to feel comfortable in the middle, and know that, even though before you used to be a specialist in something, now you are the one who’s going to empowering the specialist to do their job. And your job is actually to align the vision.
Brian: Are you seeing a difference yet in the requirements of being a good product manager for working with data, particularly AI product managers, or people working more with analytical tools? How does the job description differ a little bit for them? Are you seeing any trends?
Carlos: Yeah, absolutely. So, when we started, we were answering a lot of questions about how is product management different than project management? Or do we need Product Management at all? Now, the market has evolved so much where, if we need the product manager or not is not a question. We are seeing much more specialization within product management.
So, AI is a huge scale, and you are seeing AI product managers the same way you can see growth product managers, or product designers, or technical product managers. So, it’s becoming its own industry. And obviously, when you’re in a smaller organization, the product manager is still wearing different hats and trying to cover everything, but as you look into larger organizations or organizations that have sophisticated product teams, you will see that there are different product managers depending on the type of product, or the type of function that you are trying to cover.
Actually, this year, we released a report called The Future of Product Management. It’s free so anyone can find online and download it. And we run the survey by over 1000 product leaders in our community across the world. One of the questions we asked them was about what are the skills that product managers need to learn in order to stay up-to-date? And the number one was artificial intelligence, actually.
Brian: Mm-hm. Have you had many conversations with AI product managers to understand how is that job changing? Or what, besides a basic understanding of what the technologies are that fit under the AI umbrella because there’s many different ones, what’s different about the role for an AI product manager? Can you talk a little bit about maybe the different skills that may be required? Or what does the business need from that type of role that’s different than a traditional software product manager?
Carlos: Yeah, so I would start with, you have to be a product manager. Everyone has to have a foundation. And then, depending on the product that you’re trying to build, then it’s when you have to add layers of specific skills. So, if you’re working on a product that requires a lot of artificial intelligence, or a lot of engineering power in general, then it’s good that the product manager has some sort of domain knowledge around that. That doesn’t mean that the product manager has to be the ultimate expert in that.
Sometimes we see that product managers are not customers of the products that they’re building, but they definitely need to be passionate about the problem that they’re solving, and curious enough to dig as deep as necessary to have conversations with engineers and other specialists in the topic. So, I would say, if someone is thinking about working in product, you have to learn product first. Don’t try to learn AI and then product. But if you are passionate about AI, and this is something that you really want to learn, then it’s totally okay to start with AI. And then, hey, if you at some point feel like it is better to move more towards the middle and get an strategic overview, then you can consider product. But those two things are slightly different.
Brian: Sure. No, and I understand what you’re saying there about career path direction, but in terms of-I mean, obviously, you could fill in the blank. If I’m a supply chain product manager, well obviously, you’re going to have domain expertise, probably, in supply chain, if you’re going to call yourself that. But are there any specifics you could share about what someone that that is in this space is seeing about how being a product manager working with machine learning and AI is different than the traditional thing? Like, “I never realized how much we need to do X, Y, and Z or how the product development lifecycle is very different.” Or maybe it’s not different. I don’t know. Have you heard much about how that may be different or the same? I’m just curious about those deltas?
Carlos: Yes. So, first of all, I think the word artificial intelligence, machine learning, big data, and all of those are sometimes overused. And a lot of people tend to just throw them out there because they are buzzwords, and they may resonate or not. The reality is if you truly want to understand artificial intelligence and be eventually a good AI product manager, you need to have a technical foundation. You can’t just learn AI without any sort of technical background.
I come from an engineering background; I remember, I had a class on artificial intelligence. This is maybe 15 years ago when artificial intelligence wasn’t very cool. It was even an optional subject. And it was a lot of math and a lot of algorithms. So, if you’re really serious about working in high tech products, or products that require AI, you need to build that foundation first. I think that starting from the top and thinking, “Oh, I’m just going to learn AI because it’s cool.” It’s not going to give you really enough depth to make intelligent decisions and really empathize with your engineers at a deeper level.
Brian: Right. So, if you have that skill already, then obviously we can all go take training for our skill gap areas, and places that we want to get better at. What would be some of the skills—like, let’s say I’m a director of advanced analytics, so we’re talking about predictive analytics, machine learning, et cetera. If I already have that skill, and I don’t know what I don’t know about product management, but I’ve heard that this is an important thing, and it might be a good thing for my career, and it might help me get better at deploying better solutions inside my company, what are some of the things I would be learning if I was to go study about product management that I might not know about if I’m coming from the data perspective, or the data world, that that I don’t even know exists? Like, tell me about some of those skills that I might be learning.
Carlos: And that’s where we can help the most because at Product School we’re all about pitching product management. So, we expect that each of these students will bring their own domain expertise. In the same class, you may find experts in AI, or in data; you may find management consultants; you may find designers. Well, what the common ground for all of them is that they want to move towards the middle. They want to learn a little bit more about the areas related to product management that they are not experts in.
So, at Product School you’re not going to learn more about AI because I’m assuming that you’re already an expert. We’re going to show you how to better leverage your skills to empathize and connect with your designers, with marketers, so you can really be in the middle. And I repeat this concept a lot because that’s a very common problem that we see with specialists that try to become a generalist. In their comfort zone, they tend to go back to what they’re good at. And as a manager, we intentionally pushing you to not do it, even if you think that you can do it better than another person.
It’s more about try to empower others, try to work on the strategy, try to really align the vision and let them define the how. So, it’s basically critical with engineers and I definitely had to work a lot on that because, in engineering school, nobody really taught me anything about how to communicate or speak in public, and I find myself these days, as a CEO but in reality as a product leader, that my work is literally communicating, and making sure that everyone is aligned regardless of the domain expertise that I had before.
Brian: Mm-hm. So, if I was director of analytics, or VP of analytics, or AI, or something like this in my company, and I need to justify taking an expensive course to my management, upper management: senior-level or CXO level, what would be the value or outcomes that I would get that I would be able to communicate to them in terms of why this would be a good investment? What would the business get out of sending me to Product School when I’m already quite senior in my domain area? And particularly thinking, again, about this audience that may not be working at a tech company.
They’re not going to be—like, marketing wouldn’t even really be relevant. You might have some quote, “internal marketing” to do to help sell the changeover a little bit. But for the most part, you’re not building a revenue-generating product per se, but maybe you’re overseeing a custom application that drives a bunch of the business operations, or something like this. So, what would someone like that tell their uppers, their boss, or their management about what the value would be when they come back from the school? What would they get?
Carlos: Well, if they’re already considering it, that is a reason why, probably. And in general, what we’ve seen in organizations that are engineering-driven, or basically not product-driven, is that it’s because of the founders, they had the specific background and the company or a product was built around it and it worked, and it’s growing, but at some point, there is a there’s a lack of, really, alignment in between different functions. So, yes, let’s say you are an amazing engineer, or you built up an amazing technical product that is working and you have customers. But you can’t just continue growing by adding new features. You definitely need someone in the middle who is going to be coordinating all those efforts and building an experience for existing and new customers.
And you, at the beginning, mentioned the product mindset. I completely agree with you. This goes beyond product management. It’s really creating the mindset around, we are all building a product, even if my title is AI, or data, or product, or engineering, whatever it is, we need to understand that this goes beyond just our area of expertise. And if nobody is really taking care of that vision, if everyone is so focused on building, and the next feature, and the next data report, and the next design, whatever it is, then we’re going to find ourselves in this trap where yes, we have a monster that has a lot of different options, but who are we targeting, really, and how are we going to grow through product instead of just through making [unintelligible].
Brian: I agree. So, I’ve seen this a lot, and I feel like sometimes the role becomes—I feel like you’re managing JIRA, or you’re managing some tool and a backlog, and it gets into this process of focusing heavily on following the process a lot of the time, and sometimes I feel like what I’ve seen with product managers is they’re checked out a little bit sometimes on the big picture piece. How do you avoid doing that, so you’re not really managing consistently to engineering milestones? Did we get the release out on time, regardless of whether or not the release created any value, we’re focusing heavily on our efficiency, our burn rate, and number of tickets we banged out, and all this kind of stuff, which is important, and I understand those structures and processes can help. I feel like sometimes it can get a little bit lost, or myopic vision. Do you see this and how do you think about that? Or am I the only one seeing this?
Carlos: You’re definitely seeing this along many, many other organizations. That gap of change has to come from the top because the solution is not to just, oh, I hire a product manager and then suddenly everything works. There has to be a clear commitment and understanding that we can’t just continue growing this way. We cannot just continue growing by making just a bigger size push, and letting our engineers dictate what are the features that we’re going to solve. There has to be a common vision. There has to be an strategy around it, and someone has to own it.
And at the very beginning, especially in tech companies, the founders are acting as that product person. But it gets to a point where you just have so many engineers, and so many other things to do that you’d have to hire people to own the product. And of course, as a founder or early employee, you still have a strong opinion on the roadmap, but there has to be someone strategically owning this. And we saw that before: back in the day, product management used to be a function within marketing, or even engineering depending on the organization. Now product, it’s dedicated function, and there is a role called chief product officer that reports to the CEO. And in most cases, the CEO actually comes from a product background.
Brian: Mm-hm. I really enjoy working for those types of clients when they do because I feel like they really understand the importance, especially today when the tolerance for bad software and stuff that doesn’t work is very low. I think people have gotten so used to applications and things just working well by default, without the struggle. I think when leadership has that product mindset, they really understand the value of, what does it mean to deliver quality? How do we measure quality, and are we going to be quality-focused or not?
It’s a different lens than coming from a financials background, where I feel like there’s this void of understanding, sometimes, between user experience and how the product leadership really needs to be in charge of delivering that experience with the design group to ensure that these things get used, and to ensure that they actually turn into some type of value in the last mile. Do you think that over time we’re going to see more and more product people move into those leadership positions? The executive—do you think product management is a natural escalation path to the C suite, or not necessarily?
Carlos: It is, and it’s already happening. So, if you look at [unintelligible]. Dropbox, Airbnb, Stripe, they’re all unicorns and they’re huge—some of them are public—and their CEOs are former product managers. And that is something that it’s really, really important because it sends a very strong message from the top.
We are caring about the overall experience of the user. We are not just to build new features, or we are not just to push new clients to use this product, just because the person who signed it is different from the person who’s actually using it, which I think is the biggest problem in the B2B world. Now we see this trend where it doesn’t matter if your product is B2C, or B2B, or is an internal product; the user experience is critical.
Brian: Yeah, I think that’s changing, finally, just as I talked to a chief data product officer yesterday at a healthcare company, and he’s like, “Empathy is my number one word. I’m constantly hammering this into my staff.” I, I feel like it’s changing. But a lot of the analytics community, I feel like there’s a lot of dust still in this industry, and I hear particularly about some of the really large brick and mortar companies who still don’t think of themselves as software companies—and maybe their ultimate deliverable today, they’re building physical products or whatever—they fear this tech startup coming out of nowhere and disrupting them, but they’re not making the—it’s like, “Well, why don’t you think about copying some of the ingredients they’re using? You may not copy the cake, but you can look at what they’re doing.
And for example, there’s no product managers in your business, and there are no designers, yet you’re relying more and more on software to push the business forward. And you’re not being consistently good at delivering useful, usable solutions that actually get adopted by the business. And you’re wondering why you’re worried about some upstart? Well, look at the roles that they put as foundations and there—” I always talk about this power trio between engineering, product management, and design, and when you have those three hats together—and I’m saying engineering: this would include a data scientist today or someone, if you’re working with ML or AI, a technical lead, someone that really understands the data and the technical piece of it, working with product management and design.
That trio or quartet, if you have that and the energy is right, and the relationships are really strong, boy, you can get a lot of stuff done, and you can iterate quickly, and really produce some great stuff. So, I’m a big fan. I don’t know, I guess I didn’t have a question there. I was going off on a rant, where it’s like—just because I don’t know if you hear this at all, or if you talked to—much—to some of these companies that maybe are very early in their product management journey, about the importance of this to focus on producing useful value?
Carlos: Yeah. Well, that’s what I’m passionate about. I’ve spent many years of my life building products and also learning on the go. So, there was no structured path. And I understand that, obviously, each company is different. They come from different industries, and there are so many specifics, but there is a foundation that everyone has to agree with, as you were saying.
Like they developing this eye for user experience. Understanding that products are not just ideas and there has to be some data that backs up certain hypotheses. And also, all of that has to be connected to the business. So, if we agree that triangle, it’s product management, and product is in the middle, that’s a really good start. And then we can dig deeper and see how we can adapt certain frameworks and processes to your product and to your state of maturity because obviously, it’s not the same to create this type of product mindset in a smaller organization than in a Fortune 500.
But something that I may add here, which I think it’s really important, at least from my experience as a student. I spent a lot of time in traditional university business schools, and my biggest thing is, I want to be like most of my professors. The problem is that because they are professors. I want to apply what I’m learning, especially in tech, or in product, it’s critical. So, we definitely—we only work with product leaders who are actively working in top companies. That is Google, Facebook, Airbnb, Uber, and others. It is very important to connect reality with the school piece. So, if you want to learn more about product at an individual level, or at an organizational level, you are going to be learning directly from the people who are doing it, not from the people who say how things are done but don’t do it.
Brian: Yeah, that’s that the academic versus applied. I think that exists in the design world as well. There’s a difference between doing design in school, and doing it in a business context. And I’m sure that’s probably the same with machine learning in some of these as well. In fact, that’s a challenge in the data space. You have a lot of people, very highly trained people, often with a masters or PhD-level background in math and statistics coming into business context without that knowledge.
And so, a lot of times there’s this friction because they don’t think that they’re there to find the problems, and the opportunities to move the business forward. They’re waiting for someone to hand them a data problem, a modeling problem, and then to go off and do that work, and then there’s this, kind of, rub. So, I think part of the product management mindset, if you are coming into that area is to realize part of your job now is to be a problem finder, it’s to help set the strategy, it’s to help ensure that a model alone is not the solution. The model has to do what it was intended to do in order for it to actually be a useful solution and create value for the business. You have to elevate the perspective. Get above the—
Carlos: Exactly.
Brian: [laughs]. You got to—
Carlos: That’s exactly it. We should have you in the classroom because [laughs] that’s the whole point. And to be honest, the good news for people who come from such a strong background in data or technology is that they already did the hardest part. They already specialized and [geniuses] in one area. Now, the key is how to let go a little bit, and spend a little more time in the middle learning about non-data things, basically, design, and business. So, that is everything.
And that goes the other way around. For product people, or existing product managers or, say aspiring product managers who come from design, or business, not from data. They are not expected to become data scientists, but they are expected to develop an appreciation for data. You cannot just make decisions based on ideas, or just qualitative research. There has to be some data to back it up, and now there’s been a lot of evolution in this space.
There are so many tools out there that allow non-technical people manipulate data. You can run very complex queries without knowing SQL. You can create very complex reports, you can really understand trends. And just to give you an example, we just launched a report called The State of Product Analytics with a company called Mixpanel, and it was very refreshing to see that there are a lot of companies out there, very, very big ones that, first of all, don’t really pay that much attention to data. Like, yes, they measure the basics, but there’s a lot of work that needs to be done in order to really dig deep and understand your customers. So, I think people who come from a data background have an amazing opportunity to move towards the middle and become great product people.
Announcer: Here’s what people have been saying about the designing human-centered data products seminar.
Participant: A pretty quick and accessible snapshot of how to think about user design for data solutions products. I mean, you can go super deep on UX design; you can talk about data solutions, but how do you combine the two in a very digestible way? I think it’s a very, very solid framework to get started.
Brian: Interested in joining me and a group of fellow students in the next cohort? Visit designingforanalytics.com/theseminar for details and to learn how to get notified when pre-registration begins. I hope to see you there, and now back to the episode.
Brian: When you talk about the middle, I use that same analogy. I think about a bicycle wheel with the hub in the center and the spokes coming out. Product management is that hub, and it reports up into the business, but you have all these different spokes, QA, and software engineering, maybe data science and analytics, product design, user experience design, these are all, kind of, spokes. So, when you say moving to the middle, is that what you’re thinking about, [laughs] as well?
Carlos: Exactly, exactly. I keep repeating that message because I think it’s easy to understand. It’s not about, hey, absolutely everything goes. It’s just, you’re not going to spend all your time working in data. You’re going to need to spend time communicating with other people. And hey, leverage your data background, but don’t forget that if you want to be a leader in product, you cannot be solving these problems directly; you are going to need to empower other data people to help you with how, with the solution, and you can maximize your time defining the why and the what.
Brian: Mm-hm. Tell me a little bit—I just want to shift gears for a second. Say in the last five years, maybe ten years or something, have you seen the dance between product design and technical change? Like, the relationships there? How has design changed in the product world, or how has engineering changed from the product? Those relationships, what’s different now?
Carlos: So, traditionally most of product managers used to come from a software engineering background, at least in Silicon Valley, but that has changed. Now we see product managers coming from all types of backgrounds. Design is one of those; business; data—when I say data, in reality, it’s part of engineering or technical backgrounds. But honestly, I think we should bring more designers to the mix. I’m a big advocate of user design and developing an eye for design, and we still need more of that into the product management industry because that’s ultimately going to help build better products, and that’s something we were discussing early like if we truly believe that we’re building an experience for the user, no matter if this is a consumer product, or an enterprise product, or an internal product, someone truly has to believe in it, and care and push it. If we only have engineers in the room, who is going to advocate for the user experience?
Brian: Right, right. I think a lot of my feeling is that a lot of the designers who have moved into product roles moved into this role because of the rub with product management. The friction there, or it was too hard—they were tired of putting out stuff that they thought maybe wasn’t meeting the user experience requirements, or would stand in the way of progress. So, tell me a little bit about that; do you think that’s true? And I was really asking more about the working relationship between product management and design, and product management and engineering, how those relationships have changed not necessarily the people going into it.
But I guess that’s kind of related to it, and I wonder if that’s why designers are moving into product management is because the relationship needs work still. I don’t know. [laughs] do you think it’s gotten better? Do you think it’s gotten worse? Data, I hear, also gets in the way.
I mean, I know designers who have left big companies because, like my friend Doug Bowman, I think we talked about, leaving Google, I think it was because they’re—wanted to study the 50 different shades of blue to optimize which precise hex value of blue would convert the best. And at that point, you’re never going to leap to the next mountain with that. It’s a local maximum. You’re iterating towards a local maximum with that, but it’s never going to tell you what the next big leap is. So, data can work against it, too. So, what do you think? Tell me about these relationships, and have they changed? And is it getting better or worse?
Carlos: The good thing is that now we have options. Back in the day, at least when I was an engineer, nobody told me anything about product management. So, I felt that oh, my God, I made the wrong choice, and now I have to stick to it. Well, no. Now you can decide and both options are right, there are people who are very passionate about a specific part of the product, and they want to continue being a specialist and that’s fine. If you want to do your research about the 50 shades of blue, then you should never be a product manager. You should be an amazing designer or researcher. Or the same for engineers.
Actually, that’s what majority of people enjoy being just very good at one thing. This small percentage of people who feel that frustration of, “Hey, I spent enough time doing something very specific. But I want to have a bigger influence over the entire product. And my function just not enough. So, what are my options?” Well, good news. That option is called product, or product management. And that’s the type of people who take training, who seek help, who try. And that’s exactly what I’m talking about.
So, it’s to a specific question about the relationship between designers and product, or engineers and product: it’s critical, and that’s why empathy is very important. And that’s why it’s really good that the product managers have some experience before. Because if you just jump into the mix, and you’ve never been in an engineering room, or in an designing room, then how are you really going to empathize with those specialists if you have no idea about what’s going on, how they feel and what their motivations are? So, I’m a big believer in having some experience on the ground first as an specialist, no matter what type of specialty you like, and then move towards the middle.
And if you’re going to move toward the middle, then you are in the middle. You should try to eliminate your bias towards your previous specialty because otherwise, it’s not going to work. And you probably have to work extra hard to connect with a specialist in the areas that you are not in a specialist at. And that requires work, and it also requires some time because at the end of the day, this is an iterative process, and nobody has the answer. We know a lot of wrong answers, but you also have to put yourself out there. And the best way I’ve seen in our students, and in my experience, is to try to let things go because the more time you spend, doing what somebody else can do, the less time you have to really lead and communicate, which is what you are hired for in the first place as a product leader.
Brian: Yes, I would fully agree with that. I think if you’re spending all your—and I’ve seen this a lot, more on the engineering side, but I think anyone that’s technical and understands—especially when you think about modern data pipelines and when we’re working with advanced analytics, et cetera, there is so much plumbing to get right in order to even start talking about building the last mile, the reporting, and the modeling, and applications, et cetera. The draw to pull you back into, should this be a cloud deployment? Should this be on-prem? Should we be doing it this way? Should we use this stack or that stack? What architecture is it going to support? Blah, blah, blah.
That stuff is important, but if you’re spending all of your time arguing with your technical people on those choices, who is at the top of the ship? Who is at the wheel looking at the big picture? And I’ve seen this a lot where these long architectural arguments about stuff—because the product management came out of an engineering field, and they almost know too much about how it’s going to be implemented or could be implemented. And I get it, right? You have some experience, you’ve like, “I’ve seen this ship sink when we go this way.” But that’s not your job anymore. And I feel like delegation is such a talent. It’s a talent and a skill, you have to get really good at that letting go and, and saying, “You know what, I don’t own that decision anymore. They need to own that decision. Like they’re the head of engineering. They’re the head of design. It’s their decision. I need to move us forward at the big picture, and then we’ll revisit.” I don’t know, do you—[laughs]—is that—do you see that, too?
Carlos: That’s exactly it. So, that’s what I I love fixing. I think it’s a fascinating opportunity for individuals and for organizations to really start thinking about a team and aligning those teams around just a vision and have someone own that vision instead of not have anyone, or have just an specialist on it. And that’s actually, by the way, applies within the product organizations because when your companies are big enough, and you start having multiple products, or multiple product managers, you will realize that in a way, you have the same challenge, which is, “Okay, I have a bunch of product managers now. Who is owning the entire portfolio?”
And you will see that even within product which is already a generalist role, there are people who enjoy getting their hands dirty, and working directly with engineers, designers, and business people, and there are other type of product managers, it’s called group product managers, or VPs of product, eventually a CPO, which is more of a manager of managers. And that is the person who is trying to make sense of the overall vision between all the different products. And that’s a person who is going to focus on hiring, and culture. And there is no right or wrong answer, but I think it’s very important to know your options.
Brian: Mm-hm. Tell me what is the gap between a skilled product manager and an exceptional one? What pushes you into the exceptional category?
Carlos: So, what pushes you into the exceptional category, it’s like being obsessed with your customer; being obsessed with data. So, you cannot just say, “Oh, I’m obsessed with my customer because I think I know the customer.” You have to be checking your hypotheses, and running experiments all the time. So, experimentation is number one, and you can only do it by really caring about the user, and really caring about the data.
And then number two, it’s empathy. You have to really understand that you are not an engineer, or a data, or a designer, or a business person anymore. You are in the middle, and you have to spend time with your people.
And then the last thing, but definitely not least, is that curiosity. When you talk to product people, and you will realize that they are there for a reason. These are people who are constantly learning, but not just about their products. They’re constantly learning in general. Reading books, practicing sports, doing whatever it is, but always looking at what’s new and wanting to play around with it, just to be dangerous enough. So, I think those three areas: obsession with a customer based on data; obsession with empathy; and then obsession with learning, or just being curious are really critical.
Brian: That’s awesome. Thanks for sharing that. And would you say, is there one big problem or skill gap overall, that you see today that the field of product management needs to get better at? Does something really stick out?
Carlos: So, experimentation is something that—so data—it’s something that it’s growing for a good reason. I think before, a lot of products were made based on ideas, gut feelings, vision. Now, we’re moving more towards an agile environment where it’s still good to have big ideas, but you are going to execute in smaller sprints, increments of time. So, you are going to create a series of hypotheses, run them by the market, get some data, and then based on that, influence the next iteration. In order to do that effectively, you need to feel comfortable with data. It doesn’t mean that you need to be a data scientist, but you need to definitely have an appreciation for that instead of just ideas or gut feelings.
Brian: Yeah, I get that. I feel like in some context, though, that as much as I love the idea of what’s the shortest path to learning, the next increment of learning that we can get to inform the next choice after that, I totally get that. At the same time I feel like, especially with data products, sometimes the initial lift can be really significant. So, getting to the point where you can do smaller increments of work, there’s a challenge there. Is there any suggestions you might have for an environment like that, where—it’s kind of like thinking about what is the MVP of a rocket ship, right? Like, is it an airplane? Well, I don’t know because you’re not flying into space, so really, it’s a failure if you don’t ever get into space.
You could say, well, it just doesn’t matter. So, how do the best product managers think about this, when we’re talking about really big enterprise, really complicated systems? What does that mean when it’s not just an app, a consumer app or something, where you can deploy A/B testing, and get a million people worth of data to tell you should it be orange or blue? And should we go to this workflow or that workflow? Talk to me about that.
Carlos: I get it. And it’s almost a chicken and egg problem. Because in some cases, you cannot have that MVP or your MVP is really complex, but for a majority of cases, I’m sure there are things that you can do before you fully deploy something. And the good news is that there is a lot of tools out there that help you do that. You don’t have to build everything yourself. Meaning a lot of data, or analytic solutions, just mention a few to give an idea.
So, you have customer data platforms that allow you to ingest data directly, auto-track. You don’t have to even think about what you want to track because it’s tracking everything automatically in a raw format, but then you have your own data analytics or programmatic solutions that allow you to make sense of that data, and visualize it in a way that it’s really nice, and doesn’t require a very complex implementation. So, just to mention a few names, so in the CDP side—customer data platforms—you have a Segment or mParticle. On the product analytics side, you have companies such as Mixpanel, Heap, or Amplitude, or even Pendo, that really help product people and data people see and do things without having to build it themselves. And then you can always make that decision later on, if your product is so specific that you really need to build your own custom solution, well, that’s your choice. But at the very beginning, I definitely encourage product people to focus on what they are unique and try to rely on third-party apps, tools, or datasets so they can get to market faster.
Brian: Yes, and just a quick reminder, I think it was episode 17 of the show Experiencing Data, we had John Cutler on, actually, from Amplitude to talk about analytics, and storytelling with data, and some of these good things. So, I think those are some great suggestions for bringing—ironically pulling data out about your users. Not necessarily thinking about the data that’s actually forming the product, or the model, or the solution, but data about usage and the people that are engaging with it, so especially if you’re deploying something at scale for external customers because internally, you may be serving five or ten people as opposed to thousands or millions. So, that’s great stuff. Just one last question here. This has been a super great conversation. Thanks for coming on the show. Is there one thing that you would have changed about your own growth into the product management space? Like if you could start over and change something what would that be? Is there one thing?
Carlos: Ah, yes. And that’s why I built Product School because I think it took me too long to really know what I wanted. So, one thing that I wish I had done earlier in my career is to maybe spend less time on engineering, and spend more time on design and business, to really supplement the skill sets that I didn’t get in school. So, I could really become a full-stack product manager.
Brian: Excellent. Well, thanks for sharing that. And just in terms of following your work, I think your Twitter handle is @productschool and it’s productschool.com, right? Tell us a little bit, you have training courses there, and there’s a conference too, is that right?
Carlos: Yes, so we built the largest community for people who care about product. We are at this point over a million members, and we put together as many free resources as possible. So, conferences like you mentioned: it’s called ProductCon. It’s absolutely free. We do it six times per year. We used to do them in person in different cities, in San Francisco, New York, Los Angeles, Seattle, and London. Now we’re doing them online.
We also have books; we have a podcast. We actually released one thing called Productverse, which is exactly what we have discussed before, is the universe of products out there. So, for anyone looking for tools for data analytics, tools for user testing, tools for prototyping, you have an entire list where you can see the best solutions for each category. So, at the end of the day, I’m all about building a community and helping people grow in product. And then, in addition to those free resources, we also offer our certifications, especially for professionals. We’re very serious about getting certified, and moving to the next level, and we do that at individual level, but also at a corporate level.
Brian: Cool, excellent. Well, definitely check that out folks, productschool.com. Carlos, muchas gracias, thanks for coming on the show and talking to us about product management today. Appreciate it.
Carlos: Thank you. And let’s keep building cool stuff.
Brian: Yeah, I agree. All right, cheers.
Jana Eggers, a self-proclaimed math and computer nerd, is CEO of Nara Logics, a company that helps organizations use AI to eliminate data silos...
Dr. Murray Cantor has a storied career that spans decades. Recently, he founded Aptage, a company that provides project risk management tools using Bayesian...
