Value Driven Data Science: Boost your impact. Earn what you’re worth. Rewrite your career algorithm.

[00:00:00] Dr Genevieve Hayes: Hello and welcome to Value Driven Data Science, the podcast that helps data scientists transform their technical expertise into tangible business value, career autonomy, and financial reward. I'm Dr. Genevieveve Hayes. Today I'm joined by Miguel Curiel. Miguel is the product analytics manager at Bloomberg, where he works at the intersection of technology, data, and human behavior.
[00:00:31] He has a background in neuroscience and psychology and is currently writing a book on product analytics. In this episode, you'll learn a practical checklist for maximizing business impact through product analytics, including how to build analytics practices that drive real user behavioral insights.
[00:00:52] So get ready to boost your impact, earn what you're worth, and rewrite your career algorithm. Miguel, welcome to the show.
[00:01:01] Miguel Curiel: Genevieve, thank you so much for having me. Very excited to be here.
[00:01:04] Dr Genevieve Hayes: One of the first and most valuable lessons I learned as a data scientist was that behind every data point is a story. During the five years of my PhD, I worked with a de-identified data set. And so to me it represented nothing more than numbers on a page.
[00:01:23] So when I started my first role in insurance pricing my mindset initially remained the same. That was until my boss took me along to speak to a policy holder sitting across from one of the people my data represented. A man who was running a business on the verge of bankruptcy and my insurance premiums were causing them financial angst.
[00:01:51] That changed everything for me because seeing that man who was. Struggling to keep his business afloat, made me realize that my data wasn't just numbers on a page, it represented actual human beings who behave in sometimes unpredictable ways . i learned this lesson while working in insurance, but the same lesson is true for any product or service that humans use.
[00:02:20] And in a world where everyone's building an app or a digital product, understanding those human behaviors has become absolutely critical, which is where product analytics comes in. However, despite this being a field of growing importance, there's very little guidance on how to do it well. In fact Miguel, one of the things you told me in preparing for this episode was that there are only really four books on product analytics that you could find given this massive knowledge gap.
[00:02:50] This represents a huge opportunity for data scientists and is undoubtedly why you see value in writing your own book on this topic. So to begin with, can you explain to our listeners what exactly product analytics involves and how it differs from more traditional data science work?
[00:03:11] Miguel Curiel: Absolutely, and I. Love how you started with stories because that is crucial. And analytics, what it is the simple definition is understanding user behavior on digital products. For example, websites, apps but it's so much more than that. And. In your day-to-day, it will mostly be quantitative.
[00:03:32] So you will be understanding numbers and quantifying user behavior. So think about it for any website you go, you get to a main landing page that's what you usually call it, and you have funnels. So you progress through that website and how you click through buttons, that is your conversion rate.
[00:03:47] Those are the type of stuff you analyze and it's really easy to get lost in the numbers. But at the end of the day, to your point, you're talking about people. For example, my first job in product analytics proper was in streaming services. Something I am actually really passionate about. I'm passionate about media and stories. So that led me to having that super interesting intersection of how people consume media and then helping them have the process simpler for streaming services.
[00:04:16] You may or may not know, or the audience may or may not know that streaming services are a mess. A lot of them your favorite TV shows and movies are changing platforms. They're cracking down on password sharing, so on and so forth. So my former employer was set out to make that process simpler. At the end of the day trying to service people, and again, that's the part that is very important to remember, but is also easy to forget. So it's what I always try to get to. Who is a person you are trying to serve to? And to date, that is what I also always try to think. And behavioral science is therefore for me at the crux of it.
[00:04:50] And you had a wonderful guest who is the author of one of the best books out there on product analytics, which focuses very much on social science and behavioral science.
[00:04:58] Dr Genevieve Hayes: I take it you're referring to Joanne Rodriguez here.
[00:05:01] Miguel Curiel: Yes.
[00:05:02] Dr Genevieve Hayes: Going back to what we're talking about, so. We specialize in product analytics, in streaming services. To make that into a concrete example, if we're looking at something like Netflix, so you'd switch onto Netflix and then start clicking around to try and find something to watch.
[00:05:20] So there'd be data relating to which shows you click on? And then do you click on play? Then how many minutes do you watch of that show? Do you give up and go and watch something else? Or do you go and then watch the next episode?
[00:05:35] So I'm guessing this is creating mountains of data and then someone like you would analyze that presumably to try and optimize the amount of time people spend on Netflix or whoever you work for. Also to try and provide people with good suggestions so that they keep coming back and spend more time on Netflix.
[00:06:01] Am I correct there?
[00:06:03] Miguel Curiel: You are spot on, and I would be very fascinated by the idea of doing that type of analysis. And it can go by many names. For example, Netflix has, as far as I know, a robust experimentation program, an AB testing slash online experimentation slash conversion rate optimization program. Meaning understanding that and it sounds simple enough, but then it gets tricky because for example, let's take Netflix. How do you measure the success of search results? So, I don't know about you, but it is very common for people to go on Netflix or your favorite streaming platform and search for content. How do you measure the success of that one? Easy proxy, metric could be if people. Out of that search click on one of the results and then watch the movie. And then do they stick for the movie? but at the end of the day, those are only proxies, because we are not with that person. We are trying to and arrive at that conclusion. then again, what if they downloaded the movie or what if we don't have access to that data, or what if the tracking we have implemented broke? so there are a lot of buts in the process. So always with a grain of salt, and it usually are dealing with web analytics
[00:07:15] so there are a lot of buts. And then in the behavioral science part, there's a lot of buts in terms of how you measure it. is tricky, but yes, that is the essence of it.
[00:07:23] Dr Genevieve Hayes: How does behavioral science come into this?
[00:07:25] Miguel Curiel: Behavioral science for me, and I will take a lot from Joanne's book here. And I have it, and let me quote it. Product analytics applied data science techniques for actionable consumer insights. Highly recommend. Her proposal at the beginning of the book is have a theory of the social world.
[00:07:40] So there are countless behavioral science models to understand why we do what we do. Pick whichever you feel more comfortable with, but understand behavior from that lens. Because at the end of the day, you are trying to model behavior. You're trying to measure the success of your product. so it is really helpful to have behavioral science and it is even more important when you talk about what for me is the epitome of product analytics, AB testing because it really encompasses everything that product analytics is, and maybe testing you are optimizing for behaviors. So for example, Netflix I believe they run thousands of experiments on their streaming platform. I got one recently. They significantly modified my landing page. Like when you open Netflix for the first time. their experience was significantly revamped.
[00:08:28] I don't know if this was an AB test or full on rollout, but they did something different. I would wonder what they're trying to optimize for. They did that. What are they trying to lead me to? To click on something on the main page to scroll down. don't know. But that's up to the analyst and to the product people, decision makers behind the scenes. Whatever change you make in your product, think about what behavior it's gonna impact and how you model that relationship. That's where behavioral science comes very handy. And again, you can talk about many models and can talk about the emotions behind the scenes for people and your users. That's up to you, but very important to have a behavioral science model.
[00:09:09] Dr Genevieve Hayes: Okay. So with that Netflix example presumably they said. You, that particular landing page and some other users, some alternative landing page, and there would be an experiment going on behind the scenes, which would be presumably they'd have a hypothesis that people in one group would be more likely to click on whatever they want you to click on.
[00:09:32] Then the people in the other group, and they would be collecting data so then they could do some sort of hypothesis test to determine. Whether or not the data supports that hypothesis. Okay. And that's the behavioral science thing, because you've got that hypothesis around human behavior.
[00:09:50] Miguel Curiel: Yes,
[00:09:50] Dr Genevieve Hayes: I.
[00:09:51] Miguel Curiel: And yeah, to your point, hypothesis testing is really the backbone of it. If you come from a more traditional educational background. That's exactly what feeds into these new models. And then you have very new platforms that try to simplify that process. But it is very important to actually not skip that hypothesis part.
[00:10:08] One particular model I like to develop hypothesis is Matt Waller who also has a book but his behavioral statement is think about for a given population. are trying to optimize for this behavior as measured by this metric. And then you have slight variations to it, but in essence it's a behavioral statement that you can use as a template for your hypothesis,
[00:10:31] Dr Genevieve Hayes: Okay, and Netflix would obviously be wanting to optimize for a number of minutes spent on their website.
[00:10:37] Miguel Curiel: presumably. And that's again what I would be very curious to see. One, all of the data they have access to. The thousands surely of experiments they're running in the backend.
[00:10:46] Dr Genevieve Hayes: So this sounds like a really useful skill to have, and I can imagine that a lot of companies, particularly digital companies would benefit from having someone who can conduct these experiments, even if they're not running streaming services.
[00:11:00] Why isn't this a bigger field and why has so little been written on this topic?
[00:11:05] Miguel Curiel: That is a great question and I agree with you. I'm biased, arguably, but it is a very important skill to have and. They are transferable skills. Thankfully, a lot of people coming from academia already come with this skillset. So it's one booming thanks to products becoming, I think more democratized.
[00:11:26] So you have starting 2000 tens, probably a lot of platforms starting to create their own AB testing platforms. Which made it much more accessible. Now you have some newer platforms such as statsy or epo, which recently got acquired by Datadog, for example. so a few years ago we had the boom of many, many platforms starting to deliver their own solutions. Now they're consolidating as to why not a lot of people are writing about it still. similar case I would argue is it's a new field product analytics. The first book, and I also have it here, I don't recall the timelines, but for example, lean Analytics Joanne Rodriguez, product analytics book Flo, who is the other author of one of the product analytics book.
[00:12:09] It's called Behavioral Data Analysis with Rrn Python. those are 2014, 2015 onward. then even the most emblematic book for AV testing, for example, is Ron Coha. And I'm. Blanking on the other authors, but their book on trustworthy online experiments also came out recently, probably 2018, 2020, is very new.
[00:12:34] Dr Genevieve Hayes: Although I will argue that president Obama used AB testing to optimize his website when he was. First running for president, and that would've been 2008. So clearly these skills have been around for at least 18 years and we are dealing with statistical hypothesis testing here. So they've been around for decades before that.
[00:12:58] So, but yeah, I'd say the Obama example is probably one of the earliest prominent examples in the Internet's history.
[00:13:07] Miguel Curiel: Exactly that is a great example. But also think about who you're talking about. Obama. So that was the type of person who could have the resources to actually run AB test. And now again, with technology, I think it's only becoming easier and that's why there's been a boom and there's a great, conversion rate optimization online experimentation, AV testing, community. But it is still relatively new. Again, we're talking about 15, 20 years, but the formal science itself, statistics, it's the precursor for it all.
[00:13:38] Dr Genevieve Hayes: Yes, exactly.
[00:13:40] So you're currently writing a book on product analytics. Given what you've said it makes sense that. There is space for another book, especially since your competition is only for other books that you've identified. What sets the book that you are writing about product analytics apart from those other four books that currently exist?
[00:14:04] Miguel Curiel: Yeah, and I will preface it by saying I'm sure there are more books out there. I apologize to the authors who have written books and I haven't found you. Yet, the ones I have found I mentioned already a few lean Analytics product analytics, then by John Rodriguez, and then behavioral Data Analysis by Florent Buson. And the last one is by Timo Teal who is very active online and he talks about marketing and product analytics and self-published a book just last year, for example. So all of them focus on slightly different areas. For example, lean Analytics is catered towards startups. It's following the Lean Startup vision, then Joanne Rodriguez is very focused on behavioral science and the behavioral model behind it all, and obviously analytics included, very much so Florence Book, very academic, the most technical one I would say among them all. And then TIMOs book is very catered towards if you're a consultant and you're gonna set up a product analytics architecture for your client, then you can use the book and it's a practical guide for that. Then I'm sure there are other books. My book where it would fit in is pretty much for the aspiring product analyst or the analyst already in-house at a company or product leaders who want to integrate product analytics within their practice, within their teams, it would be. A checklist, follow these steps and you will have a model to run your organization or build your product analytics setup. you can conduct it by yourself or you can relate it to other teams. Kind of or franchising the product analytics model.
[00:15:35] And very much inspired by my own experience going into product analytics. I went and read all of these books, loved them, and then built my own model and wanna share it with the world.
[00:15:45] Dr Genevieve Hayes: Can you walk us through the key components of your checklist?
[00:15:49] Miguel Curiel: Of course it roughly follows the table of contents. So I will read it out. It starts with an introduction, for example,
[00:15:56] Dr Genevieve Hayes: I.
[00:15:56] Miguel Curiel: what product analytics is. It's a new field, I would say not a lot of people are familiar. You get that. The first important and most important element is understanding the business. Your stakeholders, your environment, your industry, et cetera. Then behavioral science, having a behavioral model as we talked about earlier. It is really, really important to understand your users from a fundamental level. you get into the more nitty gritty some will say this item, product metrics is the most important for me.
[00:16:24] It comes after business and behavior. Then you talk about product metrics such as retention average revenue per user, stuff like that. Then you do need to talk about engineering or need some baseline understanding of engineering concepts because you have to set up tracking on your website or your mobile app. There is some technical know how for that. Even if you're not an engineer, you at least know the basics. Then you talk about analytics proper. Now that you have all set up, you understand your people, you understand your business. You actually analyze data, be it descriptive statistics slicing and dicing the information or delivering predictive models, that type of things. Then the epitome that I talked about earlier, experiments, I think encompasses all of the above. You understand your business, you understand your user's behavior, you develop metrics, you analyze the data, you have an engineering infrastructure for it. And last but not least storytelling. You need to package all of this information, so how do you do it via storytelling?
[00:17:25] Dr Genevieve Hayes: So it sounds like the information you've given there, would be sufficient for someone to completely run a whole team of people who are conducting a product analytics experiment. .
[00:17:35] Miguel Curiel: That would be my hope.
[00:17:36] Dr Genevieve Hayes: Yeah, which checklist items do you see as being the most critical when it comes to creating business value?
[00:17:43] Miguel Curiel: I'm biased and my background is in psychology. I would love to say it's behavior, but really I think it's understanding the business. If you understand the business and what your organization is trying to achieve, what are your goals? You understand your. Competitive landscape, all of that, you're more than halfway through.
[00:18:05] From there stems everything else.
[00:18:07] Dr Genevieve Hayes: So it's understanding the business context based.
[00:18:10] Miguel Curiel: Yes. And obviously well not obviously, but if you're not in a corporate business, if you're in the nonprofit world, for example, the same principles apply. You want to understand what you're trying to achieve as an organization or if you're a solopreneur, same principles.
[00:18:26] Dr Genevieve Hayes: It's that data science Venn diagram that you often see, you know, subject matter expertise computer programming skills and statistical skills. You've just described those in your checklist, you need to have that subject matter expertise, which is what we just discussed. You also need to be able to do the engineering, which is your computer science side of things, and you need to analyze your data, so it's your quintessential data science skillset.
[00:18:52] Miguel Curiel: Yeah, it's exactly that and I've been thinking about the differences if there are any between, for example, product analytics and data science, the line is very blurry and with. Probably any data role, it gets blurry, got data engineering, got analytics, engineering, who's to say how many more will come? But for product analytics, I would say then if you get into the nitty gritty, the differentiator does become behavioral science and the product metrics you develop from understanding be.
[00:19:21] Dr Genevieve Hayes: It sounds like as opposed to building machine learning models, which is what some data scientists specialize in, the specialist skillset of the product analytics, data scientist is that whole AB testing, experimental side of things.
[00:19:37] Miguel Curiel: I would argue yes there will be people that say no. But I propose product analytics as a pathway or a gateway to experimentation. It's Brings everything that you need as a product analyst together.
[00:19:50] Dr Genevieve Hayes: They often say data science is the intersection of statistics and computer science. It sounds like product analytics is more skewed towards the statistical side of things because everything you talk about here is stuff that I learned during my statistical training, which I had before I went into data science.
[00:20:07] So this actually feels a lot more comfortable to me than machine learning.
[00:20:13] Miguel Curiel: Yeah. And there are teams and especially the more advanced product analytics teams that will obviously deliver predictive models and develop machine learning models. But at its core you can start with simply understanding user behavior. And again, descriptive statistics will get you. Far enough to understand your population, then you can work with more advanced knowledge and models to deliver more sophisticated solutions.
[00:20:38] And again, the bigger companies such as Netflix, I'm sure they're running a tons of behavioral models in the backend to predict what people will see next. And they're trying to nudge you that way.
[00:20:48] Dr Genevieve Hayes: And at the end of the day, it's causal analytics, so you're trying to work. At, if Netflix does this will it cause people to click on whatever they want you to click on? Are there any checklist items where you typically see data scientists tripping up and making mistakes?
[00:21:06] Miguel Curiel: Yes, especially, and I will speak from experience and I am, still work in progress, as is anyone but the business side of things, and that's why I start the book with that. And I think it's where data scientists, data analysts, product analysts, can make the most impact. Having a clear line between your work and how it impacts. The bottom line or the business. Sometimes you can have that relationship very directly. Sometimes your models are directly in the product. For example, sometimes your models are feeding the search algorithm or something like that, but oftentimes you're in the backend helping. Decision makers. So making that relationship tangible and visible again, where I think we can make the most impact. And you can come up with your own estimates of I'm creating this analysis and I'm working on analysis that can save us X amount of dollars. That is already a great approach. And the more tangible you can make it, the better.
[00:22:08] Dr Genevieve Hayes: Okay, so if that's the area where people are tripping up, that's not a technical area at all. That's a. Business area. So a natural follow on from that is what do you believe is the most valuable skill that data scientists working in that product analytics space can have?
[00:22:27] Miguel Curiel: I would say the most valuable skill is business acumen define that however you feel comfortable. If I define it, for example, understanding the business. What is a business trying to achieve? Who are your competitors? Who are you serving? Understanding everything about your business. That includes the product you're trying to deliver.
[00:22:48] Dr Genevieve Hayes: So for listeners who are in interested in incorporating product analytics, thinking into their current work, what's one step they can take tomorrow?
[00:22:57] Miguel Curiel: Read any of the books I suggested. If you're not into reading, think about key metrics for your product. Assuming you have a product or you're working for a company. And if not, think about your favorite products, the ones you use in your day-to-day life. Be that Netflix or whatever i'm sure you have access to digital environments.
[00:23:18] Think about those products you use. would you understand your own success? How would people in the backend be analyzing if you're a successful user or not? Yes, those, those would be several possible action items that people could take tomorrow.
[00:23:32] Dr Genevieve Hayes: For listeners who wanna get in contact with you, Miguel, what can they do?
[00:23:37] Miguel Curiel: LinkedIn would be the best channel for now. You can find me via my name, Miguel Curiel and keywords product analytics. Soon I will be launching new channels some digital projects in the works. But yeah, for now LinkedIn, I am very active.
[00:23:51] Dr Genevieve Hayes: Are you gonna be using product analytics on those new channels?
[00:23:55] Miguel Curiel: I have already started to use product analytics, yes.
[00:23:58] Dr Genevieve Hayes: That's good. And there you have it. Another value packed episode to help turn your data skills into serious clout, cash, and career freedom. If you enjoyed this episode, why not make it a double next week? Catch Miguel's value boost a 10 minute episode where he shares one powerful tip for getting real results real fast.
[00:24:20] Make sure you're subscribed so you don't miss it. Thanks for joining me today, Miguel.
[00:24:25] Miguel Curiel: Thank you Genevieve.
[00:24:26] Dr Genevieve Hayes: And for those in the audience, thanks for listening. I'm Dr. Genevieveve Hayes, and this has been Value Driven Data Science.