CDPs vs. data lakes: Decoding your data infrastructure | Census

Nicole Mitich
15 June 2023

Imagine, for a moment, that you're a business. Not just any business, but one that's grown beyond the point of maintaining personal relationships with every customer. 

You've got customers from all corners of the globe, each one with unique wants, needs, and buying habits. But here's the rub: How do you keep track of all these customers in a way that's meaningful and useful? Enter Customer Data Platforms (CDPs).

A CDP, at its core, is a database for customer data. Simple enough, right? 

But it's not just any database. In theory, t's a smart, sophisticated tool designed to help businesses collect, unify, and use customer data – a solution to the age-old problem of data silos and fragmented customer information. That's some elevator pitch. If only they actually followed through with their promises. 😬

In practice, the story of CDPs is a bit more complicated. While useful, CDPs are not without their share of problems and hidden costs. Most off-the-shelf CDP solutions lock you into their data-handling philosophy or fall short of being the "single source of truth" they claim to be. Moreover, the CDP industry is relatively young and still in flux, making any investment in a CDP uncertain at best.

But here’s the thing… If you're a business with a solid data infrastructure already in place, you may have a secret weapon: Your data lake. With the right tools and strategies, your existing data lake can provide many of the features of a traditional CDP, without the associated risks and costs. It's like discovering the best coffee shop in town is actually your own kitchen. ☕

So, we're diving deep into the world of CDPs to explore what they are, what they can and can't do, and how you can potentially build your own on top of your data lake.

What are CDPs?

CDPs are a type of software that collects and organizes customer data from a variety of sources to create a unified customer profile. This profile is then accessed by other systems, most often by marketers, and used to personalize communication, improve customer service, optimize marketing spend, and more. Essentially, CDPs are instrumental in facilitating a customer-centric approach to business.

But while CDPs are powerful tools, they have a number of issues that limit the scope of their applications. 👇

  1. CDPs are only as good as the data they receive. If data is inaccurate, incomplete, or outdated at the point of entry, the CDP can't fix it. So, data quality assurance remains a critical responsibility for businesses.
  2. CDPs are generally not as advanced as dedicated predictive analytics platforms. For companies looking to predict future customer behavior, they may need to invest in AI and machine learning for predictive analytics on top of their CDP.
  3. Contrary to some marketing claims, CDPs cannot be viewed as a "single source of truth" for a company's data. They are part of a broader data infrastructure and are intended to be used alongside other data systems like data lakes or data warehouses.
  4. There’s the risk of vendor lock-in. 🔒When you buy an off-the-shelf CDP, you're buying into their philosophy on handling data. This could have significant implications for how you collect and manage data now and in the future.

Still convinced a CDP is your best bet?

What are data lakes?

While a CDP provides a unified, structured view of customer data, a data lake, on the other hand, is more of a raw, unstructured storage repository that holds a vast amount of data (more than just customer data) in its native format until it's needed. 

A data lake is not specifically designed for marketing use cases, so its data often needs to be processed and structured before it can be used effectively.

Think of a data lake as a colossal library, teeming with books. It’s massive and messy, but you can find almost anything you're looking for within its stacks, including books on all of your customers, provided you're willing to scour the shelves. 📚

Unfortunately, most marketers don’t have the technical expertise to make use of a data lake to handle and activate customer data. It requires knowledge of data processing, analytics, and potentially coding or query languages.

What if you could turn your existing data lake into a CDP?

Let's take a step back and consider the bigger picture. The world of data has evolved considerably in the past decade, with data lakes becoming integral parts of any serious data infrastructure. 

Meanwhile, despite their marketing claims that they are a "single source of truth," CDPs are not central hubs. Rather, they’re larger nodes in the network of data management. They might host a significant amount of data, but that data may be incomplete and outdated. 

So, the obvious question then becomes: What if you could turn your existing data lake into a CDP? 🤔 After all, your data lake already encompasses many of the features an off-the-shelf CDP would provide. Better yet, it gives you a single source of truth to rely on.

Your data lake already stores customer data from various sources, like your CRM, website, customer service platform, and so on. Your data team already transforms this customer data to make it useful. They also already ensure that all data remains compliant with regulations like GDPR and CCPA.

And because your data team uses event-tracking tools like Segment to collect first-party data and ETL tools like Fivetran to load third-party data into the data lake, what you need is a tool powered by Reverse ETL to move data from your data lake into operational systems (e.g. Salesforce, Marketo, Facebook Ads, etc).

Census is one such tool, designed to operationalize the data in your data lake by syncing it to the apps your team uses every day.

How to roll out your own composable CDP on top of your data lake

And so we’ve arrived at the concept of a “composable” CDP. Instead of buying a monolithic CDP solution, you can build your own version by combining various tools that fit your specific needs. 🧰

So how do you use Census to build a composable CDP using your existing data lake? Let’s walk through the steps:

  1. Have a data lake. Check ✓. The data lake should be well-organized and include all relevant data about customer behavior, transactions, interactions, and other relevant information.
  2. Connect your data lake to Census. Once your data lake is set up, you can connect it to Census. Census supports several data lakes and data warehouses, including BigQuery, Redshift, Snowflake, and others.
  3. Transform your data. Before you can use your customer data effectively, it needs to be transformed into a format that's useful for your business needs. This could involve cleaning the data, aggregating it, or creating new data fields. Census provides SQL-based transformations you can use to transform your data within your data lake.
  4. Define your customer segments. Once your data is transformed, you can define different customer segments within Census Audience Hub based on various characteristics or behaviors of your customers. For example, you could create a segment for customers who have made a purchase in the last 30 days, or a segment for customers who have viewed a particular product but haven't purchased it.
  5. Sync your segments to your business tools. After your segments are defined, you can sync them to your various business tools using Census. Census supports a 130+ SaaS tools (and counting!), and you can set up the sync to run on a schedule that suits your needs.
  6. Activate your customer data. Once your customer data is synced to your business tools, you can start using it to personalize your marketing messages, improve youwide range of toolsr customer support, optimize your product, and more. The sky’s the limit. 🚀

One of the key benefits of this approach is that it allows you to maintain a true "single source of truth" for your customer data in your data lake, greatly simplifying data management and ensuring that all of your systems are working with consistent, up-to-date information.

Plus, with this composable CDP, you're not locked into a single vendor. If your needs change or if a better tool comes along, you can easily swap out one component for another, mixing and matching different tools to create a system that fits your unique requirements. 🎯

So, if you have a good data team and a well-organized data infrastructure, you can use tools like Census to create a CDP that's tailored to your business, without the limitations of an off-the-shelf solution. 

💡 If you’d like to learn more about how Census can help you build your own CDP solution out of your existing data infrastructure, schedule a demo, and we’ll talk you through it.