A Data Team of One
When Buddy was hired at Loom, he realized pretty quickly that he needed to expand on the functionality they were currently getting from Segment. “I was looking for a tool that allows you to take data captured from our product and send it to third-party sources. With Segment we couldn’t send any information outside of pure event data, which was just limiting,” says Buddy.
“For example, with Intercom, if you wanted to send a custom message based on an aggregated count of a user doing some activity or based on a specific lead score — it can't easily be done, because Segment literally needs it to be a stream of events coming in and a stream of events coming out.”
Buddy knew from experience that managing this type of data flow from the warehouse (Snowflake) to Salesforce, Zendesk, or Intercom, isn’t as simple as it could be. In his experience, it often takes several people to build custom data pipelines, which then need to be maintained and monitored independently. Which as a “‘data team of one”’ neither Buddy nor Loom had the time or capacity to take on.
“Unfortunately, a large part of our process for releasing a new feature was predicated on the use of Intercom forms,” says Buddy, “and we just weren't able to get the data from Intercom that we needed to greenlight this part of our release.”
That’s when Buddy began searching for a tool that could help him bring transformed data from the warehouse to Intercom.
Finding Census
When Buddy started looking, he had no idea if what he wanted even existed, so he asked around. “I reached out to the dbt community about whether or not someone had looked into solving this exact problem, and I was pointed towards Census which I hadn't heard of prior to that,” Buddy says. “I quickly looked into everything and I was like, “Whoa, this is kind of like Fivetran flipped around to the other side of the data warehouse. It's exactly what I'm looking for!”
Buddy could feel the impact that Census could offer right away. He says, “My whole career as a data scientist and in data engineering roles has been about setting up foundational architecture components, so I very quickly realized the value prop Census offered. I feel that anyone looking to send data to any of the connectors Census provides will quickly see the value.”
🌊 The Problem: A Sea of Tickets
Initially, Buddy began using Census to help Loom connect data to Zendesk, which ultimately allowed them to prioritize the increased volume of tickets that were coming in. “This was critical for us, especially during COVID,” said Buddy. “At that time, we had started offering the product for free to students and teachers, which was great, but we got a massive growth in users and subsequently tickets.”
Loom wanted to prioritize tickets based on the type of plan each user had, but weren’t able to do so initially because a single user can be on multiple plans. “You can't really use Segment for this,” Buddy says. “Ultimately this requires SQL to create a list that says, ‘you're a member of all these plans.’ Now let's take this list and map it into one single value.”
Instead, Census allowed Buddy to take this transformed data, relevant for ticket prioritization and easily populate the necessary fields in Zendesk. “I think Census basically made something that was impossible before very highly functional,” says Buddy. “Before we were diving through a sea of tens of thousands of tickets and which, without Census, was basically impossible to handle. Census solved some foundational problems that we needed to have fixed.”
💪 The Solution: One tool = More People
For Buddy, one of the biggest selling points of using Census was how it eliminated the need for adding more people onto his data team, without compromising on functionality.
In his experience, Buddy says he’s seen tools like Stitch and Fivetran, that get data into the warehouse, reduce the need for one to two people for most data teams. But Buddy says, “I think that same exact thing is true on the Census side, but maybe even more so.”
For Loom, Buddy found that Census was able to organize all of their data, which saved them time and the need to bring on more people. “Census allowed us to centralize everything in a single place. We already have a transformation layer, DBT, which takes hundreds, if not thousands of tables and maps them into a subset of them,” says Buddy. “Now, with Census, we can leverage that work that already exists as a result of the transformation layer. All I have to do is create a Census specific end-point, like a ‘Census Salesforce table,’ and point Census at that table. I don't need to do any engineering or anything outside of SQL to get all this done. Typically that would require a lot of engineering work and then maintenance and monitoring overhead.”
He also points out that as data teams grow, so does the amount of effort needed to get data out of the warehouse, which also won’t be the case now that they have Census. “That's where Census is the missing puzzle piece,” says Buddy, “because you're easily able to take all the work that a few people have done and just put it in the hands of your stakeholders.”
🔭 Looking forward...
Moving forward, Buddy sees the potential that Census can offer to a wide range of teams at Loom.
“We recently released our Loom for Teams product. So as we grow the team internally, our use of Census will become more and more sophisticated,” says Buddy.
“I’m actually nervous to walk the sales team through how well Census and Salesforce play together, because I know that’s going to be like turning on the fire hose.”
Buddy Marshburn, Senior Data Scientist
.svg)