“If they want to be self-sufficient, we want to help them do that.” - Brick Thompson, CEO, Blue Margin
What's the best approach to building an internal BI team?
Listen to Blue Margin Founder and CEO, Brick Thompson, and VP of Delivery Operations, Caleb Ochs, outline some of the challenges of BI self-sufficiency and their recommended approach for building an internal BI team. You can read the full transcript below.
Blue Margin offers knowledge transfer, training, and support to our clients wanting to make the transition to in-house BI, even going so far as to interview potential candidates for them. However, there are a few key considerations to take into account before drafting job descriptions.
To begin with, high salaries (Glassdoor, 2022) and employee turnover expenses significantly increase the cost of maintaining an internal team. Considering the payroll expenses needed to employ only three to five Data Architect or Data Engineer positions, it's more cost effective to partner with an external BI consultant and leverage their specialized resources. Deloitte’s 2020 Global Survey confirms this benefit of outsourcing, reporting cost reduction as a key driver of outsourcing trends and strategies (2020).
In addition to higher expenses, small, internal teams often lack the specialization and experience of a BI consultant firm that manages complex data projects day in and day out. Furthermore, we’ve observed (and experienced) building a team is a time-intensive process, taking at least six months for initial training and up to 18-24 months for full self-sufficiency.
Nevertheless, there is a certain appeal to autonomy, so Brick and Caleb walk listeners through our recommended approach of starting your team with a data visualization engineer and training them in best practices under your external partner before adding an architect, a project manager, and additional visualization engineers.
For further listening/reading:
- How to Succeed with BI: External Partner or Internal Team?
- How to Hire a Good BI Visualization Engineer
- Using BI to Optimize Your Hiring
- Glassdoor. (2022). Data Architect Salaries. https://www.glassdoor.com/Salaries/data-architect-salary-SRCH_KO0,14.htm
- Deloitte. (2020). 2020 Global Outsourcing Survey. Deloitte. https://www2.deloitte.com/us/en/pages/operations/articles/global-outsourcing-survey.html
Brick Thompson: 0:03
Welcome to The Dashboard Effect Podcast, I'm Brick Thompson.
Caleb Ochs: 0:07
And I'm Caleb Ochs.
Brick Thompson: 0:08
Hey, Caleb. So we've got an interesting one today.
Caleb Ochs: 0:11
Brick Thompson: 0:12
What are we gonna be talking about?
Caleb Ochs: 0:14
so hard to hire BI people? And... we're seeing our clients struggle with it. So, why is this happening?
Brick Thompson: 0:31
Yeah, I mean, good companies that we work with that just churn through BI resources. Not vendors, but actually employees. Just really struggling to get someone.
Caleb Ochs: 0:41
Yeah. So the conversation kind of went to a place where we've been seeing people need this, and they want to do this, they want to build their own internal team. So we started to think well, how would we do it? If we were at a company, how would we build a BI department?
Brick Thompson: 0:56
Right. And so we came up with some ideas on how we would do that. I mean, we've done that for ourselves, we built a whole company that way. It's different, though. It's there's some challenges with building a small team over building a larger one in some ways. But why don't we start with talking about why, let's say, we have a lot of really good customers, some that worked with us for many months, often over years. But why would you want to get rid of your BI vendor? If you have a good relationship and it's going well?
Caleb Ochs: 1:26
It's a good question, and it's actually kind of puzzling to me. I get it, I get why people want to build an in-house team, take this stuff in house and do it themselves. I understand the desire to do that. But I do think that there's a bit of a fallacy here around, "It's gonna be cheaper or better for us or safer," or something... There's probably all of those things to some extent in the thinking there. And it's reasonable to think that. But you know, from our side of the fence, what we're seeing is that... honestly, we've been better than any internal team we've ever seen. So we're just straight up better than them. And it's not like we're just somehow inherently better. It's because we do this day in and day out. The number of scenarios that we see, the number of people we have... 20 something engineers. All they do is BI, right. You're not going to find a 20-person BI department, especially in the companies that we work with in that middle market area. So, we're just more effective at doing this stuff. We've seen more and had more experience.
Brick Thompson: 2:44
So, I can understand why people would assume it's going to be less expensive. But our experience has been that as our clients are bringing on teams, they end up churning through employees, because it's hard to get people that are good. Some of the reasons that you gave there I think are some of the properties of a company that are different from us that make it that way. So if you only have a couple of people, they don't have a team to learn from, you know, an experienced team. They also may not get sort of the varied experience that you'd like to have to be able to tackle all these different problems. It seems a little self-serving for us to say why, but we've really thought about this. And actually, if you think about the amount of payroll costs that you need to take on to have a good internal BI team, you could make a case pretty quickly that it's going to be less expensive to have a good vendor, who's giving you the ETLs and the reporting that you need, just when you need it. One of the advantages of having a vendor is you're not paying them when they're not working on a project, whereas an employee you're always paying. But maybe you have enough work in your company where you think, "Alright, I'm gonna be able to keep three or four people busy," it still can be challenging, I think, to get it done as efficiently and as effectively because those three or four people only have their skill set, whereas a good vendor, hopefully has a larger team. And so as different problems come up, that larger team can step in and solve problems that the two or three person teams just wouldn't have the experience to go after.
Caleb Ochs: 4:22
Yeah. And sometimes the capacity. It's really interesting. And I think where I was going earlier was, it's really the safety part, right. Maybe you feel like, "We can't be tied to this third party forever," which is totally understandable. And it's probably right.
Brick Thompson: 4:43
I have those instincts myself.
Caleb Ochs: 4:45
Right. But if you really press into that, what you find is that building an internal team can be just as risky, if not more, than having a third party out there doing it for you.
Brick Thompson: 4:58
Yeah. All right. So, I think that aside, let's talk about how you would do it. Like, we want to be able to actually be really good at advising our clients on how to get self-sufficient when they want to. And so, where do you start? How do you think about it?
Caleb Ochs: 5:16
Yeah, I think all those reasons that we just laid out are why we started to think about this, because you ultimately want to help people get there, where they don't have to be tied to us forever. And we know it's not an easy thing to do, for all those reasons, and more than we just explained. So hopefully, we can start getting some of our thoughts out here, and we'll refine them as we go. And ultimately, it would be great if we could start helping people do this.
Brick Thompson: 5:42
Yeah, this may take us a couple episodes, since we're trying to keep these shorter anyway, but we can start on it today. I think one of the things that spurred me to start thinking about it and talking about it with you, and the rest of the senior team here is that, clients will ask us early on in our relationship, "Hey, can we take this over?" And our answer has always been, "Of course, we build it on your infrastructure, it'll be on your cloud tenants, we document it like crazy. We're happy to do training and handoff activities and so on." But the reality is, it's been hard to make that happen. And so, we want to figure out how to make it not hard and make it successful every time so that people aren't taking a shot at it, hiring a team and then firing them or having them quit and coming back to us, and now we're starting over again. It doesn't feel great, because if they want to be self-sufficient, we want to help them do that.
Caleb Ochs: 6:37
Right. We want to be able to answer that question with some confidence. And quite honestly, the confidence has waned over the past couple of years after watching people try this and it not working. So hopefully this will help.
Brick Thompson: 6:51
Yeah, exactly. So I think, obviously, taking over the BI requires a team. So if you don't have a BI team already in place, then you need to think about, "Who do we need, when do we need them?" If you've got a BI team in place, and you've hired us just to help augment on some stuff, this is not your problem. You've already got a team in place, and so you're bringing us into flex on time. But we're thinking about companies that don't have a team, they're starting from scratch, or they have someone who's done some dabbling in Tableau or Power BI or something, but don't really have a professional.
Caleb Ochs: 7:32
Right. And what you described, if you already have a team, that's kind of the end state, really, of the ideal. You have a team that's handling some of the things you need to do, but you can use us as that flex capacity when you need it. So, that first thing, you got to you got to build a team. And where do you start with that?
Brick Thompson: 7:53
I think you start with that visualization engineer. You could make an argument that "No, you're gonna need the data architect to do the ETLs, and get the data into a place where the vis engineer can do the reporting." But I would argue no. Keep using your vendor for that. It's a pretty specialized skill, and really what you want is to get someone who can start producing those reports, that's showing the company the ROI, and getting that person to where they're understanding the business, and the business goals and levers to be able to start actually delivering ROI for that payroll money that you're putting out.
Caleb Ochs: 8:30
Right. I think even if you don't have a vendor already, that's where you start. Because you want to get reports to the business, not spend forever dabbling in data. And that's actually one of our philosophies. Right? You always start with the reporting, not necessarily meaning you dive right into report development, but you you want to get reports out quickly. So that's definitely the right place to start.
Brick Thompson: 8:54
Yeah, because you want to get that adoption.You want to get that momentum within the company, that buy in. Okay, so you hire that visualization person, what's it going to take to get them good? And how are you going to hire a good visualization person?
Caleb Ochs: 9:08
Yeah, so luckily, we've done a podcast about that.
Brick Thompson: 9:11
Caleb Ochs: 9:14
We've put some resources out, go check out that other episode where we talked about hiring a good viz engineer.
Brick Thompson: 9:19
Yeah. So we do talk about that. I think, though, once you get the person in, and in that podcast, we really talked about how do you find that right person? How do you vet that person? And some of the resources we put in the notes for that will help with that. Still not simple, but it should help. But then once you have them on board, how do you get them up to speed with the business so that they're engaging well, and not just order takers, which is the downfall of viz engineers. That they get with a business executive, they get an order for a report, often the executive will sketch it out for him on a whiteboard, and they just go build that exact thing. And it ends up not achieving the goal of delivering ROI.
Caleb Ochs: 10:02
Yeah, that's the secret sauce, right. And hopefully you found somebody that's able to do that. But it might take some time to develop that thinking. And hopefully, if you have a good BI vendor in there already, that'd be the ideal state, then they can work through that with them, shadow, figure out how that process has gone, and then kind of adopt it as they go.
Brick Thompson: 10:25
Yeah, that would be my recommendation, actually. So even though you've got that person in house now have them work alongside an outside expert to help them get that skill set. And that might take a few months to get really good.
Caleb Ochs: 10:44
I think it would. I think it would take probably six months of shadowing. And that might be on the low side. We have some clients that do have teams, and we've come in as they've already been in place. And they're still learning from us. We've set up some special engagements just to give them some training on, "Here's the tools we use, here's why we use them." So it has to be intentional, and I do think you're probably looking at six months of that.
Brick Thompson: 11:10
Yeah. Okay, so I think the next hire is probably
Caleb Ochs: 11:11
Yeah, right. You want someone that's going to be that Data Architect. So you got someone producing reports, now you want to become self-sufficient on the ETLs, and data transforms and all that stuff. That's when you need the person that's got those deep SQL skills, or whatever platform you're working on. able to expand on the BI platform, if you've got one. If you don't, and this viz engineer is just connecting to your sources and building reports, you want someone that's going to be able to come in and say, "Okay, this is how we're going to make these reports scalable, enrich them with other data sources, mash things up. And here's how we're going to do it." That's where your data architect comes in.
Brick Thompson: 11:53
Yeah. And it's hard to know exactly when that person comes in. You could hire that person at the same time you hire that first visualization engineer. I would probably stagger them, just so that visualization engineer sort of gets a foothold and starts knowing the business so that they can help sort of get the data architect in sync with the business.
Caleb Ochs: 12:13
data is going to be structured. And... it just kind of creates a little bit of chaos that you can eliminate or reduce by stepping into it. Right. I think that would be the ideal situation. Either you hire
Brick Thompson: 12:32
Yeah, agreed. So then I think the next hire is some kind of PM. It could be a department leader, if they have the PM skills. I don't think it needs to be a full PM, because the size of the department is quite small. But I think you need somebody who is going to be able to help interface with the business, help with prioritization, with managing projects, with getting business requirements transferred in a good way to the engineers, doing communication, update meetings, and all of that stuff. You'd love to have that on day one, but I almost think it's overkill. Maybe you could borrow a PM from a different department to help with that on day one. a project manager and you, like you said, give them other responsibilities. Or maybe you can augment an existing project manager's role with taking on the BI stuff. Yeah. So at this point we're looking at three people, and I think you may even have two viz people at this point. And then actually, depending on your data environment, let's say that you're a PE owned company that is doing a buy and build strategy, you might need two data architects as well, because you're doing integrations of all the add-on companies. So you're looking at significant payroll. You're probably, I don't know, it depends on how many people and skill levels and what market you're in, but you're probably looking at half a million bucks a year, or something like that.
Caleb Ochs: 14:06
Yeah, I think that's probably a good benchmark to think about. Especially with salaries nowadays, I mean, you're probably looking at least 100k per person. Probably a lot more for the more specialized roles like a Data Architect. It's not cheap to do that.
Brick Thompson: 14:24
Yeah, agreed. Well, look, we've already gone over our time limit here. I think this might be a good place to wrap this part of the discussion, just discussing who the people are you need to hire and what is likely to cost you. I think I would want to just touch on the timeframe, just quickly. We sort of said six months to get that visualization person up to speed. I am thinking this whole process is probably no less than six months and probably more like 18 or 24 to get really self-sufficient. You've kicked out your BI vendor and you don't need to see them anymore.
Caleb Ochs: 15:00
Right. And if that's the goal, you are probably looking at something like that. 18 to 24 months. So you do this, just be ready for that. I think that's another one of the things that has been challenging for people. They don't really know what to expect when doing this, so hopefully that will help.
Brick Thompson: 15:16
Well, if you're planning to do it, let's say someone's hiring us now and saying, "Hey, we want to become self-sufficient," we should talk about that timeframe now. And if they're saying, "Yeah, I'd like it by the end of next year or middle of next year," we need to help you get that first visualization engineer hired now and have them integrated into the projects so that they're getting up to speed, so that you can get that timeline.
Caleb Ochs: 15:38
Yeah, that's exactly right. As soon as you get started.
Brick Thompson: 15:40
Yeah. Okay, there's more to talk about here. We'll come back and do some more another day.
Caleb Ochs: 15:46
Brick Thompson: 15:47
All right. Thanks.
Caleb Ochs: 15:48