Managing “Shadow AI” at your agency

The following is a computer-generated transcript. Please listen to the audio to confirm accuracy.

Chip Griffin: Hello and welcome to another episode of the Agency Leadership Podcast. I’m Chip Griffin.

Gini Dietrich: And I’m Gini Dietrich.

Chip Griffin: And Gini, I, think it’s appropriate that it suddenly got dark, in, in- Yeah. … in your office as we were trying to begin this recording- Mm-hmm … because we’re gonna be talking about being in the shadows.

Gini Dietrich: Yes, we are. In the dark. But before we do that, happy birthday.

Chip Griffin: Oh, thank you. Thank you. You’re welcome. I appreciate that. I thought maybe we’d escape that since- Nope … ’cause we were on hiatus during my actual- Nope … birthday month, so.

Gini Dietrich: Nope. Ah, come on. Yeah. It was just a few days ago. Happy belated I guess.

All right. Thank you. Especially by the time people hear this, but.

Chip Griffin: Yeah. Well, nobody will really know, so.

Gini Dietrich: I know.

Chip Griffin: I, I, appreciate it, and I mostly appreciate that you’re not singing, so,

Gini Dietrich: I can. Do you want me to?

Chip Griffin: I do not. I- Okay, well- ‘Cause, ’cause isn’t, don’t you have to license “Happy Birthday,” I think? I think that’s-

Gini Dietrich: Well, I have a different song.

It’s a song that- Oh … the kids learned in third grade that we sing instead of “Happy Birthday.”

Chip Griffin: Is, is this the dirge?

Gini Dietrich: No.

Chip Griffin: Oh. See, I like the birthday dirge.

Gini Dietrich: I don’t know the birthday dirge.

Chip Griffin: Oh, it, it, it’s, uh, it goes something like, you know, uh, another year closer to the grave or something. I mean, it’s very more…

But, the- I wouldn’t- Our kids learned it, learned it at a kid’s party, you know, 20-some years ago. Oh, that’s so good. I’ll have to look it up and share it with you. It, or- I’ll look it, yeah. I’ll delete it … or maybe, maybe Jen can do that because, and maybe sh- b- maybe she can include a link to it, in, in the show notes, ’cause it is somewhat entertaining, but I, I don’t remember the lyrics, nor do I want to try to, to invent them or sing them even if I knew them.

Gini Dietrich: All right, fine.

Chip Griffin: But, no, the, the-

Gini Dietrich: Maybe for my birthday. You have s- you have several months to, to learn it. You could sing it to me for my birthday.

Chip Griffin: Well, more than several, but yes. Anywho, so moving on from that. We, are going to talk about the shadows, the deep, dark shadows of the PR world.

No, we’re gonna be talking about, shadow AI, which is, in a lot of organizations, you have employees who are using their own ChatGPT, Claude, Gemini, whatever accounts to do work. They are not using corporate accounts. A lot of them do not understand the implications of it. The employees do not understand the implications.

The employers do not understand the implications. And I’ve seen it, either bite people or potentially bite people of late, so I thought I would raise it as something that, that owners ought to be thinking about in the current environment.

Gini Dietrich: I will tell you, not to get anyone in trouble, but almost every one of our clients, and they’re big clients, people do that.

Could be, and they’re doing it from their personal devices. It could be because it’s blocked at work. It could be because what they have at work isn’t sufficient. It could be because CoWork is significantly better than, Copilot. It, like, there are lots of reasons, but people will full on pull out their phone or their iPad and do it on their personal device at work all the time.

Chip Griffin: It is incredibly common, and, from my perspective and what I advise owners is that you’re probably not going to stamp it out.

So at a minimum, you need to educate employees- Yep … because a lot of employees don’t realize the risks in what they’re doing, and they haven’t set up any kind of a, a process to make sure that they’re, you know, doing it as intelligently as possible at least, if they’re- Right … gonna use their own accounts.

And, look, I, mean, I, don’t necessarily think that, that it’s awful for them to be doing this. I wouldn’t try to eliminate it. But you do need to educate them because a, a lot of times when I talk with rank and file employees of agencies, they don’t realize that, for example, that their personal accounts, by default, all of the data leaks out for training purposes.

And it’s in many cases, depending on which tool you’re using, as simple as a toggle to make- Yep … sure that that information- Yep … does not go straight into the training sets- Yep … of these providers. And that’s the default setting on your corporate accounts if you’ve got them as an agency, but it is not the default on most personal accounts.

And so just that one change can make a big difference. But I have yet to talk to any individual employee using their individual account who knows that that’s even something they can or should do.

Gini Dietrich: Yeah. I agree with you. I think that having just a really easy AI policy is the right way to go because I don’t– I think you’re right.

I don’t think you’re gonna stamp it out. I mean, heck, I use my personal one for lots of stuff too. Mostly because I trained it before we had the corporate account, and I don’t wanna go back and retrain what I’ve already worked, what I’ve already created. So, but we just created a one-page, like, bullet points AI policy that says exactly what you said, like toggle it off.

If you’re gonna be vibe coding, here’s what you need to be thinking about. You know, all of the things. I think it’s probably 12 bullet points. It’s not a huge lift, but it just helps them understand. And one of the things that we did is we spent some time building the cowork Instance for the organization, and it has, you know, our OKRs and our plan and our vision and, you know, all of the stuff, our brand kit, all of the stuff.

So it’s easier for, my team now, it wasn’t six months ago, but now it’s easier for my team to use the company one because it has all of that stuff in there and it’s already been trained.

Chip Griffin: Yep. Yeah, I mean, it’s, you know, so I think there’s, this is a multi-stepped process. You know, part of it is the policy, part of it is education.

Gini Dietrich: Yep.

Chip Griffin: Part of it is you should have a standard corporate tool that you are using. It doesn’t mean that it’s the only one, but, it should be– If you do that, it will allow you to do a lot of the information sharing, skill sharing, things like that, that ensure the consistency across the organization and make sure that everybody doesn’t have to reinvent the wheel constantly.

But I think the you brought up two things that I think are worth exploring more. One probably is a separate episode in addition to this, but the other, I think, makes sense within this context. The one to sort of put a pin in and come back to is your fear of moving from your personal one to the corporate one because of the lost, you know, memory and context and all of that kind of thing.

And, I think that there is, I, I, think there’s a lack of knowledge in the agency community generally about how best to capture all of the, the knowledge and things that you’re developing alongside these AI tools and making it as portable as possible. So, you know, one of the things I’ve been focused on in recent months is really making sure that, that my second brain, if you will, in AI is portable across multiple tools.

And so I’ve now built it so, for example, I have one brain that I share across both my Claude and ChatGPT accounts-

Gini Dietrich: Yep. I do the same thing. Yep.

Chip Griffin:… so that you have consistency, and, I think it’s worth sort of exploring that, maybe not in technical detail, but, you know, how you can share it amongst your own accounts, how you can share it more effectively with your colleagues and coworkers and that kind of thing.

So I think that’s worth revisiting in more depth because I think there, particularly as we become more AI forward as agencies generally, that’s something that becomes increasingly valuable and increasingly important. But the, piece that, that I think really fits into this shadows discussion is the vibe coding, and we see a lot of agency employees doing vibe coding, which is fantastic.

I am, I’m a huge advocate of it. I think that if you want to get ahead in professional services generally, in the agency world specifically, you need to learn how to do at least basic vibe coding. That said, I think people do not understand all of the risks and complexities associated with vibe coding.

Gini Dietrich: Right.

Chip Griffin: And, so, there are a lot of basics that, that agencies need to be thinking about here, and again, it comes into that education and training piece in working with employees, because most of them don’t understand if you mess something up and you are not doing version control or keeping backups, you can be, you know, really up a creek without a paddle.

Yep … and yes, you can reconstruct it in some fashion oftentimes by going back in the conversation, but it’s not simple. And because most people who are doing vibe coding today do not have any previous programming experience, they don’t understand these concepts like version control and being able to roll back easily and, all of the things that geeks like me who’ve been coding since, you know, the early ’80s get and understand.

And, and, that Git reference, by the way, was for those of you who do actually know your coding stuff- … because it’s a great repository tool that most of you who are doing vibe coding should be looking into because it- Yeah … is really helpful-

Gini Dietrich: Yep …

Chip Griffin: in terms of making sure that when you have an oopsie, you can solve it and fix it easily.

But a lot of people if, if they’re doing this on their own devices, they may not even have just regular backups of this stuff because maybe they’re saving their regular agency work to a Dropbox account or OneDrive or whatever you’re using. But whatever they’ve set up to use with Coworker or Codex or whatever, they may not have that in one of those directories that is by default syncing to the cloud and getting updated.

And so you wanna make sure that you’re helping them understand that that needs to be part of the process, because how awful would it be if you just lost all of this work that you had been doing? And so much of what you do with Coworker, Codex is device specific, and you cannot access anywhere else. I suspect that will change over time.

I suspect that, that those will become, you know, more cloud-like, and we’ve already seen Claude, for example, merging the chat and cowork functions somewhat in their app. Yep … and, so we’ll see a place where I think you can just share it across devices, but right now it is frustratingly difficult for those of us with multiple devices to manage.

I’ve got two PCs and a laptop and other and, and so I’ve actually written my own systems for converting the code so that it all is accessible elsewhere. But if you don’t know how to do these things, you could be in a real world of hurt if your computer crashes and you’ve got no backups.

Gini Dietrich: Yeah, and I will say that, not that I speak from experience or anything, but it only has to happen to you once, and then you learn very quickly how to make all of that happen.

Because you’re right, I, I was vibe- I love to vibe code. It’s one of my favorite things to do, but the very first time I did it, I didn’t know I was supposed to do version control or any of that stuff. Right. I didn’t know anything about Git. I didn’t know any of it. I do now.

Chip Griffin: Yep. And look, that’s, how most of us learned the hard way in the olden days of writing this code.

You’re like, “Oh, shoot, I wish I had had a copy of that.” Yeah. You know? And, in the old days, our backups were printouts, right? Because you- Right … there was no way… Like, when I did computer coding on a cassette tape, you know, there was no real way to make a copy of that easily, so, you know, we would just, you know, hit print, and, you know, on a little dot matrix printer we’d have a copy of the code so we could retype it if we absolutely had to.

Gini Dietrich: That’s so funny. Ugh …

Chip Griffin: not ideal. A lot easier to do things today.

Gini Dietrich: Not ideal, no.

Chip Griffin: but, you know, those are the kinds of things, and, if we’re all going to become programmers of a sort, we need to be thinking about that. We need to be thinking about, how do you properly test and maintain some of this stuff that we’re creating?

Because it’s super easy to vibe code the first version of something. It’s a lot harder to handle the maintenance that’s required on it, you know, when connections to data sources break or technology- Yep … evolves- Yep. Yep … or those kinds of things. Yep. Most, of the people who are vibe coding don’t do anything in terms of security testing of the, the code that they’re writing.

And, if that vibe coded thing ties into other systems, which many times they do, you may have created, an opening that you’re not aware of into your back-end systems that could be problematic. And so these are all things that we need to increase the level of education about so that our teams are at least thinking about these things.

I’m not saying they’re gonna solve them all. They’re not gonna… We’re still gonna have issues that crop up. But we’ve got to be doing more to try to educate owners, employees, and everybody else involved in the process.

Gini Dietrich: Yeah, absolutely. I really think that starting with an AI policy is the right thing to do.

Don’t make it overly complicated. Like, when we started, we had this big, like, AI policy legal packet. And I was like, “This is way too much. Way too much.” Like, people are not going to absorb that. So I used my AI to dumb it down, for lack of a better term, and, you know, really to highlight the things, and then I went through myself and said, “Okay, great.

These, this is a good start. Now I need to add this, this, and this,” just based on how I see people using it. So, I also think that shadow AI is one thing, but there are plenty of people who are not using it at all. And I was actually kind of shocked to find that in my own organization, like, really? O-Okay.

So we’ve done a little bit of, you know, we do a, we do micro-learning sessions every week. So we’ve done a little bit of AI micro-learning just to show people, like, it’s not just for a blog post draft or helping you refine an email. Like, it can h- it could be a thinking partner. It can help you with these things.

And so we’ve been doing some micro lessons on that too to just help them understand that this isn’t gonna take your job. We still need you to do your work. We still need your brains for all of this, but this will make you more effective and more efficient.

Chip Griffin: Yeah. Although I, I, have taken to becoming much scarier, and I’m telling people it is gonna take their job if they are not using it effectively themselves.

Gini Dietrich: If they don’t use it. No, I agree with that.

Chip Griffin: Yeah. But, I agree with you. Yeah. There are, there are a lot of folks in the agency community who are not using AI beyond very rudimentary use.

Gini Dietrich: Yeah.

Chip Griffin: And, and I, I don’t think there are very many who aren’t using it at all, but there are a lot who are using it more like a, you know, a replacement for Google or something to, you know, give them a quick draft of a blog post or an email or something like that.

And, you really need to be taking advantage of it at a much higher level if you want to be successful in really any kind of knowledge work moving forward. And- Yeah … and I think people have a relatively short window to get up to speed on this. I think we’re talking, you know, a year, two years tops.

Gini Dietrich: If– Yeah, yeah. I think two years is being generous.

Chip Griffin: And, I think if, if you are, if you are not, if a year from now you are not actively using AI every single day in a really intelligent way, I don’t know that you have a future.

Gini Dietrich: I would agree with that. I would agree with that. And I think you have to use it, to your point, in a really intelligent way.

I just answered, or I just had a conversation with a Forbes reporter who’s writing about, there’s a term for it… I can’t think of the term right now. There’s a term for you using it as a thinking partner, using AI as a thinking partner, and he was telling me that almost nobody does that, and I was like Really?

Like, that’s the only thing I use it for is, you know, here’s what I’m thinking, poke some holes in it, play devil’s advocate, tell me, you know, what’s strong, what’s weak, what I need to think through more effectively. It has helped me with … I mean, I, I think I’ve mentioned before, I call it my co-CEO, and I’m like, “Okay, here’s today’s challenge.

Here’s what I’m thinking. Here’s the documentation. Here’s the backup. Help me think this through.” And it’ll be like, “What about this, and what about that?” And I’ll say, “Well, no, I think you’re wrong about this,” and, “What about that?” And, like, we have ongoing conversations about things, and I don’t … From what he was saying, like, almost nobody uses it that way, and I think that is the real value because it helps you operate at a completely different level.

Chip Griffin: Yeah. I mean, you’ve got … You have to, and, it, feels weird, but you have to treat these tools as if they are actual employees, consultants, whatever you wanna call them. And, it, it absolutely feels weird to anthropomorphize a chatbot. And, certainly there are ways to, to go way overboard, and y- you know, you hear these, you know-

Gini Dietrich: You’re not gonna fall in love with it

Chip Griffin: these really, really weird stories of, of what people have done. And, you can sort of … You know, the more time you spend with them, you s- you can kinda understand how it, you know, for the right personality, maybe it kind of veers down those- … creepy paths. I’m not encouraging that. No. I’m not encouraging that.

Gini Dietrich: No, no.

Chip Griffin: But, but you’re absolutely right. You have to be having meaningful conversations about the work that you do, your strategies. They are incredibly good at poking holes, incredibly good at helping you to think through things. I mean, I, have spent probably, I wouldn’t say an inordinate amount of time, but a lot of time having it challenge me.

I have Claude interview me on a regular basis on different topics so that it can build its knowledge, because I’ve built a whole second brain operating system kind of thing. Yep, yep. I’ve had it mine through, and, maybe, this is another episode at some point where you and I can talk about some of the systems that we’ve put in place as examples, because I know from our previous conversations there’s some overlap, but also different ways that we do things.

But you know, I, I’ve got 20-plus years of, a, a digital footprint, and AI is great at mining through that. And so I’ve had it do that so that it can, it knows more about me. And so when it pushes back, it pushes back with specific examples, and it will say, “Well, when you did this in, you know, 2007, you know, this was the decision you made. You know, why isn’t that relevant here?” Or things like that.

And it’s weird at first, but, you’re ne- you wouldn’t even find an employee who could do that because none of them in, in all likelihood have been with you for 20 years. Yeah. And so having that available to you is something that you just should not be passing up, and we want to encourage our employees who probably don’t have a 20-year footprint like we do to be trying to find ways to do it, and we want to try to facilitate them using all of the tools at their disposal.

So we certainly need to avoid the knee-jerk reaction when you hear an episode like this, shadow AI, oh my God, I need to shut this down. I can’t- No … I can’t have employees doing– You cannot have that reaction. Mm-mm. And you should not have, while you should have an AI policy, it should be simple- Yeah … and clean.

We can’t go back to the early days of social media policies that, that organizations tried to put in place. And again, we’ve been around a long time, so we’ve seen this movie before. And some of the social media policies that people were putting in place 20 years ago were absolutely bonkers and unnecessary.

And I fear that we’ll see some of the same thing on the AI front. Yes. And part of this, by the way, is, with all due respect to our lawyer friends, sorry, Sharon, don’t talk to your lawyer first about this. You can talk to your lawyer about it, but, lawyers are naturally risk-averse, right? And so if you, if you put this in the hands of your lawyer, particularly if it’s not someone like Sharon who has deep experience in the agency world, they’re gonna sit down and they’re gonna say, “Oh, you need to say no to this, and this, and this, and this.” And all of a sudden, nobody’s able to even use AI in a meaningful way.

Gini Dietrich: Yeah, yeah.

Chip Griffin: And so you, you’ve got to try to put reasonable safeguards in place, reasonable policies, but I think the most important is the education piece.

Yes. If you educate people, they are much more likely to make the right decision. It’s not guaranteed, but it’s more likely, and right now we’re at a place where we’re not doing the right level of education of our teams. And part of that is because a lot of owners don’t actually know a lot of what we’re talking about.

I mean, I think, you know, you and I- Yep … are certainly at the leading edge- Yep … for a lot of folks in the agency community. We need to get more people at that same level and let it flow through to their teams to make sure that we really are leveraging this technology for all that it can do.

Because it, I mean, I, I have n- I have not been this enthusiastic about a piece of technology in the world of PR and communications at least since the beginning of the World Wide Web in the mid-’90s.

Gini Dietrich: Yeah, I agree with you. And, like, the– I know I’ve said this before, but the amount of work and the productivity that I’ve been able to achieve, I honestly don’t know how I did my job without it.

It’s, it is, it’s next level. And you know, there are some weekends where I’m so excited about something that I’m working on, it might be vibe coding or something else, that I will literally sit in front of my computer all weekend and just, like, in the zone because I’m so excited about it. So I think that there’s a big opportunity here for you to explore and to understand and to change the way that you do your business, run your business in a really fun and effective way.

Chip Griffin: Well, and that’s probably yet another ep- episode topic, going forward in, in trying to figure out how you invest your time in AI, because it is, it is just as easy to go down unproductive rabbit holes because- Yeah … they are fun. Sure. Right? And so I, I often find myself sitting there saying, “Do I really need to build this? Is this- … is this really helpful?”

You know, I… and it, it, reminds me a little bit of, woodworking, which is something that I’ve done for- Yeah, yeah, yeah … for many, many years. But, like many woodworkers, I have probably built more things for my shop than I have to use in my house, right? So you, you spend so much time, you know, building workbenches and cabinets and jigs and all of that kind of stuff, which are all really cool, but at the end of the day, they’re not the things that you, you know, it’s not the furniture or the built-ins or whatever that you can use around the house.

And so we need to be careful that we don’t get so enthusiastic, that we’re only building for that. So making those decisions about where does AI actually help and where is it, you know, kinda using it for the sake of using it, is, is something to be paying attention to as well.

Gini Dietrich: Yeah, totally agree.

I love it. I’m, I’m a big fan. Love, love, love it. Love.

Chip Griffin: And so we’ve, after our summer hiatus, we sat here and we said, “You know, we should’ve been spending more time thinking of topics to come up with.” And, and so instead we kind of pick a random topic to go with, and we’ve come up with multiple episodes, for future discussions, so.

Gini Dietrich: I wrote them all down too. So they’re in our document, so we have-

Chip Griffin: Out of the corner of my eye, I can see that our shared document- … has been, been being updated. I, I cannot update while we’re talking because I have one of those really loud clickety-clack keyboards. And so it would, overwhelm the audio here.

Dun, dun, dun, dun, dun. Because I, I, like the noisiest possible keyboard you possibly can have. So. I love it. Anyway, on that note, I, I think it’s probably a good time to, wrap up. We can come out of the shadows with AI. Maybe it will, you know, the storm will pass, in Chicago, and it will get a little bit brighter for you in your office as well. It still seems like it-

Gini Dietrich: It’s like nighttime …

Chip Griffin: it must be pretty dark there. Yeah, crazy. If you’re, if you’re not watching us on video, and you really should watch us on video, because it is so compelling to see us and not just listen to us. But it, it definitely looks dark there. So on that note, we will wrap up here. I’m Chip Griffin.

Gini Dietrich: I’m Gini Dietrich.

Chip Griffin: You… Did you forget who you were? There was, there was a long pause there.

Gini Dietrich: It’s because you, there’s a delay with you.

Chip Griffin: Oh, okay. That’s good to know.

Gini Dietrich: Yeah, yeah.

Chip Griffin: On that note, it depends.