In this episode of the Certified Skills in AI and Tech Podcast, we speak with T. Scott Clendaniel, Director of Data & Analytics and AI expert, who has been a leader in the field since 1986. With experience at top companies like 3M, Biogen, Mercedes, and Morgan Stanley, Scott provides valuable insights into the evolving world of AI and data science. He serves on multiple boards, mentors upcoming talent, and has lectured at Johns Hopkins and the University of Maryland. Scott takes the time to discuss AI’s impact, its practical applications, and the future of data analytics.
You’re the director of data analytics and AI. What is AI doing to your world? How has AI changed the way you think about your space?
I’m happy to participate, but I come as myself. I do not represent Gartner or endorse any products. We take our impartiality very seriously, so we always start any interview with that.
So, how has artificial intelligence affected my space? I started in the field in 1986, just past my 30th anniversary. I’ve been involved in advanced analytics most of the way through. The biggest change has been that for two decades, I was begging people to notice data science and machine learning. People were resistant and didn’t believe it could do anything. They often saw us as geeks who were bothering them.
Now, the pendulum has swung the other way. I’ve seen advertisements for electric toothbrushes and toasters that include artificial intelligence. It’s like we’ve gone too far to the other side.
Generative AI has a lot of potential, but I think the hype has gotten ahead of our ability to maintain and regulate it. The media hasn’t explained some of the risks of this technology. My fear is that as we slide down the other end of the hype cycle, there are going to be some really unhappy customers.
There’s a lot of conversation right now about software engineers and what their roles will look like. A lot of coding is being automated with tools like ChatGPT and Copilot. Software engineers, who were once in high demand, are now worried about job security because major companies are laying off software engineers. They’re thinking, “Where can I go to create an impact?”
I think it’s really important to be careful when making predictions. As Yogi Berra said, “Predictions are always dangerous, especially when they’re about the future.” I don’t think software engineers are going anywhere; their roles are just going to change. It’s going to be more about assisted or augmented intelligence than pure artificial intelligence for generating and maintaining code.
The best use cases I’ve seen involve the software engineer developing an initial script, then passing it off to an AI tool for review, feedback, and further questions. For example, “I see this error here; I’m not sure what it means or how to resolve it.” There was an interesting article last week saying that because ChatGPT is primarily based on data prior to September 2021, its ability to pass the leaked code tests— which became popular—was because it memorized the answers. If you test it against more recent data, the accuracy rate drops significantly.
Remember what generative AI is actually doing: it’s taking a group of tokens and predicting the next most likely word or phrase. There isn’t intelligence in the sense of a tiny programmer inside the computer thinking about what to do next. It’s much simpler than that. Calling it artificial intelligence is a bit dicey because when we hear “intelligence,” we assume there’s some wisdom or thought process, which isn’t really there. It will be interesting to see what happens and whether we get better at making predictions.
One of the great ways to start a bar fight among people in the field is to make them argue about what intelligence means in the first place, and then stand back because things can get dicey. Whether we’re going to have artificial general intelligence or not, we’ll see. Will we have something that mimics it in the future? Probably. Are we going to have the rise of the machines? Probably not, in my opinion.
I’m more concerned about bad actors controlling technology than about technology developing consciousness on its own. Bad actors have been around for a long time, and we know they’re out there and how they work. That’s my bigger concern in the short run.
If we look at companies, those that you advise and others, everyone is at least dabbling with ChatGPT or some type of generative AI by now, but the field is so much bigger than that.
What’s your general advice for companies regarding AI in the short term?
Boring artificial intelligence is profitable artificial intelligence. The more we chase after shiny objects, the more trouble we’re likely to get into. Plain old machine learning can be run in Excel right now. We don’t need generative tools or copilots for that. If we focus on projects like fraud detection, predictive maintenance, or response rates to an email offer—things that have been around since Sears pioneered them in the ’50s—those continue to be very profitable, very controllable, low-risk, and relatively robust. You don’t need all these other elements for that.
My big question to people is: if you’re going into generative AI now, why weren’t you using those other techniques before? What is the ROI you’re expecting from generative AI that you wouldn’t have gotten from the prior techniques? And why do you feel now is the moment to be in rather than out? These are important questions to think through.
If organizations haven’t taken care of their data governance, it doesn’t matter what technology you put on top of it; you’re still going to have problems. It’s cliché but true: crawl, walk, run. Start with the basics, especially on the expense side, then move forward. Having multiple AI tools at $20 each for every employee is an unbearable burden for most organizations, and many haven’t demonstrated that the ROI is there.
Look at the Goldman Sachs research that came out two weeks ago. And regarding the engineers currently in positions, some may be worried, some may not be, but the advice for them is to recognize that AI isn’t going away. Machine learning will evolve, get better, go through the hype cycle, and likely a valley of disappointment before hitting its stride, but it’s not going anywhere. Just like the Internet, it’s going to be ubiquitous.
What should people think about their current skill set in light of AI’s evolution?
We need to agree that the traditional model of education—four years in high school, maybe four more in college, and then you’re done—is no longer relevant. Whatever we decide today, it will be different in five years. It’s time to think about ongoing upskilling. What’s my personal plan for continued growth? How will I practice these skills? Because whatever we choose, it will change.
In the Gartner peer community, there are trainings and certifications in AI, which are crucial. Upskilling isn’t going anywhere. The real problem is that organizations are still operating under assumptions from the industrial revolution. Generative AI may not change that much, except to highlight the need for continual development and improvement. Organizations can’t just send employees to a single training and think they’re done. The world doesn’t work that way. Skills development is needed for people of all backgrounds.
Are things getting harder with AI’s impact on marketing and content creation?
Yes, especially in marketing. It has become more competitive because so many people are using AI to create content. You still have to be creative and get people’s attention, which has become more challenging. AI has lowered barriers to entry. Now, anyone can create something creative without needing a huge budget or extensive skills.
The fundamentals of marketing—understanding your target and crafting your message—haven’t changed. AI just changes how we execute those ideas. The focus now will be more on the message and less on the tools used to create it.
What about AI’s role in product development?
AI does offer opportunities to learn about user behavior and create new products. But understanding customer behavior has always been important. It’s less about how big your cloud is or which algorithms you use and more about making better decisions. Most organizations haven’t been willing to truly understand their customers and create products that meet their needs. Adding “with artificial intelligence” to a product doesn’t necessarily make it better.
Bill Gates once said you could take the worst idea, add “on the Internet,” and make money until the bubble bursts. Today, “with artificial intelligence” has become the new buzzword. But I don’t think my toothbrush is doing much more with AI than it did before. The real opportunity is that AI lowers barriers for non-technical individuals to interact with technology. We’ve made complexity as simple as sending a text message, which opens up a lot of possibilities.
It sounds like lowering barriers to entry is a major theme in our conversation. Does that democratize business?
Lower barriers to entry can be good or bad. They can democratize business, but they can also rush products to market without proper testing. Automation existed long before generative AI, so this isn’t entirely new. Companies should be careful what they wish for because democratizing technology also means more competition and potential pitfalls.
Kryterion’s Approach to AI-integrated Test Development and Delivery
At Kryterion, we clearly see the opportunities and disruption that AI is bringing to our industry. Our top priority is to provide clients with the best mix of security, innovation, service, and value in our tools for test development and delivery. AI is quickly becoming an integral technology in our products and direction. The path we are on with AI encompasses the immediate benefits it provides while maintaining a sharp awareness of its evolution allowing us to ensure our products continue to meet or exceed the needs of our clients.
Please contact us to set up a meeting so we can further discuss how important AI is to our future and our clients.
Want to keep up with insights on AI in the testing industry? Listen to the Kryterion podcast
About Kryterion Inc.
Kryterion Inc. is a global leader in innovative testing and credentialing solutions, helping organizations across various sectors develop and manage their assessments with our advanced test development platform and multi-modal delivery solutions. Established in 2001, Kryterion offers secure, integrated services and extensive support, enabling clients to demonstrate skills and achieve world-class careers.
Ready to take the next step to start or grow a program for your organization? Click here to connect with a Kryterion expert.
