Cracking the Code on Building an AI-Powered Skills Taxonomy with Brian Ray, Global Head of Data Science and AI at Eviden

Felicia Shakiba  00:04

Hello everyone. I'm Felicia Shakiba, and you're listening to the CPO PLAYBOOK podcast. Join me and my guests as we feature insightful conversations with HR leaders, people scientists, and executives from diverse industries and functions, offering valuable perspectives on the future of work. Discover a unique outlook on navigating the complexities of the modern-day working world, exploring innovative strategies in talent management and corporate culture from the Chief People Officer's perspective. Tune in to stay ahead of the game when it comes to all things people-related.

Felicia Shakiba  00:45

Organizations that have undergone multiple acquisitions face a particularly daunting challenge, the absence of a comprehensive skills taxonomy. This deficiency not only hinders workforce agility, obstructs informed decision making and undermines overall efficiency, but it also compounds the complexities brought about the disparate company cultures and structures and vision possessing a dynamic and adaptive system capable of evaluating present skill levels and anticipating future requirements. That is the ultimate aspiration. The savings in terms of HR and leadership efficiency spanning from hours to potentially years would be beyond measure. 

In our upcoming interview with AI expert, Brian Ray, Global Head of Data Science and Artificial Intelligence at Eviden, we delve into strategies for constructing an AI driven skills taxonomy tailored to the intricacies of companies navigating multiple acquisitions are just for themselves offering insights that are invaluable in surmounting these multifaceted challenges. 

Felicia Shakiba  01:57

Brian, thank you so much for being with us today. Welcome. 

Brian Ray  02:01

You bet. Thanks for having me. 

Felicia Shakiba  02:03

If I were to embark on constructing an AI powered skills taxonomy within an organization, where would I start? 

Brian Ray  02:11

Well, I mean, you must have some historical data in most organizations, from the folks who are working for you today. So I would start there, look at what data you have available. If it be, you know, resumes or job descriptions, or whatnot, start with that, and then move quickly on to where you want your capabilities to move to. So do you have any business objectives as a business that you want to get into a certain area? Do you have any business problems need solved? What sort of capabilities and skills will support that movement? 

Brian Ray  02:48

Once you have a general idea of that data, it's always a good idea to do an exercise an analytical exercise to look at what key phrases occur most frequently throughout that data, and they segment based off of title and other aspects of it. And then you move on from there. And eventually, what you'll end up with through that process is a taxonomy that clearly displays the different areas your business operates in today, as well as some of the areas where you want to move to. 

Felicia Shakiba  03:19

Having clean data, in the very beginning, is essential in order to even start this exercise. 

Brian Ray  03:25

Yeah, I mean, you're dealing sometimes with data that isn't as precise as other data, it's not like physics data, where you're just dealing with numbers. Sometimes when you're dealing with human data, you're dealing with titles and descriptions that need disambiguated. And that's something to keep in mind that there is no perfect clean data. But when you do an exercise, and you have enough quantity of data, the noise goes away, and the signal starts coming up. And you start seeing clearly what traits identify certain positions and roles and things like that. 

Felicia Shakiba  04:00

And so when you're delving into the intricacies of data requirements, what specific datasets would be important for this AI driven skills taxonomy to effectively analyze, you know, what are the diverse range of data sources we would use? 

Brian Ray  04:18

Yeah, good question. I mean, you can rely on your historical candidate data, which would be CVs and resumes. And also correspondence around the first lines of communication with folks who have not hired yet, you can also look at ONET occupation codes, those are available as their open source or other occupation codes that set out there to help you segment different lines of business and different capabilities. 

Brian Ray  04:46

And it's also interesting to look at data as a timeline. So for instance, it's very difficult to do, but if I'm able to notice that someone has done a particular job for a while much longer than they have done these other jobs, it's a good idea to weigh that to take into consideration that this person may have held this position for 10 years, versus another candidate that may have held that same position for 10 months. So you always want to contextualize the data, when you're looking at people data, you want to be able to understand that in the context of the knowledge that they may have gained throughout that process. 

Felicia Shakiba  05:27

I'm just thinking about the skills or where we would house that type of data, and it could be how long has someone you know, sat in a specific role? Or would you be able to capture the top three skills that someone particularly has, you know, where would that data sit normally for an organization? Or how would we be able to capture it if an organization didn't have it? 

Brian Ray  05:50

Yeah. And I think it's grouped largely by capabilities, I believe. And if I was going to stand back and look at the whole pyramid of what a business is trying to accomplish, that's really important, in human capital to understand what is the business trying to do? I like to look at it through the lens of business use cases to start and a business use case could be something, you know, like, we want to be able to quickly adjust claims in the insurance industry. And that's a business use case - we want to increase the ability to do that. And when I look at that pyramid, I can say, Well, okay, that's a use case that has an ROI around it or return on investment and has a risk around it. And I can measure those things, I compare those to other use cases. Now where capabilities come into play, whether they're using solutions on top of that functionality or not your capabilities as a business is your ability to address that use case and to be able to solve it essentially. 

Brian Ray  06:50

So I have a capability of looking at documents, automated or manually. That's a capability, and then where do these capabilities derive from? They derive from skill. You're capable of doing something because you have the skills to do it. So that whole pyramid of going use case, you know, addressing that to a business problem, and then solutions that you may use to address that problem. And then you know, capabilities and then skills within there, that's an important pyramid of things that most businesses are, are trying to get our arms around the whole stack. 

Brian Ray  07:22

They're trying to get it around the ability to do that, generally speaking, and that's - they sometimes look to partners to do it. Or they sometimes look to build more skills. So that means hiring or building skills with existing staff, how do you learn? How do you learn how to do that in your business, and then deliver it? There's also some overarching skills sometimes that'll drive, you know I always say project management and things like that those are these overarching skills that kind of help encapsulate some of those other capabilities help you move forward across capabilities. So those are important too from an operational standpoint. 

Felicia Shakiba  07:57

And I would imagine that leaders from various divisions would need to converge to shape this taxonomy. Those are the SME's the subject matter experts. So could you share, what kind of input would an organization need from them? And how might this collective dynamic enhance the overall accuracy of what we're trying to measure? Or even predict? 

Brian Ray  08:22

Yeah, I think that, you know, I call it human in the loop to some degree. So if I am an organization, and the capabilities either sit within a certain department - let's say it's a large organization, then those are the folks that are helping solve the problem, that may be different from the department that's receiving that solution. The people receiving that solution, the groups may be a completely different department with a completely different skill set. And the question that always comes into play is where do those two things converge? 

Brian Ray  08:57

How do you make sure that the folks who are able to solve the problem have the right understanding of the problem they're solving, and then also, the people who are receiving the benefit of that solution have the right understanding whether that solution is actually going to solve their problem or not, you know, from a technical perspective. 

Brian Ray  09:18

So when you look at a taxonomy, those words may be different. You know, those words may be different, when you're looking at it through the lens of being the recipient, internal customer was a term that used to use a lot. If I'm the internal customer of that capability, I may use different words to describe what I'm getting versus if I'm a producer of that solution. And if there are different words, but in theory, they're talking about the same thing, how do you marry those. And what we start doing is we start talking about grouping of these abstract concepts, and call them topics. So if you have a topic model, you're talking about grouping of topics that you get away from the specific words, but you look at a pool of words that may describe a solution or capability, or skill. 

Brian Ray  10:05

And you look at how that pool of words used and contacts with each other - they're talking about the same thing. And you know, the the most simplest example I could give that thinking would be if I were to take all the news articles, in the last 10 years, the typical example in natural language processing, I take all of the news articles, and I use unsupervised AI or machine learning to group them, I would notice that news articles about sports, very naturally grouped with other news articles about sports. And same with news articles about the government, or politics, and so on and so forth. And the natural way for those things to group together is exactly how you want to look at topics when you're talking about grouping around capabilities and skills, is that you want to group them by frequency. 

Brian Ray  11:02

And you want to understand there's an area here that's very interesting, in fact we did that for ourselves that Eviden. We said, Okay, we want to double check to make sure that the candidates that are coming in, are being interviewed by the right department. So we started, not deterministically. But just as advice to those looking at it, the humans, giving them some heads up of where that candidate based off correspondence or the resume and everything else, where would that candidate fall? Would they fall into the HR department themselves? Or would they be in the legal department or they'd be in the data science department or engineering? And we've came up with some surprising results occasionally, it's like we were looking at this candidate through this lens, they applied for this area, but they really fit in better in this other area. 

Brian Ray  11:50

Maybe we should ask them why are they looking for a job change? Are they looking for a career change or are they mis-categorized? If you see for instance, that they're interested in AI all over their resume, but they're applying for a job as a business intelligence officer or something like that, let's just say doesn't line up one to one, it's a good idea to go fact check that and go figure out if that person is really looking for a job in AI, or if they're looking for that business intelligence job. And that's a good example of topic modeling, because you're understanding the general topic of the information they provided, and you're modeling around it. But it's fairly abstract, because you're only looking at these combinations of skills and words and things like that. So you're trying to understand based off of that, what a machine would recommend that person -  what would be expected of it. 

Felicia Shakiba  12:46

And you're marrying that suggestion from just looking at the resume or just grabbing the data from the resume? 

Brian Ray  12:55

Right, right, you're taking the terms used in the resume and how they're used, sometimes you contextualize it. You're looking at those key words, and you're seeing how often they fall in the bucket you expect them to fall into, it's just a statistical thing. It could also be by the way, it doesn't have to be just the resume. It could also be the letter, introductory letter, and also other correspondence they've had with the company so far, you know, the words that they've used. And, you know, if you get really wise about it, and you have their permission, if you're recording like even this recording, right now, you could take all the words we've spoken so far, and probably predict that we're talking about CPO PLAYBOOK, and you can predict those things just because of the frequencies of the words that we use. 

Brian Ray  13:44

Now the risk is, is that sometimes that words used in resumes can be used to unfairly categorize someone, and you don't always want that you want to sometimes be cautious of classifying into job roles and things like that. And there's several ways to handle that sort of thing. One thing we'd, like is occupation codes I mentioned before, you know, if you could have that as the middle taxonomy that you, you create your own way of mapping to that you can handle that you're not misusing some information, blindly marrying them with a job requirement, versus being able to handle it as a skill versus a words of that could be found in description of a school that you went to, or something like that, that has nothing to do with their skills. 

Felicia Shakiba  14:32

Interesting. Yeah, because the resume might put you in one direction. And that's not really not authentically, how they perform...

Brian Ray  14:39

Yeah, you're just blindly looking at words and you could have words in the title of some jobs you've held or something like that - have nothing to do with what you want to do or where you want to go. 

Felicia Shakiba  14:47

Right. So we are not currently able to decipher whether someone is a master at a skill or low level at the skill, we're really just looking at the words that are being used. And then essentially, later on, it is still important to leverage the data not to make decisions, but rather to help guide us to make decisions with additional data. 

Brian Ray  15:09

Yeah, generally, there are some exceptions to that. My friend John, I went hiking with yesterday, he's an orthopedic surgeon. And if you look at his CV, there are probably skills that do point towards expertise. There, you know, years, there are certain things in that aspect that do signify that this person is an MD, they have that initials in their name, and so on and so forth. 

Felicia Shakiba  15:32

So levels implied with the validity of what someone might be a master of. 

Brian Ray  15:39

Yeah, yeah. And it could be assumed, like same with you know, any other credentialing title. And that could happen more explicitly. If the resume contains specific certifications, like for instance, you know, I take some cloud certifications, sometimes get those, you can then assume that I have some level of expertise and there's someone more direct one to one mapping with a skill set versus if I'm talking about my paragraph, I write in my resume about what I did on a job. That's not as deterministic as if I have this explicit. I got this certification in this particular technology or something like that. 

Felicia Shakiba  16:16

Got it. And then you touched on the fairness of the data. And so I'm curious to learn, what measures are you putting into place to mitigate the biases or the inaccuracies? 

Brian Ray  16:31

Fantastic question, and we take it very seriously to I mean, the the one way that we've approached this is what you can do with a job requirement and you can do with a resume and those are two different ends of the spectrum, right? That requirement tells you what you're hiring for the resume tells you this is what I've done. And the goal ultimate goal there is that those you marry those who marry those somehow the problem with using clear form tax to do that, that hasn't been curated, that people haven't agreed on that haven't looked at is that it could do it on any arbitrary thing. It could be mistaken. It could be something that isn't what they meant. But if you do the due diligence of marrying those with specific codes, like, ONET occupation codes or skills that you've already curated very carefully, you can generate those taxonomy attributes before the resume. And you can do that for this job [requisition]. 

Brian Ray  17:30

And then you're just marrying skills to skills, you're not marrying text descriptions, descriptions, you're marrying those actual labels that you've come up with about that position, to labels about that candidate. And that's a much more fair way of doing it. 

Brian Ray  17:46

The challenge also is is because if I mentioned computers, 15 times and my resume, and someone else mentions computer only once in their resume, does that mean I'm better than computers? No, it doesn't. So you always kind of have to do your due diligence. Still today, you still have to figure out what does that context really mean. Now? Is AI getting smart enough to do some of that? Yeah, it is. But I don't think that it would be smarter without the humans reviewing that. So I think that as we go along, we always treat machine learning and AI as assistive technology. And that decisive in that sense, so that you get the right answers, and you grow with the machine, not against it or something like that. 

Felicia Shakiba  18:29

And what about accommodating future skill demands? You know, a lot of organizations are planning for the future talent needs an overall workforce planning trajectory. So how might this taxonomy possess the agility to adapt and learn in order to stay ahead of upcoming skill trends. And on that note, I'm just curious, if you could name like, what are the tools that you're using in order to build this as well? 

Brian Ray  18:54

Yeah, yeah, sure. First off is the first concept you're talking about, we call that drift, concept drift, or, you know, skill drift in this case. And what drift is, is essentially where the outside world changes, based off of the process. It changes, despite the process you've already put in place already. 

Brian Ray  19:15

So when you put that process in place, you've validated that those skills are relevant, and everything's working great, you can take some data, you can prove that it's working out, you can just, you know, you can look at it again and say, Okay, this works. This is working today, as I know it. Now, the process didn't break what changed was outside of this process, the world outside changed, right? So now there's these new skills. And it no longer does this perfect marriage anymore because these skills are unknown, - it's unseen data, and a regular review process. Now, if you're doing it automated with like machine learning, and things like that in AI, then there's a retraining process that's very explicit that you go through. 

Brian Ray  19:58

And most practitioners are trained on that. And if it's a manual process, you have to do the same thing, you have to do a review and update process as frequently as needed. How frequently do you know that it's needed, you can do some sampling along the way, and see if drift starts to occur. And you as you can tell, it just starts to curve, because you can measure your, okay, how this how would this work with this new set of data, And then you see it drifting in the wrong direction, eventually, you say, Okay, I'm gonna rebuild this thinking, or this model. And I'm gonna reapply it to the new world. And then you go, and you rinse and repeat. 

Brian Ray  20:34

So it doesn't ever - the review process never goes away. Now, I mentioned before human in the loop, you build that into your process, then some of that does help. It helps when you're constantly reviewing good results, as well as bad results, saying, Well, I see that you identified this person has the skill, but we found that that's untrue. Add that to my database of things. 

Brian Ray  20:56

Tool wise, it really depends on the sophistication of the organization, where if I an analyst, and I just get out of college, and I, I want to I picked up some Python in college, and I want to use some language processing tools off the shelf, I could use LDA and LDAvis, and NLP and NLTK. And I could, I could use some very basic tools to do some general analysis.  But that changes drastically when you're talking about, in my case, I'm 100,000 people organization, and you need really solid, sophisticated tools that can be automated and updated. And that's when you start getting into cloud technology like Google Cloud Platform, and AWS and Microsoft technology are some of the vendors and even open source tools as well. 

Brian Ray  21:44

And then when you get into the Gen AI Space, which is fairly new, which could be useful here, by the way, I can classify resumes better with Gen AI, which was very relevant here and I can also interact with candidates differently that I you know, I'm talking about the large language models that have been released recently, and there's various flavors of those, some are proprietary, most are and then some are open source. It's complicated to keep up with, it's fast moving, but the value proposition is there. So you can interact differently with candidates and also classify documents, more accurately with less data, and things like that. 

Brian Ray  22:18

So there is a value proposition to that tool set, but it comes with a cost to get up to speed on it also, a cost as you run it. The values there- so many businesses, I have to gather around that quite a bit and figure out if it's worth it or not some cases, so they have to pick the right use cases, they also have to look at the risk involved. So there's risk involved with some of these processes, the predictions, right, so any prediction has risks. They could be wrong. But you got to figure out how that's going to impact this type of thing. And the last thing I should mention is regulation could likely change too. Regulations already changed in Europe around the space. And it's changing very quickly here in the US as well. 

Felicia Shakiba  22:59

I could only imagine where we will be a year from now regarding regulations. But I think it's still exciting to take the journey. And so there's there's another thought I have, which is once the taxonomy is built, and it serves as an identification tool to fill current roles or even future roles within the organization, on that line of thinking, would this AI taxonomy, personalized skill development recommendations? And would it align with personal and organizational objectives?

Brian Ray  23:34

Yes, I love this topic. So when we were looking, we did some work for some state departments, and we were looking at helping find jobs. Now in theory, exactly that I can see where there's a gap pretty easily. Now, if I'm really smart, now, I will really want to help the population find jobs, I would do just that I would say, Okay, well, I'm seeing a skill gap here, I see this person applied for this job. Let me create another aspect of this taxonomy that has more to do with recommended trainings on how to get there, how do you build that skill? Should I go back to school? Should I take a course? Should I work on my speaking skills on my own? Or do I just need to elaborate more on my next rendition of my resume? If I found a potential experience that you could elaborate on - and maybe that'll fill that gap too? But absolutely. So there is a really a helpful aspect of understanding, statistically, what's missing there. That's real helpful. 

Brian Ray  24:41

So like, I would like to know that for myself too like, what am I missing in my training and development plan to get to- I just recently take on new rule of evidence that's global and pretty wide, sweeping... Well, please tell me what skills do I need to work on a do my job better? You know, I would do it in a heartbeat. 

Brian Ray  25:01

It's hard to figure out sometimes when if you're not getting any recommendations or assistance on that. And that machine can do it for me, I would probably take it up, I would take up on it, I would say okay, well, if he points me to a specific course, and only takes a few hours to especially, you know, and I can go online and take that for free. Even better yet, you know. So I think you're onto something here, I think it really allows you to to understand what next steps should be taken. In fact, that's one of the models that we develop sometimes as next best actions NBA, not the basketball, but next best actions allows you to help predict what are the next actions a person should take? And there's no reason that can't be someone looking for a job. 

Felicia Shakiba  25:37

That's awesome. I guess that's the future.

Brian Ray  25:41

Yeah, absolutely. Well, it's, it's really, it's really helpful in that sense, too. Because then you're also getting to see more honestly and transparently, potentially your competition in the workspace, right? 

Brian Ray  25:56

What is the person who didn't achieve this job? What did they do differently? People keep asking, Well, why can't I get this job? Why can't I be promoted even if I have a job? Why can't I get a raise? You know, people keep asking these questions. And to have some logical statistical basis that gives them a reasonable answer is extremely helpful to alleviating that frustration, and to do something about it and to go, you know, address it personally. 

Brian Ray  26:22

Or organizationally, if I noticed that there's a lot of folks that are missing the skills needed to build those capabilities to develop that, I could survey that. I could figure out, you know, through a learning and development process, what people are lacking. I could start treating that as an initiative saying that we're doing that right now. We're saying that we want to have 25,000 Eviden employees very knowledgeable about Gen AI by Q2. And how? So how do you do that? Well, you got to know where you're at today and you got to know what's available and you got- it's a bunch of tricky stuff to do that. But I believe a lot of organizations are in the same boat we are. 

Felicia Shakiba  26:58

Brian this was very insightful I'm so glad we have you on the show. Thank you so much for your time. 

Brian Ray  27:05

You bet, thanks for having me. I'm looking forward to hearing it.

Felicia Shakiba  27:09

That's Brian Ray, Global Head of Data Science and Artificial Intelligence at Eviden.

If you liked today’s episode, we have more podcasts on innovative HR strategies, talent management, organizational culture, and more, and how to navigate the complexities of modern-day HR. Find them, at CPO PLAYBOOK dot com slash podcasts or search CPO PLAYBOOK on Apple Podcasts, Spotify, or wherever you listen. Thanks for listening to the CPO PLAYBOOK podcast - we’ll be back with a new episode next time. I’m Felicia Shakiba.

If you love CPO PLAYBOOK, the best thing you can do to support us is to become a subscriber. You can do that at CPO PLAYBOOK dot com slash podcast. That’s CPO PLAYBOOK dot com slash podcast. If there’s an episode you loved, please share it with a friend. And if you have an idea you would like us to talk about or a guest you’d like to nominate, visit CPO PLAYBOOK dot com slash contact us to suggest an idea.

‍