Posts
Tech & Innovation Blog
Meet Roberto! One of the brilliant ADP Machine Learning developers behind our new payroll app, Roll™ by ADP®
Machine Learning, Brazil Labs, Roll by ADP
We had a chance to catch up with Roberto at ADP’s Brazil Innovation Lab in Porto Alegre. He shares his career journey, why he chose ADP, what keeps him excited, and why it’s a great time to work in machine learning.
On February 25, 2021, ADP unveiled an exciting and groundbreaking new payroll app called Roll™ by ADP® to help small businesses run payroll anywhere, anytime, quickly, and compliantly with no experience needed. The app’s artificial intelligence-backed conversational interface allows users to complete payroll on their mobile phones in less than a minute simply by texting, “Run my payroll.” Full press release. Watch a quick video of the app.
We had a chance to catch up with Roberto, a Machine Learning developer and one of the brilliant minds at ADP’s Brazil Innovation Lab in Porto Alegre.
Congratulations to you and your team on the launch of Roll™ by ADP®! We’d love to learn a bit about you. How long you’ve been at ADP, what brought you here, and what do you do here?
Yeah, sure. I’m working with machine learning here and part of Brazil’s Innovation Lab. I’ve been with ADP for three and a half years, so I started in 2017. I worked previously at HP, you know, the printer company, right? In their research lab.
I came here to start building the chatbot—a product complete within itself. A system where we can leverage the intelligence to make life easier for people that are using it. I’ve used chatbots, and sometimes they can be painful. Our job is to take the pain away. During development, we closely followed what our clients saw in production and what they said. When they are happy, that made us happy. We tried to understand our pre-production clients, make sense of what we learned, and iterate improvements before he launched.
So, our team is global and is split between here, the US, and India. We have about 13 people in Porto Alegre, but only four are working on just machine learning. We have around 32 people in Roseland, New Jersey, and about 20 colleagues in India. Our job here is to take care of the chatbot and help customers when they have questions. It’s kind of like using Alexa or Siri. When users ask questions, the AI is doing other things while trying to reply.
We’re also trying to extract insights from what customers are doing. For example, when you hire someone, we get the information behind the scenes, and then we do some tricky calculations to assist. The bot checks on things like gender and pay equity and offers data-driven insights to the client. For instance, in this location, you should offer a higher salary.
Tell us a bit about your career journey.
Sure. It’s a little bit messy. I’m an electrical engineer and worked a little bit in the automotive industry. I started as a hardware engineer working for Johnson Controls on a project for Fiat. Then I moved to a semiconductor company as an engineer and spent some time there. After that, I decided to move into technical marketing.
From there, I decided to get a master’s degree in Technology Management. I’ve lived with my wife in Lausanne – Switzerland, for two years. That was the initial plan. Then I got a job at Texas Instruments in technical sales. We stayed three more years before moving back to Brazil in 2015 and getting a job with HP. That was a big shift. I went from technical sales to software engineer. I had a colleague there that was working on machine learning. I fell in love with it, and I studied more about it. Then I got this opportunity at ADP to work 100% on machine learning. That’s why I came here. We pay 1 in 6 people in the US. There’s a lot of data here and good stuff we can do with it. So, I’ve been here since 2017.
I’m 41, almost 42, now. I have a daughter (Gabriela), she’s one year old. She is definitely my biggest project!
What still excites you about working here?
The team still energizes me. Before the pandemic, I enjoyed working with people globally and meeting the US teams in person when we still could travel to New York. We are trying to build this culture of applying available technologies and bridge the gap between open source and what folks in academia are doing with practical, real-life applications in our product, Roll™ by ADP®. Using this outside perspective, we filter what makes sense into our products to mature our technology.
I think the dynamics and openness of the machine learning domain are really driving the market right now. There’s a lot available in open source, and it’s our job to be up to date on the latest developments. It’s an exciting time to be working in machine learning.
Tell us a little about your project.
We beta tested with clients for almost two years. Last year, we did many internal demos based on our work with a gourmet ice cream company recommended by our Business Anthropologist, Martha Bird. We expanded and started working with our Small Business Services group and built our client list to 70 before we launched in February.
As we scaled for our GA release, we matured the product using input from a small number of clients. ADP’s executive team was happy with the product, and yeah, we hope people like what they see. As I mentioned, I go into production logs every week and see what customers are saying. Sometimes you get some nice comments, which is lovely, right? People talking to the chatbot and just saying, “Thank you!” I love seeing that. Martha measured pre-release net promoter scores (NPS), and they were really good. But we will keep the ball rolling and bring new features to future releases.
If someone asked you why they should choose ADP over other tech companies, what would you say?
I can say one of the things comparing ADP with the other companies where I’ve worked, and maybe it’s just specific to our product or my leader, but something I value a lot is openness. When I worked at other companies, there were a lot of layers. I think people are pretty open here also in terms of technology choices. I know that engineers like to experiment and test to see if stuff works. We try to do that here, experiment with things.
We are shifting from a service company to a more technology-oriented company. Here in Brazil, we are trying not only to apply technology but also to share ideas across the company. 
We created a machine learning discussion group. There are about 12 of us. We discuss papers, review articles, create challenges to learn new skills. We sometimes do presentations, attend or present at conferences. Everything is online today, which makes it easier. We get to exchange ideas and nurture our learnings across teams. We’ve discussed starting to produce some technical articles, and I’m happy that we can use the tech.adp.com blog to share them in 2021. I wish I had more time to write, but I don’t have as much time with my little one.
We also did our first internal developers conference in 2020, and I presented Uncertainty in Deep Learning. It was an amazing experience, great to share, but also to get feedback.
When I mention that we do these things during interviews and other things we are trying to do, candidates like this. I know in some companies people work in silos, but you cannot do that here in Brazil. We share as much as we can here. The openness I mentioned, it’s important.
Above, you mentioned exploring open-source and academia. Are there any projects outside of ADP that excite you right now?
Great question. Yes, there’s an open-source project called Open Mined and a course I’m interested in related to privacy with machine learning. The program is called “The Private AI Series.” Facebook is one of the course sponsors. They have a framework behind the scenes that helps people take care of customer privacy. In case you are interested, here’s the link: https://courses.openmined.org/.
Our team also continues to study and review new technologies. We are following Harvard CS224W online for graph neural networks and Causal Inference (lots of interesting applications will come out of this domain for sure!). For neuro-linguistic programming (NLP), we follow a vibrant startup and open source community called Huggingface. (https://huggingface.co/).
One last question! If you could advise your younger self or someone starting their career, what would you say?
Be inquisitive. Study. Help others.
Thanks for your time, Roberto!
Tech & Innovation Blog
ADP’s Annual Developers Conference Video
Engineering, Innovation, What We Do
Accessible Video Controls
GPT Connect: ADP’s Developer Conference
[MUSIC PLAYING]
[TEXT] GPT Connect. ADP’s Developer Conference. Cool People.
[SPEAKER 1] They built it, they ran it, and they owned it.
[TEXT] Cool people connecting with cool stuff. 3 days. 4 tracks – artificial intelligence and machine learning, product and user experience, clean code and reliability, and product showcases.
[DESCRIPTION] Slides filled with different speakers and information and graphs flash across the screen.
[TEXT] 5 keynotes.
[DESCRIPTION] Photos of 5 speakers.
[TEXT] Ian Swanson, Worldwide Leader, Artificial Intelligence and Machine Learning, Amazon Web Services (AWS).
[IAN] AI and machine learning have become more accessible to all businesses — champion an ML culture throughout all roles at ADP.
[TEXT] Teresa Torres, Product Discovery Coach for Product Talk.
[TERESA] Bringing the product manager, the design lead, and the tech lead together to be jointly responsible for making discovery decisions.
[TEXT] Andy Hunt, Author of the Pragmatic Programmer, Agile Manifesto Co-Author and Musician.
[ANDY] Learning and communicating, that’s exactly what we do all day. We communicate to other people and we learn new things. A team is a complex adaptive system of exploration and dialogue.
[TEXT] 114+ speakers, 20,000+ sessions views, unlimited connections…
[ANDY] We have amazing teams doing amazing work all around the world.
[SPEAKER 5] End to end, we have an opportunity to directly touch the lives of 2.9 billion people.
[SPEAKER 6] We’ve got to live in this world right now, right? We will actually be able to get much more done because of our converged system.
[SPEAKER 7] GPT is the backbone of the company. ADP’s data is so massive in terms of representative of the overall economy.
[SPEAKER 8] And how we can connect our innovations and our teams to shape that.
[TEXT] Innovating. Connecting. Developing. Designing. What’s Next? GPT Connect. Ready to design what’s next? Visit tech.adp.com/careers.
[LOGO] ADP, Always Designing for People.
[TEXT] ADP and the ADP logo are registered trademarks of ADP, Inc. All other marks are the property of their respective owners. Copyright 2020 © ADP Inc.
[MUSIC PLAYING]
More than anything, over the past year, we’ve learned the power of connections! To keep our global teams connected, a group of Global Product & Development (GPT) associates raised their hands to lead ADP’s global, three-day, all-virtual developers conference. Check out our sizzle reel!
Tech & Innovation Blog
Podcast: Fireside chat with Don Weinstein, CVP, Global Product & Development on his career and the latest tech happenings at ADP
Career Path, Fireside Chat, Latest Tech
ADP Technologist, Nik Palmer, hosts an engaging chat with our Global Product & Techology leader, Don Weinstein, and the executive sponsor for our Generations Business Resource Group. Don chats about his career path and gives some advice. He also shares the latest tech happenings at ADP, machine learning, and our evolving products. The Generations BRG focuses on creating connections between emerging and established professionals.
PODCAST TRANSCRIPT
[LOGO] ADP, Always Designing for People
[NICK PALMER] Howdy and welcome to the GENcast podcast. My name is Nick Palmer and I will be your host for this episode with Don Weinstein, Chief Product and Technology Officer here. Don is also the Executive Sponsor of Generations.
His role is somewhat unique as most companies separate product and technology. And what’s also interesting is that Don has a background in aerospace engineering and consulting prior to coming to ADP to work in corporate strategy and technology. Don, welcome, and thanks for sitting down with us.
[DON WEINSTEIN] Thanks, Nick. It’s great to be here.
[NICK PALMER] So to get started, let’s find out a little bit about you and tell us a little bit about what generation you identify with most closely and what are your standout roles.
[DON WEINSTEIN] Great question. I’m Generation X through and through. I do like from time to time looking at those different examples about the– or the differences in the different generations. And the one that always stuck out to me with Generation X being kind of a sandwich generation.
Stuck in between the boomers and the millennials is just kind of being the pragmatists in the room and figuring out how we help move things forward and make progress and maybe less about it was stuck between the me generation and the we generation before and after us.
[NICK PALMER] Building bridges versus building walls.
[DON WEINSTEIN] I like that. I hadn’t heard that, but I’ll quote you on that going forward. It’s really about just again, not trying that overly call attention to ourselves as a generation as much as just moving things forward. I was going to say, my top two stand out roles are advisor provider, which I was told was a somewhat unusual combination, but that’s all I know.
[NICK PALMER] Right on. Yeah, I don’t think I’ve met one of those yet. So that’s interesting. So tell us a little bit about some technology happenings here at ADP. If our listeners haven’t had a chance to listen to your recent HRExaminer executive podcast with John Sumser, we’ll make sure that we put a link up there and encourage them to take a listen.
In that podcast, you cover some excellent material there about how ADP is addressing diversity and inclusion via our products and our client services, most of which is just given away for free. Can you share a little bit about the GTP diversity and inclusion strategy and its intended impact for both clients and associates?
[DON WEINSTEIN] Yeah, I’d be glad to. And I think the first thing to peel back on is this isn’t some new idea that just came to us because of obviously what’s been happening, the increased focus we’ve seen based in large part on the recent tragic incidents with George Floyd and Breonna Taylor and Ahmaud Arbery, which I think have created a renewed focus on the problems and the challenges that diversity has and that we have with diversity as a country and in particular in the workplace.
So I am glad for that renewed focus. But this is something we’ve been thinking about for and working on for a while. I’d point you back to, it was 2016 when we launched our Pay Equity Explorer and actually was recognized as an awesome new technology for 2016 by HR Executive Magazine the HR Tech Conference.
And in particular, what made the Pay Equity Explorer great or an awesome new technology is it took this long founded challenge that we know about, disparities and discrimination and pay practices in the workplace, and took a new angle to it where we could leverage ADP’s vast treasure trove of data so we could understand industry benchmarks, both outside an organization, as well as inside an organization, and in addition to which we can apply machine learning algorithms to help identify not just– you see the classic studies that say, OK, in this context, women get paid 82 cents on the dollar.
It’s a constantly moving benchmark so I don’t remember what the latest is. But that was a relatively recent one. And it’s important. And it’s important to know. But it’s not actionable. OK. So what do I do about it? More importantly, where do I get started?
And so the Pay Equity Explorer was able to break that down using machine learning to crawl through a client’s data and identify very specific, this individual in this job relative to this internal and external market benchmark is being underpaid by this exact amount and here’s how you can take care of that.
And I think that was a real breakthrough in terms of attacking that problem. And we looked at that across all dimensions. We looked at it by gender, we looked at it by race, we looked at it by ethnicity, because trying to get multi-dimensional on the problem.
And that was really just the first such example. We followed that up with a couple of more products, one of which we focused on including a little about diversity and inclusion and inclusiveness. We’ve been very focused on accessibility in our products.
And I’m talking about making content more accessible in particular for folks who may be visually impaired or other types of impairments that would prevent them from being able to access their normal workplace tools.
So our core application, My ADP, that’s used by employees and managers, as well as our ADP mobile application, we’ve been embedded enhanced usability and accessibility controls. We’ve been doing that for years. I’m actually trying to remember when we started. It’s been so long that we’ve been at that I couldn’t I couldn’t tell you off the top of my head, but it’s been a while.
[NICK PALMER] I know I’ve supported AudioEye for a while now.
[DON WEINSTEIN] Exactly. AudioEye as a partner, we’ve worked with on both to consult with us on how to design our products to make them more accessible, and then they have plug-ins that handle all different types of impairments and get us beyond level two on the web content accessibility guidelines.
And those are just a couple of examples. I could drone on forever. You probably don’t want me to. We’re continuing to push the ball forward. We had a couple of things in the works. We’re working on some new diversity inclusiveness dashboards with our data cloud team to give clients better analytics about what they’re doing, multiple different metrics around– you’ll get the entire talent lifecycle.
Are we recruiting diverse talent pools? How are we rewarding? How are we retaining? How are we promoting? And taking all of that through a D and I lens. We’ve kind of already started working on that next iteration when obviously the external environment kicked it up a notch in terms of our priority level.
And you had asked the question not only from an external angle but also from an internal angle. So I’m super excited that we’ve been able to partner with our internal ADP D and I team, in particular Aisha who is our Chief Diversity Officer, to have ADP be client number one for any and all new products that we want to push in this area.
And actually have been having great success with Sreeni Kutam about making ADP client number one for any of our enterprise HR products. And diversity inclusiveness is just the latest example that we’re putting a lot of energy into right now.
[NICK PALMER] As we move from a service company to a technology company, it’s interesting to see that dynamic come into play and adoption of that agile startup mentality of using what we build internally. That’s awesome. So, tell us a little bit about your path towards executive leadership and how that shaped your approach towards management of people and product.
[DON WEINSTEIN] Yeah, I’ve had somewhat of a circuitous path. You pointed out in the intro that I was an aerospace engineer by training. I started my career at General Electric. And I was working in the satellite telecommunications side of the house.
So I worked on such projects as the Dish Network and GPS. So if you’re using Waze on your way home at some point or the next time you’re in the car, you’re welcome. And truthfully, a lot of people worked on that project. You can imagine I was just one of many really brilliant and talented folks who did that.
But it was fun. It was an exciting place to start a career. But how I got from there to the payroll and HCM industry, it was not a linear trajectory whatsoever. I was bought, sold, merged, acquired six different times in my first several years. GE at some point decided to get out of that business.
They sold it to Martin Marietta. Martin Marietta merged with the Lockheed Company, became Lockheed Martin. Lockheed Martin shut down the facility I was working at. There were 5,000 people there when I started. And I was the 10th to last to walk out the door and shut the place down.
And now it’s just an industrial brownfield site that’s been sitting there unoccupied for 20 plus years, which is kind of sad. What I learned in some of that– first thing, so, coming out of that, I didn’t know what to do with myself.
So what do you do when you’re a 20-something engineer and you don’t know what to do with the rest of your life? I went back to school and got an MBA. And then when I finished up my MBA, I still don’t know what I wanted to do with myself?
So what do you do when you’re an overeducated 20-something who doesn’t know what to do? You go into consulting and tell other people what to do. I don’t know what to do with my life, but let me tell you what to do with yours.
Consulting was a good experience for me in that up until that point, I had only known one industry, one business, and one functional area. So it really helped me broaden myself out. And I think if there’s one or two takeaways I could take away from that experience, it’s I had the opportunity to work in a lot of different industries on a lot of different problems, and having that cross-functional type of experience I think was useful.
Even starting my career at GE, they really encourage some rotational type assignments. I have noticed within ADP, we probably don’t do that as much as other organizations. Also I made a stop at IBM for several years. And we used to joke at IBM, it stood for I’ve Been Moved.
But at ADP, I think we have folks who tend to spend more time in one area. And it’s great for developing deep expertise in that area. And some of the stuff that we do, you have to be pretty deep and expert to do it.
But I think also we can benefit and folks can benefit from taking that kind of side step rotational assignment to learn about the business through another lens and then come back. It’s something I’ve been encouraging within my technology organization now. Really, two things.
One is getting more rotations within technology, including on my leadership team. So the simple way we think about it is I’ve got the applications side of the house, as you said, the product side. And then the traditional infrastructure, the technology side.
So we’ve been rotating people within products, moving from one product, maybe moving from a shared product like our identity management or reporting solution into a market facing product like RUN or Workforce Now or something like that.
Gives people a different perspective. Moving between the application side and the infrastructure side of the house. So we’ve done a handful of rotations there. And I think we’ll do some more. And also encouraging bi-directional folks from the technology organization to roll out into business type roles and vise versa.
Folks who are in the different sides of the business who have interests or an aptitude for technology to come do a tour of duty in the technology world and then go back to their functional areas with a greater appreciation for what we’re doing here.
It’s something, like I said, I was able to benefit and take advantage of early on in my career. And I think that’s something that I’m trying to encourage others at least within my scope of influence in GPT to take advantage of as well. Does that make sense?
[NICK PALMER] Yes, it does. That’s great. I think that I’m going to quote you on the merged, bought, sold, and acquired. Yeah. One of the things that you said in there that was particularly interesting was about that tour of duty in rotation.
And that’s something that I’d like to make sure that we’ve heard is covered again and mentioned again because that’s something that we’ve heard from our membership, saying how should I approach the career path and moving forward?
Should I jump around and try lots of different things? Or should I stick and go deep expertise? And I’ve obviously stuck and done the deep expertise side of things. But I tell people all the time, jumping around and gaining a broad perspective, that’s also equally valid.
They are opposite sides of the same coin. And as long as you’re holding that coin and paying attention to it, either one can afford you great insight and knowledge and career path opportunity. So thanks for sharing the opposite side of the coin from what I typically do.
[DON WEINSTEIN] Yeah, and if I could just put a nuance on that, I like to think about it as a major and a minor. Because I wouldn’t want to create the false expectation. I think too much jumping around, also a bad thing, right? I think you need to stay long enough in an area to develop a certain level of depth and expertise.
And that’s it. That’s your major. And then when you have that depth of expertise in a certain area, then when you go out into another area, you may be a novice in that new found area, but you can be valuable to the folks there because you bring the expertise about something else that they don’t know about.
So for instance, do a typical one again in my area, is if I’ve got somebody who’s super knowledgeable about infrastructure and then they go do a rotational assignment on the app side of the house, they may not know everything there is to know about app development, but they can bring to that team their richness and depth of experience on infrastructure that is going to be beneficial to the team.
And then potentially, like I said, not then move on to the next thing and the next thing and the next thing. But do that as sort of like a side step and then come back to infrastructure and be a better infrastructure engineer because you understand more now how the application that your infrastructure is supporting.
So a little different than saying jump around as much as you do a sidestep, come back. Maybe apply what you learned. Maybe do another one, come back. So I think it’s useful for somebody to have a major. I’ll tell you, my life experience, as I said, I started out as an engineer. Spent several years in engineering.
Coming out of engineering, I went into consulting and I got to apply that in a bunch of different areas. Then I did some work in strategy. Came back to product. Then I came to ADP. Well, did a long tenure in project management.
Went out to strategy. So I was a Chief Strategy Officer for a couple of years. Came back to GPT. So not floating all over the place, but sidestep, come back, sidestep, come back, learn a new skill, come back and apply it kind of model, if that makes sense.
[NICK PALMER] Right, and I think how I would encapsulate that nuance that you’re talking about is you have to do the tour of duty. You have to learn something, not be a tourist and jump around. So there is a difference in terms of your depth and understanding that you gain in a tour of duty versus just being a tourist to a new location.
[DON WEINSTEIN] Very well said, yes.
[NICK PALMER] So let’s get into the big elephant in the room. One of the biggest hurdles we had in late FY 20 was the rapid change to remote work modality. The business continuity team recently discussed this undertaking in their internal webinar.
And you talk about the international effort in depth on the HRExaminer podcast. The company saw some wonderful productivity engagement through the process. And as associates continue to work remotely and some areas slowly begin the process of returning to the office, what kinds of productivity and technology challenges do you see arising?
[DON WEINSTEIN] Yeah, it’s a good question. And something we’re thinking a lot about these days, of course. The first thing I observed is as we moved everybody remote, there was this certainly surge of connectivity going on that causes a lot of technical challenges. And now we’re keen thinking already ahead to like, what the year end is going to look like and is that going to be another surge.
But the productivity question is I think the one that’s even more interesting because you see different views out there and different scenarios about, well, are we more productive in a remote setting or are we less productive? That’s a question that comes up from time to time. And I’ll share just a couple of dynamics.
And these are anecdotes that I’m observing. I don’t claim to have the full settled science on is work from home more or less productive yet. But the one thing I noticed is in the past where we had most folks working in an office and then a minority working from home, I thought it was hard to be in that subset minority of folks who weren’t in the office, versus when everybody is remote, it seems like, if you’re having a meeting and there’s 12 people in the meeting and nine of them are in the room and then three are on the phone or the video, you’d see that the folks who were on the phone or the video, every once in a while somebody was like, oh, stop, wait.
Folks on the phone, do you have anything to add? Because it almost becomes an afterthought. Whereas when everybody’s remote, I actually think it makes that aspect of it work a little bit better. But now that we’re talking about reopening, and by the way, just for full disclosure, I’m in sitting in Roseland right now because part of the very first wave of US pilots, Roseland was the first office and I was in the first cohort of folks who raised my hand and said, sure, I’ll come back into the office.
So I’m in Roseland right now. But some of our offices around outside the US have already started to open up. Shanghai was the first and a handful in Europe as well. But I think it’s a challenge when you’re in this partial hybrid state, right? So if everybody’s remote, I think the collaboration works better.
And in particular getting more voices heard. But when some folks are remote and some folks are in the office and you have this partial stasus going, I’m not sure that that works as well. And that’s something that we have to be very mindful of as we hopefully at some point start to transition back.
And the second thing I’d point out is where we have mature teams of people, I’ve got a mature team of folks, they’ve been working together for a while, they know each other, they’ve been working on their products for a while, they know the products. I think those teams going remote were capable of doing so.
And it was almost a seamless transition. Now we’re at the point, it’s July, and this is the time of year when we have our new college grads start showing up. So I’ve got close to 100, I’ve got 150 total who will be showing up here over the next several months. And so the ability to onboard and assimilate those new hires into the company– so first they have to learn about the company, about the products, about the other team members.
For most of them, it’s going to be their first real work experience. And the ability to manage through that in a remote setting I think is a little bit trickier. And it’s one that I don’t know that we’ve cracked the code on, but we’re mindful of now that we’re going through a virtual onboard process.
So I think those are some real challenges that we need to be really taking seriously here the longer that this stretches out. I don’t know what that would be like to onboard as a brand new, fresh out of college graduate, my first real job, my first company, and I’m remote for six months. Nobody knows. I mean, I saw Google came out and announced they’re going to be remote at least till next summer. Not saying that they’re right, but it just shows you where things are moving.
[NICK PALMER] Yeah, the vision is definitely down the road as opposed to next week or next month.
[DON WEINSTEIN] That’s right. Definitely doesn’t feel like it’s around the corner.
[NICK PALMER] And I think the key thing, though, that you bring up is that we’re mindful of it. We’re aware that this is a potential issue. And unless we are aware of it, we can’t really do anything about it. So it’s good to hear that we’re paying attention and monitoring, even if we don’t have all the answers right now. We’re closely watching and observing to see what those answers might become and reveal.
[DON WEINSTEIN] Exactly.
[NICK PALMER] So in terms of that, how do you see the business resource groups and Generations playing a role in this remote and hybrid environment?
[DON WEINSTEIN] It’s a great question. It’s an important one. And I’ll go back to the beginning of my career and tie that into why I always felt so passionate about Generations and the work you all are doing here. And that is that I felt like I did have a good onboarding experience when I first started my career back at GE.
Again, I came in as a brand new, fresh faced engineer right out of college and got to work on some pretty complex stuff. But I was lucky to really have– and this gets the spirit of Generations. We brought people in together as a class. So we had a cohort of people I came with, that I bonded with, and some of whom are still my closest friends to this day. So I had a lot of peer interaction and learning through it together.
And then we had senior level mentoring. And I remember a couple of mentors in particular who really took me under their wing and showed me the ropes a little bit and helped me really accelerate my career by being able to learn from the wisdom of some of the folks who had been there together and enjoy the camaraderie aspect of being part of a group of people and not feeling like I was at it alone.
It’s one of the things we’ve tried to bring to, as I mentioned, and we called our GPT Development Program when we onboard the new college graduates. We bring them in as a class and try and create that spirit. And we assign them mentors from within the broader organization. And as I look at where we are as an organization right now in terms of moving folks out of the office remote, I think it’s been about 20 or so weeks here in the US since we went to remote work.
And again, I know there was this initial surge of productivity because people were certainly, we had a lot of work to do as we were responding to various aspects of the COVID crisis from a regulatory or compliance aspect. And there was just the change element there, I’d say, almost like a mini little euphoria.
I’m speaking for myself now. I’ve been coming into an office for 30 years and I never really had an opportunity to work from home for any kind of extended period. So it was just different. You get a little bit of an, I’ll call it an adrenaline rush.
But I’ve seen and I’ve also read some interesting articles by others about how that phenomenon played out in a number of different organizations. But after that initial adrenaline rush, now we’re settling in for the long haul. And how do we keep folks engaged?
How do we avoid burnout in a world where if you’re like me, the whole work life balance, whatever it was went away because I wake up and get my first cup of coffee and, boom, I’m right at it. And then I just keep going all day, all night. That’s not a recommendation or a lifestyle tip, but it makes it harder.
And so there is some notion that the longer this drags on, the current scenario, you’re worried about either, you’re worried about burnout, you’re worried about fatigue factors, you’re worried about people losing engagement, losing connectivity to their team or to their organization.
And so I think the work that business resource groups do is probably more critical now in this type of environment than ever before in terms of creating a sense of affinity and shared purpose with people who have common interests and common objectives that span organization or function or job, but rather say, we have connectivity at a purpose level here.
And so I’m thankful that our ADP, that our HR organization had the foresight to start us down this path several years ago. And we’ve got this kind of infrastructure in place now today. Because I believe it’s critical to helping folks navigate the current crisis from a personal work life relationship aspect.
[NICK PALMER] Right on. Thank you very much, Don. We appreciate your time here today. And I wanted to thank you personally for your guidance and suggestions for the Generations Group. I know that we are looking forward to an amazing FY 21.
One of the things when we connect and we talk about is books and self-improvement that we’re undertaking currently. So would you like to leave the Generations members any suggestions on what you’ve been reading or studying lately?
[DON WEINSTEIN] Yeah, I’d be happy to. I’ll just share with you what I’ve been reading. I don’t know how helpful it is. But I’ve become quite a big fan of this– is an academic, a historian at Oxford, Yuval Noah Harari.
He’s written a few books. Sapiens was a history of humankind. Homo Deus was a kind of a forward looking one. And the current book that he’s got out there is called 21 Lessons for the 21st Century. And it couldn’t, in my opinion, couldn’t be more timely.
It was sort of a little dark, if I would offer, and maybe it’s in keeping with the times. It’s funny, he published it before the pandemic or some of the unrest that we’re seeing happen. But seems to have a little bit of– seemed a little prescient in that.
But I can skip you through some of it because there are like 20 chapters like, yep, that’s where we’re at. That’s not working, that’s not working, that’s not working. And I get to the end, I’m like what’s the answer? So what do we do? And his only answer is meditate.
[NICK PALMER] I’m a big fan of those books. I would recommend them to all of our Generations members. I know we’ve shared a couple of those in previous quarterly book roundups. So thanks for the plug on those.
[DON WEINSTEIN] Absolutely. This is the generation that’s going to carry us forward. So meditating and being a little contemplate, not the worst idea right now.
[NICK PALMER] Thank you very much, Don. We appreciate your time today. And we will look forward to having you back sometime in the future.
[DON WEINSTEIN] Thank you. My pleasure.
[MUSIC PLAYING] [LOGO] ADP, Always Designing for People.
[TEXT] ADP and the ADP logo are registered trademarks of ADP, Inc. All other marks are the property of their respective owners. Copyright © 2020 ADP, Inc.
AI and Data Ethics: 5 Principles to Consider
As organizations develop their own internal ethical practices and countries continue to develop legal requirements, we are at the beginning of determining standards for ethical use of data and artificial intelligence (AI).
In the past 20 years, our ability to collect, store, and process data has dramatically increased. There are exciting new tools that can help us automate processes, learn things we couldn’t see before, recognize patterns, and predict what is likely to happen. Since our capacity to do new things has developed quickly, the focus in tech has been primarily on what we can do. Today, organizations are starting to ask what’s the right thing to do.
This is partly a global legal question as countries implement new requirements for the use and protection of data, especially information directly or indirectly connected to individuals. It’s also an ethical question as we address concerns about bias and discrimination, and explore concerns about privacy and a person’s rights to understand how data about them is being used.
What is AI and Data Ethics?
Ethical use of data and algorithms means working to do the right thing in the design, functionality, and use of data in Artificial Intelligence (AI).
It’s evaluating how data is used and what it’s used for, considering who does and should have access, and anticipating how data could be misused. It means thinking through what data should and should not be connected with other data and how to securely store, move, and use it. Ethical use considerations include privacy, bias, access, personally identifiable information, encryption, legal requirements and restrictions, and what might go wrong.
Data Ethics also means asking hard questions about the possible risks and consequences to people whom the data is about and the organizations who use that data. These considerations include how to be more transparent about what data organizations have and what they do with it. It also means being able to explain how the technology works, so people can make informed choices on how data about them is used and shared.
Why is Ethics Important in HR Technology?
Technology is evolving fast. We can create algorithms that connect and compare information, see patterns and correlations, and offer predictions. Tools based on data and AI are changing organizations, the way we work, and what we work on. But we also need to be careful about arriving at incorrect conclusions from data, amplifying bias, or relying on AI opinions or predictions without thoroughly understanding what they are based on.
We want to think through what data goes into workplace decisions, how AI and technology affect those decisions, and then come up with fair principles for how we use data and AI.
What Are Data Ethics Principles?
Ethics is about acknowledging competing interests and considering what is fair. Ethics asks questions like: What matters? What is required? What is just? What could possibly go wrong? Should we do this?
In trying to answer these questions, there are some common principles for using data and AI ethically.
- Transparency – This includes disclosing what data is being collected, what decisions are made with the assistance of AI, and whether a user is dealing with bots or humans. It also means being able to explain how algorithms work and what their outputs are based on. That way, we can evaluate the information they give us against the problems we’re trying to solve. Transparency also includes how we let people know what data an organization has about them and how it is used. Sometimes, this includes giving people an opportunity to have information corrected or deleted.
- Fairness – AI doesn’t just offer information. Sometimes it offers opinions. This means we have to think through how these tools and the information they give us are used. Since data comes from and concerns humans, it’s essential to look for biases in what data is collected, what rules are applied, and what questions are asked of the data. For example, if you want to increase diversity in hiring, you don’t want to only rely on tools that tell you who has been successful in your organization in the past. This information alone would likely give you more of the same rather than more diversity. While there is no way to completely eliminate bias in tools created by and about people, we need to understand how the tools are biased so we can reduce and manage the bias and correct for it in our decision making.
- Accuracy – The data used in AI should be up to date and accurate. And there needs to be ways to correct it. Data should also be handled, cleaned, sorted, connected, and shared with care to retain its accuracy. Sometimes taking data out of context can make it appear misleading or untrue. So accuracy depends partly on whether the data is true, and partly on whether it makes sense and is useful based on what we are trying to do or learn.
- Privacy – Some cultures believe that privacy is part of fundamental human rights and dignity. An increasing number of privacy laws around the globe recognize privacy rights in our names and likeness, financial and medical records, personal relationships, homes, and property. We are still working out how to balance privacy and the need to use so much personal data. Law makers have been more comfortable allowing broader uses of anonymized data than data where you know, or can easily discover, who it’s about. But as more data is collected and connected, questions arise about how to maintain that anonymity. Other privacy issues include security of the information and what people should know about who has data about them and how its used.
- Accountability – This is not just compliance with global laws and regulations. Accountability is also about the accuracy and integrity of data sources, understanding and evaluating risks and potential consequences of developing and using data and AI, and implementing processes to make sure that new tools and technologies are created ethically.
As organizations develop their own internal ethical practices and countries continue to develop legal requirements, we are at the beginning of determining standards for ethical use of data and AI.
ADP is already working on its AI and data ethics, through establishing an AI and Data Ethics Board and developing ethical principles that are customized to ADP’s data, products and services. Next in our series on AI and Ethics, we will be talking to each of ADP’s AI and Data Ethics Board members about ADP’s guiding ethical principles and how ADP applies those principles to its design, processes, and products.
Read our position paper, “ADP: Ethics in Artificial Intelligence,” found in the first blade underneath the intro on the Privacy at ADP page.
How to select, gather, clean, and test data for machine learning systems
https://explore.adp.com/spark3/how-data-becomes-insight-the-right-data-matters-454FC-31577B.html
How Data Becomes Insight:
The Right Data Matters
By SPARK Team
What goes into selecting, gathering, cleaning and testing data for machine-learning systems?
It’s not enough to have a lot of data and some good ideas. The quality, quantity and nature of the data is the foundation for using it effectively.
We asked members of the ADP® DataCloud Team to help us understand what goes into selecting, gathering, cleaning and testing data for machine-learning systems.
Q: How do you go from lots of information to usable data in a machine-learning system?
DataCloud Team: The first thing to figure out is whether you have the information you want to answer the questions or solve the problem you’re working on. So, we look at what data we have and figure out what we can do with it. Sometimes, we know right away we need some other data to fill in gaps or provide more context. Other times, we realize that some other data would be useful as we build and test the system. One of the exciting things about machine learning is that it often gives us better questions, which sometimes need new data that we hadn’t thought about when we started.
Once you know what data you want to start with, then you want it “clean and normalized.” This just means that the data is all in a consistent format so it can be combined with other data and analyzed. It’s the process where we make sure we have the right data, get rid of irrelevant or corrupt data, that the data is accurate and that we can use it with all our other data when the information is coming from multiple sources.
A great example is job titles. Every company uses different titles. A “director” could be an entry-level position, a senior executive, or something in between. So, we could not compare jobs based on job titles. We had to figure out what each job actually was and where it fit in a standard hierarchy before we could use the data in our system.
Q: This sounds difficult.
DataCloud Team: There’s a joke that data scientists spend 80 percent of their time cleaning data and the other 20 percent complaining about it.
At ADP, we are fortunate that much of the data we work with is collected in an organized and usable way through our payroll and HR systems, which makes part of the process easier. Every time we change one of our products or build new ones, data compatibility is an important consideration. This allows us to work on the more complex issues, like coming up with a workable taxonomy for jobs with different titles.
But getting the data right is foundational to everything that happens, so it’s effort well spent.
Q: If you are working with HR and payroll data, doesn’t it have a lot of personal information about people? How do you handle privacy and confidentiality issues?
DataCloud Team: We are extremely sensitive to people’s privacy and go to great lengths to protect both the security of the data we have as well as people’s personal information.
With machine learning we are looking for patterns, connections or matches and correlations. So, we don’t need personally identifying data about individuals. We anonymize the information and label and organize it by categories such as job, level in hierarchy, location, industry, size of organization, and tenure. This is sometimes called “chunking.” For example, instead of keeping track of exact salaries, we combine them into salary ranges. This both makes the information easier to sort and protects people’s privacy.
With benchmarking analytics, if any data set is too small to make anonymous ― meaning it would be too easy to figure out who it was ― then we don’t include that data in the benchmark analysis.
Q: Once you have your initial data set, how do you know when you need or want more?
DataCloud Team: The essence of machine learning is more data.
We want to be able to see what is happening over time, what is changing, and be able to adjust our systems based on this fresh flow of data. As people use the programs, we are also able to validate or correct information. For example with our jobs information, users tell us how the positions in their organization fit into our categories. This makes the program useful to them, and makes the overall database more accurate.
As people use machine-learning systems, they create new data which the system learns from and adjusts to. It allows us to detect changes, see cycles over time, and come up with new questions and applications. Sometimes we decide we need to add a new category of information or ask the system to process the information a different way.
These are the things that both keep us up at night and make it exciting to show up at work every day.
Learn more by getting our guide, “Proving the Power of People Data.”
Tech & Innovation Blog
Why I think ADP is one of the best places for ML engineers and Data Scientists
Why ADP, ML and Data Science, Careers
When first asked to write an article for ADP’s tech blog, I had flashbacks to working on my dissertation, and it was, to put it delicately, one of my worst nightmares.
I mean, don’t get me wrong, I am proud of my work and forever thankful to my advisors for pushing me, but writing is not one of my natural abilities. Nevertheless, the request came at a rather interesting time for me, so I said yes. But let me take a step back.
“One day, the AIs are going to look back on us the same way we look at fossil skeletons on the plains of Africa. An upright ape living in dust with crude language and tools, all set for extinction.”
From one of my all-time favorite movies, that quote has been stuck in my head for more than half a decade. The first time I heard it, the quote resonated with the young geek in me and triggered my curiosity and desire to understand Artificial Intelligence (AI). That, in turn, pushed me to pursue a master’s degree and kickstart a career as an ML engineer. My years of research taught me that we are far from AI overlords, but the quote changed the lens with which I view the world.
So, I mentioned above that the request to write this article came at an interesting time for me. Why? I’m currently building a language model that can write meaningful phrases and sentences—as if written by a human being (where was this when I was writing my dissertation?!)
Natural Language Generation captured my interest at ADP when I discovered all the time and effort our client service associates put into crafting documents for our clients. I asked myself, “If we’re building machines to converse with us, why can’t we have them write for us, too?” Not only would that yield consistency in the quality and tone of our client responses, but for people like me, it may reduce an associate’s angst over a potentially time-consuming task and improve job satisfaction. That sounded like a win-win.
As I worked on the model, a friend joked that I was probably wasting my time on a project that my organization may never adopt. I disagreed. I’m blessed to work for wonderful, supportive leaders. Since I started at ADP, both my director and vice president have always encouraged me to challenge the status quo. Did I always succeed? Nope, but they created a safe space where I could take risks. Sometimes I fail, and that’s OK. It’s worth it to try.
I started working for ADP’s Retirement Services organization almost two years ago, thanks to a fantastic director who believed in me and gave me an opportunity despite my minimal experience. It was at a time when ADP ambitiously sought to build AI-centric products to make our client experience better. As a budding ML engineer, this was my happy place.
Although ADP has been around for over seven decades, a few years ago, we refocused on incorporating AI into our core strategy. This shift presented engineers with Machine Learning and Data Science backgrounds a unique opportunity. Sadly, for my peers at other companies, things they tell me they often face are a lack of opportunity, lack of problems to solve, and a limited scope due to the maturity of their company systems. You won’t find those things here.
We are still in an evolving space and actively innovating, which creates a ton of opportunity. I may be biased, but I think ADP is one of the best places for ML engineers and data scientists that love to innovate to grow their careers. Why? Besides a strong support system from senior leadership, we have a corporate focus to infuse AI into our products along with an unending stream of potential products and solutions to create.
Some parts of our company are still in the nascent stages of leveraging machine learning to improve our products. You may not find a lot of opportunities to build products from the ground up (although we are working on several!) inside a Fortune 500 company like ADP, but many also don’t have what we uniquely offer. ADP pays over 20% of the working population in the United States, giving ML engineers and data scientists a rare chance to work with some of the industry’s biggest datasets.
As an ADP ML engineer, I get the best of all worlds. I get to research and implement solutions for relevant problems and issues that impact the working world. For example, my team is currently tackling one of the biggest financial challenges in the country: retirement preparedness. We’re using comprehensive datasets from different organizations to enable us to teach people better financial planning habits and demonstrate the impact of those lessons on their financial future. I love to say we are, “Helping America Retire Better.” Every extra year of planned retirement that we deliver to people makes me happy. Impacting people’s lives through my work is what motivates me to come to work every day.
But it’s not all rainbows and unicorns. This article wouldn’t be complete and would be slightly disingenuous if I didn’t talk about the challenges. Let’s be realistic. Everyone faces challenges at work.
One problem I see is that people love the hyper buzzwords: AI, Machine Learning, Deep Learning, Data Science, oh my! But often, people don’t always see the value in the ideation phase. One of the great things about ADP is our culture of encouraging innovation that helps engineers move forward. Yes, maybe there were times people were wary of an idea, but no one ever discouraged me from working on a proof of concept.
Another challenge has to do with our scale, which is sometimes a blessing and a curse for ADP. Because of it, we need to work with teams across the organization and deal with conflicting opinions and priorities. Leaning into our core value of working as “One ADP,” many times, this helps us to resolve these issues, but it might take a few less-than-fun meetings or calls. These challenges can sometimes be annoying, and they take resilience to navigate through, but thanks to my amazing team and leadership support, I’ve never felt helpless or demotivated.
So, what do you say? Does this sound like a place for you? I’ll end by simply saying: give us a try. Apply and interview. I promise, once you meet us, you’ll understand why people stick around for a long time. I mean a really long time. Some of the smart and awesome engineers I work with had the pleasure of seeing the original Star Wars…in the movie theatre (no, I mean the first time!). Our multigenerational workforce is one of the things that makes this place culturally rich and diverse, but no less fun.
Ciao!
PS: The natural language model I’ve been working on wrote this article, so I hope you enjoyed it!
PPS: Just kidding. The model did generate some of the sentences I used in this piece, and hopefully, someday, it will be able to write an entire blog post for me!
Sanjay Varma Rudraraju is an Application Developer at ADP based in New Jersey.
