For Intuit®, powering prosperity for our millions of customers around the world is not just a mission statement, it’s a way of life for Intuit employees (all 10,000+ in 20 locations in nine countries). Every year, CEO Sasan Goodarzi bestows the CEO Leadership Award on four employees who exhibit these four mission-based behaviors:
- Focusing on the largest growth bets and key innovation priorities.
- Being customer obsessed.
- Driving a data-driven culture.
- Raising the bar on accountability.
There were dozens of nominees in 2020. Ido Mintz, an Intuit principal data scientist, was one of the honored awardees because of his determination to solve an issue that caused refund delays for TurboTax® customers.
What was the problem he solved?
According to the CEO Leadership Award committee, customers were experiencing funding delays when they mistyped their bank accounts for a refund in TurboTax. There was no immediate notification of this error, causing disruption of up to eight weeks of delayed funding.
Ido applied an AI deep-learning algorithm to reduce payment failures. The AI model Ido and the team developed called Verum (“valid” in Latin) reverse-engineer bank account number patterns to predict the likelihood of each account number to be invalid. This resulted in instant notification of this possible error and reduced invalid account number issues by 55 percent, preventing the delay of more than $140 million in tax refunds this past season. Per Ido’s co-workers, “The effort didn’t exist before Ido was involved, and it wasn’t on any roadmap. Together with Tom Yacobi, an Intuit program manager, he went to San Diego and they both didn’t stop until the magic happened.”
Who is Ido?
The Intuit Developer team sat down with Ido to find out more about who he is and what lessons he learned during the development process.
Q: Can you fill us in on your career journey before joining Intuit four years ago?
Ido: Holding a bachelor’s and a master’s Degree in economics Ben Gurion University, I’m in essence an economist. I started my career at the Israeli Antitrust Authority where, for five years, I led several initiatives to transform the Israeli credit card and banking industries. This includes the transition toward contactless payments and increasing the use of debit transactions in Israel. I have also led research and strategy in the retail department of Union Bank, a local bank in Israel.
At Intuit, I serve as a principal data scientist, developing AI solutions and leading mission-based teams to make our customer experience smarter and simpler. I am also a member of the Intuit AI architecture committee.
Q: What lessons did you learn while developing your AI solution?
Ido: The main lesson I learned in the two-year journey prior to winning the award is that “the secret of success is our ability to create large-scale cooperation,” as stated by Professor Yuval Noah Harari.
This means that it is not enough to have an awesome cutting edge AI solution to create significant customer impact. While the AI solution is the beating heart, what makes it come to life is collaboration with product managers, engineers, analysts, and other data scientists to make the solution more accurate, robust, and timely available. To drive effective collaboration, I learned that two fundamentals must be achieved: alignment and trust.
Alignment is what drives others to follow you to achieve success, and trust is what allows them to believe you can actually deliver the success. Missing one of those ingredients will, in most cases, break the cooperation and eventually cause the failure of the mission.
The first fundamental for creating large-scale cooperation is alignment. People are very busy. Each of us has tasks and obligations, so there needs to be an agreed-upon goal to foster new collaboration.
#1: From my experience alignment is formed well when:There is a well-defined customer problem. An important observation is that value is created when we manage to reduce customer pain. At Intuit, our products are built using a Design for Delight (D4D) methodology. This process makes sure we are working on the most important customer problems. An essential condition for alignment is that we all understand and agree upon our common purpose.
#2: There is a clear definition of the current and desired (ideal) states. Driving meaningful change is a long journey. When we ask people to join us on this journey, there needs to be an agreement where we are and where we’re trying to go. It should also be clear that the journey is worth the effort. Within the Intuit AI organization, we summarize the customer problem and ideal state in a <from> <to> format that makes it clear how solving the customer problem will create the desired result: “<Customer> is trying to gain/do this <benefit> but is unable to/hindered because of <problem>.
Intuit/Intuit AI will help deliver this <benefit> by <how improvement achieved>, which will lead to <improvement from value, to value>, which will be delivered by <delivery date> in support of <company goal>.”
#3: There is a quick proof of concept using experiments. Experiments are extremely important in helping gain trust that a basic implementation of the solution can partially solve the customer problem.
#4: There are well-defined success metrics (KPIs). Everybody knows how important clear and well-defined KPIs are. These should be broken down into: solution metrics, customer metrics, and company metrics.
The development cycle is described well in Lessons Learned Leading AI teams by Intuit Data Science Group Manager Shir Meir Lador. Before starting to make actual decisions using the AI model, we monitor model health and performance in silent production and evaluate the impact on business metrics. Then, we A/B test (action on part of the population to evaluate impact in a controlled experiment).
I found the following actions can help significantly to gain trust between team members and stakeholders:
- Build trust over time. Weekly recurring meetings create a contract that the team is here for individual team members to address any issues, working closely together with accountability to make progress relatively quickly.
- Lean in to transparency: Documenting decisions/results builds credibility in the team’s ability to deliver, and drives data driven decisions. When the data is available and accessible, every member can raise hard questions and challenge decisions.
- Respect your team members and build upon agreements: Pre-wire meetings when needed and avoid escalations. Remember these are your partners and solve conflicts inside your four walls.
- Give credit where credit is due: Provide constructive feedback; I like using the State – Behavior – Impact (SBI) model. This is a useful tool as it concentrates on the observed behavior and its implication rather than emotions. It is also focused on what can we do better in the future rather than whose fault it is. Also take time to recognize employees for going above and beyond themselves.
- Begin with team empathy: Understand your teammates’ personal background, challenges, and goals and integrate that into expectations.
Congratulations to Ido on this well-deserved award! His customer-obsessed and data-driven solution will continue to help Intuit power prosperity for small businesses and customers around the world.
During our interview, Ido quoted Intuit Chairman of the Board and Former CEO Brad D. Smith, so we’ll leave it with you today: “We measure success in the only way that matters – through the eyes of the customer.”