*Disclaimer: due to confidentiality agreements, I have not shared any classified information, screenshots, or documentation.

Allstate was an amazing opportunity to follow my time at GEICO because Allstate’s assistants had the inverse of GEICO’s challenges. Whereas GEICO had one complex bot, Allstate had many different teams working on about a dozen different assistants. This role gave me a consulting-type experience and significantly grew my stakeholder management skills.

Though I did flex my design and NLU skills at Allstate, I’m choosing to highlight the managerial impact I had. I believe growth as a designer isn’t just about getting better at designing; it’s about impacting the organization around you because that organization is going to ultimately determine the success of your designs.

Highlights

  • Designated CxD for the following bots:

    • Customer Claims

    • Claimant Claims

    • Body Shop

    • Agent Concierge

    • Internal IT Help Desk

    • Customer Service

    • Customer Concierge

  • Integrated RAG prompt design for the first time ever in an Allstate bot

  • Increased claims NLU accuracy by 27 percentage points

The Goal

My directive for Allstate was to bring order and CxD best practices to the teams I worked with. After a time of understaffing, this was the first time some of these teams received a dedicated conversation designer. As such there were many best practices to implement and trust for me to earn.

The Challenges

With seven different teams and bots to manage—each designing, building, planning, and collaborating differently—I knew that establishing baseline practices across bots was essential for improving organizational efficiency and ensuring a consistent customer experience.

After taking the time to connect with disparate product owners, business architects, developers, and AI trainers, I identified the following opportunities for improvement:

  • Design documentation was outdated or formatted inappropriately.

  • NLU models were not always created by linguists or AI trainers, leading to gaps in baseline elements like confidence scores and inadequate testing.

  • Team structures and roles did not align with product team or conversational AI best practices, with business architects or product managers independently creating designs from scratch.

  • Miscommunication between leadership and vendors resulted in an ineffective backlog.

  • A lack of user testing led to decision-making based on internal opinions rather than user-centric insights.

  • Ineffective technology (a topic modeler for PDFs being used for the bot) was leading to low containment and user dissatisfaction

The Solutions

Each challenge required a lot of time and relationship management. Many of my coworkers had been working at Allstate for a decade or more and were confident that their way was best. I took the time to get to know each stakeholder individually to earn their trust. This was the most important thing I could have accomplished because it’s what made the following improvements possible with very little (proud to say) push back:

  • Design documentation:

    • Moved all chatbot documentation from legacy software like Axure and Miro to Voiceflow

    • Established pattern libraries and an agreed-upon way of designing amongst fellow designers and devs

  • NLU models:

    • Instituted confidence score thresholds, quarterly health checks, held-out test sets, and testing procedures for all lacking bots

    • Improved claims NLU by 27 percentage points through trained utterance analysis, balancing, and semantic matching

  • Team structures and roles:

    • Provided teams with a best-in-class team structure that emphasized clear roles and responsibilities for business, design, engineering, research, and product/project managers

  • Miscommunication between leadership and vendors:

    • Cleaned backlogs and re-prioritized features based on independent research and vendor recommendations

    • Created roadmaps and sprint schedules that emphasized meaningful CxD updates

  • A lack of user testing:

    • Wrote a comprehensive, repeatable user testing process and documentation for all teams to use

    • Began with moderated user test with internal employees with great success, leading to adoption by multiple teams

  • Ineffective technology:

    • Persuaded product manager of internal bot to experiment with and adopt RAG solution

    • Advised senior leadership on RAG’s capabilities, limitations, and risks

    • Created the conversational prompt design for the model