Event-Driven Call Transcript Classification Pipeline with Vertex AI and Gemini

We designed and implemented a fully event-driven architecture on Google Cloud Platform that processes call transcripts in real-time:

The core of the solution leveraged Vertex AI and Gemini large language models to analyze call transcripts:

• Model Selection: Utilized Gemini for its superior ability to understand conversational context and nuances in call transcripts
• Prompt Engineering: Designed specialized prompts for each classification task using LangChain optimized to achieve high performance in transcript analysis
• Multi-label Classification: Developed four distinct classification chains
• Safety Mechanisms: Configured safety settings to enable business classification while preventing inappropriate content
• Data Preprocessing: Implemented text normalization, special character removal, and standardization techniques
• Validation System: Created a fuzzy matching validation process to correct classification outputs that didn't match predefined categories
• Pipeline Automation: Built a three-step Vertex AI Pipeline that:
1. Exports transcript data from BigQuery
2. Processes transcripts through classification models
3. Loads enriched data back to BigQuery for activation
• Model Versioning: Established CI/CD processes for prompt updates

This approach allowed us to process thousands of call transcripts weekly with minimal human intervention, achieving high classification performance while maintaining flexibility to adapt to changing business needs.

Implementing this event-driven ML solution required overcoming several complex technical hurdles:

• Transcript Quality Issues: Call transcripts often contained noise, brand mentions, and speech disfluencies that reduced classification accuracy. I developed specialized preprocessing techniques to clean and normalize the text before analysis.
• Event-Driven Architecture: Ensuring reliable real-time processing with minimal latency required careful design of the Pub/Sub topic configuration, subscription handling, and error management to prevent message loss.
• Authentication & Security: The solution required secure authentication between multiple GCP services. I implemented service account management with appropriate IAM permissions and Secret Manager integration for API credentials.
• Pipeline Orchestration: Building a reliable KubeFlow pipeline involved complex component design, artifact passing between steps, and ensuring consistent data formats throughout the workflow.
• LLM Prompt Engineering: Developing effective prompts for the Gemini model required extensive testing and refinement to achieve high classification accuracy across diverse call scenarios and product types.
• Validation & Error Handling: LLM outputs sometimes contained unexpected formats or classifications. I implemented a fuzzy matching system that could correct outputs to match predefined categories without human intervention.
• Cost Optimization: LLM inference at call volume scale could become expensive. I implemented batching and caching strategies to reduce API calls and optimize the overall solution cost.
• Data Privacy Compliance: Call transcripts contained personally identifiable information (PII). The solution needed robust security measures to ensure GDPR and CCPA compliance throughout the data processing pipeline.

Several key architectural and modeling decisions were made to ensure the solution was scalable, cost-effective, and optimized for real-world call data:

• Event-Driven Architecture: Chose an event-driven design using Pub/Sub and Cloud Functions to enable low-latency, real-time processing and avoid unnecessary compute costs associated with batch jobs.
• Model Selection: Selected Gemini 1.5 Pro for its strong zero-shot performance and contextual understanding of noisy, real-world transcripts, outperforming traditional classification models in preliminary evaluations.
• Prompt Design Strategy: Used LangChain to modularize and chain prompts, enabling rapid iteration, reusable templates, and better debugging across multiple classification tasks.
• Pipeline Modularity: Structured the Vertex AI Pipeline with clearly separated steps for ingestion, classification, and enrichment to improve observability and maintainability.
• Fallback Handling: Anticipated classification edge cases by integrating fuzzy matching and validation layers, trading minor recall for significantly improved precision and reliability.