Resume

Building and maintaining data infrastructure powering real-time and batch analytics across the organization.

• Replaced expensive batch RDS snapshot ingestion with real-time CDC → BigQuery across 800+ PostgreSQL/MySQL tables, improving data latency while reducing monthly spend by $8,000.
• Upgraded Airflow from Composer 2 to Composer 3, modernizing Python runtimes and system packages while maintaining backward compatibility for 100+ DAGs.
• Led cost observability initiative — implemented monitoring dashboards and review cycles, reducing BigQuery slot usage by 527 hours/day and saving $1.5K/month via table partitioning.
• Deployed Apache Spark on GCP to enable DBT Python models, unlocking flexible transformation logic previously impossible due to BigQuery's SQL-only limitation.

Led infrastructure-level data platform work: cost optimization, real-time processing, and tooling for data science teams.

• Achieved $7,400/month cost reduction by analyzing network traffic with Datadog and migrating Trino on Kubernetes from multizone to active-standby architecture.
• Improved clickstream analysis by migrating from batch to real-time with Kafka, Spark Streaming, and Databricks.
• Created Databricks notebook templates with integrated MLflow tracking, enabling data scientists to rapidly prototype and monitor experiments.
• Built a custom Trino logging plugin in Java to track query ownership and identify high-cost users.

Designed ingestion frameworks and data transformation pipelines, with focus on reliability and self-service for data consumers.

• Developed a config-driven ingestion framework (Strategy pattern + FastAPI) integrating 10+ sources: PostgreSQL, MongoDB, Jira, Zendesk, CleverTap, and Facebook Ads.
• Reduced manual COGS calculation from 3 days to under 1 hour by implementing Slowly Changing Dimension (SCD) logic in dbt models.
• Applied IaC with Terraform to automate provisioning of Spark clusters and AWS IAM roles, ensuring environment parity.

Contributed to platform reliability and code quality initiatives on the data engineering team.

• Consolidated 2,000+ lines of user-contributed PySpark code from Databricks notebooks into a shared library.
• Developed CI/CD validation for dbt and Looker to enforce partitioning standards and prevent inefficient deployments.