Responsibilities include:

• Reduced RDS pipeline steps from 5 to 3 by eliminating redundant copy steps and developed a Terraform module for automating GCP Storage Transfer creation.
• Minimized load on the production database by migrating recurring ad-hoc analytics queries to BigQuery

Responsibilities include:

• Setup data infrastructure
• Cost optimization
• Knowledge sharing

Accomplishments:

• Achieved $7,400 monthly cost reduction by transitioning Trino from multizone to active-standby architecture.
• Provisioned Kubernetes cluster from scratch, and migrated Trino from EMR, resulting in 80% cost reduction
• Developed a Web UI to team’s data platform using Flask API and Retool, reducing manual ticket by 90%.
• Implemented instance right-sizing strategies and reserved instances, resulting in a $2,300 monthly cost reduction

Accomplishments:

• Accelerated user journey analysis by 10x by shifting from batch event tracking to real-time using Kafka.
• Optimized pipeline queue time by 60% using Airflow TaskGroup to group and manage heavy workload tasks.
• Automated Spark clusters provisioning using Terraform reduced processing time from 2 days to 10 minutes.
• Developed a system to ensure heavy dbt pipelines includes partitioning key, minimizing full table scans by 20%

Accomplishments:

• Introduced Presto long query alerting system that reduced p99 query time from 2 minutes to 30 seconds.
• Integrated 2000+ lines of custom Python ETL code contributed by multiple users into a centralized library.
• Developed a Web UI to team’s data platform using Flask API and Retool, reducing manual ticket by 90%.
• Ingested 100+ tables from PostgreSQL and MongoDB across the companies into data lakehouse.

Accomplishments:

• Increased redeem token insertion operation throughput on share and win service by using bulk-insertion.
• Researched groupcache peer-to-peer caching library as Redis alternative to be used for internal services.
• Tech stacks used: Golang, MySQL, MongoDB, Google PubSub, Redis

Accomplishments:

• Researched and refactored Cicil Jobs module to Clean Architecture.
• Tech stacks used: Golang, RabbitMQ, RedisResearched

Accomplishments:

• Learned fundamental web development and software engineering concepts using Clojure.
• Tech stacks used: clojure, luminus, compojure, pedestal, swagger.