dikshant sharma

About

Software Engineer focused on data platforms, backend infrastructure, and AWS-native systems. At BigBasket, architected a 30–40 TB Apache Iceberg lakehouse replacing Redshift, cutting infrastructure costs by ~$36K/month and reducing CDC/data replication latency from 24 hours to under 10 minutes while improving data accuracy to ~99.9%.

Experience

• Software Engineer (Intern → Full-Time) · BigBasket01-01-2025

  • Architected a 30–40 TB, 3-tier medallion Iceberg lakehouse on S3 with Trino, dbt, and AWS Glue metastore, replacing Redshift across 800+ production tables. • Eliminated ~$36K/month in infrastructure costs: $35K Redshift licensing plus on-prem Debezium/Kafka transformer servers replaced with managed and serverless AWS services. • Reduced data replication latency from 24 hours to under 10 minutes by engineering a CDC pipeline (AWS DMS Serverless + Spark), raising data accuracy to 99.9%+ across 400+ RDS MySQL tables. • Defined tiered SLA architecture: Bronze layer guaranteed at <15 min; Gold data marts served at 10 min / 1 hr / 6 hr / 24 hr cadences. • Orchestrated production Airflow DAGs on EKS with dbt-Cosmos integration and CI/CD-gated DAG deployment for modular pipeline execution. • Migrated a 2,000-line reconciliation SQL workflow to 10+ modular dbt incremental models using Iceberg MERGE, unblocking independent analytics iteration. • Configured Trino resource groups on a 6-node EMR/EKS cluster, resolved JVM OOM/spill failures, and set up CloudWatch + PromQL consumer lag alerts. • Deployed OpenMetadata on Kubernetes with SSO auth, enabling 4–5 cross-functional teams to self-serve data catalog, lineage, and data quality alerts.

• Student Developer — NU Student App · NIIT University08-01-2023 – 01-01-2024

  • Built a cross-platform Flutter app serving 1,000+ daily active users with Firebase auth, REST API integration, SQLite offline storage, and anti-proxy attendance locking.

Education

• NIIT University, Rajasthan, IndiaBachelor of Technology (B.Tech), Computer Science and Engineering

Similar talent on Pangea

Arjan A.An experienced full-stack developer adept at optimizing backend systems, and developing dynamic and user-friendly web interfaces.Toronto, ONAdnan A.Senior Software Engineer (Backend & Platform) | Distributed Systems, Kubernetes, Cloud InfrastructureIslamabad, PakistanAbhishek C.Versatile software engineer skilled in front-end, full-stack and mobile development.Hyderabad, TelanganaMeghana G.Senior Data Engineer | Analytics Engineering & ELT Pipelines (dbt, Snowflake, AWS)Boston, MA