12+ years1 TB data mart~63 operating companies24x faster

I build the data platform a business runs on. Then I make it faster, cheaper, and easier to trust.

Hi, I'm Vishal Kumar Sarvade, a Technical Architect at Grid Dynamics with 12+ years in data engineering. Today I own a 1 TB Amazon Redshift data mart that about 63 operating companies report from, and I lead the ten engineers who keep it running.

I also:

Portrait of Vishal Kumar Sarvade
Selected work

Problems I've solved, with the numbers

Real work for a global direct-selling business and, earlier, a healthcare technology company. Client names stay private; the numbers are real.

Performance

Twelve hours to thirty minutes

A critical aggregation workload took about 12 hours. I rebuilt it on Redshift and reworked how it executes. It now finishes in 30 minutes.

Redshift · SQL · Stored procedures · Sort and distribution keys
Performance

Ten seconds became fifteen minutes

Write jobs on Redshift Serverless slowed from about 10 seconds to 10 to 15 minutes. I traced it to base capacity being set equal to the maximum, and recommended RA3 for writes with a Serverless workgroup for reporting.

Redshift Serverless · RA3 · Data sharing
$5K/ month
saved by consolidating an ETL Works instance and S3 storage
Cost

Saving money, and knowing when not to sign

Consolidation saved about $5,000 a month. On a 3-year Redshift reservation, I found the cost model had never included the biggest lever, a move to Graviton-based RG nodes. I advised against signing either option on the table.

Redshift · Reservations · ETL Works · S3
Modernization

Four clusters, no more sync jobs

A four-cluster Redshift estate, one producer and three consumers, stayed in step through sync jobs. I moved it from DC2 to RA3 and set up data sharing. The sync jobs are gone and consumers read live data.

Redshift RA3 · Data sharing · DC2 migration
5
on Members
2
on Orders
operating companies now served from Aurora MySQL, off AS400
Modernization

Off DMS, and off AS400

Replication from Aurora MySQL ran on AWS DMS, with maintenance pauses, full reloads after failures and binlog workarounds. I replaced it with Zero-ETL, which ended all three. The same pipeline now carries the move off AS400, routed country by country.

Aurora MySQL · Zero-ETL · AWS DMS · AS400/DB2
Modernization · In progress

110 jobs, moved one at a time

I built the capacity and pricing plan for moving 110 Informatica Cloud jobs to ETL Works, presented the options to the client and won the project. The move runs from September 2026 to March 2027, in two phases and one job at a time, so nothing breaks on the way over.

Informatica Cloud (IICS) · ETL Works · Redshift
278
dictionary sheets in one searchable portal
7
sheets in each Looker lineage report
Governance

Making the data findable

A static, S3-hosted metadata portal built from the business's data dictionary, with quality gates in the build. A Python tool on the Looker SDK that traces every dashboard to the column. A job metadata framework that tracks every pipeline across Informatica, ETL Works and Zero-ETL.

Python · Looker SDK · Amazon S3 · Redshift
Batch by batch
every load validated · rejected rows returned as CSV
Governance

Sensitive records, moved carefully

I moved government IDs, bank account details, retail and customer service records from CSV files and MySQL into Aurora MySQL. Every batch was checked for accuracy, and rejected rows went back to the customer as CSV.

Aurora MySQL · MySQL · SQL · PII handling
4 GB
member loads from CSV extracts · plus providers, claims, authorizations and eligibility
Modernization · 2014 to 2018

A healthcare migration, rule by rule

For a healthcare technology company, I led the move of member, provider, claims, authorization and eligibility records off legacy systems with SQL Server stored procedures and SSIS, validating every business rule and running the gap analysis with the client.

SQL Server · T-SQL · SSIS
The platform I run

One warehouse for a global business

I architected this data mart and lead the team that runs it. Every dashboard and report the business produces reads from it. Simplified, with client names removed.

200+tables in one star schema
150+Informatica and ETL Works jobs
100+Looker dashboards maintained
10engineers I lead and mentor
Sources1 of 4
Commerce platformJava, Spring Boot, Angular, MySQL and Kafka microservices
JoinShoppingAdminCorporateCMSCustomer ServiceReporting
Aurora MySQLOperational data, the new source of truth
AS400 / DB2Legacy source, being retired market by market
PostgreSQL and flat files
Ingestion2 of 4
Redshift Zero-ETLAurora MySQL to Redshift, replaced AWS DMS
Informatica Cloud100+ task flows, moving to ETL Works
ETL Works50+ jobs today, 110 more on the way
Country-based routingEach market reads from AS400 or Aurora until it moves
Warehouse3 of 4
Amazon Redshift RA3 producerStar schema · 1 TB · 200+ tables
Data sharingThree consumer clusters read live data, no sync jobs
Job metadata and catalogPipelines tracked; tables and columns documented
Used by4 of 4
Looker100+ dashboards maintained, 40+ built
Power BI and Microsoft FabricSemantic models, Copilot-built reports
NL-to-SQL agentPlain-English questions, read-only
M365 CopilotFabric data agents, in progress

Standards I setNaming, keys, required audit columns, distribution style, documentation and access grants. Every database change traces to a Jira ticket and a Git commit.

Who I work withThe client's CTO, business analysts, and the application, production support and legacy teams I coordinate releases with.

How I run the teamSprint planning, Scrum calls and architecture reviews with a ten-person offshore team, turning business requirements into delivered Jira tickets.

AI, applied

AI where it removes real work

I don't start with the model. I start with the question people keep asking, the business rule behind it and the data that answers it. Once I understand the root cause, I automate the part that repeats.

Asking the warehouse a question in plain English

I built an LLM-based NL-to-SQL agent over the data mart and its Looker tables. It can only read. It blocks PII, forces the filters every query needs, and pins down what "last quarter" means instead of guessing.

The chat front ends add a trust layer: a confidence score, a plain account of how the question was read, and the SQL it ran, right there on screen.

SELECT onlyPII blockedMandatory filtersPeriod disambiguation14 production SQL templates
Two n8n AI agents

One answers the team's recurring FAQs; the other does first-pass data analysis.

Built
PDF reports to Power BI, with Claude

Took report requirements, built the data mart tables and semantic models, generated Power BI and HTML reports, validated them against the database and presented them to the customer.

Delivered
Informatica observability dashboard

Pipeline health in one view. Built with Claude Code.

Built
Data dictionary app

Puts the business data dictionary in front of the team in one searchable place. Built with Codex.

Built
Estimation app

Structures delivery estimates, validation effort, risks and deployment steps. Built with Codex.

Built
Power BI as a Looker alternative

Six dashboards, three of them built with Copilot, plus Microsoft Fabric set up. Presented to CTO and VP audiences.

Proven out
M365 Copilot on governed data

Fabric data agents so Microsoft 365 Copilot answers from Power BI semantic models.

In progress
Experience

Twelve years, two employers, one long thread

I've worked on the same global data platform since 2018, first as an engineer building its pipelines and now as the architect who owns it.

Apr 2023 to presentCalifornia

Technical Architect

Grid Dynamics

I own the data architecture for a global direct-selling business: the Redshift data mart, its ingestion, its standards and its AI layer. I lead a ten-person offshore team and work directly with the client's CTO and business teams.

  • Rebuilt a critical aggregation workload from 12 hours to 30 minutes.
  • Replaced AWS DMS with Zero-ETL, then used it to start moving off AS400.
  • Moved four Redshift clusters from DC2 to RA3 with data sharing.
  • Planning and leading the move of 110 Informatica jobs to ETL Works.
  • Built the NL-to-SQL agent, two AI agents, and the metadata, lineage and job-tracking tools.
  • Cut about $5,000 a month by consolidating an ETL Works instance and S3 storage.
  • Ran proofs of concept on Snowflake, Airflow, PySpark and Fivetran, and wrote the BRD and TRD for a PySpark aggregation pipeline on Redshift.
Jan 2022 to Mar 2023Florida

Associate Technical Architect

NextSphere Technologies

Turned business requirements into solution designs and delivery plans. Reviewed the team's work, cleared blockers and coordinated production fixes. Worked with application teams on design and release support so data integration kept pace with platform changes, and with client teams on options and trade-offs, in plain terms.

Jan 2020 to Jan 2022Clearwater, Florida

Senior Software Engineer

NextSphere Technologies

Led a small team that built end-to-end pipelines from PostgreSQL and AS400 into Amazon Redshift with Informatica Cloud and ETL Works.

  • Tuned Redshift through distribution strategy, sort keys and regular maintenance.
  • Tuned the Informatica Secure Agent for maximum task-flow capacity.
  • Automated the daily data-statistics report and the audit checks.
  • Ran job setups, the data dictionary, connections, configuration changes, backups and deployments.
Mar 2014 to Jan 2020Hyderabad, India

Associate Technical Lead

NextSphere Technologies

Grew from Associate Software Engineer to Associate Technical Lead. Led a healthcare data migration from 2014 to 2018, then joined the e-commerce data platform I still work on today.

  • Guided the offshore production support group and reviewed code.
  • Built ETL pipelines into Amazon Redshift and audit jobs against the legacy platform.
  • Earned repeated client thanks for pinning down exact root causes during UAT.
How I work

Six habits behind the work

A technical decision is a business decision.

The architecture comes with the TCO comparison, the FTE math and the cost model.

Every number gets checked.

If I can't trace a figure back to its source, it stays out of the deck.

Root cause first.

Mismatched reports, failed pipelines and slow queries get investigated before anyone proposes a fix.

Draw it for the person who approves it.

My architecture diagrams are made so an executive can follow the lineage.

Decide, then move.

A structured comparison beats an endless vendor evaluation.

Explain it myself.

I won the ETL Works migration by presenting the options to the client directly.

Toolkit

What I use in production

Warehouse and databases

Amazon RedshiftRA3Redshift ServerlessData sharingZero-ETLWLMSort and distribution keysAurora MySQLPostgreSQL / PL/pgSQLSQL Server / T-SQLMySQLDB2 / AS400S3 data lake

Integration and pipelines

Informatica Cloud (IICS)Secure AgentIICS REST APIETL WorksAWS DMSHVRSSISChange data captureBatch and near-real-time loads

Modeling and governance

Dimensional modelingStar schemasData catalogsLineageData qualityPII controls

Analytics

Looker / LookMLPower BIMicrosoft FabricSemantic models

AI and automation

NL-to-SQLn8n agentsClaude CodeCodexMCPStrands Agents SDKMicrosoft CopilotFabric data agents

Languages and cloud

SQLPythonAWS: S3, EC2, RDS, Lambda, IAM, Secrets Manager, API GatewayAzure

Ways of working

GitJiraAgile and ScrumCode reviewRelease management

Evaluated in proofs of concept

SnowflakeApache AirflowPySparkApache SparkFivetran
Writing, learning and building

Outside the day job

Daily

Field notes on data and AI agents

A short, sourced note each day on how companies are putting AI agents into production, plus longer posts on Redshift and data architecture.

On LinkedIn

  • RedshiftTop-K queries on Redshift and how zone maps help
  • AIRouting prompts across models, drawing on the RouteLLM and FrugalGPT research
  • n8nHow I structure n8n agent workflows
  • LearningGraduating from the BeSA Agentic AI cohort

Learning and events

  • May 2026Builders of Tomorrow AI Super Hackathon, Computer History Museum. Built a video-generation pipeline on BytePlus Seedance 2.0.
  • Spring 2026BeSA Batch 9, a six-week AWS Agentic AI cohort.

Built in public

  • ToolETL platform comparison: total cost of ownership across Informatica Cloud, Talend Cloud and ETL Works. Open-source.
  • AgentsA five-agent content pipeline: intake, research and calendar, image generation, review, publish.
  • n8nA storefront with an embedded AI chat assistant
  • DashboardA daily scrum status board that tracks changes from one day to the next.
About

The longer version

I started in 2014 in Hyderabad, writing SQL Server stored procedures for a healthcare migration. Members, providers, claims, eligibility. It taught me early that a migration is really a long list of business rules, and every one of them has to be proven.

In 2018 I moved onto an e-commerce data platform for a global direct-selling company, and I'm still on it. I've worked on it across two employers, first as an engineer building pipelines out of AS400 and PostgreSQL, and today as the architect who owns it. It's a 1 TB Redshift data mart with more than 200 tables, and about 63 operating companies report from it.

A lot of my best work is plain engineering. A job that took 12 hours now takes 30 minutes. Four clusters share data instead of syncing it. A replication pipeline no longer needs babysitting. A 3-year reservation didn't get signed because the cost model left out the option that mattered most.

More recently I've been putting AI to work on the same platform: an NL-to-SQL agent that shows its SQL, agents that handle the questions my team gets every week, and Copilot running on governed semantic models.

I lead a team of ten. I care about standards, root causes and numbers you can trace back to their source.

Contact

Let's talk data platforms.

The fastest way to reach me is email or LinkedIn.