Rachit Sharma in a navy hoodie on a misty hilltop
My story

I don't measure engineering by the code written.

I measure it by the complexity simplified, the effort eliminated, and the impact created. I'm a data engineer at Celebal Technologies in Jaipur, working on enterprise data platforms, migrations and automation, and on the pre-sales conversations where those solutions get shaped.

Before that I spent three years at Infosys making operations run themselves, and started out at IBM building chatbots. On my own time, I build tools and products: some for me, some for friends, some that might become something bigger.

My story

Let's go back in time.

Dec 2024 to now
01 / 04
  1. Dec 2024 to now

    Data consultant

    Celebal Technologies, Jaipur
    • Tech lead on a SAS to Databricks modernisation: 110 legacy jobs, ~30% faster processing, ~25% cost savings.
    • Metadata-driven Kafka ingestion, Databricks Asset Bundles deployments and ingestion monitoring for a data platform shared by several business units.
    • Pre-sales and solution engineering for retail, logistics, supply chain, banking and automobile clients, including a framework for sharing data from AWS to Azure Databricks.
    • Built Splunk to ClickHouse and Snowflake to ClickHouse migration accelerators that run discovery, lineage, conversion, deployment, data load, reconciliation and reporting.
  2. Aug 2021 to Nov 2024

    Specialist programmer (data)

    Infosys, Pune
    • Built Automated IntelOps: automated more than 90% of manual job monitoring on an on-premise Hortonworks platform, with a Grafana dashboard and automatic ServiceNow ticketing.
    • Scheduled daily stakeholder reports, saving about three hours of effort a day.
    • Hackathon winner, recognised for automation coverage and the monitoring dashboard.
  3. Mar 2021 to Aug 2021

    Intern, chatbot development and data engineering

    IBM
    • Built a RASA chatbot that cut onboarding time by 40%, with a Slack integration that improved timely attendance updates by 30%.
    • Automated data extraction and cleansing, reducing manual effort by 80%.
  4. 2017 to 2021

    Bachelor of Technology

    SRM University, Kattankulathur
    • Four years of engineering, graduating with 83.2%.
    • Finished alongside an internship at IBM.
How I work

Think deeply. Engineer intelligently. Automate relentlessly. Build continuously.

Think in systems

Not isolated tasks. Every pipeline sits inside a process, a team and a business decision.

Automate the repetitive

If people do the same thing every day by hand, that is a design problem waiting to be solved.

Build it once, reuse it

Accelerators and utilities beat solving the same problem for the hundredth time.

Use new technology with intent

AI and emerging tools are useful when they remove real friction, not when they decorate a slide.

Multiply other people

The best tooling makes a whole team faster, not just the person who wrote it.

Measure outcomes

Complexity simplified, effort eliminated, impact created. Not lines of code.

Capabilities

Tools in service of systems.

In client delivery

Used in the projects on this site.

Languages and processing

PythonSQLPySparkApache Spark

Lakehouse

DatabricksDelta LakeMedallion architectureDatabricks Asset BundlesData modellingPerformance tuning

Governance

Unity CatalogRow and column securityLineageAudit

Streaming and cloud

Apache KafkaMetadata-driven ingestionAWS S3GlueLambdaRedshift

Also worked with

Worked with across projects and practice.

Databricks and orchestration

Azure DatabricksDelta Live TablesDatabricks WorkflowsAutoloaderSpark SQLAirflow

Cloud and Hadoop

AthenaCloudWatchADLSAzure Data FactoryHadoopHiveBash

Data and analytics

Confluent KafkaParquetSnowflakeClickHouseSplunkPower BIAmazon QuickSight

Exploring now

Where my side projects are taking me.

Product building

Apps that run locallyLocal AI for data processingMigration utilities
Certifications
  • Databricks Certified Data Engineer ProfessionalVerify ↗
  • Databricks Certified Data Engineer AssociateVerify ↗
  • Databricks Certified Generative AI Engineer AssociateVerify ↗
  • AWS Certified Solutions Architect AssociateVerify ↗
Education
  • Bachelor of Technology, SRM University, Kattankulathur
    2017 to 2021, 83.2%
Beyond the keyboard

Building the engineer, out loud.

A 30-day speaking challenge on Instagram, to get clearer and more confident at explaining technical ideas. Clear, educational and conversational, with spontaneous topics on purpose.

Day 6