Moving 110 legacy SAS jobs onto a Databricks lakehouse
As tech lead, I led the modernisation of 110 legacy SAS jobs onto Databricks, and built the tooling that made the migration faster to run and easier to trust.
- Client
- A large financial services enterprise
- Company
- Celebal Technologies
- Role
- Tech lead
- Focus
- Enterprise modernisation
The brief
Modernise a portfolio of 110 legacy SAS jobs onto Databricks on AWS, including the reports the business relies on.
What I owned
Technical leadership of the migration: solution design, the migration approach, validation of migrated outputs, and performance optimisation on the new platform.
What I built
- Migration accelerators that took 15 days out of the migration timeline.
- Reusable report utilities that reproduce Excel reports in their exact layouts, cutting manual reporting effort by about 30%.
- Lakehouse pipelines in PySpark on Delta Lake, organised with a medallion architecture and governed through Unity Catalog.
How I think about it
A migration is more than code conversion. The business keeps running on the old system while the new one is built, so the real work is reproducing behaviour people already trust, proving it matches, and then making it faster. The accelerators and report utilities exist because the same problems kept appearing job after job, and solving them once was cheaper than solving them 110 times.