We are seeking a Senior Data Engineer to manage the end-to-end data engineering lifecycles, Vocus platforms, and partner with stakeholders to translate complex needs into scalable AWS and Databricks architectures. You won't just build pipelines, you will design the platform's future, optimise distributed systems, champion best practices, and mentor your peers. Key Responsibilities Design robust, scalable data architectures, gather stakeholder requirements and deliver clear technical solutions Maintain and optimise all data-related AWS services, ensure high availability, security, and performance of cloud data infrastructure Manage both AWS and Databricks environments strictly using Terraform and automate infrastructure provisioning and updates Build, maintain, and execute robust CI/CD pipelines and automate code testing and deployments for seamless delivery Build end-to-end AWS and Databricks solutions, create reliable batch, near real-time, and real-time ingestion pipelines, tune Databricks clusters, and eliminate any bottlenecks in distributed systems Design high-performance data model leveraging Medallion architecture principles Build automated data quality, job performance reports, etc using AI agents Should be able maintain and optimize data projects built in Google cloud platform Foster a culture of technical excellence including train, mentor, and guide fellow data engineers Skills & Experience 5-10 years of hands-on Data Engineering Deep expertise in AWS, with solid exposure to GCP Mastery of Databricks, Spark architecture, streaming, and cluster configuration Production-grade Python and PySpark skills Advanced SQL for complex querying and manipulation Proven hands-on experience with CI/CD and Terraform Expert in modern data warehouse and data lake modelling techniques.
