Data Engineer – AWS & Databricks

We are looking for a Data Engineer to design, build, and enhance scalable data pipelines and Lakehouse platforms using AWS and Databricks for a major organization in the banking/financial services sector.
This role offers the chance to work on large-scale, enterprise data platforms with strict standards for security, governance, and performance.

Key Responsibilities

• Design, implement, and support ETL/ELT pipelines on Databricks using PySpark, Spark SQL, and Delta Lake

• Develop robust data ingestion solutions leveraging AWS services, including:

  • S3, Glue, Lambda, and Step Functions

  • Streaming platforms such as Kafka/MSK or Kinesis

  • Connectivity with on-premise systems, RDS, and Redshift

• Orchestrate and automate data workflows using Databricks Workflows, Airflow, or equivalent tools

• Improve Lakehouse efficiency through partitioning strategies, Delta optimization, and compute/cost tuning

• Build and maintain data quality controls, monitoring, and production issue resolution processes

• Enforce data governance and security standards, including PII handling, access management, and audit compliance

• Work closely with data analysts, data scientists, and business teams to deliver reliable, business-ready datasets

Requirements

Must-have

• 2–5+ years of hands-on experience in Data Engineering

• Strong practical experience with:

  • AWS (S3, Glue, Lambda, IAM, Step Functions)

  • Databricks (PySpark, Delta Lake, Workflows)

  • Python and SQL

• Solid understanding of data modeling concepts (relational and dimensional)

• Experience working with large-scale, distributed data platforms

Nice-to-have

• Background in banking or financial services (e.g., core banking, payments, lending, regulatory reporting)

• Experience with real-time/streaming data processing (Kafka/MSK, Kinesis, Spark Structured Streaming)

• Familiarity with data governance and catalog solutions (Unity Catalog is a plus)

• Infrastructure-as-Code experience (Terraform or AWS CDK)

• Understanding of security and compliance requirements in regulated environments