About the role

We are looking for a Middle Data Engineer to help modernize a 15-year-old data warehouse into a governed Databricks Lakehouse. You will build batch and streaming pipelines with PySpark and Delta Lake, following a medallion architecture across bronze, silver, and gold layers. This role also uses AI tools like Claude and GitHub Copilot to speed up development.

What you will do

  • Design, build, and operate batch and streaming data pipelines on Databricks using PySpark, Delta Lake, and Databricks Workflows.
  • Model and maintain a medallion (bronze/silver/gold) architecture serving analytics, reporting, and machine learning consumers.
  • Migrate legacy ETL and data warehouse workloads onto the Lakehouse with validated data parity and minimal business disruption.
  • Use Claude or Github Copilot as a development accelerator, generating code scaffolding, writing and reviewing tests, creating documentation and prototyping solutions.
  • Write clean, well-tested Python and SQL; maintain high standards through code review and documentation.
  • Optimize Spark jobs and Delta tables for performance and cost, including partitioning, clustering, caching, and cluster sizing.
  • Implement data quality, lineage, and governance controls using Unity Catalog and automated validation checks.
  • Debug, troubleshoot, and resolve pipeline failures, data defects, and production incidents.
  • Collaborate with DevOps, platform, and analytics engineers on observability, security, and compliance best practices.

Must haves

  • 3+ years of professional experience in data engineering, featuring direct expertise with Apache Spark and cloud-based data architectures.
  • Strong hands-on experience building data pipelines with Databricks, Apache Spark (PySpark), and Delta Lake.
  • Advanced SQL and Python, with strong data modeling skills across dimensional and Lakehouse patterns.
  • Experience with streaming ingestion using Structured Streaming, Auto Loader, Kafka, or Event Hubs.
  • Experience with workflow orchestration (Databricks Workflows, Airflow, or Azure Data Factory).
  • Experience with legacy platform migrations, ETL modernization, or managing data hygiene when porting old systems.
  • Strong problem-solving, collaboration, and communication skills.
  • Familiarity with Unity Catalog, data governance, access control, and PII handling.
  • Experience with dbt or an equivalent transformation framework.
  • Familiarity with secure coding standards and industry security best practices.
  • Experience delivering production data platforms at scale.
  • Upper-intermediate English level.

Nice to haves

  • Experience with Infrastructure as Code (IaC) using Terraform and CI/CD using Azure DevOps.
  • Experience working with relational databases (specifically PostgreSQL) and data persistence concepts.
  • Familiarity with logging and monitoring tools (e.g., Dynatrace, CloudWatch, Databricks system tables).
  • Experience working in Agile or team-based development environments preferred.

Perks

  • Growth without limits: build your skills through mentorship, internal TechTalks, challenging projects, and a dedicated annual learning budget
  • Competitive compensation: get recognition that reflects your skills and impact, with regular performance and compensation reviews
  • Flexibility: work 100% remotely with flexible hours that support focus, autonomy, and a healthy work rhythm
  • Meaningful, modern projects: build impactful products using modern technologies alongside global teams and leading brands
  • Collaborative culture: join a supportive environment with zero micromanagement where ideas are welcomed and contributions are recognized
  • Well-being & support: access local well-being programs and people-focused support tailored to your location

Didn’t find the perfect fit?

Subscribe to get notified about new roles that match your skills and interests.

Get role alerts

Hiring process timeline

We designed our hiring process to be fast, transparent, and convenient — so you always know what to expect and can move through the steps without unnecessary delays.

lightning-iconA quick note

After applying, keep an eye on your inbox — including your spam folder. Occasionally, emails from our hiring platform, LaunchPod, may land there.

1

Tell us about yourself

Submit a short application form.

2

Pass a quick test

Pass a 30’- 60’ test.

3

Record a short video

Introduce yourself on video to fast-track your application process.

4

Meet our team

Once you pass the video review, grab a spot on our calendar for a technical interview.

5

Get an offer

Welcome to AgileEngine!

FAQ

Have any questions?

What is the work format and schedule?

We are a remote-first company, so all our positions are remote. Schedules are flexible, but the main requirement is to have an overlap with your client’s schedule to ensure smooth collaboration. Specific details depend on the project and are discussed during the interview process.

What level of English is required?

We look for Upper-Intermediate (B2) proficiency or higher. Since you will be collaborating with international teams and global clients (including Fortune 500 companies), English is a part of your daily work.

Does AgileEngine provide work equipment?

It depends on your location. We provide equipment in Ukraine, Poland, Argentina, Colombia, Mexico, Brazil, Guatemala, Portugal, Spain, and India. In the USA, equipment is usually provided by the client; otherwise, you will be expected to use your own setup.

What opportunities for professional growth do you offer?

We support continuous growth through personalized development paths tailored to each expert. Also, you can expect:

  • An annual learning and development budget
  • Internal workshops, tech talks, and mentorship programs
  • Opportunities to switch projects or grow into new roles over time

What does a typical team look like, and what tools do you use?

Teams vary by project, but you’ll usually work with a mix of experts across different fields, along with a delivery or project manager who supports collaboration and onboarding.

For day-to-day work, teams commonly use tools like Jira and Google Workspace, along with communication platforms such as Google Chat. Depending on the client, you may also work with Slack, Confluence, Notion, or similar tools.

Need help?

Want more details? See full FAQ

If something doesn’t work or you have questions regarding your application process, drop us a line.

    Scroll to Top