Lead Backend/Data Engineer

Department: Engineering
Specialization: Data Engineer
Experience: Lead Senior
Technologies: Apache API AWS Hadoop Java Spark SQL
Client: Indeed
Special referral bonus: No
Hot position?: Not Hot
Work model: Remote
Technical flow: Data Engineer
Engineering technical flow: Data Engineer
The world’s number 1 highest-traffic job website, Indeed.com has 250 million unique monthly visitors and 9.8 jobs posted every second. A giant on a mission to help people find jobs, Indeed works with product teams in Austin, Tokyo, Seattle, San Francisco, Singapore, and Hyderabad.Ready to help Indeed improve people’s lives, one job at a time?On behalf of Indeed, AgileEngine is looking for a Lead Data Engineer to develop and enhance data pipelines for the internal Indeed data product. The ideal candidate’s prior work experience reflects high standards for engineering with proven capabilities. The candidate possesses the ability to take on important business challenges and execute them with precision and quality.

Requirements

  • 4+ years experience in software development in preferably Big Data Projects like Hadoop eco-systems.
  • Should have expereicne with Kafka, Spark.
  • Practical experience implementing Apache Spark jobs: knowing its features on partitioning, grouping, joins, importing the data into the cluster, exporting the data from the cluster.
  • BS in Computer Science or equivalent practical experience
  • Practical experience with Java (ideally 2+ years).
  • A plus if one has experience working with AWS, specifically AWS EMR (or ability to pick it up fast), although not mandatory.

Nice to have

  • Apache Hive
  • Apache Pig
  • Apache Airflow

Responsibilities

  • The project is to create a proof of concepts and productinize solution for migration of several core indexes builders to AWS, to increase stability of the builder and thus reliability of its data.
  • Migrate Indeed search index builders from old framework to Apache Spark and AWS;
  • Providing Apache Spark best practices and mentoring.
  • Producing high-quality code
  • Communication with Engineering & Product management
  • Alignment with existing development teams

We offer

  • Interesting and challenging tasks
  • Flexible work schedule
  • Zero bureaucracy
  • Friendly and a very skilled team with great corporate culture and mentorship (visit us and see it yourself)
  • US democratic management style
  • Opportunities for self-realization, professional and career growth
  • Cool events and team activities
  • Professional workshops and training, a great engineering culture

About the project

Since 2004, Indeed has given job seekers free access to millions of jobs from thousands of company websites and job boards. As the leading pay-for-performance recruitment advertising network, Indeed drives millions of targeted applicants to jobs in every field and is the most cost-effective source of candidates for thousands of companies.

About AgileEngine

Founded as a dev tool vendor with a 2-person team

2010

Opened a dev center in Ukraine

2012

Pivoted into outsourced product development

2014

Launched mobile and UI labs

2015

Got our first Inc. 5000 award

2016

Opened a dev center in Argentina

2017

Became a top-3 ranked custom software developer in DC, Ukraine, and Argentina

2019

Became the #1 software development company to hire in 2020

2020

Opened new dev centers in Mexico and Colombia, counting 500+ experts

2021

How we lead

A company where experts grow, hone their skills, and do what they love, AgileEngine is guided by these principles:

Stay agile and embrace changes

Thrive in a results-driven culture with individual autonomy

Innovate with fellow experts in a no-blame environment

Learn from mistakes and move on

Foster mutual trust and support

Our geography

UTC-5
WASHINGTON DC USA
UTC-5
MIAMI USA
UTC-6
MEXICOMexico
UTC-5
ColombiaColombia
UTC-3
BrazilBrazil
UTC-3
ArgentinaArgentina
UTC+2
UkraineEurope
UTC+1
PolandEurope
UTC+0
PortugalPortugal
UTC+5:30
IndiaIndia

Apply for this position

Allowed Type(s): .pdf, .doc, .docx