What you will do
- Design and develop the data’s team micro services – Java services running on K8S;
- Scale our data pipeline to support the processing of billions of events in both low-latency real-time and T+1 batch using advanced technologies like Clickhouse, Spark, and Kafka;
- Tackle data duplication, velocity, schema adherence (and schema versioning), high availability, data governance, and more;
- Develop and maintain our data pipeline written mostly in Java and running on K8S in a micro-service architecture;
- Plan and communicate integrations with other teams that consume the data and use it for insights creation;
- Ongoing improvement of the way data is stored and served. Improve queries and data formats to make sure the data is optimized for consumption by a variety of clients;
- Query optimizations in our data warehouses, ensuring data completeness and reliability at scale.
Must haves
- BSc. in Computer Sciences from a top university, or equivalent;
- Strong background as a software engineer with at least 3+ years experience with Java;
- Experience in data engineering and data pipeline development;
- Proven experience with relational and non-relational DBs. Expert. Proficient in SQL and query optimizations – preferably in clickhouse;
- Experience with monitoring systems (either Prometheus, Grafana, Zabbix, Datadog);
- Experience in object-oriented development. Should have strong software engineering foundations;
- Curiosity, ability to work independently and proactively identify solutions;
- Excellent verbal and written communication skills in a remote environment;
- Upper-intermediate English level.
Nice to haves
- Experience working in low-latency, real-time systems processing billions of events a day;
- Experience with data-engineering cloud technologies as Apache Airflow, K8S, Clickhouse, Snowflake, Redis, Spark, Caching technologies and/or Kafka.
AgileEngine is an Inc. 5000 company that creates award-winning software for Fortune 500 brands and trailblazing startups across 17+ industries. We rank among the leaders in areas like application development and AI/ML, and our people-first culture has earned us multiple Best Place to Work awards.
If you’re looking for a place to grow, make an impact, and work with people who care, we’d love to meet you!
About the project
The benefits of joining us
Professional growth
Accelerate your professional journey with mentorship, TechTalks, and personalized growth roadmaps
Competitive compensation
We match your ever-growing skills, talent, and contributions with competitive USD-based compensation and budgets for education, fitness, and team activities
A selection of exciting projects
Join projects with modern solutions development and top-tier clients that include Fortune 500 enterprises and leading product brands
Flextime
Tailor your schedule for an optimal work-life balance, by having the options of working from home and going to the office – whatever makes you the happiest and most productive.
Your AgileEngine journey starts here
2 min
Tell us about yourself
2 sec
Confirm requirements
30 - 60 min
Pass a short test
5 min
Record a short video
→ Introduce yourself on a video, instead of waiting for an interview
Live interview
Ace the technical interview with our team
→ Schedule a call yourself right away after your video is reviewed
Live interview
Final interview with your team
→ Get to know the team you will be working with
Get an offer
As quick as possible
