Data Engineer (Middle/Senior) ID34675

• What you will do

• Collaborate with engineers, architects, data scientists, analytics teams, and business stakeholders in an Agile setup;
• Build, and support both Cloud and On-Premise enterprise data infrastructure;
• Design and implement robust, reusable, and scalable data pipelines to automate ingestion, processing, and delivery of both batch and streaming data (structured and unstructured);
• Develop data APIs and delivery services to support critical business processes, analytical models, and ML applications;
• Assist with selecting and integrating data tools, frameworks, and applications to extend platform capabilities;
• Apply best practices for enterprise data management, including master data, reference data, metadata, data quality, and lineage.

• Key competencies

• Strong problem-solving and critical-thinking abilities;
• Proactive, ownership-driven approach to work;
• High attention to detail and quality;
• Ability to break down complex tasks and prioritize effectively;
• Comfortable working both independently and in a team setting;
• Clear and structured communication skills.

• Must haves

• 4+ years of experience as a Data Engineer;
• Hands-on experience with Python;
• Proven experience building data lakes, cloud data platforms (using GCP or AWS), ETL/ELT, and data integration;
• Proficiency with Snowflake and Apache Airflow;
• Expertise in developing distributed data processing and streaming frameworks (e.g., Apache Spark, Apache Beam, Apache Flink);
• Strong knowledge of NoSQL database technologies (e.g., MongoDB, BigTable, DynamoDB);
• Proficiency in build and deployment tools (e.g., Visual Studio, PyCharm, Git/Bitbucket, Bamboo, Maven, Jenkins, Nexus);
• Experience with containerized microservices and REST/GraphQL API development.

• Nice to haves

• Experience with Kafka, Cloud Pub/Sub;
• Expertise in database design methodologies (e.g., RDBMS, Document DBs, Star Schema, Kimball Model);
• Familiarity with integration and service frameworks (e.g., API Gateways, Apache Camel, Swagger, Zookeeper, messaging tools, microservices);
• Exposure to CI/CD tools and processes (e.g., Jenkins, Docker, Containers, OpenShift, Kubernetes);
• Experience working with financial services or wealth/asset management projects.

AgileEngine is one of the Inc. 5000 fastest-growing companies in the US and a top-3 ranked dev shop according to Clutch. We create award-winning custom software solutions that help companies across 15+ industries change the lives of millions.

If you like a challenging environment where you’re working with the best and are encouraged to learn and experiment every day, there’s no better place — guaranteed! 🙂