Data Engineer ID43407

• What you will do

• Develop and maintain secure and efficient data pipelines using DBT, PySpark, and Python in an Indeed-supported cloud environment, integrating all HR data sources to enable analytics and insights for HR COEs and HR leadership;
• Build the infrastructure and tooling required for optimal extraction, transformation, and loading (ETL) of data from various sources using Python, DBT, Terraform, and AWS technologies such as Glue, EMR , and S3;
• Identify, design, and implement internal process improvements, including automating manual workflows, debugging and optimizing data delivery, and re-architecting infrastructure for enhanced scalability;
• Apply the Data Quality Framework in the HRDM repository by developing unit tests, and creating both manual and automated test cases;
• Monitor data pipelines to ensure a minimum of 99.5% data availability;
• Create and maintain data models that are intuitive and easily consumable by business partners;
• Write and update project documentation (most data ingestion scripts are written in Python).

• Must haves

• 4+ years of professional experience as a Data Engineer;
• Proficiency in Python for object-oriented programming and data scripting;
• Hands-on experience with Snowflake;
• Strong experience with SQL;
• Experience using DBT for data transformations;
• Experience with Airflow;
• Familiarity with REST APIs for integration and data ingestion;
• Experience working with Google BigQuery;
• Proven track record of handling large-scale and sensitive data securely;
• Experience with real-time data processing (e.g., from Google Sheets or APIs);
• Strong documentation skills;
• Advanced English communication skills.

AgileEngine is one of the Inc. 5000 fastest-growing companies in the US and a top-3 ranked dev shop according to Clutch. We create award-winning custom software solutions that help companies across 15+ industries change the lives of millions.

If you like a challenging environment where you’re working with the best and are encouraged to learn and experiment every day, there’s no better place — guaranteed! 🙂