Hadoop Data Engineer

Title

Job Description

Skillsets

Typical Experience

Feature Engineer (Sr)

• Design and implement scalable, reusable feature pipelines (batch and real-time)
• Develop complex feature transformations and advanced data modeling logic
• Optimize feature performance, latency, and cost efficiency
• Ensure feature quality, validation, and SLAs (freshness, accuracy, reliability)
• Collaborate with Data Science and ML Engineering teams to align features with use cases
• Contribute to feature store architecture and standards
• Mentor Feature Engineers and promote engineering best practices
• Support production deployment, monitoring, and incident resolution

Technical Skills

Programming: Advanced Python and SQL

Distributed Processing: Spark / Flink (large-scale data processing)

Feature Engineering: Advanced transformations, feature design patterns

Data Modeling: Complex transformations, aggregation strategies

Feature Stores: Hands-on with platforms such as Hopsworks, Feast, SageMaker

ML Lifecycle Understanding: Feature importance, model input optimization

Data Quality & Validation: Drift detection, validation frameworks

Platform & Engineering

CI/CD pipelines and automated testing

Cloud platforms (Azure / AWS / Google Cloud Platform)

Monitoring, observability, and production debugging

Performance tuning and scalability optimization

Soft Skills

Technical leadership and mentoring

Cross-team collaboration (Data Science, MLOps, Platform)

Strong problem-solving and optimization mindset

Ability to translate business use cases into feature logic

• 3 10+ years in Data Engineering, Feature Engineering, or ML Engineering
• Proven experience designing production-grade data/feature pipelines
• Strong track record in scalable distributed data systems
• Experience working in enterprise AI/ML platforms or feature stores
• Prior mentoring or technical leadership experience