Lead Data Engineer

Exp. Required: 8+ Year’s

Must Have Technical/Functional Skills

• 8+ years of data engineering experience, including technical leadership of complex data platforms or pipelines.

• Demonstrated proficiency processing unstructured and multimodal data, including video, images, LiDAR, GNSS, IMU, and telemetry.

• Proven experience designing architectures and pipelines for AI training dataset creation at scale.

• Advanced Python and SQL skills with hands-on experience in distributed processing and workflow orchestration.

• Experience with cloud data platforms, object storage, lakehouse patterns, metadata, lineage, governance, and dataset versioning.

• Strong understanding of data quality, observability, performance engineering, and production support.

Roles & Responsibilities

• Own the end-to-end architecture for ingestion, transformation, synchronization, validation, and publishing of multimodal sensor datasets.

• Define engineering standards, reusable frameworks, data contracts, metadata, lineage, and quality controls.

• Lead design reviews and guide engineers on implementation, testing, troubleshooting, and performance optimization.

• Partner with AI, perception, annotation, platform, and business stakeholders to prioritize requirements and milestones.

• Establish monitoring and operational controls for pipeline reliability, throughput, traceability, and cost.

• Lead root-cause analysis and remediation for complex data and pipeline defects.

• Maintain architecture documentation, technical roadmaps, delivery risks, and knowledge transfer.

Generic Managerial Skills, If any

Technical leadership, work planning, mentoring, stakeholder communication, design governance, risk management, and cross-functional coordination.