Senior Iceberg DBA

Job Title: Senior Iceberg DBA

Location: Remote Role

Interview: Video Interview

 

Description:

sistency, and availability across all Lakehouse workloads
• Maintain operational integrity for datasets at multi-terabyte to petabyte scale

Advanced Table Management & Optimization
• Execute advanced Iceberg table maintenance and optimization strategies:
o Compaction (minor/major) and small file mitigation
o Snapshot expiration and metadata compaction to control metadata growth
o Orphan file cleanup (vacuum) to maintain storage efficiency
• Optimize data layout and performance through:
o File size tuning and distribution strategies
o Partition evolution and pruning optimization
o Clustering and ordering techniques (e.g., Z-ordering or similar patterns)

Data Modeling Standards & Lakehouse Design Alignment
• Support and enforce data modeling best practices aligned with:
o Normalized data structures (3NF) for source-aligned datasets
o Medallion architecture (Bronze / Silver / Gold layers) for curated data flows
• Ensure Iceberg table design aligns with:
o Data ingestion patterns (raw vs curated layers)
o Downstream consumption and performance requirements
• Assist in structuring datasets to balance:
o Data integrity and normalization
o Query performance and analytical efficiency
• Work with data engineering teams to ensure consistent implementation of layered data architecture across multiple applications

Multi-Engine Query Performance & Consistency
• Ensure consistent and performant query behavior across:
o Spark (CDE)
o Hive / Impala (CDW)
• Troubleshoot and resolve:
o Query performance bottlenecks
o Metadata inconsistencies across engines
o Inefficient execution plans and scan patterns

Hive & Teradata Modernization Support
• Play a key role in enterprise data platform modernization (Hive and Teradata → Iceberg)
• Support:
o Schema alignment and data type mapping
o Data validation and reconciliation
• Troubleshoot migration-related issues and ensure post-migration stability and performance

Metadata & Data Lifecycle Management
• Manage Iceberg metadata to ensure:
o Efficient scaling and performance
o Consistent table state across engines
• Execute lifecycle operations:
o Data retention and archival policies
o Snapshot lifecycle management and cleanup
o Time-travel optimization and maintenance

Production Support, Incident Resolution & On-Call
• Provide L2/L3 support for data-related production issues across Iceberg-based Lakehouse workloads
• Participate in on-call rotation to support critical data platforms and ensure timely response to incidents
• Respond to and resolve P1/P2 production incidents within defined SLAs, minimizing impact to downstream applications and reporting
• Troubleshoot:
o Data inconsistencies and reporting discrepancies
o Query failures and performance degradation
• Perform root cause analysis (RCA) and implement preventive measures to avoid recurring issues
• Collaborate with platform and application teams during incident triage and resolution
Security & Data Governance Support
• Support fine-grained access control using:
o Ranger policies and RBAC
• Own and ensure data validation, reconciliation, and accuracy between source and Iceberg datasets
• Ensure secure and compliant access to data across applications

eye