Senior DevOps / Data Platform Engineer

 Job Responsibilities:

• Infrastructure & Automation: Build, automate, and maintain AWS infrastructure (S3, networking, security) using Terraform to minimize manual deployment tasks.
• Data Ingestion & CDC: Support the onboarding of SQL Server and MongoDB data sources into the data lake. Configure, deploy, and troubleshoot Fivetran connectors and CDC-based database replication for both initial loads and ongoing pipelines.
• Data Storage Engineering: Build and maintain data lake storage patterns using Amazon S3 and Parquet.
• Pipeline & Platform Utilities: Develop Python scripts and utilities for operational validation, platform automation, schema checks, and deployment tasks.
• CI/CD & Operational Health: Build and maintain CI/CD pipelines for data platform components. Implement monitoring, alerting, logging, and health checks across the pipeline.
• Troubleshooting & Operations: Diagnose issues across source databases, Fivetran connectors, AWS infrastructure, and downstream pipelines. Document deployment procedures, runbooks, and configuration standards.
• Client Onboarding: Support production deployments and the onboarding of additional client environments.

Job Description:

• AWS Expertise: Hands-on experience with S3, VPC/networking, IAM security, and infrastructure automation.
• Infrastructure as Code (IaC): Strong experience with Terraform and automated environment provisioning.
• CI/CD: Experience building and maintaining pipelines (e.g., GitHub Actions, GitLab CI, Jenkins).
• Python Development: Strong Python scripting capabilities for automation, data validation, and platform tooling.
• Database Experience: Practical experience working with SQL Server and MongoDB.
• Data Storage: Deep understanding of Parquet file formats and S3 data lake storage patterns.
• Operations & Observability: Solid background in production operations, logging, alerting, and end-to-end incident troubleshooting.

Preferred Skills:

• Hands-on experience with Fivetran (connector deployment, CDC configuration, and troubleshooting).
• Experience with Snowflake or data lakehouse architectures, table formats, and data catalogs.
• Familiarity with database replication technologies and high-volume production databases.
• Experience supporting multi-tenant or multi-client enterprise data platforms.

Education:

• Bachelor's or Master’s degree in Computer Science, Computer or Electrical Engineering, Mathematics, or a related field.