Back to Jobs
Recently postedTechnology

Principal Software Engineer (Data Platform)

S Linx LLCDallas, TX🇺🇸United States

Quick overview

SeniorityLeader
Work modeOn Site
LocationDallas, TX, United States
SQLApache SparkAWSETLAirflowApacheData PipelineKafkaPythonKubernetesCI/CD

Job description

Position: Principal Software Engineer – Data Platform
Work Location: Dallas, TX & San Antonio, TX
Work Mode: Closer to one of these locations, requires to be onsite 3 days a week
Duration: Minimum 1 Year
Key responsibilities:
  • Define technical direction and engineering standards for production-quality Python applications, reusable components, packaging, configuration, logging, testing, debugging, and maintainability.
  • Provide architecture leadership for scalable ETL/ELT pipelines, Spark-based processing, workflow orchestration, and Kubernetes-native data workloads.
  • Well versed with streaming applications Kafka.
  • Able to build optimized pipelines to ingest data.
  • Establish reference patterns and reusable frameworks that improve developer productivity, platform consistency, self-service, and operational efficiency.
  • Drive architecture and engineering decisions that improve data pipeline reliability, performance, scalability, observability, and recoverability.
  • Lead resolution of systemic software, pipeline, infrastructure, performance, and data-quality issues spanning multiple technology layers and teams.
  • Establish cloud-native development, CI/CD, DevOps, source-control, code-review, and engineering-governance practices for the Software Engineering workstream.
  • Guide Software Engineers and Senior Software Engineers through complex design decisions and ensure solutions remain maintainable, supportable, and reusable.
  • Partner with data consumers, application teams, architecture, and platform engineering to ensure data-platform capabilities support enterprise use cases and reliable dataset delivery.
  • Establish responsible patterns for AI-assisted software development and support solutions, with appropriate technical validation and engineering controls.
  • Lead capability transfer and documentation so client teams can operate, extend, and support delivered solutions without vendor dependency.
Required skills and experience:
  • Strong Python software development experience, including object-oriented programming, classes, modules, functions, parameters, exception handling, and reusable component design.
  • Proficiency with Python logging, configuration management, command-line parameters, debugging, and error-handling patterns.
  • Experience writing unit, integration, and regression tests for Python applications and transformation logic.
  • Knowledge of software design principles, maintainable coding practices, source control, code reviews, and Python packaging.
  • Strong SQL development and query optimization.
  • Data engineering and ETL/ELT pipeline development experience.
  • Experience with Apache Spark and distributed data processing.
  • Python experience for data transformation and automation.
  • Experience with workflow orchestration platforms such as Apache Airflow or similar.
  • Knowledge of data modeling and data warehousing concepts.
  • Experience with Kubernetes and containerized data workloads.
  • Experience with cloud-native platforms and AWS services.
  • Experience with CI/CD and DevOps practices.
  • Experience with observability, monitoring, and operational analytics.
  • Knowledge of data quality, lineage, and governance practices.
  • Demonstrated focus on automation and operational excellence.
  • Expert-level software engineering and architecture capability with demonstrated ability to establish Python, data-platform, cloud-native, and DevOps patterns and standards across teams.
  • Demonstrated enterprise data-platform engineering leadership, including complex architecture decisions, systemic problem solving, technical governance, and capability transfer.

Similar jobs