Technical Lead/Sr SW Engineer – Infrastructure Telemetry & Capacity Planning Platform
Quick overview
Job description
Location: San Francisco, CA (Onsite)
Duration: Contract - W2
Pay: $95/hr W2
Job Description:
We are seeking an experienced Technical Lead contractor to deliver hands-on software development for our Infrastructure Telemetry and Capacity Planning platform. This engagement covers the collection, streaming, processing, and persistence of telemetry data from various sources that contains utilization of servers and GPUs across our fleet, enabling efficient querying and dashboarding for capacity planning. The ideal contractor hits the ground running, engages actively in architecture design discussions, and drives execution with minimal ramp time.
- Scope of Engagement — The contractor will serve as an active technical contributor and platform specialist working closely with cross-functional teams, focused on the design, development, and delivery of an end-to-end telemetry pipeline that turns raw server and GPU metrics into a reliable, queryable capacity planning backbone.
- Telemetry Pipeline Design & Architecture — Participate in end-to-end solution design for streaming telemetry ingestion from servers (utilization, memory, thermal, power, and workload metrics), contributing to data flow models, schema design, and error-handling strategies. Recommend appropriate patterns (streaming vs. batch, event-driven ingestion, EMR-based processing) based on data volume, cardinality, and latency requirements for capacity planning use cases.
- Development & Implementation — Develop production-grade ingestion and processing pipelines following team coding standards, peer review practices, and CI/CD workflows. Build streaming collectors/agents for server telemetry, implement processing jobs on EMR (Spark), and persist processed data in NoSQL stores like Trino-queryable Lakehouse formats (e.g., Parquet). Write adequate test coverage and instrument pipelines with logging and monitoring for operational observability.
Required Qualifications:
- 10 years of software development experience, with demonstrated technical execution on complex, large-scale data or infrastructure platforms
- Strong proficiency in Java and/or Python; hands-on experience with Spark and distributed data processing
- Deep, hands-on experience with AWS EMR (or Spark) for large-scale data processing pipelines
- Experience with streaming data ingestion technologies (e.g., Kafka, Flink) for high-volume telemetry data.
- Solid experience with NoSQL data stores (e.g., DynamoDB, MongoDB) OR Lakehouse/query engines such as Trino, including schema and partitioning strategies for efficient querying at scale
- Experience collecting and processing infrastructure telemetry using tools such as Prometheus, Grafana.
- Proven expertise in cloud-native architecture — high availability, extensibility; AWS preferred
- Familiarity with containerized deployment environments (Kubernetes, Docker) and CI/CD pipelines
- Strong experience using AI-assisted development tools (e.g., Cursor, Windsurf, or equivalent) to accelerate delivery and maintain code quality
- Strong analytical and problem-solving skills with the ability to operate independently in an ambiguous environment
Preferred Qualifications:
- Working knowledge of capacity planning concepts — utilization forecasting, resource trending, and translating telemetry into actionable capacity signals
- Experience building dashboards and visualization layers for operational and reporting.
- Experience with Agile development methodologies and DevOps practices.
Will your resume get past this employer's screening? Check your resume against this job, free
Similar jobs
- DL
Swine HerdsmanRecently posted
Danner Livestock · Chalmers, IN · Hybrid · Posted 19 Nov 2026 🇺🇸
PigSalary not listed - EX
IT Inventory AssociateRecently posted
Excelraise LLC · Kennesaw, GA · On-site · Posted 20 Nov 2026 🇺🇸
Microsoft ExcelMicrosoft OfficeInventory ControlSalary not listed - BT
Product AnalystRecently posted
Boston Technology Corporation · Columbus, OH · Hybrid · Posted 20 Nov 2026 🇺🇸
Business AnalysisSalary not listed - SS
Websphere Integration Tech-Remote(USA)-W2 Role -Exp 12+yrsRecently posted
SSTech LLC · United States · Remote · Posted 20 Nov 2026 🇺🇸
MicroservicesGitAWS+16Salary not listed - SO

Performance ConsultantRecently posted
System One · Fort Meade, MD · On-site · Posted 20 Nov 2026 🇺🇸
Stakeholder ManagementPortfolio ManagementRisk Management+2Salary not listed - SH

Set Your Own Schedule - Shop and Deliver GroceriesRecently posted
Shipt · Ogden, IL · Hybrid · Posted 20 Nov 2026 🇺🇸
AndroidSalary not listed - SH

Set Your Own Schedule - Shop and Deliver GroceriesRecently posted
Shipt · Champaign, IL · Hybrid · Posted 20 Nov 2026 🇺🇸
AndroidSalary not listed - SH

Set Your Own Schedule - Shop and Deliver GroceriesRecently posted
Shipt · Seymour, IL · Hybrid · Posted 17 Nov 2026 🇺🇸
AndroidSalary not listed - TD

.NET ARCHITECTRecently posted
The Dignify Solutions, LLC · United States · Hybrid · Posted 20 Nov 2026 🇺🇸
DockerMicroservicesSQL+7Salary not listed - AC

ADS & ADAS TechnicianRecently posted
AVA Consulting · Ann Arbor, MI · Hybrid · Posted 20 Nov 2026 🇺🇸
ADASASE CertificationVehicle DynamicsSalary not listed - TE
Administrative Athletic ManagerRecently posted
TECKNOMIC LLC · Washington, DC · On-site · Posted 20 Nov 2026 🇺🇸
CanvaSchedulingInstagramSalary not listed - MU

AI-Enabled Remediation Engineer(Linux)Recently posted
MphasiS Corporation USA · Alpharetta, GA · Hybrid · Posted 20 Nov 2026 🇺🇸
DockerGitAWS+16Salary not listed