Data Engineer with Clickhouse

Job Description

Mandatory Primary Skills: Hadoop, Python, Databricks, Click house DB , ADLS, SQL + Azure cloud technologies above all with Azure (such as ADF, Databricks, Log Analytics, Logic App, Key Vault, Blob Storage, ADLS, etc.)

Secondary Skills: Snowflake ,

Detailed JD as below.

• We run a near real-time analytics platform for mobile network data, ingesting billions of rows a day from Azure Data Lake/HDFS into Click House and serving subscriber-level queries back to internal APIs in tens of milliseconds. We are expanding the platform footprint to support additional use cases by adding more datasets
• Data Engineer would work on: extending the Python ingestion framework that loads from Azure Data Lake/HDFS; table design covering partitioning, sort keys, TTL and materialized views; operational diagnosis of merge behaviour, part counts and replication lag; and integration with Snowflake and Databricks Iceberg alongside the primary feeds.
• We need strong analytical SQL and experience with a columnar or MPP database — Click House, Big Query, Redshift or similar.
• Python sufficient to own a production ingestion framework rather than write scripts.
• Comfort at hundreds of billions of rows, since the failure modes differ from smaller systems.
• Cloud object storage and Parquet. Enough Kubernetes to tell whether a problem is the database or the platform underneath it.
• Strongly preferred: direct Click House experience, particularly Merge Tree engines and aggregate function states, and telecom data such as DNS , IPFIX, etc.
• Click House specifically is uncommon, and we would rather have a strong engineer from an adjacent system than a weak one with the exact keyword.