Principal Data Engineer

Real-time market data at scale

S&P Global Platts / UBS

Market data does not wait for a nightly load. AWS data-lake ingestion, real-time and batch, with Kafka Streams, Flink and Spark Streaming landing in Cassandra, Neo4j and MongoDB, at S&P Global Platts, UBS and Nuclear Risk Insurers.

Real-time and batch: AWS data lake
Streaming: Kafka / Flink / Spark
Stores: Cassandra / Neo4j / MongoDB

What was built

AWS data lake with real-time and batch ingestion. Kafka Streams, Flink and Spark Streaming into Cassandra, Neo4j and MongoDB. The same pattern across S&P Global Platts, UBS and Nuclear Risk Insurers.

AWS · Kafka Streams · Flink · Spark Streaming · Cassandra · Neo4j · MongoDB

Bring one workflow. We will tell you how we would approach it.

Book a 30-minute call