Kafka why

  • Distributed, resilient, architecture, fault tolerant
  • Can scale to millions of messages
  • High performance - Latency less than 10ms - Real time

Use cases

  • Messaging system
  • Activity Tracking
  • Gather metrics from many different locations
  • Application log gathering
  • Stream processing (with the kafka streams api or spark)
  • De-coupling of system dependencies
  • Integrations with spark, flink, storm, hadoop, and many other big data technologies

Who uses kafka?

  • Netflix - uses kafka to apply recommendations in real-time while you are watching tv shows.
  • Uber uses kafka to gather user, taxi and trip data in real-time to compute and forecast demand and compute surge pricing in real-time
  • Linked in uses kafka to prevent spam, collect user interactions to make better connection recommendations in real time.
  • Kafka is only used as a transport mechanism! Developers still need to write code to do these things.

comments powered by Disqus