Monitor Kafka Streams Applications in Confluent Cloud

Confluent Cloud provides tools to monitor and manage your Kafka Streams applications.

Prerequisites

  • Access to Confluent Cloud.
  • A provisioned Kafka cluster in Confluent Cloud.

Step 1: Create a Kafka Streams application

  • To create a Kafka Streams application, follow the steps in Kafka Streams Quick Start.
  • Or you can run an existing Kafka Streams application that uses Apache Kafka® topics in Confluent Cloud.

Step 2: Monitor the application

  1. Log in to Confluent Cloud Console and navigate to your Kafka cluster.

  2. In the navigation menu, click Kafka Streams.

    The Kafka Streams page displays a list of all the Kafka Streams applications running in your Kafka cluster. The displayed metrics are aggregated across all the Kafka Streams applications in your Kafka cluster.

  3. In the list, find the application you want to monitor. You can search for the application by name.

    The displayed metrics include:

    • Application Name: The name of the Kafka Streams application.
    • Client version: The version of the client used by the application.
    • Running threads: The number of threads running in the application.
    • Total production: The total number of messages produced by the application in the last minute.
    • Total consumption: The total number of messages consumed by the application in the last minute.
    • Total lag: The total lag of the application.
  4. Click View consumer group to open the Consumer Lag page.

    The Consumer Lag page displays the consumer group metrics for the application.

    The displayed metrics include:

    • Consumer group ID: The ID of the consumer group.
    • Total messages behind: The total number of messages behind for the consumer group.
    • Consumer lag: The lag for each topic partition.

    For more information, see Monitor Confluent Cloud Clients.