Steven Matison

Hello, My name is Steven Matison and I am a Senior Solutions Engineer at Cloudera, Inc. Check out my journey with Cloudera Data Platform (CDP).

3 minute read

The Data In Motion team is thrilled to announce the release of Cloudera Streams Messaging - Kubernetes Operator 1.4 and the exciting new Cloudera Surveyor! This release marks a significant step forward in simplifying and enhancing your Apache Kafka management experience on Kubernetes, focusing on intuitive user experience and powerful new capabilities.

Release Highlights

Introducing Cloudera Surveyor: Your New Go-To for Seamless Kafka Management on Kubernetes

Cloudera Surveyor is a brand-new, intuitive graphical interface built to streamline the monitoring and management of your Apache Kafka clusters running on Kubernetes. As a core component delivered with Cloudera Streams Messaging - Kubernetes Operator 1.4, Surveyor empowers you with:

  • Effortless Management: Move beyond the command line with a modern, easy-to-use UI for all your Kafka operations.
  • Multi-Cluster Visibility: Manage and monitor multiple Kafka clusters from a single, centralized console.
  • Actionable Insights: Get real-time health and performance metrics that help you quickly identify and resolve issues.
  • Enhanced Security: Benefit from built-in TLS, robust LDAP authentication, and enforcement of Kafka ACLs.
  • Enterprise Scalability: Designed to handle large-scale deployments, Surveyor scales to manage hundreds of Kafka clusters, brokers, and topics.
  • Broad Kafka Compatibility: Manage diverse Kafka deployments consistently across on-premises, cloud, or Kubernetes environments.

Cloudera Streams Messaging - Kubernetes Operator 1.4: The Foundation for Your Kubernetes-Native Kafka

Cloudera Streams Messaging - Kubernetes Operator 1.4 provides the robust and reliable foundation necessary for deploying and managing Kafka clusters, including the new Cloudera Surveyor. This update brings continued stability and improvements to your containerized Kafka deployments, ensuring a seamless experience for upgrades and operations within your Kubernetes environments.

Unified Data In Motion Portfolio: Beyond Kafka

This launch further strengthens Cloudera’s comprehensive Data In Motion portfolio. While Cloudera Streams Messaging - Kubernetes Operator and Cloudera Surveyor significantly enhance your Kafka experience, our solutions span the entire real-time data lifecycle. Our portfolio also includes powerful offerings for data ingestion and transformation with Apache NiFi, and advanced stream processing and analytics with Apache Flink, providing a complete ecosystem for your event-driven architectures.

Who Benefits from Cloudera Surveyor and When?

Cloudera Surveyor is designed to bring clarity, control, and efficiency to anyone managing Apache Kafka on Kubernetes, whether you’re an administrator, a DevOps engineer, or a developer.

For Kafka Administrators & SREs (Site Reliability Engineers):

  • Daily Operations & Health Monitoring: Easily oversee the health of all your Kafka clusters, brokers, and topics from a single dashboard. Quickly spot anomalies, monitor resource utilization, and ensure the continuous availability of your streaming infrastructure.
  • Proactive Troubleshooting: When issues arise, drill down into performance metrics, consumer group lag, and topic details with an intuitive UI. Identify bottlenecks, misconfigurations, or problematic consumers faster, significantly reducing mean time to resolution (MTTR).
  • Scaling & Capacity Planning: Gain insights into topic growth, partition distribution, and broker load to inform your scaling decisions and optimize resource allocation across your Kubernetes environment.

For DevOps & Platform Engineers:

  • Streamlined Deployments & Upgrades: While Cloudera Streams Messaging - Kubernetes Operator handles the heavy lifting, Surveyor provides immediate visual confirmation of your deployed Kafka clusters, enabling quick validation post-deployment or upgrade.
  • Consistent Multi-Cluster Management: For organizations running numerous Kafka clusters (e.g., for different departments, environments, or applications), Surveyor offers a centralized management plane, drastically simplifying oversight and ensuring consistent configurations and security policies.

For Developers working with Kafka:

  • Understanding Topic & Consumer Behavior: Quickly inspect topics, view message counts, and monitor consumer group progress. This is invaluable for debugging applications, verifying data flow, and understanding how new features or changes impact the Kafka ecosystem during development and testing phases.
  • Security & Access Management: Easily review and manage Kafka ACLs directly through the UI, ensuring that applications and users have appropriate access to topics and consumer groups, without resorting to complex command-line operations.

Resources

Ready to explore Cloudera Streams Messaging - Kubernetes Operator 1.4 and Cloudera Surveyor? Find out more here:

New - Cloudera Surveyor Documentation

New - Cloudera Streams Messaging - Kubernetes Operator 1.4 Release Notes

Cloudera Streaming Product Page

Accelerate Streaming Pipeline Deployments with New Kubernetes Operators (webinar recording)

As always, check out the entire DOCS for the CSM Operator.

You may also enjoy