Create Cassandra Integration

[andresrc]

Metrics

The integration should collect performance characteristics of a Cassandra cluster for the following:

• System metrics
  Disk usage, CPU usage, Memory usage (JVM heap, non-heap, and total memory used by Cassandra).
• GC metrics
  Total number of collections that have occurred (since JVM start), estimated cumulative collection elapsed time (since JVM start), last collection duration.
• Compaction
  Compaction metrics for knowing when to add capacity to your cluster.
• Thread pool and read/write latency statistics
  Increases in pending tasks on thread pool statistics can indicate when to add additional capacity.
• Table statistics
  Provides a number of metrics that are important for monitoring performance trends.

More information can be found in Cassandra monitoring docs.

Logs

Parse Cassandra system logs and ensure that the extracted fields are ECS compatible. See the example grok patterns here.

Package / Dataset creation or update checklist

This checklist is intended for Devs which create or update a package to make sure they are consistent.

All Changes

• Change follows development guidelines
• Supported versions of the subject being monitored are documented
• Supported operating systems are documented (if applicable)
• System tests exist
• Documentation
• Fields follow ECS and naming conventions
• At least a manual test with ES / Kibana / Agent has been performed.
• The required Kibana version is set to the lowest version used in the manual test.

Dashboards

• Dashboards exists (if applicable)
• Screenshots of added / updated dashboards
• Datastream filters added to visualizations

Log datasets

• Pipeline tests exist (if applicable)
• Test log files exist for the grok patterns
• Generated output for at least 1 log file exists

Metric datasets

This entry is currently recommended. It will be mandatory once we provide better support for it.

• Sample event (sample_event.json) exists

New Packages

• Screenshot of the Fleet "Add Integration" Page.

[elasticmachine]

Pinging @elastic/integrations (Team:Integrations)

[mtojek]

Referring to our conversation today about Jolokia endpoint.

I did a quick investigation on the Beats side and we have ActiveMQ, Tomcat and Kafka modules using Jolokia endpoint to scrap metrics. In all cases we don't differentiate level keys, but just assume the flat model:

See:

Tomcat requests
Tomcat threading
Tomcat memory
ActiveMQ broker
Kafka consumer

In terms of Cassandra integration (org.apache.cassandra.metrics:type=Table,keyspace=*,scope=*,name=*) I guess it would be valuable to let the user define these levels using UI (default: verbose ones disabled), and consider keyspace, scope and name as standard fields (I hope they can be scraped using Jolokia input).

[akshay-saraswat]

Cassandra Package has been merged. We are tracking the promotion of this package via the following Github issue. Closing this issue to remove redundancy and confusion.
#1438

[akshay-saraswat]

My bad. I misunderstood. That's a meta issue, we still need this card to track the progress of integration promotion. Reopning it.

[masci]

Package is on production EPR, closing