If you start it using the standard startup script, Cassandra will listen for connections on port 8080 (port 7199 starting in 0.8.0-beta1) to view and tweak variables which it exposes via JMX. This may be helpful for debugging and monitoring. See also JmxGotchas.
The MemtableThresholds page describes how to use Jconsole as a client for this.
Domain |
Source Location |
Type |
Keyspace |
Cache |
Attributes |
Operations |
./concurrent/JMXEnabledThreadPoolExecutorMBean.java |
|
|
||||
|
||||||
./concurrent/JMXEnabledThreadPoolExecutorMBean.java |
|
|
||||
|
||||||
./concurrent/JMXEnabledThreadPoolExecutorMBean.java |
|
|
||||
|
||||||
./concurrent/JMXEnabledThreadPoolExecutorMBean.java |
|
|
||||
|
||||||
./concurrent/JMXEnabledThreadPoolExecutorMBean.java |
|
|
||||
|
||||||
./concurrent/JMXEnabledThreadPoolExecutorMBean.java |
|
|
||||
|
||||||
./concurrent/JMXEnabledThreadPoolExecutorMBean.java |
|
|
||||
|
||||||
./concurrent/JMXEnabledThreadPoolExecutorMBean.java |
|
|
||||
|
||||||
./concurrent/JMXEnabledThreadPoolExecutorMBean.java |
|
|
||||
|
||||||
./concurrent/JMXEnabledThreadPoolExecutorMBean.java |
|
|
||||
|
||||||
./concurrent/JMXEnabledThreadPoolExecutorMBean.java |
|
|
||||
|
||||||
./concurrent/JMXEnabledThreadPoolExecutorMBean.java |
|
|
||||
|
||||||
./concurrent/JMXEnabledThreadPoolExecutorMBean.java |
|
|
||||
|
||||||
./concurrent/JMXEnabledThreadPoolExecutorMBean.java |
|
|
||||
|
||||||
./concurrent/JMXEnabledThreadPoolExecutorMBean.java |
|
|
||||
|
||||||
./concurrent/JMXEnabledThreadPoolExecutorMBean.java |
|
|
||||
|
||||||
./concurrent/JMXEnabledThreadPoolExecutorMBean.java |
|
|
||||
|
||||||
./concurrent/JMXEnabledThreadPoolExecutorMBean.java |
|
|
||||
|
||||||
./db/CommitLogExecutorServiceMBean.java |
|
|
||||
|
||||||
./db/CompactionManagerMBean.java |
|
|
BytesCompacted |
|||
|
||||||
./gms/FailureDetectorMBean.java |
|
|
|
|||
./service/StorageServiceMBean.java |
|
|
CurrentGenerationNumber |
|||
clearSnapshot |
||||||
./service/StorageProxyMBean.java |
|
|
RangeOperations |
|||
|
||||||
./streaming/StreamingService.java |
|
|
||||
All org.apache.cassandra.concurrent Types are thread pool executors which have Attributes ActiveCount, CompletedTasks and PendingTasks. They are viewable from the command line with {{'}}nodeprobe tpstats{{'}}.
They are generally defined as follows :
Type |
Units |
Range |
Notes |
int |
tasks |
integer >=0 |
The current number of running tasks. |
Type |
Units |
Range |
Notes |
long |
tasks |
integer >=0 |
The number of completed tasks since executor creation. |
Type |
Units |
Range |
Notes |
long |
tasks |
integer >=0 |
The number of tasks waiting in the queue to be executed. |
This thread pool is single threaded.
Per src/java/org/apache/cassandra/service/AntiEntropyService.java,
AntiEntropyService encapsulates "validating" (hashing) individual column families, exchanging MerkleTrees with remote nodes via a TreeRequest/Response conversation, and then triggering repairs for disagreeing ranges. Tree comparison and repair triggering occur in the single threaded AE_SERVICE_STAGE. |
This pool has 4 threads, which is hardcoded per src/java/org/apache/cassandra/service/StorageService.java,
private static int consistencyThreads_ = 4; // not configurable |
The CONSISTENCY-MANAGER executor is responsible for passive read repair, for example after a read with ConsistencyLevel.ONE.
Per src/java/org/apache/cassandra/service/StorageService.java,
/* This thread pool does consistency checks when the client doesn't care about consistency */ |
This thread pool is single threaded. Code is defined in src/java/org/apache/cassandra/io/DeletionService.java. This executor is responsible for deleting files, such as obsolete sstables and obsolete commitlog segments.
This thread pool is multi-threaded, with between 1 and Runtime.getRuntime().availableProcessors() threads in the pool.
See Also : FLUSH-WRITER-POOL and MEMTABLE-POST-FLUSHER. Per src/java/org/apache/cassandra/db/ColumnFamilyStore.java,
* submitFlush first puts [Binary]Memtable.getSortedContents on the flushSorter executor, * which then puts the sorted results on the writer executor. This is because sorting is CPU-bound, * and writing is disk-bound; we want to be able to do both at once. When the write is complete, * we turn the writer into an SSTableReader and add it to ssTables_ where it is available for reads. * * For BinaryMemtable that's about all that happens. For live Memtables there are two other things * that switchMemtable does (which should be the only caller of submitFlush in this case). * First, it puts the Memtable into memtablesPendingFlush, where it stays until the flush is complete * and it's been added as an SSTableReader to ssTables_. Second, it adds an entry to commitLogUpdater * that waits for the flush to complete, then calls onMemtableFlush. This allows multiple flushes * to happen simultaneously on multicore systems, while still calling onMF in the correct order, * which is necessary for replay in case of a restart since CommitLog assumes that when onMF is * called, all data up to the given context has been persisted to SSTables. |
This thread pool is multi-threaded, with between 1 and Runtime.getRuntime().availableProcessors() threads in the pool.
See Also : FLUSH-SORTER-POOL and MEMTABLE-POST-FLUSHER. Per src/java/org/apache/cassandra/db/ColumnFamilyStore.java,
* submitFlush first puts [Binary]Memtable.getSortedContents on the flushSorter executor, * which then puts the sorted results on the writer executor. This is because sorting is CPU-bound, * and writing is disk-bound; we want to be able to do both at once. When the write is complete, * we turn the writer into an SSTableReader and add it to ssTables_ where it is available for reads. * * For BinaryMemtable that's about all that happens. For live Memtables there are two other things * that switchMemtable does (which should be the only caller of submitFlush in this case). * First, it puts the Memtable into memtablesPendingFlush, where it stays until the flush is complete * and it's been added as an SSTableReader to ssTables_. Second, it adds an entry to commitLogUpdater * that waits for the flush to complete, then calls onMemtableFlush. This allows multiple flushes * to happen simultaneously on multicore systems, while still calling onMF in the correct order, * which is necessary for replay in case of a restart since CommitLog assumes that when onMF is * called, all data up to the given context has been persisted to SSTables. |
This thread pool is single threaded. Executor for the Gossiper.
This thread pool is single threaded.
Per src/java/org/apache/cassandra/db/HintedHandOffManager.java,
* For each table (keyspace), there is a row in the system hints CF. * SuperColumns in that row are keys for which we have hinted data. * Subcolumns names within that supercolumn are host IPs. Subcolumn values are always empty. * Instead, we store the row data "normally" in the application table it belongs in. * * So when we deliver hints we look up endpoints that need data delivered * on a per-key basis, then read that entire row out and send it over. |
This thread pool is single threaded.
See Also : LB-TARGET. Per src/java/org/apache/cassandra/service/StorageLoadBalancer.java,
/* This thread pool is used for initiating load balancing operations */ |
This thread pool is single threaded.
See Also : LB-OPERATIONS. Per src/java/org/apache/cassandra/service/StorageLoadBalancer.java,
/* This thread pool is used by target node to leave the ring. */ |
This thread pool is single threaded.
This thread pool is single threaded.
See Also : FLUSH-SORTER-POOL and FLUSH-WRITER-POOL. Per src/java/org/apache/cassandra/db/ColumnFamilyStore.java,
* submitFlush first puts [Binary]Memtable.getSortedContents on the flushSorter executor, * which then puts the sorted results on the writer executor. This is because sorting is CPU-bound, * and writing is disk-bound; we want to be able to do both at once. When the write is complete, * we turn the writer into an SSTableReader and add it to ssTables_ where it is available for reads. * * For BinaryMemtable that's about all that happens. For live Memtables there are two other things * that switchMemtable does (which should be the only caller of submitFlush in this case). * First, it puts the Memtable into memtablesPendingFlush, where it stays until the flush is complete * and it's been added as an SSTableReader to ssTables_. Second, it adds an entry to commitLogUpdater * that waits for the flush to complete, then calls onMemtableFlush. This allows multiple flushes * to happen simultaneously on multicore systems, while still calling onMF in the correct order, * which is necessary for replay in case of a restart since CommitLog assumes that when onMF is * called, all data up to the given context has been persisted to SSTables. |
This thread pool is multi-threaded, with between 1 and Runtime.getRuntime().availableProcessors() threads in the pool.
Per src/java/org/apache/cassandra/net/MessagingService.java,
// read executor puts messages to deserialize on this. |
This thread pool is single threaded.
This thread pool is mutli-threaded and has between 1 and Runtime.getRuntime().availableProcessors() threads in the pool.
This thread pool is multi-threaded and has between 1 and concurrectWriters threads in the pool.
Per src/java/org/apache/cassandra/concurrent/StageManager.java,
private static int concurrentWriters_ = 32; |
This thread pool is multi-threaded and has between 1 and concurrectReaders threads in the pool.
Per src/java/org/apache/cassandra/concurrent/StageManager.java,
private static int concurrentReaders_ = 8; |
This stage is single threaded.
Type |
Units |
Range |
Notes |
int |
tasks |
>=0 |
The number of tasks which are currently executing. |
Type |
Units |
Range |
Notes |
long |
tasks |
>=0 |
The number of completed tasks. |
Type |
Units |
Range |
Notes |
long |
tasks |
>=0 |
The number of tasks waiting in the queue to be executed. |
Type |
Units |
Range |
Notes |
java.lang.Long |
bytes |
integer >=0 |
The number of bytes successfully compacted. |
Type |
Units |
Range |
Notes |
java.lang.Long |
bytes |
integer >=0 |
FIXME?? : The total size of the SSTables involved in the current compaction. |
Type |
Units |
Range |
Notes |
java.lang.string |
name |
Any valid Column{{`Family name |
The name of the Column}}`Family currently being compacted. |
Type |
Units |
Range |
Notes |
int |
SSTables |
>=0 |
The maximum number of SSTables in the compaction queue before compaction kicks off. |
Type |
Units |
Range |
Notes |
int |
SSTables |
>=0 |
The minimum number of SSTables in the compaction queue before compaction kicks off. |
Type |
Units |
Range |
Notes |
int |
tasks |
>=0 |
The number of tasks waiting in the queue to be executed. |
Dump endpoint arrival windows to a file in /var/tmp, per src/java/org/apache/cassandra/gms/FailureDetector.java:
FileOutputStream fos = new FileOutputStream("/var/tmp/output-" + System.currentTimeMillis() + ".dat", true); |
Type |
Units |
Range |
Notes |
int |
Generation Number |
>=0 |
The number of the current generation in the Gossiper. |
Type |
Units |
Range |
Notes |
java.util.Set |
Nodes |
n/a |
A set of the nodes which are visible and live, from the perspective of this node. |
Type |
Units |
Range |
Notes |
Example |
java.util.Map |
Nodes,Disk Usage |
n/a |
A map of which nodes have what level of load. |
{10.0.0.130=107.86 GB,10.0.0.41=125.82 GB,10.0.0.176=117.47 GB,10.0.0.15=68.65 GB,10.0.0.91=148.25 GB,10.0.0.165=247.33 GB] |
Type |
Units |
Range |
Notes |
Example |
java.lang.String |
Disk usage |
n/a |
The amount of load on the node being queried. |
127.7 GB |
Type |
Units |
Range |
Notes |
Examples |
java.lang.String |
Operation mode string |
n/a |
A string describing the current operation mode. (FIXME: wikilink on operation mode?) |
Decommissioned, Joining: getting bootstrap token, Joining: getting load information, Leaving: streaming data to other nodes, Normal, Joining: sleeping " + RING_DELAY + " for pending range setup, Leaving: sleeping " + RING_DELAY + " for pending range setup |
Type |
Units |
Range |
Notes |
Example |
java.lang.String |
key range start position |
n/a |
A string describing the start of the range of keys this node is responsible for on the ring. |
|
Type |
Units |
Range |
Notes |
Example |
||
<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="093d369a-b21e-4d82-9d4f-d28292ffdc99"><ac:plain-text-body><![CDATA[ |
java.util.Set |
Nodes |
n/a |
A set of the nodes which this node knows about which are currently unreachable. |
[10.0.0.1,10.0.0.2] |
]]></ac:plain-text-body></ac:structured-macro> |
Arguments |
Return Type |
Notes |
n/a |
void |
Clear all the snapshots for this node's keyspace. |
Per src/java/org/apache/cassandra/db/Table.java :
String snapshotPath = dataDirPath + File.separator + name + File.separator + SNAPSHOT_SUBDIR_NAME; ... FileUtils.deleteDir(snapshotDir); |
Arguments |
Return Type |
Notes |
n/a |
void |
Instruct this (live) node to remove itself from the token ring. |
This operation may fail if :
Arguments |
Return Type |
Notes |
n/a |
void |
Trigger a cleanup compaction. Goes over each file and removes the keys that the node is not responsible for, and only keeps keys that this node is responsible for. |
Arguments |
Return Type |
Notes |
n/a |
void |
Trigger a major compaction (of all SSTables on disk). |
Argument p1 |
Argument p1 Type |
Argument p2 |
Argument p2 Type |
Return Type |
Notes |
Example |
tableName |
java.lang.String |
columnFamilies |
java.lang.String... |
void |
Flush all memtables for a table and column families. |
|
Argument p1 |
Argument p1 Type |
Argument p2 |
Argument p2 Type |
Return Type |
Notes |
Example |
tableName |
java.lang.String |
columnFamilies |
java.lang.String... |
void |
Trigger proactive repair for a table and column families |
|
Argument p1 |
Argument p1 Type |
Argument p2 |
Argument p2 Type |
Return Type |
Notes |
Example |
tableName |
java.lang.String |
token |
java.lang.String |
java.lang.String |
Attempts to return N endpoints that are responsible for storing the specified key i.e for replication. |
|
Argument p1 |
Argument p1 Type |
Argument p2 |
Argument p2 Type |
Return Type |
Notes |
Example |
tableName |
java.lang.String |
token |
java.lang.String |
java.lang.String |
Returns the N endpoints that are responsible for storing the specified key i.e for replication. |
|
Argument p1 |
Argument p1 Type |
Return Type |
Notes |
Example |
keyspace |
java.lang.String |
java.lang.String |
For a keyspace, return the ranges and corresponding hosts for a given keyspace. |
|
Arguments |
Return Type |
Notes |
n/a |
void |
Generate new auto-assigned token for this node, between the two most heavily loaded nodes. |
Argument p1 |
Argument p1 Type |
Return Type |
Notes |
keyspace |
java.lang.String |
void |
Move this node to a new token, specified as the argument. |
This operation may fail if :
Argument p1 |
Argument p1 Type |
Return Type |
Notes |
keyspace |
java.lang.String |
void |
Remove the specified token from the ring. Used on a live node to remove the token of a dead node from all nodes in the ring. |
This operation may fail if :
Argument p1 |
Argument p1 Type |
Argument p2 |
Argument p2 Type |
Return Type |
Notes |
Example |
classQualifier |
java.lang.String |
rawLevel |
java.lang.String |
void |
Set the level of logging in Log4j. |
DEBUG INFO WARN ERROR FATAL ALL OFF - http://jakarta.apache.org/log4j/docs/api/index.html |
Argument p1 |
Argument p1 Type |
Return Type |
Notes |
Example |
tag |
java.lang.String or null |
void |
Take a snapshot of all keyspaces on this node and optionally name it with a (non-null) tag name. |
|
Argument p1 |
Argument p1 Type |
Argument p2 |
Argument p2 Type |
Return Type |
Notes |
Example |
tableName (Keyspace Name) |
java.lang.String |
tag |
java.lang.String or null |
void |
Take a snapshot of a keyspace and optionally name it with a (non-null) tag name. |
|
Type |
Units |
Range |
Notes |
long |
Operations |
>=0 |
The number of range operations since executor start. |
Type |
Units |
Range |
Notes |
long |
Operations |
>=0 |
The number of read operations since executor start. |
Type |
Units |
Range |
Notes |
double |
Microseconds |
>=0 |
The latency of range operations since the last time this attribute was read. |
See Also : RecentReadLatencyMicros, RecentWriteLatencyMicros.
Per src/java/org/apache/cassandra/utils/Latency{{`Tracker.}}`java/getRecentLatencyMicros :
long ops = opCount.get(); long n = totalLatency.get(); return ((double)n - lastLatency) / (ops - lastOpCount); ... lastLatency = n; lastOpCount = ops; |
Type |
Units |
Range |
Notes |
double |
Microseconds |
>=0 |
The latency of range operations since the last time this attribute was read. |
See Also RecentRangeLatencyMicros, RecentWriteLatencyMicros.
Type |
Units |
Range |
Notes |
double |
Microseconds |
>=0 |
The latency of write operations since the last time this attribute was read. |
See Also : RecentRangeLatencyMicros, RecentReadLatencyMicros.
Type |
Units |
Range |
Notes |
long |
Microseconds |
>=0 |
The latency of all range operations since executor start. |
Type |
Units |
Range |
Notes |
long |
Microseconds |
>=0 |
The latency of all read operations since executor start. |
Type |
Units |
Range |
Notes |
long |
Microseconds |
>=0 |
The latency of all write operations since executor start. |
Type |
Units |
Range |
Notes |
long |
Operations |
>=0 |
The number of write operations since executor start. |
Type |
Units |
Range |
Notes |
Example |
java.lang.String |
Status string |
n/a |
A string describing the current state of the StreamingService. |
Flushing memtables for <tableName>, Performing anticompaction, Sending a stream initiate message to <target>, Done with transfer to <target> |
Type |
Units |
Range |
Notes |
Example |
java.lang.Set |
Nodes |
n/a |
A set of the nodes this node is currently streaming to. |
|
Type |
Units |
Range |
Notes |
Example |
java.lang.Set |
Nodes |
n/a |
A set of the nodes this node is currently streaming from. |
|
Argument p1 |
Argument p1 Type |
Return Type |
Notes |
Example |
node |
java.lang.String |
java.Util.List |
Given a node as an argument, return a List of filenames, current position in the file and how many total bytes are expected in the stream. Shows incoming streams only. |
|
See Also : getOutgoingFiles.
Per src/java/org/apache/cassandra/streaming/StreamingService.java :
files.add(String.format("%s %d/%d", f.getFilename(), f.getPtr(), f.getExpectedBytes())); |
Argument p1 |
Argument p1 Type |
Return Type |
Notes |
Example |
node |
java.lang.String |
java.Util.List |
Given a node as an argument, return a List of filenames, current position in the file and how many total bytes are expected in the stream. Shows outgoing streams only. |
|
See Also : getIncomingFiles.
https://c.statcounter.com/9397521/0/fe557aad/1/|stats