Collect TaskTracker Shuffle Output Bytes

Ambari.SCOM.PerformanceRule.TaskTrackerShuffleOutputBytes (Rule)

This rule collects number of bytes produced by shuffle running on Hadoop TaskTracker.

Knowledge Base article:

Summary

Collects number of bytes produced by shuffle running on Hadoop TaskTracker.

Element properties:

TargetAmbari.SCOM.HostComponent.TaskTracker
CategoryPerformanceCollection
EnabledTrue
Alert GenerateFalse
RemotableTrue

Member Modules:

ID Module Type TypeId RunAs 
DS DataSource Ambari.SCOM.Module.HostComponentMetricsPerformanceDataSource Default
WriteToDB WriteAction Microsoft.SystemCenter.CollectPerformanceData Default
WriteToDW WriteAction Microsoft.SystemCenter.DataWarehouse.PublishPerformanceData Default

Source Code:

<Rule ID="Ambari.SCOM.PerformanceRule.TaskTrackerShuffleOutputBytes" Target="Ambari.SCOM.HostComponent.TaskTracker" Enabled="true" ConfirmDelivery="false">
<Category>PerformanceCollection</Category>
<DataSources>
<DataSource ID="DS" TypeID="Ambari.SCOM.Module.HostComponentMetricsPerformanceDataSource">
<IntervalSeconds>900</IntervalSeconds>
<TimeoutSeconds>300</TimeoutSeconds>
<PropertyName>mapred.shuffleoutput.shuffle_output_bytes</PropertyName>
<CounterName>Shuffle Output Bytes</CounterName>
</DataSource>
</DataSources>
<WriteActions>
<WriteAction ID="WriteToDB" TypeID="SC!Microsoft.SystemCenter.CollectPerformanceData"/>
<WriteAction ID="WriteToDW" TypeID="SCDW!Microsoft.SystemCenter.DataWarehouse.PublishPerformanceData"/>
</WriteActions>
</Rule>