com.twitter.heron.integration_test.core

Class AggregatorBolt

java.lang.Object

com.twitter.heron.api.topology.BaseComponent

com.twitter.heron.api.bolt.BaseRichBolt

com.twitter.heron.integration_test.core.BaseBatchBolt

com.twitter.heron.integration_test.core.AggregatorBolt

All Implemented Interfaces:

IBolt, IRichBolt, IComponent, IBatchBolt, ITerminalBolt, java.io.Serializable

public class AggregatorBolt
extends BaseBatchBolt
implements ITerminalBolt

A Bolt which collects the tuples, converts them into json, and posts the json into the given http server.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
AggregatorBolt(java.lang.String httpPostUrl)

Method Summary

Modifier and Type	Method and Description
void	declareOutputFields(OutputFieldsDeclarer outputFieldsDeclarer) Declare the output schema for all the streams of this topology.
void	execute(Tuple tuple) Process a single tuple of input.
void	finishBatch() Invoke finishBatch() when the bolt is done, e.g.
void	prepare(java.util.Map<java.lang.String,java.lang.Object> map, TopologyContext topologyContext, OutputCollector outputCollector) Called when a task for this component is initialized within a worker on the cluster.
void	writeFinishedData() Invoke to write all aggregated data to the destination Destination can be http URL, local file, hdfs, etc.

Methods inherited from class com.twitter.heron.api.bolt.BaseRichBolt

cleanup

Methods inherited from class com.twitter.heron.api.topology.BaseComponent

getComponentConfiguration

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface com.twitter.heron.api.topology.IComponent

getComponentConfiguration

Constructor Detail

AggregatorBolt

public AggregatorBolt(java.lang.String httpPostUrl)

Method Detail

finishBatch

public void finishBatch()

Description copied from interface: IBatchBolt

Invoke finishBatch() when the bolt is done, e.g. receives corresponding terminal signals

Specified by:

finishBatch in interface IBatchBolt

prepare

public void prepare(java.util.Map<java.lang.String,java.lang.Object> map,
                    TopologyContext topologyContext,
                    OutputCollector outputCollector)

Description copied from interface: IBolt

Called when a task for this component is initialized within a worker on the cluster. It provides the bolt with the environment in which the bolt executes.

This includes the:

Specified by:

prepare in interface IBolt

Parameters:

map - The Heron configuration for this bolt. This is the configuration provided to the topology merged in with cluster configuration on this machine.

topologyContext - This object can be used to get information about this task's place within the topology, including the task id and component id of this task, input and output information, etc.

outputCollector - The collector is used to emit tuples from this bolt. Tuples can be emitted at any time, including the prepare and cleanup methods. The collector is thread-safe and should be saved as an instance variable of this bolt object.

execute

public void execute(Tuple tuple)

Description copied from interface: IBolt

Process a single tuple of input. The Tuple object contains metadata on it about which component/stream/task it came from. The values of the Tuple can be accessed using Tuple#getValue. The IBolt does not have to process the Tuple immediately. It is perfectly fine to hang onto a tuple and process it later (for instance, to do an aggregation or join).

Tuples should be emitted using the OutputCollector provided through the prepare method. It is required that all input tuples are acked or failed at some point using the OutputCollector. Otherwise, Heron will be unable to determine when tuples coming off the spouts have been completed.

For the common case of acking an input tuple at the end of the execute method, see IBasicBolt which automates this.

Specified by:

execute in interface IBolt

Parameters:

tuple - The input tuple to be processed.

declareOutputFields

public void declareOutputFields(OutputFieldsDeclarer outputFieldsDeclarer)

Description copied from interface: IComponent

Declare the output schema for all the streams of this topology.

Specified by:

declareOutputFields in interface IComponent

Parameters:

outputFieldsDeclarer - this is used to declare output stream ids, output fields, and whether or not each output stream is a direct stream

writeFinishedData

public void writeFinishedData()

Description copied from interface: ITerminalBolt

Invoke to write all aggregated data to the destination Destination can be http URL, local file, hdfs, etc.

Specified by:

writeFinishedData in interface ITerminalBolt