Quick Start Guide

Run a single-node Heron cluster on your laptop

The easiest way to get started learning Heron is to install and run pre-compiled Heron binaries, which are currently available for:

  • Mac OS X
  • Ubuntu >= 14.04

For other platforms, you need to build from source. Please refer to Heron Developers.

Step 1 — Download Heron binaries using installation scripts

Go to the releases page for Heron and download two installation scripts for your platform. The names of the scripts have this form:

  • heron-client-install-0.14.0-PLATFORM.sh
  • heron-tools-install-0.14.0-PLATFORM.sh

The installation scripts for Mac OS X (darwin), for example, would be named heron-client-install-0.14.0-darwin.sh and heron-tools-install-0.14.0-darwin.sh.

Once you’ve downloaded the scripts, run the Heron client script with the --user flag set:

$ chmod +x heron-client-install-VERSION-PLATFORM.sh $ ./heron-client-install-VERSION-PLATFORM.sh --user Heron client installer ---------------------- Uncompressing...... Heron is now installed! Make sure you have "/usr/local/bin" in your path. ...

To add /usr/local/bin to your path, run:

$ export PATH=$PATH:/usr/local/bin

Now run the script for Heron tools (setting the --user flag):

$ chmod +x heron-tools-install-VERSION-PLATFORM.sh $ ./heron-tools-install-VERSION-PLATFORM.sh --user Heron tools installer --------------------- Uncompressing...... Heron Tools is now installed! ...

To check Heron is successfully installed, run:

$ heron version heron.build.version : 0.14.0 heron.build.time : Tue May 24 22:44:01 PDT 2016 heron.build.timestamp : 1464155053000 heron.build.host : ${HOSTNAME} heron.build.user : ${USERNAME} heron.build.git.revision : be87b09f348e0ed05f45503340a2245a4ef68a35 heron.build.git.status : Clean INFO: Elapsed time: 0.001s.

Step 2 — Launch an example topology

If you set the --user flag when running the installation scripts, some example topologies will be installed in your ~/.heron/examples directory. You can launch an example topology locally (on your machine) using the Heron CLI tool:

$ heron submit local \ ~/.heron/examples/heron-examples.jar \ # The path of the topology's jar file com.twitter.heron.examples.ExclamationTopology \ # The topology's Java class ExclamationTopology \ # The name of the topology --deploy-deactivated # Deploy in deactivated mode INFO: Launching topology 'ExclamationTopology' ... [2016-06-07 16:44:07 -0700] com.twitter.heron.scheduler.local.LocalLauncher INFO: \ For checking the status and logs of the topology, use the working directory \ $HOME/.herondata/topologies/local/${ROLE}/ExclamationTopology # working directory INFO: Topology 'ExclamationTopology' launched successfully INFO: Elapsed time: 3.409s.

This will submit the topology to your locally running Heron cluster but it won’t activate the topology. That will be explored in step 5 below.

Note the output shows if the topology has been launched successfully and the working directory.

To check what’s under the working directory, run:

$ ls -al ~/.herondata/topologies/local/${ROLE}/ExclamationTopology -rw-r--r-- 1 username role 2299 Jun 7 16:44 ExclamationTopology.defn -rw-r--r-- 1 username role 5 Jun 7 16:44 container_1_exclaim1_1.pid -rw-r--r-- 1 username role 5 Jun 7 16:44 container_1_word_2.pid drwxr-xr-x 11 username role 374 Jun 7 16:44 heron-conf drwxr-xr-x 4 username role 136 Dec 31 1969 heron-core -rwxr-xr-x 1 username role 2182564 Dec 31 1969 heron-examples.jar -rw-r--r-- 1 username role 5 Jun 7 16:44 heron-executor-0.pid -rw-r--r-- 1 username role 0 Jun 6 13:33 heron-executor.stderr -rw-r--r-- 1 username role 17775 Jun 7 16:44 heron-executor.stdout -rw-r--r-- 1 username role 5 Jun 7 16:44 heron-shell-0.pid -rw-r--r-- 1 username role 5 Jun 7 16:44 heron-tmaster.pid drwxr-xr-x 25 username role 850 Jun 7 16:44 log-files -r--r--r-- 1 username role 4506 Jun 8 12:05 metrics.json.metricsmgr-0.0 -rw-r--r-- 1 username role 5 Jun 7 16:44 metricsmgr-0.pid -r-xr-xr-x 1 username role 279 Dec 31 1969 release.yaml -rw-r--r-- 1 username role 5 Jun 7 16:44 stmgr-1.pid

All instances’ log files can be found in log-files under the working directory:

$ ls -al ~/.herondata/topologies/local/${ROLE}/ExclamationTopology/log-files total 1018440 -rw-r--r-- 1 username role 94145427 Jun 8 12:06 container_1_exclaim1_1.log.0 -rw-r--r-- 1 username role 75675435 Jun 7 16:44 container_1_word_2.log.0 -rw-r--r-- 1 username role 187401024 Jun 8 12:06 gc.container_1_exclaim1_1.log -rw-r--r-- 1 username role 136318451 Jun 8 12:06 gc.container_1_word_2.log -rw-r--r-- 1 username role 11039 Jun 8 11:16 gc.metricsmgr.log -rw-r--r-- 1 username role 300 Jun 7 16:44 heron-shell.log -rw-r--r-- 1 username role 29631 Jun 7 16:44 heron-ExclamationTopology-scheduler.log.0 -rw-r--r-- 1 username role 2382215 Jun 7 15:16 heron-stmgr-stmgr-1.username.log.INFO -rw-r--r-- 1 username role 5976 Jun 7 16:44 heron-tmaster-ExclamationTopology2da9ee6b-c919-4e59-8cb0-20a865f6fd7e.username.log.INFO -rw-r--r-- 1 username role 12023368 Jun 8 12:06 metricsmgr-0.log.0

Step 3 — Start Heron Tracker

The Heron Tracker is a web service that continuously gathers information about your Heron cluster. You can launch the tracker by running the heron-tracker command (which is already installed):

$ heron-tracker ... Running on port: 8888 ... Using config file: $HOME/.herontools/conf/heron_tracker.yaml

You can reach Heron Tracker in your browser at http://localhost:8888 and see something like the following upon successful submission of the topology: alt tag

To explore Heron Tracker, please refer to Heron Tracker Rest API

Step 4 — Start Heron UI

Heron UI is a user interface that uses Heron Tracker to provide detailed visual representations of your Heron topologies. To launch Heron UI:

$ heron-ui ... Running on port: 8889 ... Using tracker url: http://localhost:8888

You can open Heron UI in your browser at http://localhost:8889 and see something like this upon successful submission of the topology: alt tag

To play with Heron UI, please refer to Heron UI Usage Guide

Step 5 — Explore topology management commands

In step 2 you submitted a topology to your local cluster. The heron CLI tool also enables you to activate, deactivate, and kill topologies and more.

$ heron activate local ExclamationTopology $ heron deactivate local ExclamationTopology $ heron kill local ExclamationTopology

Upon successful actions, a message similar to the following will appear:

INFO: Successfully activated topology 'ExclamationTopology' INFO: Elapsed time: 1.980s.

For more info on these commands, read about topology lifecycles.

To list the available CLI commands, run heron by itself:

usage: heron ... Available commands: activate Activate a topology deactivate Deactivate a topology help Prints help for commands kill Kill a topology restart Restart a topology submit Submit a topology version Print version of heron-cli For detailed documentation, go to http://heronstreaming.io

To invoke help output for a command, run heron help COMMAND. Here’s an example:

$ heron help submit usage: heron submit [options] cluster/[role]/[environ] topology-file-name topology-class-name [topology-args] Required arguments: cluster/[role]/[env] Cluster, role, and environ to run topology topology-file-name Topology jar/tar/zip file topology-class-name Topology class name Optional arguments: --config-path (a string; path to cluster config; default: "$HOME/.heron/conf") --config-property (key=value; a config key and its value; default: []) --deploy-deactivated (a boolean; default: "false") -D DEFINE Define a system property to pass to java -D when running main. --verbose (a boolean; default: "false")

Step 6 — Explore other example topologies

The source code for the example topologies can be found on GitHub. The included example topologies:

  • AckingTopology.java — A topology with acking enabled.
  • ComponentJVMOptionsTopology.java — A topology that supplies JVM options for each component.
  • CustomGroupingTopology.java — A topology that implements custom grouping.
  • ExclamationTopology.java — A spout that emits random words to a bolt that then adds an explanation mark.
  • MultiSpoutExclamationTopology.java — a topology with multiple spouts.
  • MultiStageAckingTopology.java — A three-stage topology. A spout emits to a bolt that then feeds to another bolt.
  • TaskHookTopology.java — A topology that uses a task hook to subscribe to event notifications.

Troubleshooting

In case of any issues, please refer to Quick Start Troubleshooting.

Next Steps