TUTORIAL: Getting Started with Alluxio &
5 MIN TUTORIAL
Among the many ways to deploy Alluxio on AWS, one of the simplest approaches is to use AWS CloudFormation. With a few clicks, all the resources needed for an Alluxio cluster will be automatically modeled and provisioned. You can define the type of Alluxio cluster you want and how you want to configure it. This tutorial outlines the steps to use the Alluxio CloudFormation template to provision a cluster, including setting up a cluster in high availability mode.
Familiarity with Cloudformation and EC2 is helpful but not required. The tutorial launches an Alluxio cluster from the perspective of a user with a newly created AWS account.
Note that the launched instances do not qualify for free usage tier. The default instance type we will be using, r4.xlarge, costs only 7 cents an hour.
LAUNCH AN ALLUXIO CLUSTER
This section outlines how to launch an Alluxio cluster with a single master node.
Step 0: Create an EC2 Key Pair
Note: If you have valid EC2 KeyPair already, skip ahead to step 1.
Step 1: Choose the CloudFormation Template to Launch
Step 2: Specify Stack Details
Step 3: Stack options
Step 4: Review and Launch
STEP 5: Explore the Alluxio cluster
After the Alluxio cluster is created successfully, we can interact with the cluster now!
The following instructions are different based on whether or not you choose high availability mode.
Restart Alluxio cluster
To restart the Alluxio cluster, run the following commands in each master node as the
$ /opt/alluxio/bin/alluxio-start.sh -a master $ /opt/alluxio//bin/alluxio-start.sh -a job_master $ /opt/alluxio//bin/alluxio-start.sh -a proxy
And run the following commands in each worker node as the
$ /opt/alluxio/bin/alluxio-start.sh -a worker $ /opt/alluxio//bin/alluxio-start.sh -a job_worker $ /opt/alluxio//bin/alluxio-start.sh -a proxy
Launch an Alluxio cluster with spot worker instances
By default, all instances inside the Alluxio cluster will be launched on demand. EC2 instances can also be launched as spot instances, saving a significant portion of the instance cost at the risk of having these instances terminated and reclaimed by EC2 at any time. Alluxio masters are critical to the cluster and should not be launched as spot instances. In contrast, worker instances fit the use case of spot instances because new workers can register themselves to the cluster and old workers will be marked as lost. Because the addition and loss of workers do not affect basic Alluxio functionality, we can support launching Alluxio clusters with spot instances for workers.
Creating an Alluxio cluster with spot worker instances requires a different Alluxio CloudFormation Template. The template url is https://alluxio-public.s3.amazonaws.com/cft/AlluxioCFTSpotInstance.json
The configuration is the same with the addition of the parameter WorkerSpotPrice, which specifies the maximum hourly price that you are willing to pay for spot instances.
Note that the price should be set according to the worker instance type. If the price is too low, Alluxio workers may not be fully provisioned and the stack will show as