Tachyon 0.7.0 Release

We are excited to announce Tachyon v0.7.0, our largest release to date with a large number of new features, significant code base improvements, and a several stability enhancements and bug fixes. Major features and updates are listed below.

Downloads for the release can be found here: Downloads.

Native S3 Under Storage Support & Under Storage System Pluggability

Tachyon v0.7.0 provides a native S3 connector which can be used to communicate with any under storage that exposes the S3 interface. The under storage component has become modularized, simplifying the process of implementing new types of under storage connectors. The native S3 integration is a good example of the ease of implementing an under storage system submodule.

Swift Openstack Under Storage Support

Swift as another under storage system that is supported in the new release. The Swift under storage connector enables Tachyon to be easily deployed with any systems using Open Stack Swift as storage. A sample configuration for Swift is provided at conf/tachyon-env.sh.swift.

Remote Write Support

Remote memory write for Tachyon is now available, allowing any Tachyon client to store data directly into Tachyon without needing a local Tachyon Worker. Tachyon clients are no longer required to be co-located with Tachyon workers to enable writes into Tachyon storage. Remote writes in Tachyon allow greater flexibility in deployment environments.

Pluggable Evictors and Allocators

In this release, Tachyon exposes a richer set of internal information to customize allocators and evictors, allowing allocation and eviction policies to be highly customized according to the workloads. At the same time, these components are better modularized , and much easier to implement.

One Command Cluster Deployment

The deploy module now allows users to easily start an AWS or virtualbox cluster running Tachyon with a variety of under storage systems. The process has been greatly simplified, and setting up a cluster should take less than ten minutes. The module also provides the option to bundle additional systems that would run on top of Tachyon, such as Apache Spark. The deployment module provides an interface to define custom under storage or higher level systems to be deployed.

Metrics Reporting

Tachyon v0.7.0 comes with metrics reporting for clients, workers, and master. Metrics can be exposed through several configurable options, including JMX, Graphite, or web servlet. A sample configuration can be found at conf/metrics.properties.template. This new feature helps system admins manage and maintain Tachyon system, and also helps application developers optimize performance and troubleshoot.

Worker Web UI & UI Improvements

A worker web UI is now available, displaying worker specific information such as which file’s blocks are available in the Tachyon storage of the worker. This provides a significantly easier way for administrators to get ad hoc updates on the state of their cluster. Other improvements have been made to the UI with new features such as the log browser, which also greatly aids in cluster maintenance.

Enhanced Configuration

Tachyon v0.7.0 has the option to set parameters of Tachyon systems through a configuration file tachyon-site.properties as an alternative to specifying command line java options. Configurations are now much easier to manage, update, and validate with the new configuration file format.

Acknowledgements

  • Abhiraj Butala – Modularization of Tachyon Core, Code improvements
  • Albert Chu – Code and documentation improvements
  • Austin Chungath – Code improvements
  • Bin Fan – Modularization of Tachyon Core, Worker Improvements, Code and Documentation Improvements
  • Calvin Jia – Modularization of Tachyon Core, S3 Under Storage Connector, Worker Improvements
  • Carson Wang – Metrics Reporting, Worker Improvements, UI Features, Max Free Allocation Policy
  • Chang Cheng – Deploy Module Improvements, Worker Improvements, LRU Eviction Policy
  • Chaozhong Yang – Code improvements
  • David Capwell – Logger improvements
  • David Zhu – UI Features, Code improvements
  • Dina Leventol – Remote Reader improvements
  • Dibyendu Bhattacharya – Documentation improvements
  • Dong Lin – Code improvements
  • Elena Viter – Tachyon Client improvements
  • Gene Pang – Remote Write, Data Server improvements, Worker Improvements
  • Gil Vernik – Swift Under Storage Connector
  • Guoquan Sheng – Code improvements
  • Haoyuan Li – Modularization of Tachyon Core, Worker Improvements, Code and Documentation Improvements
  • Huang Xiaomeng – Code improvements
  • Henry Saputra – Tachyon Configuration
  • Jin Xin – Code improvements
  • Ku Liu – Code and documentation improvements
  • Kun Xu – Code improvements
  • Luo Li – Worker Improvements, Code improvements
  • Mingfei Shi – Worker Improvements, Code improvements
  • Nan Dun – Code and documentation improvements
  • Peng Shu – Worker Improvements, UI Features
  • Pengfei Xuan – Code and documentation improvements
  • Qianhao Dong – Worker Improvements
  • Qifan Pu – Code improvements, Worker Improvements
  • Rob Vesse – Modularization of Under Storage, UI Features, Code and documentation improvements
  • Robert Navado – Configuration Improvements
  • Rong Gu – Worker Improvements
  • Shaoshan Liu – Code and documentation improvements
  • Shiqing Fan – Code improvements
  • Siyuan He – Code improvements
  • Taka Shinagawa – Documentation improvements
  • Tian Chen – Tachyon Shell improvements
  • Tianyin Xu – Worker Improvements, Documentation improvements
  • Tingjun Xu – Deploy Module improvements
  • Vaishnav Kovvuri – Documentation improvements
  • Wang Haihua – Remote Reader improvements
  • Wang Tao – Dataserver improvements
  • Zhaohui Zhang – Code improvements
  • Zhongyi Xie – Code and Documentation improvements