How do you offload workloads from Hadoop?

What is Apache Hadoop If you’re new to building big data applications, Apache Hadoop is a distributed framework for managing data processing and storage for big data applications running in clustered systems. It consists of 5 modules – a distributed file system (aka HDFS or Hadoop Distributed File System), MapReduce for parallel processing of datasets, … Continued

Tags: , ,

How do you orchestrate data between disparate storages?

As the data ecosystem within enterprises grow larger and larger, not only do we see an increase in total data volumes but also an increase in the disparate storage systems in which they are housed. The challenge then becomes how do different applications and teams have an efficient way of being able to access data … Continued

Tags: ,

Why am I seeing AWS S3 500 / 501 errors?

Increasingly S3 is being used as a data store for analytical and machine learning workloads. This means that it is very easy to generate a massive amount of get operations and request data from S3. For example: a couple of commands can launch a 1000 node cluster of AWS EMR service with the Spark or … Continued

Tags: ,