Today EMC announced upcoming updates to OneFS, the operating system for the Isilon scale-out storage platform. The new version of the software will offer new capabilities including:
- Support for Hadoop’s HDFS 2.0
- Object Storage
- Extended RESTful APIs with support for EMC Atmos, Amazon S3, and OpenStack’s Swift
- Support for EMC’s newly-announced software-defined storage platform, ViPR
- Online Block-Level Data Deduplication
- New Audit Security features
I’ll break down each of the new features below.
Hadoop HDFS 2.0
Isilon OneFS will allow for simultaneous support for HDFS 1.0 and 2.0, allowing the same data to be exported either way. This will not only allow customers an easy way to test new Hadoop distributions before actually converting to 2.0, but will also allow that conversion to go smoothly with minimal downtime.
The new Hadoop integration will offer full support for Pivotal HD (formerly Greenplum HD).
You can argue that Isilon offers object storage today through the REST APIs added in OneFS 7.0, but the new update will expand upon this functionality and bring true object store capability to the Isilon platform.
In the update Isilon incorporates a REST Object Access to Namespace interface which will enable automated provisioning and management of storage. Adding this support for objects allows customers to consolidate their object and file storage into a single, easy-to-manage, scalable system. Object access will occur with the same high performance as Isilon file access.
Extended REST APIs
The new extended REST APIs in the updated OneFS allow for easy cloud-extensibility of the Isilon platform. By integrating full support for:
- EMC Atmos
- Amazon S3
- OpenStack Swift
customers will be able to easily move both files and objects between their Isilon cluster and any of these cloud platforms.
EMC Software-Defined Storage — ViPR
EMC’s newly-announced software-defined storage layer, ViPR (formerly known as “Project Bourne”) will act as the go-between allowing object access from Isilon to the platforms mentioned above.
Online Block-Level Data Deduplication
The new version of OneFs adds simple block-based deduplication of stored data. Administrators can easily set deduplication policies based on individual directiories or storage pools.
Data deduplication will be done post-process, which means that new files and objects will initially be stored on Isilon full-size. Later, a background job will scan these files and objects for duplicate blocks, both within the individual entity, but also across all deduplicated entities throughout the entire Isilon cluster.
The deduplication granularity is a fixed block size of 8KB. Depending on the nature of the dataset, EMC predicts customers can expect up to a 30% reduction in need storage capacity, power and cooling, and rack space needed.
The update to OneFS will add new enhancements to help ensure regulatory compliance. The main addition enabling this will be full integration with the EMC Common Event Enabler (CEE). CEE enables audit applications to record all file accesses occuring via SMB and/or CIFS. This further allows for easy report generation and chain-of-custody tracking. With these features, customers wil be better able to meet regulations such as HIPAA and Sarabanes-Oxley. At least as importantly, customers will be able to demonstrate to an auditor that they are, in fact, in compliance with these regulations.
As part of the announcement, EMC spoke of collaboration between Isilon and VCE. Existing VCE partners will be able to sell Isilon alongside a Vblock as part of an overall solution, for example, to provide home directory space for a Vblock acting as VDI infrastructure.
EMC said that in the future the Vblock and Isilon will be able to be managed through a single interface, so I expect that a future release of the VCE Vision management software will offer support for managing Isilon clusters.
EMC says the new version of OneFS will be delivered “later this year”. Earlier rumors had placed the target date as June, but I’ve been unable to confirm that.