Why you need a file system storage architecture, and what to look for
by Stefan Bernbo, founder and CEO, Compuverde
Organizations are looking for ways to scale to meet exponentially increasing data storage needs. The digital transformation of both business and society have pushed traditional storage architectures past their limits.
Even if organizations had the budget to keep scaling linearly, it would take too long to meet current needs. Companies need solutions that won’t break the bank or take a long time to implement.
Vertical storage architecture contains bottlenecks that slow performance to an unacceptable level – even with the addition of multiple servers.
However, because software-defined storage (SDS) separates the programming that controls storage-related tasks from the physical storage hardware, it significantly reduces costs associated with hardware.
In this scenario, fewer, less-expensive servers can be used to improve both capacity and performance. Administration is simplified and made more flexible and efficient. SDS enables users to allocate and share storage assets across all workloads. For this reason, the industry has embraced SDS.
Gartner recently reported that by 2020, anywhere from 70 to 80 percent of unstructured data will be stored and managed on lower-cost hardware supported by software-defined storage.
What’s in a File System?
While it is widely understood that unstructured data is best managed with a file system, for some reason, many SDS offerings focus solely on block or object store. Few offerings focus on file systems or do them well. Without a file system overlaying this data, it becomes very difficult to manage the data.
Here’s a quick review of the purpose of the three kinds of storage:
- Block: Considered the foundation of storage, block is used for storing virtual machines or databases, However, you need files as well to deal with all the unstructured data.
- Object: This type of storage is used for machine-to-machine/IoT transactions and other applications that require extreme scalability. However, it isn’t much better than block when it comes to managing data.
- File systems: Though it gets less press, file storage is the best for handling unstructured data.
SDS providers are not ignorant of the need for file systems, so some claim to provide file system with their offerings. However, these file systems are usually based on Samba (see below) and fail to provide some of the features most Windows users are used to.
Because so many users find themselves in need of a file system after purchasing an SDS solution, they turn to open source software like Samba, which enables support for SMB and allows end users to access and use files on the company’s intranet or network. However, providing file services through Samba often means going without needed features.
File systems don’t stand on their own; they also need file-related features to deal with unstructured data. These features include:
- Retention, allowing you to automatically create a single folder or a hierarchy of folders on file servers, which can be deleted according to assigned policies.
- Snapshot, a read-only copy of the contents of a file system or independent file set taken at a single point in time. When a snapshot of an independent file set is taken, all files and nested dependent file sets will be included in the snapshot.
- Tiering uses a policy to let you to define where a specific file is to be placed and if and when the file will be migrated between file system pools. You can define both file placement and migration policies. By using a policy, you create a filter that designates a specific file type to a particular tier. Tiered storage is more efficient and boosts performance.
- Quota, to help keep track of the amount of storage being used. You can set a soft limit quota that will warn you when part of a file system is close to reaching its storage limit but still allow data to be saved. If you set up a hard limit quota, after the quota is reached, no new data can be saved.
Eighty percent of data is unstructured, and only file system storage can manage that data. It remains a conundrum of the industry that SDS vendors, fully aware of this fact, typically offer an anemic file system. Customers look elsewhere for help, but freeware file systems don’t provide all the needed features.
A comprehensive solution to meet today’s storage needs includes all three storage types, able to handle structured and unstructured data. Don’t just take the provider’s word for it when they say there’s a file system; take the time to check under the hood to ensure you’re getting a solution that will meet your storage demands.