Falconstor Community

You are here: FalconStor Blog Virtual Tape Library (VTL) Deduplication - The Power of Flexibility

Deduplication - The Power of Flexibility Featured

Rate this item
(1 Vote)
Today’s experienced IT staff with responsibility for storage know the importance of data deduplication and understand that there are trade-offs among the various deduplication options that are available in the market. It is important to select the type of deduplication that best meets their unique and changing  requirements, so I will discuss these.  It is important to note that these options should be available at the job level, so that IT staff can easily make adjustments for the many types of data they manage. To keep this blog reasonably short, we won’t attempt to define the types, but please drop us a note if you have questions on definitions.

Inline deduplication has the primary benefit of minimizing storage requirements, reducing them by as much as 40 percent. It is ideal for small storage configurations or environments where immediate replication is desired. For the highest performance levels, a recommended best practice is to use flexible deduplication policies to leverage post-process deduplication for the initial backup (for speed), and then switch to inline deduplication for subsequent backups.

Post-process deduplication is ideal when your key goal is to back up as quickly as possible. As its name implies, it occurs after the backup process completes, thus it can be scheduled to run at any time.

Concurrent deduplication runs in a similar fashion to post-processing, but it starts as soon as the first set of records has been written and runs concurrently with backup. Deduplication engines start working immediately, making full use of available CPU resources. This is highly suitable for clustered VTL environments. Because replication starts sooner, data can be quickly recovered from a remote site.

FalconStor Turbo deduplication improves the performance of post-process or concurrent deduplication through a unique hybrid approach that leverages inline deduplication processes from within the VTL itself rather than the deduplication engine. FalconStor Turbo deduplication works best in large and clustered environments with sufficient CPU power to support inline pre-processing. FalconStor Turbo deduplication reduces the I/O required for deduplication, improving the performance of post and concurrent processing by as much as 300 percent.

No deduplication can be used on data that does not deduplicate effectively or data that is exported to physical tape. Examples include image files and data that is pre-compressed and encrypted. By selectively “turning off” deduplication, you can save deduplication cycles and focus on deduplicating the data that will yield the highest overall ratios and value.

As with most things in life, “one size does not fit all.”  This is especially true for storage management and data deduplication. Take the time to understand your needs and then pick a product like FalconStor VTL Deduplication that does not handcuff you to only one type of deduplication.
Gary Parker

Gary Parker

FalconStor Product Marketing Manager

E-mail: This e-mail address is being protected from spambots. You need JavaScript enabled to view it
More in this category: « Prev Next »