News
December, 2011 How-To Bulletin #2
Posted by Jordan Steinke on 12 December 2011 10:17 AM
Introduction:

To all of our customers and reseller partners;

Winter is thought of as a time when everyone can relax and slow down, worrying about that looming deadline after the New Year. Yes, we rolled our eyes at that too. Our industry demands that we meet every deadline head-on, and this means that our SANs must always be at peak performance.  This month has two features to keep the proverbial fires burning through the coldest months of the year.

This month's main story from Roger Beck features our powerful DataMover, and should prove to be of great value to anyone dealing with file transfers from internal or external devices. 

As in our previous issue, our director of sales, Shawn Clark, has a brief message for everyone, and we've persuaded our Software Quality Assurance Engineer, Ryan, to write a short blurb about all of the exciting changes he has seen over the course of User Guide editions for DataMover.

Firstly, please read on below for Roger's DataMover trick.
How to Feature By Roger Beck:

Situation:
I have to transfer data from my local drive or an external drive into my workspace. This should go blazing fast but sometimes I wish I could throttle the transfer speed because I don't want to affect my other activities. I've tried several tools like rsync, cvcp or optimized graphical Windows copy tools. Even worse, I have to optimize my copied data after the transfer to get good performance for playback. 

Cause:
As said last month for bccp already, most bundled graphical or command line based tools are not sequence aware, they hand the content over how it's laid down on the source file system. There is no real rule how files on a non-sequence aware file system get allocated, hence this issue affects all OS platforms and none is better than the other. In addition to commercial copy tools you can add the fact, that they split the file into several chunks to be able to use more then one copy thread which boosts the performance. Of course however, those tools have to assemble the transferred blocks at the workspace. While there is no control which block arrives first, your single file becomes truncated and will end up with many many extents.

Solution:
Bright's DataMover is a powerful and highly configurable command line engine which will manage your data in the way you want it. There are several advanced options which can increase or decrease the performance of the data transfer as you need it. Of course DataMover is sequence aware to guarantee the best layout, literally you get what you see, not just filtered by a file system to sort it first and make it look nice.

The basic copy command which keeps the permissions, ownership and works recursively would be
bdm -rp /mnt/disk_drive/project/shot-1 /mnt/ol01/vip/trailer/.

To speed up the transfer you can increase the number of buffers and threads. Those options will depend on the client itself like OS, memory and such.
bdm -rp -b 13m -t6 /mnt/disk_drive/project/shot-1 /mnt/ol01/vip/trailer/.
-b 13m = use a buffer of 13mb which matches almost a single 2K dpx file
-t 6 = use 6 worker threads to be used

Now imagine the workspace is busy and all resources are more or less assigned to the attached clients but you need to ingest data and you want to get this job done before you forget to do it.
bdm -rp -M 50m  /mnt/disk_drive/project/shot-1 /mnt/ol01/vip/trailer/.
-M 50m = use only 50Mb/s for the data transfer and get the job done so I can focus on other tasks.

In some cases the workspace is quite churned, with fragmented, randomized and interleaved files, and transfer speeds might be slow. The pre-allocation process takes a long time.  The workspace should have been manually optimized if BrightClip isn't installed. However, sometimes there isn't time for optimizing the workspace or dedicated folders and the performance can become bad. Even under these conditions, DataMover has an option which will give some benefit.
bdm -rp -b13m --no-contig /mnt/disk_drive/project/shot-1 /mnt/ol01/vip/trailer/
--no- contig = don't look for the largest contiguous chunk and hand the files as fast as possible and in sequence over to the workspace.   

Conclusion:
These examples show just a few of the available options in DataMover, and there are almost no limits to creating scripts around your common tasks and letting your creativity explore the advantages of this robust command line engine.


Back to Contents
A Brief History of DataMover By Ryan Spano:

I've been given the chance to feed extensive product information into Bright's user guide documentation. It's a great opportunity to complement the knowledge base material we have on our support site, and yet provide it in a more formal manner. We have a new document format that looks great (the first example of it being the DataMover 10-003.A user guide available in the Downloads section of the support site). We look forward to providing Bright documentation in this well-presented format with more usage details about our products.

For example, soon we'll have a command option which will be added to BDM that allows the user to select a data transfer algorithm that makes use of specific StorNext file system features. Using this new option will ensure that the best method is being used for moving or copying large media files.  We're hoping to release this software update soon, along with an updated user guide to include a description of the new command option.

We can't wait to provide more helpful information to our customers as new and revised documents are released.

Keep watching for BTI announcements; next year looks exciting and my personal goal is to help translate that into a great product experience for all of you.


Back to Contents
A Note from Director of Sales, Shawn Clark:

As we approach the end of another exciting year at Bright Technologies, we would like to thank everyone for their continued support of Bright products and offerings. We strive to be innovative in creating solutions to enhance our user’s experience, and we look forward to advancing our technology even further in the new year. Be on the lookout for new releases as we stand behind our commitment of improving and enhancing professional media workflows by making daily production more predictable, productive and profitable. Thank you again, and we wish everyone great success in 2012.

-Shawn

Back to Contents
Afterword:

Ryan has made an interesting point, and we feel it definitely deserves to be mentioned again.  DataMover has changed considerably from editions released even this time last year.  We would like to encourage everyone to contact support or sales and ask about obtaining a trial version of the latest version.  We feel it is definitely worth your time.



We've been pretty secretive about our development process for the new line of our BrightDrive platforms, and we may have given away a few pieces of information here and there, but you can find all the details on our website by clicking the Procyon logo on the right.  Our new HA platform, among many things, is extremely compact taking up a mere 3U footprint.  It is also more energy efficient and heat efficient.  This small newsletter does not do its feature set justice, please visit our website or contact sales@4bright.com for all the details about this exciting opportunity for your upgrade plans.  Oh, but before we forget to mention it, the Procyon is available now!


We'd like to remind everyone about a few of our ongoing offers to under support customers:
  • G1 Server Rebuilds to update to StorNext 3.5.2 (where applicable)
  • 30 Day Trials of DataWrangler
  • Admin GUI For G2 Users and G1 Users
  • Monitoring Tool For G2 Users
We hope you enjoyed last month's first how-to bulletin about BrightClipCopy, questions implementing the tip in November's issue or this issue can be directed to support@4bright.com or through the ticketing system.

Our next January How- To will feature another useful nugget of information, please look for that towards the beginning of the month.

Sincerely,

Bright Technologies
Description: BTI Logo
Bright Technologies, Inc.
10405 Double R Blvd
Reno, NV 89521
P +1 775.823.9002
F +1 775.201.0027
www.4bright.com
support.4bright.com
sales@4bright.com

Watch our new BrightDrive Movie!
http://www.4bright.com/brightdrive-movie.html
 

Back to Contents