Storj and Object Mount
In my last article about Storj, I focused on the economics of managing a cloud archive and the rich benefits Storj can offer both in base cost and operational efficiency. This time around I wanted to take some time to look at the advantages Storj’s client side mount tool Object Mount can bring to both migration strategies and day to day operation. As well as considering the challenges of migrating a legacy archive onto a cloud first platform like Storj.
A platform to monetise your content
If your archive is a source of income, then the availability of your archived data needs to be at the heart of your operation. Poor performance and organisation have a direct impact on customer experience and lost time in admin or management, both of which have a less visible but very real impact on your bottom line.
With constant innovations in content analysis, you need to have fast and easily administered access to your archive to take advantage of these services to make your archive work harder for you.
With centralised cloud providers such as AWS, providing data access to users and services can be challenging with highly trained staff required to manage access and privileges via identity management services like AWS’ IAM.
With Storj, user permissions are managed through a user-friendly UI. Your data is just as secure with encryption for all data at rest, and MFA quick and easy to configure, you just don’t need a degree to deploy it.
With Storj and their POSIX compliant mount client Object Mount, you can have the best of both worlds. Storj has a fully AWS compliant S3 API to allow any service with S3 integration to hook straight into your buckets. Object Mount adds to this functionality by giving Mac, Windows and Linux clients local mount access to your S3 data. Object Mount allows your S3 storage to act like a local disk with full file locking capabilities. Intelligent caching of hot data reduces latency for frequently accessed files and most importantly, Object Mount is optimised for large files, pre-fetching data into your cache bringing performance to media playback and processing from S3 sources.
The revolutionary part of the Object Mount and Storj workflow is that you can have direct access to your archive data for collaborative editing, SaaS AI and transcoding services on any platform. Object Mount even works brilliantly deployed in a docker environment to provide fast data access to dynamic micro services.
The ideal back end to your store front
The ideal end game of a well-managed, monetised archive is to have a cost effective, highly available online storefront, with a great user experience.
Solutions like VIDA, recently adopted by BBC Worldwide, gives you exactly that and with Storj as a back end you can enhance an already great platform with the elevated resilience, availability and performance of Storj.
Remember that Storj distributes data across a global network with no bottlenecks for disparate regions. Your content is available globally with faster retrieval speeds and increased reliability for storefront customers accessing high-resolution product images, videos or any media asset, and with your data spread and protected across hundreds of data centres across the globe you have removed any single point of failure.
Your dev and admin team will also be filled with joy as Storj’s S3-compatible API allows for seamless integration with existing e-commerce and DAM platforms without major workflow disruptions and no invisible tax on API calls that you get with AWS S3.
This combination of affordability, performance, and security makes Storj a natural choice for a scalable, high-performance DAM backend supporting a global storefront like VIDA.
And if your operation is not direct sales but focused on processing or re-versioning then all the above benefits apply to Media and Project Asset Management platforms too!
Example MAM/Storefront back-end
That all sounds good, but I can’t face the migration.
Even when there is a compelling cost and performance argument to move your archive to a new platform, there is no bigger turn off than a challenging migration, particularly if you have a data on multiple systems and drives.
Because Storj have POSIX compliant access through Object Mount as well as AWS S3 API, the process of moving your data into a Storj environment can be navigated with far less pain than other platforms.
Executing a migration is a serious business and spending some time on the planning is key to success. Fortunately, at Tyrell we have the experience and tools that can take the pain out of the process and take advantage of Storj’s unique features.
Unstructured data on drives in cupboards and camera cards in that drawer in the corner can be the most anxiety inducing format an archive can take. With fast desktop access to the Storj platform through Object Mount the copy process can be made as painless as possible, although it’s likely some poor assistant is still going to have to translate that old excel sheet into a folder structure. Although it’s not all bad news for the intern, once transferred Storj’s flexible access methods and performance mean that it’s a great platform for AI tools to run image recognition, audio to text transcription on your files and deeply enrich your metadata.
Structured data on an LTFS system or another cloud location is ripe for a well-designed analytics tool like StorageDNA’s DNAfabric or Cloudfirst.io .
For small and medium sized organizations StorageDNA is a fantastic option to get a balanced view of your storage estate. DNAfabric is specifically designed to interrogate large and disparate pools of data, and their respective cost profiles help you get a complete view of just how well your storage is working for you. The Tyrell team can help you use this intelligence to rationalise and transfer your data and reduce both the headache and cost of a significant migration.
If you are an enterprise level business then inevitably something more turnkey is going to be needed, that’s where Cloudfirst come in. By providing an in depth audit of legacy archive storage, Cloudfirst can give you a clear written TCO report setting out the ongoing and projected costs of keeping your on-premise archive running, alongside a fully cost projected view of what that same data would look like in the cloud or in a hybrid deployment. This is not just a bean counting exercise. Cloudfirst dig into the operational impact of transitioning away from existing infrastructure to give you a holistic view of where you are and what transitioning your archive could look like. With major media archive migrations under their belt including a 30PB migration for the BBC, they are a fantastic choice for any media enterprise to help explore cloud for archive.
Pre-Migration Analysis AKA, what don’t you need
It seems obvious that it’s pointless moving redundant or duplicated data into any archive, least of all one billed on utilisation. But many users would be shocked at the levels of data duplication present in many storage pools.
Analysis, be that automated from StorageDNA Fabric, or turnkey from Cloudfirst is crucial in making sure your new platform is not bloated with costly, redundant data.
Don’t start before you get your price, particularly when dealing with centralised cloud
It’s not really in a consultant’s nature to say don’t take my word for it… But never just take our word for it! Real world cost analysis is essential in making an informed decision on your storage strategy and StorageDNA Fabric and Cloudfirst can help get your projections as accurate as possible, so you can feel confident you’re making the right call before hitting the transfer button.
DNAfabric offers tools to analyse and compare storage costs across various cloud providers and tiers. For example, how can you minimise the API call fees when pulling data from Glacier Deep? You may be able to save money or enter a new price tier by bulk retrieval. DNAfabric and the Tyrell engineering team can build you accurate cost models to inform your planning.
For those with a need for a more granular projection, Cloudfirst take a significantly deeper dive and produce a 5-year projection. Empowering you to make informed decisions based on specific needs. If your data is at significant scale then this kind of detail is essential to make informed decisions.
Keep your business going
The biggest fear for many when they hear the word migration on any system is disruption to business. Making a migration smooth and transparent to the user is an art form.
DNAfabric employs file transfer acceleration to enhance transfer speeds, ensuring that data moves as swiftly as possible from AWS S3, AWS Glacier or in fact any service to Storj. You can utilise this to minimise the delta between the source and destination of your migration by making the transfer quick. You also have check sum tools to ensure your data is secure in its new home before you retire your legacy archive
Again Cloudfirst goes further, fully managing the transfer while ensuring your business continuity, taking the pressure off your technical teams and allowing them to concentrate on the most important thing- your customers.
Data flow for a small/medium scale migration
In conclusion
If you have a large on premise LTO archive, significant cloud S3 presence or find yourself sifting through that cupboard of Lacie Rugged drives 3 times a week, the agility and cost benefits of Storj and Object Mount have the potential to not only save you significant sums of money but revolutionise the way you access, manipulate and monetise your content.
If you share this conclusion, then you can rest assured that Tyrell, along with our partners Storj, StorageDNA, and Cloudfirst.io, can make the transition process simple and efficient.
Storj as the storage at the heart of your infrastructure
If you would like to learn more, or see a demonstration of the power of Storj and Object Mount and discuss what it can do to unlock your archive workflows please get in touch with the Tyrell team.
Email: sales@tyrellcct.com
Tel: +353 (1) 888 1118 or +44 (0) 207 148 6201
