Decentralizing Data Storage With Storj: 10 Times Faster, 50 Percent Cheaper
Cointelegraph caught up with James Prestwich, Storj Co-Founder, an open source and cloud storage platform, that can’t be censored, monitored or have downtime.
Look at this decentralization movement gaining in scope and marching across the industries. Uberization is unstoppable and we certain about it. Data storage market is getting largely influenced by the trend as well. Cointelegraph caught up with James Prestwich, Storj Co-Founder, an open source and cloud storage platform, that can’t be censored, monitored, or have downtime. In an interview with Cointelegraph Prestwich shared his views on the ongoing trend, the place of Storj in this decentralization revolution, and some of the security concerns.
Uber for data storage
Developers are diving into exploring ways to allow sharing of computing power, hard drive space and bandwidth, it is interesting to watch this trend developing further, and Cointelegraph asked Storj where they see this trend going further.
Looking at the projects trying to build a platform for computing power sharing, Prestwich emphasizes that they are taking on a difficult task.
He explains:
“There’s no efficient proof for arbitrary computations. This means that if you don’t trust the person performing the computation, you can’t believe the result. Some problems, like Proof of Work, Equihash, or sorting a list, have results that are much easier to check than create. But most problems are very difficult to ‘prove’ the result of. And if you’re paying someone to do work for you, you’d really rather get the right answer. So there’s a trust problem there.”
While we might still be a few years away from seeing supercomputers working, distributed storage is already here.
He continues:
“Storj is real, and it works. Computing is likely still a few years out. You might see some application-specific tools but there’s a major breakthrough required before secure distributed computing is real. Storage is really the low-hanging fruit here. It’s expensive, necessary everywhere and easy to verify. Both Storj and Sia use auditing schemes that are fast and compact, so you can be 100 percent sure that a file is being stored: there is no trust required.”
Storj is the first decentralized, end-to-end encrypted cloud storage that integrates Blockchain technology and cryptography to allow for higher security of data. It started as a platform to allow for crowdsourced P2P data storage. By decentralizing the data storage, Storj team was aiming at building a service similar to existing, but faster, cheaper, and more secure.
Thus, guys are aiming at building a service that is 10 times faster and 50 percent less expensive than traditional cloud storage solutions. It can be achieved by taking advantage of parallel downloads.
Prestwich explained to Cointelegraph:
“Rather than getting the entire file through a single download, Storj takes a more Bittorrent-like approach. Shards of the file come in from different farmers all at once, as fast as you can download them, and the file is rebuilt on the fly. If you get 10 people delivering you shards at once, they can upload faster than you can download.”
Therefore users get faster service, however, sometimes slow famers can be a bottleneck. Over time Storj has learned how to identify slow farmers and optimize shared distribution to ensure everybody gets the speed they want. For that Storj has hired a full-time data scientist who is working on this problem and is planning to add few more people to the team in the nearest future.
Security breaches are inevitable: how to minimize the consequences
Following numerous recent data leaks and security breaches, the question of ensuring privacy and security of the stored data inevitably come to the fore. Storing data in a decentralized fashion, how one can be sure that the devices that are used for storing data are hack-proof? Prestwich explained to Cointelegraph:
“We’re not building a fortress for your data - Storj is a next-generation adaptive distributed system. Storj works whether or not the farming nodes are secure. To protect the data, it’s encrypted before it enters the network. So right away, no one can read it without that encryption key.”
In the Storj platform all data is sharded before being sent out to renters of the hard drive space (or farmers) and each shard is sent to a separate node. As Prestwich explains, it is already hard enough to find all the shards, even if someone possesses the key. Redundant shards are created via a process called Reed-Solomon erasure coding, even if a few shards disappear, one can still retrieve the file and redistribute new shards. Storj is trying to make sure data is secure even if many farming nodes are compromised. Assuming that it will happen sooner or later, developers are putting systems in place now to beat it.
ICO industry has exploded
Storj started as an open source project in 2014 and was launched with a crowd sale. Storj has designed its own token to act as a means of exchange to buy and sell space within the network. SJCX token allowed users to purchase storage and bandwidth, and get paid for renting out hard drives and bandwidth. Storj managed to raise half a million dollars through a crowd sale and kickstarted the project and launched Storj Labs Inc, a company built on top of the Storj platform to provide additional services and support. Since then the ICO industry has definitely evolved into a more established setup. Prestwich says that ICO industry has really exploded within the last few years. He explains:
“We were one of the first token sales back in the day and the entire market was much smaller then. It’s fascinating to see projects with a white paper and an explainer video raising millions of dollars in such a short amount of time. We hope there will be some major successes coming out of it, but it’ll likely be awhile before any real results come in.”
New tools, partnership with Microsoft and more
Since its inception, Storj has been on a mission to transform the centralized system of data storage by building a platform where no one except the owner of the data can get an access to it. Prestwich shared some of the key achievements the team accomplished in the last years:
“Storj has been several years of solid research, development, trial, error, and constant work. We’re competing heads up with household names and that’s a little daunting. We’re way behind in terms of tools and brand awareness. That being said, we’ve come a long way. We designed a novel interactive storage audit protocol. We built an effective publish/subscribe system on a peer-to-peer network. We bootstrapped a distributed network to thousands of nodes, storing petabytes of data. We built a high-quality library to upload/download files in C, and used it to create a major integration (TBA soon). We built a SaaS billing platform from scratch. We published everything as free and open source software so that everyone can benefit from this work.”
Developers have figured out how to make Storj content delivery work natively in browsers, they are still working on it, but it is already available to be tested. Beside that Storj has partnered with Microsoft Azure and Heroku to deploy Storj development tools. Prestwich shared that the team will be expanding these tools over time, as well as bringing new partnerships and integrations to new platforms.
At the moment, they are adding certificate generation automation for farmers, to make it work over TLS, and designing new and expanding existing platform tools.
Prestwich explained:
“We’re years behind the rest of the object storage world and we’ll be cranking out tools to catch up. More libraries in more languages. More interfaces. More management options. Ethereum migration and related smart contracts. We have plans and initial dev work done on some awesome smart contracts to enhance Storj’s functionality.”
Migration to Ethereum
Prestwich shared that over the past couple of months there have been numerous discussions within the Storj team and community about the Counterparty platform. A few important aspects came to the fore, including the need to significantly update the official Counterparty wallet software, a significant increase in the transaction processing times due to issues with backlogs in Bitcoin Blockchain and increase in the transaction fees. Additionally, there has been uncertainty over the future of other features following the multisig ceasing to function because of Bitcoin making multisig transactions non-standard, a decision to migrate the token powering the Storj platform from Counterparty to the Ethereum ERC20 token standard.
As Prestwich shared with Cointelegraph, the use of an Ethereum-based token offers clear solutions to some of the problems:
“Development is extremely active. Features are added regularly and bugs are dealt with quickly. Ethereum transactions are usually confirmed in seconds to minutes, while Ethereum fees are currently negligible. The ERC20 standard provides out-of-the-box interoperability with a wide variety of wallets and tools. Ethereum’s smart contract platform can be leveraged to build a more robust and decentralized storage system, opening exciting opportunities to provide data services to smart contracts and Dapps. As a result, we will be migrating SJCX to a new Ethereum-based ERC20 token that we feel better serves the Storj ecosystem and its stakeholders.”
Storj is about to embark on a token sale to spread awareness of the project, expand the ecosystem and distribute the tokens people need to interact with the platform. As explained by Prestwich the proceeds of the sale will go a long way to helping Storj realize their vision of an open, accessible, distributed storage system. For more details about the upcoming token sale, please visit Storj web page.
Try hard not to use Blockchain
In the current Blockchain revolution, opinions are largely divided. Some think that Blockchain technology is being overhyped, while others believe that it is actually underhyped and the actual impact of it will be much greater than we can imagine. Prestwich explained to Cointelegraph:
“We have some complicated feelings about this. We’re all fascinated by Bitcoin, Ethereum, and other Blockchains, and we’re building something incredible with them. That being said, here’s our advice for people thinking about using a Blockchain: Try very very hard not to. Blockchains are expensive, inefficient, and hard to scale. Blockchains are brand new, so there’s no technical, legal or financial roadmap. Everyone is still trying to figure out what they’re good at, and there will be a lot of projects that fail because they used a Blockchain when they didn’t need one.”
Storj has limited their use of Blockchains to things they think they’re good at payments between machines, provable computing, trustless interaction. The team has focused on building a distributed system, and Blockchains are just a tool helping to achieve that. He continues:
“We think that Blockchains will change the world. Everyone says they cut out the middleman, well, we’re going to realize that in these technical and financial systems humans are mostly just middlemen moving information around. They open the doors to a new generation of machine-to-machine transactions. Entire automated marketplaces trading without human intervention. It’s a long road to get there. And it will be a transition made in fits and starts. But it’s inevitable.”
One doesn’t need to understand Blockchain
There is also an opinion that wider adoption of Blockchain technology is often hindered by its being too difficult to understand. But do we need to understand how the technology works to get how and why it is so beneficial? Prestwich shared his opinion:
“One of the well-known things about cryptography is that you should never write your own encryption algorithm. It’s complex and easy to make a subtle mistake. Mathematicians make cryptographic libraries. Developers use those libraries without understanding them to serve users. Blockchains are the same way. A highly specialized group can make tools that developers use to serve users. Most developers will never need to understand this technology.”
At the end of the day, all technology works this way - plumbing, electricity, physics, marketing - we can get all the benefits without users ever knowing it exists.
