NeoFS and Filecoin Comparison
Introduction
With the development of decentralized services and the launch of new solutions in the field of distributed data storage, more and more attention is focused on this topic. The first overview of existing decentralized storage solutions can be found in the previous article by NeoNewsToday about services such as NeoFS, StorJ, Sia, and FileCoin. In this review, we want to focus on comparing NeoFS and Filecoin.
NeoFS is a distributed, decentralized object storage network developed by Neo SPCC. NeoFS Nodes are organized in a peer-to-peer network that takes care of storing and distributing user’s data. Any Neo user may participate in the network and get paid for providing storage resources to other users, or store his data in NeoFS and pay a competitive price for it. Users can reliably store object data in the NeoFS network and have a transparent data placement process due to the decentralized architecture and flexible storage policies.
Each NeoFS Node is responsible for executing the storage policies that the users select for geographical location, reliability level, number of nodes, type of disks, capacity, etc. Thus, NeoFS gives full control over data to users. Deep Neo Blockchain integration allows NeoFS to be used by Decentralized Applications (dApps) directly from NeoVM on the Smart Contract code level. This way dApps are not limited to on-chain storage and can manipulate large amounts of data without paying a prohibitive price.
NeoFS has a native gRPC Application Programming Interface (API) and popular protocol gates such as AWS S3, HTTP etc, allowing developers to easily integrate their existing applications without rewriting their code. Using this set of features, it’s possible, for example, to use a dApp’s Smart Contract to manage monetary assets and data access permissions on NeoFS and let users access that data using a regular Web Browser or mobile application. NeoFS in the public network does not use its own token, unlike all competitors, but uses the Neo network GAS token.
Filecoin is a peer-to-peer network that stores files, with built-in economic incentives to ensure files are stored reliably over time. In Filecoin, users pay to store their files on storage miners. Storage miners are computers responsible for storing files and proving they have stored the files correctly over time. Anyone who wants to store their files or get paid for storing other users’ files can join Filecoin. Available storage, and the price of that storage, is not controlled by any single company. Instead, Filecoin facilitates open markets for storing and retrieving files that anyone can participate in.
Filecoin includes a blockchain and native cryptocurrency (FIL). Storage miners earn units of FIL for storing files. Filecoin’s blockchain records transactions to send and receive FIL, along with proofs from storage miners that they are storing their files correctly.
Major differences
Network and objects’ operations
The Filecoin network is built on the idea of developing a layer of payment and control over IPFS, and is a blockchain that allows you to connect users and storage nodes, as well as reward nodes involved in various administrative functions. The main task of Filecoin is to control every object uploaded by the user through the blockchain. Since the Filecoin network, due to architectural features, cannot automatically control the distribution of objects on the network before placing an object, the user himself chooses with whom he will store a copy of the data. At the same time, all this happens through Storage Market OrderBook, which in the latest versions of Filecoin is proposed to be moved outside the Filecoin blockchain network in order to reduce the load on the blockchain (which affects single points of failure in the form of OrderBook services). But deals are still made on the blockchain itself. In addition, at the moment, there are no data replication mechanisms in Filecoin in case of storage node failure. The user himself must control this and renegotiate the smart contracts for storage instead of the lost ones. However, Filecoin data recovery is promised in the future.
After the user has entered into a smart contract with the storage node, he can upload the data. The upload result will be recorded in the blockchain. Thus, the preliminary actions for each upload of an object require at least two transactions on the blockchain. Considering the average block time of 30 sec in Filecoin mainnet, this can be a serious problem and a source of high delays when you need to perform a large number of PUT operations on very small objects.
For GET operations, the user must first contact the Filecoin network (the Retrieval Provider node) to find out where the object is and create a payment channel. After that, the object will only be downloaded in parts with intermediate payment. This also requires additional time costs and the need to pay for obtaining information. Information fees can be a major obstacle to using Filecoin in real-world applications. At the same time, storing the addresses of objects in the Filecoin network is entirely the responsibility of the user. Because of all of the above, working with data can take a long time to both receive and load data. Based on current high-level benchmarks, PUT steps are estimated to take around ~5–10 minutes for a 1MiB file in the Filecoin network and the same is true for data retrieval.
NeoFS was originally created as an object storage network in order to compete with not only decentralized but also distributed storage systems such as Swift, CEPH, and others. One of the goals was to cover all kinds of data operations. The main challenge was to minimize the number of network requests and minimize the impact of the block acceptance time. The NeoFS network is based on different principles, which are based on the development and adaptation of the СRUSH (Controlled Replication Under Scalable Hashing) protocol to work in a decentralized setting, as well as its adaptation for the possibility of describing any node attributes, whether it be a geographic location or a type of disk. In NeoFS, the network is presented in the form of a multigraph (Network map), which allows the use of mathematics (placement policy for the container and the placement function for the object) without requiring additional requests to the network to determine where to look for the object or where it should be placed. In this case, the result is the same for all network nodes and the recipient of the object can also check the correctness of the storage request without additional network requests. The ability to define a container (like an s3 bucket) allows you to quickly perform object search operations and work with objects, even if the user does not store their identifiers. Also, the container (as a group of nodes) itself controls the compliance of the storage policy and is responsible for replicating objects in the event of node failure.
The incentive model and pricing follow free-market principles. Each node declares how much it wants to receive as a reward for data storage services for the upcoming epoch. The placement function considers the declared price and prefers nodes with better prices. However, because nodes in the Network Map declare key-value attributes describing their parameters, such as geographical location, type of storage, capacity, and other things also used in Storage Policy, it leaves room for nodes that want a higher price, if they provide better or unique services to still get into the placement function result.
In short, if the node is too greedy it does not get users and rewards. To charge higher prices and make more profit, the node needs to satisfy market demand or provide something unique. This allows the creation of a system with free-market principles without the order book bottleneck.
Object properties
The Filecoin protocol is supposed to work with data by ID only.
In the NeoFS protocol, work with metadata is also available. The object consists of system and custom users’ metadata and the object’s payload. Thus, the NeoFS protocol supports operations such as searching through the metadata of user objects or filtering rules in the ACL to give access to objects with certain metadata. This allows building any application for working with data to get objects from a container by their metadata.
In Filecoin, the minimum duration for a deal to store data is set in the miner’s “ask”. There’s also a practical limitation because sectors have a minimum duration (currently one month). In NeoFS, storage does not have the limitation of a minimum or maximum storage time for an object unless the user himself specifies it. Payment for NeoFS storage is made once per epoch (for a short period of time, on the test network it will be approximately equal to 6 hours).
Data security
Data security is critical for any data storage system. It is important to define whether a user has access to a particular piece of information or not.
Filecoin doesn’t have any mechanism to control data access rights and permissions. Anyone who knows the object ID can download the data. The only way to regulate access is to create own contracts in the Filecoin blockchain for object placement, access, verification, and payment. However, this looks like an almost impossible complex operation — since it will require the user, bypassing the order book, to look for storage nodes himself and directly communicate with them off-chain to agree to work with custom untrusted contracts. It also requires a lot of development work from the user to implement and verify these contracts.
At the same time, NeoFS fully supports access control inside the NeoFS network protocol with a flexible multi-level ACL system. Only a combination of various mechanisms can effectively provide it, and yet ACL remains the key one.
Access Control Lists specify users’ IDs and the rights they can have, namely to read (search through the container) or to write (other object operations). While receiving a request, any object server gets a container and compares the sender (the first element in the chain of signatures) and the container’s ACL. The container’s ACL covers all objects therein. Thus, the owner of the container obtains full control and sets certain permissions authorizing defined groups of users only.
To obtain information on authorization rules, NeoFS uses a flexible ACL system involving basic ACL and extended ACL, if it is allowed. In NeoFS, you can differentiate access rights both for the container (basic ACL) and for a specific object or a group of objects (extended ACL), united by any arbitrary attribute. Also, it is possible to define access rights for each specific operation.
Basic and extended ACL together uses multiple parameters, thus, providing greater control. In this way, the owner of the data has complete control over who has access to it.
Smart-contract integration
Like similar storage systems, Filecoin only allows distributed applications to integrate with it at the level of storing an object’s ID in a smart contract. This provides verifiability for the object, but all other application logic (uploading, downloading, and processing) can only be implemented externally to the contract itself. Inevitably, it introduces a single point of failure and prevents the creation of truly decentralized or trustless applications.
The possibility of working directly with smart contracts on the Neo network was a key consideration when designing the NeoFS architecture. Support for NeoFS protocol has been added into Neo 3 native Oracle contract, allowing NeoFS objects to be used inside smart contracts.
For example, a contract could make the decision to transfer tokens or otherwise change its behavior based on the contents of an object stored in NeoFS. Alternatively, a smart contract could create new objects in NeoFS based on the result of its own internal logic.
Neo SPCC also plans to add a layer for decentralized data processing in the future, providing computation in addition to storage. This will make it possible to perform complex calculations on data inside smart contracts without the prohibitive on-chain cost. For example, you could upload a photo to NeoFS, process it with a neural network to confirm authenticity, and trigger a subsequent transfer of tokens. All this would be performed inside one smart contract on the Neo network, leveraging the cost efficiency NeoFS storage network (and future data processing) network.
The Neo SPCC team views this full decentralized stack as the future for FinTech and DeFi blockchain applications
Demo services
Filecoin and NeoFS have their own demo services for loading objects into a decentralized distributed storage system via an HTTP gate — Send.NeoFS (NeoFS) and File.video (Filecoin).
For Send.NeoFS any file may be uploaded with a user-specified lifetime of 15 minutes, 1 hour, 24 hours, or 48 hours. File uploads are free during the testing phases, with a file size limit of 50Mb.
File.video only accepts video files with a 30MB limit. But does not announce the storage time limit. Also, in this demo service, the object is downloaded (GET) not from the Filecoin network, according to their description, but from the IPFS nodes, which are used as “hot” storage and are a third-party service for Filecoin protocol — Powergate. Powergate is an attempt to solve the slow procedure of downloading an object from the Filecoin network through the use of additional IPFS nodes as a data caching layer.
Conclusion
The Filecoin network protocol is well suited for storing large objects as a cold storage backup.
NeoFS is well suited for working with application data and is equally well adapted both for a big data stream of small objects and for working with large cold data. Access control systems and metadata make NeoFS an efficient and flexible storage platform for creating applications.
