Qloud lite paper

Flexible, lightweight, decentralized data storage & transfer network

Created by Chunk Works

Qloud is a decentralized data storage and transfer network that addresses the limitations of existing solutions and provides a future-proof alternative. With its cutting-edge technology and flexible design, PIPE offers improved data retrieval and scalability, providing a superior user experience. The key to its flexibility is the PIPE Extension System (PES), which allows for easy addition of new features and extensions to the core network, ensuring compatibility with future advancements in technology. Additionally, PIPE’s Hybrid Cryptography, flexible redundancy, customized node deployment, indexing options, and integrated DNS manager provide users with the ability to choose their desired level of security, redundancy, and performance. Overall, PIPE is a state-of-the-art decentralized storage solution that provides the security, scalability, and adaptability required to meet the industry’s ever-changing demands.
ptive and capable of accommodating the evolving needs of its users.

PIPE is made up of three main components:

• PIPE Core (PIPE-rs)
  This is the main library responsible for handling all aspects of networking, database management, data recovery, and storage management.
• PIPE main extension
  The extension is tasked with the validation of payments (smart contracts) and the validation of storage.
• PIPE-Node
  This is a fully functional default node that includes both the PIPE core and PIPE main extension, allowing for participation in the mainnet or the creation of private decentralized networks.

Figure 1: PIPE Core

It is important to note that the PIPE Core is designed to be separate from the business logic, such as the validation of payments and storage, which are handled by the PIPE main extension. This modular design enhances the overall functionality and versatility of the network.

The PIPE main extension is designed to be flexible and able to accommodate additional functionality as required. TangleHUB, the creator of PIPE, can extend the PIPE main extension if deemed beneficial to the overall network. Additionally, the PIPE Extension System (PES) enables developers to freely create and add extensions to PIPE Core by constructing a Rust project and integrating it into the pipe-rs framework. This approach enables the network to remain adaptive and capable of accommodating the evolving needs of its users.

Qloud revolutionizes the decentralized storage space with its cutting-edge design that addresses the most pressing issues in the industry. Our core design has been heavily influenced by the following issues:

Challenges

Trade-offs between user-centric and function-centric designs
PIPE has created a user-centric design that focuses on the needs and wants of users, such as better usability, increased satisfaction, and rapid innovation, among others. However, a user-centric approach might also lead to limited scalability as the particular set of features is only relevant to certain groups.

Rapid industry developments

The public cloud and in particular the decentralized storage alternative industry is rapidly evolving, making it difficult to predict future user expectations. To ensure a design that is future-proof, it is essential to stay current with industry developments and to design for scalability and flexibility. What is the best approach to designing a decentralized data storage and transfer network that can adapt to future developments and user needs?

Challenges in integrating additional features

A decentralized storage network must incorporate essential functionalities such as data distribution, encryption, integrity, retrieval, incentivization, indexing, and decentralized control, among others in its core product in order to provide a seamless and efficient user experience. By including these components in its core, users will have access to all necessary tools and features without the need for additional add-ons or third-party solutions.

The integration of these components also ensures that they are designed to complement each other and work seamlessly together, resulting in a more robust and reliable system. Additionally, this approach eliminates the need for users to navigate multiple different interfaces or platforms in order to access all the features they require, which is crucial for providing a streamlined and efficient user experience while simplifying the overall functionality of the system.

Solution

The PIPE solution is a cutting-edge technology that is designed to meet the needs of modern society. It is made up of a lightweight and flexible core that can adapt to a user’s specific needs.
The architecture of PIPE is extendable, meaning it can be easily customized and built upon to accommodate the needs of any particular use case. This makes it a versatile and scalable solution that can be used in a variety of settings, from small businesses to large enterprise environments. With its combination of adaptability, scalability, and customization, the PIPE solution is set to change how businesses approach technology.

PIPE’s unique combination of features and capabilities sets it apart from other solutions on the market. Let’s explore the components that make PIPE stand out.

PIPE Extension System (PES)

The PES (PIPE Extension System) allows developers to extend and customize the functionality of the core technology. By constructing a Rust project and integrating it into the PIPE-rs framework, developers are able to freely create and add extensions to the PIPE Core. The components that are included in an extension are optional, providing developers the flexibility to build the functionality that best suits their use case.

One optional component is Hooks, which give the ability to interact with specific points within the PIPE Core. Another component is the Storage Engine, which can be tailored to meet the needs of the specific use case. The Storage Engine can use RAM, a filesystem, or a custom solution. Connection Capabilities are another optional component, with options including P2P, Websocket, or a custom solution. Additionally, Cryptography is another optional component, with options including Ed25519, Falcon-512, or another (standard) cryptographic algorithm.

These customizable components, combined with the ability to easily integrate with the PIPE-rs framework, make the PES a powerful tool for developers looking to extend the functionality of the PIPE Core. The PES makes it easy for developers to build and deploy innovative solutions, delivering the agility and scalability that organizations require in today’s fast-paced technology landscape.

Hybrid Cryptography

The security architecture has been designed to provide maximum flexibility and adaptability to meet the unique needs of different use cases. One aspect of this security flexibility is the ability to choose from a variety of cryptographic algorithms, giving users the freedom to select the algorithms that are most appropriate for their needs. This ability is referred to as Hybrid Cryptography and provides a level of control over the security process, allowing users to optimize the security for their specific requirements.

In addition to algorithm selection, PIPE also provides “verifier flexibility”, allowing the verifier to accept or decline the cryptographic algorithms used to secure the information. An example of this would be the ability of a small IOT device to verify the proof of data coming from a node. As the data is received, the device can read the algorithms being used, and decide which algorithm it is capable of handling. This allows the security solution to be optimized for the specific capabilities of each device.

In an ideal scenario, all available algorithms would be verified, providing a layered approach to security. If one algorithm is marked as “invalid”, the others can provide additional protection, ensuring the security of the signature even if one algorithm is compromised.

Redundancy

Data redundancy is managed through the creation of parity chunks, also known as shards in the field of erasure coding. The number of parity chunks created is dependent on the desired level of redundancy and can be specified by the uploader. PIPE does not duplicate data by default, as this is deemed the responsibility of the client. Control over the number of parity chunks will be provided through a user-friendly interface, or fine-grained control via an API or terminal program.

However, the PIPE framework is designed to manage data recovery in the event that a parity chunk goes missing. The exact mechanism for this recovery is currently under discussion, but the solution will attempt to restore the missing chunk, by reading from the remaining chunks, reconstructing the data, and sending the chunk to a storage node.

Customized node deployment

Customization is provided by the ability to compile different functions into each PIPE-node. This allows for flexibility in deployment, as a node can be designed to run on a variety of platforms, including resource-constrained embeddedmodular devices (i.e. IoT), web browsers, and enterprise-grade servers.
Indexing

The ability to customize chunk addresses during file upload in PIPE opens up new possibilities for efficient and innovative file management. With the option to assign a chunk address based on personal preferences, as opposed to being limited to a content-based ID, PIPE broadens the scope of file organization and retrieval:

• Users can create their own decentralized index by attaching keywords to larger texts for easy searching.
• Migration from or increasing performance of other decentralized storage networks to PIPE is made simpler, as the previous solution’s content ID can be easily attached.

The fact that the chunk address is user-defined and not generated by the system eliminates the need for storage and retrieval of complex addresses, as the chunk address can be generated based on a set of instructions or conditions. This means that in PIPE, it is possible to retrieve data from a channel using only the private key, greatly simplifying key storage and long-term data retrieval.
For example, Data Channels use a simple integer as a “page number” which is then translated internally into the chunk address.

PIPE DNS Manager

PIPE works with a variety of devices, including embeddedmodular devices, web browsers, and enterprise-grade servers. However, nodes within PIPE don’t have any information about other nodes; only about their neighbors. To connect to the network, a public key must be generated and a connection made to a subset of the network. If the node doesn’t have an SSL certificate, it cannot connect to the web browser, creating a limitation. To solve this issue, a gateway could be created as a common solution. However, this also creates further limitations such as centralization, censorship, and performance issues, which contradicts the simplicity of PIPE by adding another layer.

For these reasons, PIPE created the PIPE DNS manager. This allows users to access PIPE nodes or content directly through their browsers. Browsers require SSL certificates and a domain name, so the PIPE DNS manager creates a subdomain to request an SSL certificate, which is then linked to a PIPE node. This solution maximizes decentralization and eliminates bottlenecks while being easy to use. It eliminates the limitations that come with using a gateway, such as centralization, censorship, and performance issues.

In this chapter, we outline the key design decisions that have been discussed and considered for the technical design of our project. These decisions reflect our team’s considerations on what is necessary for an effective and efficient technical design.

Some of these decisions have already been implemented in the current design, while others may still require further evaluation and refinement. This chapter serves to provide insight into the thought process and considerations behind our technical design, and to showcase our commitment to ensuring a well-designed and practical solution.

Etc.