Technical overview

All the features mentioned aren’t separate features but byproducts of a single core algorithm behind the Chunk Works Platform. This is fundamentally different from traditional approaches where you have:

  • Separate RAID solutions for disk protection
  • Different tools for anti-ransomware
  • Distinct load balancing tools like Edge servers

System characteristics

Our innovative system design ensures every component functions as a full node, including browsers running WebAssembly.
End users can perform local recovery operations independently, while unified code manages both load balancing and recovery functions. Rather than relying on multiple layered solutions, the system operates as a single integrated platform.

Network architecture

The system uses an innovative reverse peer-to-peer concept, contrary to conventional P2P networks, which require nodes to advertise their content. Our approach uses a deterministic algorithm to predetermine content location, so that nodes can route data chunks without complete network visibility. It also uses zero-knowledge proof technology to ensure complete anonymity.

Routing mechanism

The routing system is built on game theory principles, intelligently directing data chunks to the nearest known node to their destination. Each network hop progressively moves data closer to its final destination. This enables immediate parallel downloads without requiring preliminary file location searches, allowing the system to initiate transfers even before determining exact file locations.

Network architecture

The system employs a reverse peer-to-peer network architecture, fundamentally different from traditional P2P networks like BitTorrent or Limewire. Instead of nodes advertising their content, which creates excessive traffic and statistical complexity, the Chunk Works Platform uses a deterministic algorithm to predetermine content location. This approach enables immediate chunk routing without requiring knowledge of the entire network, implementing zero-knowledge proof principles for enhanced anonymity.

Efficient routing

The routing mechanism is elegantly simple, based on game theory principles. When transmitting data, chunks are routed to the closest known node towards the destination. Each network hop brings the data progressively closer to its final destination, creating an efficient routing path without the overhead of constant content advertising.
This architecture enables immediate parallel downloads without the traditional requirement of first locating files across the network. The system can initiate transfers instantly, even without knowing exact file locations, resulting in faster and more efficient data handling while maintaining security and reducing network overhead.

Chunk storage requirements

The storage system operates on a fundamental chunk-based architecture. The core requirements for any storage backend are minimal, consisting primarily of two essential functions: chunk storage and chunk retrieval. Additional optional functions enhance network performance, particularly the ability to retrieve random chunks. This random access capability is especially crucial for the storage check functionality, which verifies data integrity and availability across the system.

Advanced (re-)distribution algorithm

Using typical XOR-based calculations for chunk distribution could result in uneven distribution. The Chunk Works Platform implements a proprietary linear distance calculation algorithm. This ensures equal chunk distribution across all nodes regardless of their recognition of each other. The algorithm enables automatic rebalancing without requiring data re-upload when adding new nodes. The system can start with a single node and scale up seamlessly as requirements change.

Nodes

The node system is designed with intentional non-interactivity at its core. Rather than providing a fixed, standard interaction method, the system maintains a flexible, modular approach. This fundamental design choice allows the node to remain a pure backend component while interaction methods can be added as needed.

Node management and configuration

The platform incorporates extensive customization capabilities at the node level. Administrators can configure various aspects including host settings, port configurations, authentication methods, and Docker parameters. The storage engine can be tailored to specific requirements, providing flexibility in how data is managed and stored across the infrastructure.

Integration capabilities

When specific interaction needs arise, such as S3 compatibility, these can be activated as extensions on top of the core system. This modular approach means that functionality can be loaded and enabled based on specific use cases, rather than burdening the system with unnecessary features by default. The system maintains its optimizations and core benefits regardless of which interaction layer is chosen.
The system’s flexibility extends to seamless integration possibilities. It can be incorporated into websites without requiring any client-side installation, operating transparently in the background. This makes it possible to utilize the system’s full capabilities while maintaining a low profile, with end users potentially unaware of the underlying technology powering their experience.

File System emulator

The Chunk Works File System serves as a filesystem emulator for the Chunk Works network. At its core, the network operates purely with chunks rather than files, with file handling simulated at a higher level. This design choice keeps the core system lean and reduces potential security vulnerabilities. The system offers flexibility for different use cases – from enterprise users working with raw chunks to businesses needing a more user-friendly, Dropbox-like interface with advanced security features.

Security by design

The system operates on a fundamental principle that transforms how network security is approached: any discrepancy in network operation, regardless of its source or nature, is treated as a potential threat. This eliminates the traditional approach of creating specific responses to known threats, instead implementing a universal response protocol for any deviation from normal operation. Whether the disruption comes from planned maintenance, hardware failure, or a malicious attack, the system’s response remains consistent and predictable.

Automated recovery and distribution

When a disruption occurs, the system initiates its recovery protocol immediately. There’s no waiting period and no need for manual intervention – the moment a server becomes unavailable, the system automatically assumes data unavailability and begins recovery procedures. This immediate response ensures minimal downtime and maintains continuous data integrity across the network.
The recovery process itself is fully automated, with remaining active nodes in the network instantly redistributing affected data among themselves. This dynamic rebalancing maintains the system’s required redundancy levels and ensures continuous data availability even during node failures. The self-healing nature of the network means that when nodes return to service, data is automatically rebalanced to restore optimal distribution.

Comprehensive protection

The automated response to any form of unavailability creates a self-healing network that maintains data integrity and availability without human intervention. This comprehensive approach provides robust protection against various threats while simplifying system administration and reducing the need for manual recovery procedures. The system’s ability to handle unknown future threats without protocol updates, eliminate human decision-making delays, and maintain constant data availability makes it uniquely resilient in today’s rapidly evolving threat landscape.

Post-quantum encryption

Post-quantum transition presents a significant challenge, with many organizations facing small but critical bottlenecks in their infrastructure – like legacy Windows XP systems or embedded devices with secure elements incompatible with post-quantum cryptography. The iterative approach offered by the Chunk Works Platform allows organizations to begin their transition before resolving all these obstacles.

Future-ready architecture

What sets Chunk Works’ approach apart is its forward-looking design. Rather than just addressing the quantum computing threat, the system takes it one step further and is designed to handle any future security challenges. When new threats emerge, such as AI-discovered vulnerabilities in encryption algorithms, transitioning to new security measures requires just a simple configuration change, unlike traditional systems that would require another complete infrastructure overhaul.
This creates a system inherently prepared for both known and unknown threats. By treating every anomaly as a security concern, it’s automatically positioned to handle emerging threats that haven’t even been conceived yet. The forward-thinking design eliminates the need to update threat response protocols as new types of attacks or failures emerge. The result is a robust, self-maintaining system that provides consistent protection while significantly reducing administrative overhead.

Crypto-agility

Crypto-agility is a system’s ability to quickly switch cryptographic algorithms, keys, or protocols. This is essential for future-ready data security, especially when encryption is at risk of being compromised or outdated.

Chunk Works’ truly crypto-agile system streamlines all key management, making it easy to quickly change cryptographic keys and can combine multiple types of encryption for better security. Ease-of-use helps it adapt to threats fast, reducing potential damage from breaches. By adopting such true crypto-agility, organizations can easily safeguard the confidentiality, integrity, and availability of data against evolving threats and vulnerabilities, regardless of changes in the threat landscape.

Hybrid Encryption

The system’s unique approach embeds the different algorithm choices and parameters within public keys, enabling seamless hybrid cryptography. This means a post-quantum-ready actor can communicate with other actors using only classical cryptography without requiring them to upgrade. The hybrid cryptography library automatically handles encryption according to each party’s capabilities and preferences, validating overlapping security preferences when possible.1
This flexibility extends to both software and hardware implementations. While many secure elements lack post-quantum capabilities and may continue to do so (similar to IPv6’s prolonged adoption challenges), Chunk Works’s approach allows organizations to implement post-quantum security where possible while maintaining compatibility with legacy systems. This pragmatic approach enables progressive adoption without requiring a complete system overhaul.

Seed-based security

Cryptographic algorithms tend to have different, incompatible implementations, making it hard to switch between algorithms. Chunk Works offers a unique method of key derivation, based on a seed, which removes the requirement to remember or store multiple keys, lowering the risk of unintentional exposure. This approach is particularly useful in ecosystems where ease-of-use and resource efficiency are essential, like edge, IoT, or embedded devices linked to IT infrastructures.

This seed-based security enables interoperability between algorithmic implementations, streamlines cryptographic output for crypto-agility, requires storing only one secret regardless of the amount or types of cryptography used, and provides simple, granular control over data access. The modularity of the architecture ensures adaptability to evolving cryptographic technologies, enabling seamless updates and easy integration of new algorithms while maintaining operational consistency across various algorithms with hybrid cryptography.

Deployment

Chunk Works deployment tool is specifically designed for system administrators rather than end users. It has the capability to log into network servers and generate its own keys, all while operating without external dependencies—except for the necessary UI libraries for different operating systems. As a deliberate security measure, the deployment tool cannot delete data, ensuring that even if the system were to be compromised, an attacker couldn’t perform mass data deletion.

Deployment options

The deployment tool comes in multiple variations to suit different needs. Besides the UI version for desktop administration, it offers a command-line version for advanced system administrators and an API version that can run as a server. This flexibility allows MSPs/MSSPs to offer the deployment tool as a service to their customers. The system eliminates the complexity of traditional tools like Kubernetes and Ansible, simplifying deployment to a straightforward process regardless of the installation environment.

Large scale deployment considerations

When scaling to larger deployments (e.g., three nodes with 10x14TB drives each), the system takes a unique approach to storage management. Rather than implementing LVM-based software RAID or creating multiple mount points, the Chunk Works Platform handles redundancy at the application level. The system offers two chunk storage methods: standard and attestation, with attestation providing linear distribution across multiple nodes.

Node and redundancy management

Each node represents a failure domain, meaning redundancy must occur across nodes rather than within them. The system uses a sophisticated parity mechanism rather than simple replication. With 33% redundancy (e.g., the recommended setting for datacenters), if 100 chunks exist, 50 parity chunks are created, totaling 150 chunks. This allows one-third of the nodes to fail while maintaining data integrity.

Redundancy calculations

The system calculates required nodes using the formula: Total nodes = 2 / (1 – redundancy percentage). For example, with 33% redundancy, three nodes are required. At 50% redundancy, the system switches from parity to direct copying, as creating copies becomes more efficient than calculating parity chunks. The system always rounds up chunk calculations to ensure proper redundancy, even handling edge cases like prime number chunk quantities.

Node size optimization

Rather than deploying large servers with multiple drives (e.g., 10x14TB), the system can benefit from smaller servers with less storage per unit. This approach provides better distribution of CPU, memory, and network resources. One recommended configuration is creating multiple VMs, each with a single disk, which provides both performance benefits and implicit redundancy protection.

Multi-tenant architecture

The system supports both shared and dedicated deployments. In shared mode, multiple clients can utilize the same nodes, with client-side cryptography ensuring separation. The S3 API implementation is always shared, using a unique cryptographic system where access keys and secret keys incorporate Chunk Works API keys for authentication and identity management.

Management

The Chunk Works Dashboard is designed for service providers managing customer networks. From a single interface, administrators can deploy multiple nodes simultaneously, update them, and take them offline when needed.

Each network represents a customer’s group of Chunk Works nodes. The management portal is intended for administrators of specific networks. The system eliminates the need for administrators to manage multiple tabs and server connections, streamlining the entire management process. The interface shows nodes with their IP addresses, algorithms, and includes monitoring capabilities.

Monitoring and recovery

The recovery and rebalancing processes are visible through the Liftoff interface, showing node status and storage utilization changes during recovery operations. When a node fails, the system displays the redistribution of data across remaining nodes, typically visible as percentage increases in utilization on surviving nodes.

Network management features

  • Administrators can manage multiple networks from one local location
  • The system uses configuration files that can be automatically generated
  • Direct filesystem access through Chunk Works filesystem using hardcoded seeds
  • Support for multiple nodes across different locations/providers/countries

Storage and data management

  • Data storage uses various backends including filesystem, memory, and S3
  • Uses Docker containers with attached storage volumes
  • Supports both flash storage and HDD
  • Data persistence even after container destruction
  • Storage backend options include local filesystem and Amazon S3 API

Deployment and Security

  • Networks can be created and managed via API
  • Includes monitoring capabilities for node capacity and security threats
  • Non-destructive management system – cannot permanently delete data
  • Network destruction only removes nodes, preserving underlying data
  • Supports customer offboarding while maintaining data integrity

Technical specifications

The system provides comprehensive support for both x86 and ARM architectures, with native compilation for each platform. The ARM implementation notably offers significant hardware cost reductions, making it an attractive option for cost-conscious deployments. This architectural flexibility forms the foundation of the system’s broad compatibility across different hardware environments.

Rust

The software is built in Rust. We view it as the essential programming language for handling lightweight software, which enables efficient operation including with legacy systems. This addresses the key challenge of legacy system integration. It also solves issues with servers that might not meet new requirements, such as transitioning to post-quantum cryptography. Traditionally, this would require updating your entire server infrastructure and all protocols. However, our approach is different: you can keep your existing server infrastructure and applications because our software adapts and can be installed on all existing servers and legacy applications when needed.

Rust as a programming language enforces a specific way of thinking through its built-in checks and structure. There are two critical issues that Rust simply prevents by design:

Race conditions

Race conditions typically occur when a program runs on multiple CPUs simultaneously, causing potential timing conflicts where one process might finish before another. These are eliminated by Rust’s design. The language’s ownership system and borrowing rules make it impossible to create these conditions.

Memory-related bugs

Memory-related bugs are prevented through Rust’s memory safety guarantees. The compiler enforces strict rules about memory allocation and access, making common vulnerabilities like buffer overflows impossible.