Distributed hash table

« Back to Glossary Index

History and Properties of Distributed Hash Tables
– DHT research was motivated by peer-to-peer systems like Freenet, Gnutella, BitTorrent, and Napster.
– Napster used a central index server, making it vulnerable to attacks and lawsuits.
– Gnutella used a query flooding model, which was less efficient than Napster.
– Freenet employed key-based routing, clustering similar files on the same set of nodes.
– DHTs combined the decentralization of Freenet and Gnutella with the efficiency of Napster.
– Autonomy and decentralization are key characteristics of DHTs.
– DHTs should be fault-tolerant, handling continuous node arrivals, departures, and failures.
– Scalability is important, allowing efficient operation with thousands or millions of nodes.
– DHTs minimize coordination between nodes, reducing the work required for membership changes.
– Some DHT designs focus on security and anonymity, although less common than in other P2P systems.

Structure and Keyspace Partitioning in Distributed Hash Tables
– DHTs consist of an abstract keyspace, such as a set of 160-bit strings.
– Keyspace partitioning schemes distribute ownership of the keyspace among nodes.
– Overlay networks connect the nodes and enable key lookup.
– To store a file, its filename is hashed to generate a key, which is sent to a participating node.
– Retrieving a file involves hashing the filename to find the node responsible for the key.
– Most DHTs use consistent hashing or rendezvous hashing to map keys to nodes.
– Consistent hashing and rendezvous hashing minimize remapping of keys during node changes.
– In consistent hashing, the circular keyspace is divided into contiguous segments owned by nodes.
– Rendezvous hashing assigns keys based on the highest random weight.
– Both algorithms ensure that adding or removing a node affects only adjacent keys.

Applications of Distributed Hash Tables
– DHTs serve as infrastructure for various services like anycast, distributed file systems, and domain name services.
– Notable networks using DHTs include BitTorrent’s distributed tracker, the Kad network, and Freenet.
– DHT technology is adopted by BitTorrent and the Coral Content Distribution Network.
– DHTs enable peer-to-peer file sharing and content distribution systems.
– DHTs can be used for instant messaging, multicast, and building search engines like YaCy.

Overlay Networks and Algorithms in Distributed Hash Tables
– Each node maintains a set of links to other nodes (neighbors or routing table).
– These links form the overlay network.
– Nodes pick neighbors based on the network’s topology.
– All DHT topologies ensure that for any key k, each node either owns k or has a link to a node closer to k.
– Key-based routing is used to forward messages to the owner of any key k.
– Aside from routing, algorithms exploit the overlay network structure for message dissemination.
– Structella implements flooding and random walks on a Pastry overlay.
– DQ-DHT implements a dynamic querying search algorithm over a Chord network.
– These algorithms are used for overlay multicast, range queries, and collecting statistics.
– They leverage the structure of the overlay network to optimize message distribution.

Security and Techniques in Distributed Hash Tables
– DHTs are inherently more resilient against hostile attackers compared to centralized systems.
– Byzantine fault tolerance can defend against the Sybil attack in DHTs.
– Whanau is a DHT designed to be resistant to Sybil attacks.
– Incorporating social trust relationships into the system design can circumvent the Sybil attack.
– Effective defenses against Sybil attacks are an ongoing research topic in security conferences.
– DHT security techniques surveyed by Urdaneta, Pierre, and van Steen.
– Novel architectures for P2P applications: the Continuous-Discrete Approach.
– Dipsea: A Modular Distributed Hash Table.
– Structured overlay for heterogeneous environments.
– Self-Chord: A Bio-Inspired P2P Framework for Self-Organizing Distributed Systems.

A distributed hash table (DHT) is a distributed system that provides a lookup service similar to a hash table. Key–value pairs are stored in a DHT, and any participating node can efficiently retrieve the value associated with a given key. The main advantage of a DHT is that nodes can be added or removed with minimum work around re-distributing keys. Keys are unique identifiers which map to particular values, which in turn can be anything from addresses, to documents, to arbitrary data. Responsibility for maintaining the mapping from keys to values is distributed among the nodes, in such a way that a change in the set of participants causes a minimal amount of disruption. This allows a DHT to scale to extremely large numbers of nodes and to handle continual node arrivals, departures, and failures.

DHTs form an infrastructure that can be used to build more complex services, such as anycast, cooperative web caching, distributed file systems, domain name services, instant messaging, multicast, and also peer-to-peer file sharing and content distribution systems. Notable distributed networks that use DHTs include BitTorrent's distributed tracker, the Kad network, the Storm botnet, the Tox instant messenger, Freenet, the YaCy search engine, and the InterPlanetary File System.

Distributed hash tables
« Back to Glossary Index

Submit your RFP

We can't wait to read about your project. Use the form below to submit your RFP!

Gabrielle Buff
Gabrielle Buff

Just left us a 5 star review

Great customer service and was able to walk us through the various options available to us in a way that made sense. Would definitely recommend!

Stoute Web Solutions has been a valuable resource for our business. Their attention to detail, expertise, and willingness to help at a moment's notice make them an essential support system for us.

Paul and the team are very professional, courteous, and efficient. They always respond immediately even to my minute concerns. Also, their SEO consultation is superb. These are good people!

Paul Stoute & his team are top notch! You will not find a more honest, hard working group whose focus is the success of your business. If you’re ready to work with the best to create the best for your business, go Stoute Web Solutions; you’ll definitely be glad you did!

Wonderful people that understand our needs and make it happen!

Paul is the absolute best! Always there with solutions in high pressure situations. A steady hand; always there when needed; I would recommend Paul to anyone!

facebook
Vince Fogliani
recommends

The team over at Stoute web solutions set my business up with a fantastic new website, could not be happier

facebook
Steve Sacre
recommends

If You are looking for Website design & creativity look no further. Paul & his team are the epitome of excellence.Don't take my word just refer to my website "stevestours.net"that Stoute Web Solutions created.This should convince anyone that You have finally found Your perfect fit

facebook
Jamie Hill
recommends

Paul and the team at Stoute Web are amazing. They are super fast to answer questions. Super easy to work with, and knows their stuff. 10,000 stars.

Paul and the team from Stoute Web solutions are awesome to work with. They're super intuitive on what best suits your needs and the end product is even better. We will be using them exclusively for our web design and hosting.

facebook
Dean Eardley
recommends

Beautifully functional websites from professional, knowledgeable team.

Along with hosting most of my url's Paul's business has helped me with website development, graphic design and even a really cool back end database app! I highly recommend him as your 360 solution to making your business more visible in today's social media driven marketplace.

I hate dealing with domain/site hosts. After terrible service for over a decade from Dreamhost, I was desperate to find a new one. I was lucky enough to win...

Paul Stoute has been extremely helpful in helping me choose the best package to suite my needs. Any time I had a technical issue he was there to help me through it. Superb customer service at a great value. I would recommend his services to anyone that wants a hassle free and quality experience for their website needs.

Paul is the BEST! I am a current customer and happy to say he has never let me down. Always responds quickly and if he cant fix the issue right away, if available, he provides you a temporary work around while researching the correct fix! Thanks for being an honest and great company!!

Paul Stoute is absolutely wonderful. Paul always responds to my calls and emails right away. He is truly the backbone of my business. From my fantastic website to popping right up on Google when people search for me and designing my business cards, Paul has been there every step of the way. I would recommend this company to anyone.

I can't say enough great things about Green Tie Hosting. Paul was wonderful in helping me get my website up and running quickly. I have stayed with Green...