Distributed computing

Introduction and Overview of Distributed Computing
– Distributed system components are located on different networked computers.
– Components communicate and coordinate their actions by passing messages.
– Distributed computing is a field of computer science that studies distributed systems.
– Challenges of distributed systems include maintaining concurrency, overcoming the lack of a global clock, and managing independent failures.
– When a component fails, the entire system does not fail.
– Distributed systems are groups of networked computers with a common work goal.
– Concurrent computing, parallel computing, and distributed computing have overlap.
– Parallel computing is tightly coupled, while distributed computing is loosely coupled.
– In parallel computing, processors have shared memory, while in distributed computing, each processor has its own memory.
– Distributed systems are represented as a network topology, while parallel systems have direct access to shared memory.

History and Evolution of Distributed Computing
– Concurrent processes communicating through message-passing originated in the 1960s.
– Local-area networks like Ethernet were the first widespread distributed systems.
– ARPANET, the predecessor of the Internet, introduced in the late 1960s.
– ARPANET e-mail was an early large-scale distributed application.
– Distributed computing became its own branch of computer science in the late 1970s and early 1980s.
– Andrews (2000) discusses the historical development of distributed computing.
– Banks (2012) provides insights into the secret history of the internet and its founders.
– Tel (2000) explores the evolution and advancements of distributed algorithms.
– Ohlídal et al. (2006) discuss the application of evolutionary computing in distributed systems.
– Real Time and Distributed Computing Systems (PDF) provides an overview of distributed computing systems.

Architectures and Models in Distributed Computing
– Hardware and software architectures are used for distributed computing.
– CPUs are interconnected with networks at a lower level.
– Processes running on CPUs are interconnected with communication systems at a higher level.
– Distributed programming falls into architectures like client-server, three-tier, and peer-to-peer.
– Architectures can be categorized as loose coupling or tight coupling.
– Parallel algorithms in shared-memory model allow all processors to access a shared memory.
– The parallel random-access machine (PRAM) model is commonly used.
– Shared-memory programs can be extended to distributed systems with communication encapsulation.
– Parallel algorithms in message-passing model involve designing the network structure and program execution for each computer.
– Boolean circuits and sorting networks are used in this model.

Applications and Use Cases of Distributed Computing
– Distributed systems are used when data needs to be shared between multiple computers in different locations.
– They allow for larger storage, faster compute, and higher bandwidth than a single machine.
– Distributed systems provide more reliability as there is no single point of failure.
– They are easier to expand and manage compared to a monolithic uniprocessor system.
– Using a cluster of low-end computers can be more cost-efficient than a single high-end computer.
– Vigna and Casey (2015) explore the impact of cryptocurrencies and blockchain on distributed computing.
– Quang Hieu Vu et al. (2010) discuss the principles and applications of peer-to-peer computing.
– Lind and Alm (2006) present a database-centric virtual chemistry system for distributed computing.
– Chiu (1990) proposes a model for optimal database allocation in distributed computing systems.
– Elmasri and Navathe (2000) provide an overview of distributed database systems.

Algorithms, Techniques, and Challenges in Distributed Computing
– Lynch (1996) describes the fundamentals of distributed computing.
– Peleg (2000) provides an introduction to distributed systems.
– Ghosh (2007) discusses the challenges and benefits of distributed computing.
– Keidar (2008) explores the concepts and principles of distributed algorithms.
– Papadimitriou (1994) presents a comprehensive overview of distributed computing.
– Andrews (2000) discusses various algorithms and techniques used in distributed computing.
– Arora and Barak (2009) explore the trade-offs between bit, message, and time complexity in distributed algorithms.
– Cormen, Leiserson, and Rivest (1990) discuss distributed algorithms and their complexities.
– Herlihy and Shavit (2008) provide in-depth coverage of concurrent programming and synchronization in distributed systems.
– Lynch (1996) discusses the challenges and future directions in distributed computing.