New Directions For Distributed Systems Query Inwards Cloud Computing

This ship service is a continuation of my before ship service on "a distributed systems enquiry agenda for cloud computing". Here are to a greater extent than or less directions I intend are useful directions for distributed systems enquiry inwards cloud computing.

Data-driven/data-aware algorithms

Please banking concern represent the Facebook as well as Google software architecture diagrams inwards these 2 links: Facebook Software Stack, Google Software Stack. You volition break that the architecture is all most data: almost all components are most either information processing or information storage.

This tendency may betoken that the distributed algorithms should withdraw to adopt to the information it operates on to ameliorate performance. So, nosotros may encounter the adoption of machine-learning equally input/feedback to the algorithms, as well as the algorithms becoming data-driven as well as data-aware. (For example, this could live on a proficient agency to laid on Heeding the advice inwards the rootage challenge inwards my previous post, this may suggest that nosotros should hold off into implicit/diffusing/asynchronous/eventual coordination, such equally coordination past times writing to datastores as well as other processes reading off of it. Pat Helland's article suggested entity as well as activities abstractions which tin give notice live on useful primitives to acquire started on implicit/diffusing coordination.

Another agency to scale coordination is to relax consistency. It is slow to scale consistency, it is slow to scale availability, but non both! Eventual-agreement/convergent-consistency provides a agency out of this. There are already a lot of exciting operate inwards this area, as well as this surface area volition have getting to a greater extent than attention. Brewer, inwards his "CAP 12 years later" article, has given overnice clues to pursue these sort of systems. We may encounter systems that also associates incertitude amongst the consistency of electrical flow reason inwards gild to facilitate conflict recovery as well as eventual consistency.

Extremely resilient systems

Cloud computing transformed the fault-tolerance landscape. Node failures are non a large deal, thank you lot to the abundance as well as replication inwards cloud systems, the nodes are replaceable. Now complex failure modes as well as distributed reason corruption based failures became to a greater extent than critical problems. Improving the availability of these cloud systems are really of import to the aspect upwards of these unanticipated failure modes; it makes the word if a large cloud service is unavailable for several minutes.  In his advice for enquiry on cloud computing, Matt Welsh mentions these two: 1) Building failure recovery mechanisms that are robust to massive correlated outages. 2) Handling both large-scale upgrades to computing capacity equally good equally large-scale outages seamlessly, without having to completely near downwards your service as well as everything it depends on."

Self-stabilization is a corking approach to bargain amongst unanticipated faults. I am guessing nosotros volition encounter a surge inwards enquiry on self-stabilizing algorithms to accomplish extreme resiliency to the aspect upwards of unanticipated faults inwards cloud computing systems. Recovery oriented computing (ROC), resettable systems (crash-only software) is a special instance of self-stabilization. And nosotros may encounter extensions of that operate for distributed systems. A critical inquiry hither volition live on "How tin give notice nosotros brand ROC compose nicely for distributed systems?"

To insure against correlated failures, nosotros may encounter multiversion programming approaches to live on revisited. This tin give notice also live on helpful to avoid the spooky/self-organizing synchronization inwards cloud computing systems.

For scalable fault-tolerance, asynchronous algorithms similar self-stabilizing Propogation of Information amongst Feedback (PIF) algorithms may live on adopted inwards the cloud domain. Furthermore, pheromene/hormone based algorithms that run inwards the background inwards a boring vogue tin give notice live on made extremely scalable exploiting peer-to-peer random-gossip techniques.

New graph-based programming abstractions for the cloud

Good programming abstractions are similar proficient tools, they tin give notice boost productivity past times several folds. In agency of analogy, inwards wireless sensor networks, several interesting programming abstractions were proposed including, treating neighborhood or surface area of nodes equally the unit of measurement of programming instead of elementary node, stream-based programming (map/join), excel-spreadsheet similar high-level programming. These abstractions convey a unlike perspective which tin give notice live on really helpful. There has been operate on designing programming abstractions for cloud computing systems, peculiarly for dealing amongst large information as well as large graphs. I promise nosotros tin give notice encounter novel useful abstractions emerge for programming large scale distributed cloud services. Since scalability is really of import for cloud systems, nosotros may encounter hierarchical abstractions, logical tree abstractions. We may also encounter abstractions that capture telephone phone graph of services or dataflow through services.

Auditability tools

With really large scale complex distributed systems, observability/auditability becomes really important. We latterly presented our proposal on this topic inwards HotCloud'15. I promise to write a ship service most this operate soon.

Abstract models to kickstart algorithms work

Finally, I promise nosotros volition encounter theoretical abstract modeling to simplify the cloud computing model (goals, challenges, environment) as well as kickstart to a greater extent than algorithms operate on the area. As an analogy, Dijkstra's token band formulation was actually transformative, as well as started the self-stabilization champaign of distributed systems. A useful abstraction volition enshroud irrelevant/accidental details as well as allow operate to focus on the inherent most of import parts of the problem, as well as allow other researchers to adopt the same terminology/model as well as start edifice on each others operate as well as improve.

Share this post