Paper Summary. Real-Time Automobile Learning: The Missing Pieces
This paper, dated March 11, 2017 on arxiv, is from UB Berkeley. Here is Prof. Michael Jordan's Strata Hadoop conference utter on this.
The newspaper commencement motivates the demand for real-time auto learning. For this it mentions in-situ reinforcement learning (RL) that closes the loop past times taking actions that touching on the sensed environment. (The minute paragraph mentions that such RL tin live on trained to a greater extent than feasibly past times using simulated/virtual environments: past times commencement trying multiple actions [potentially inwards parallel] to meet their touching on inwards simulation earlier interacting amongst the existent world. Again this requires real-time surgical operation every bit the simulation should live on performed faster than real-time interaction.)
Based on this application scenario, hither are their desired requirement from the ML platform.
R1: depression latency
R2: high throughput
R3: dynamic business creation (RL primitives such every bit Monte Carlo tree search may generate novel tasks during execution)
R4: heterogeneous tasks (tasks would convey widely dissimilar execution times together with resources requirements)
R5: arbitrary dataflow dependencies (BSP doesn't cutting it)
R6: transparent fault-tolerance
R7: debuggability together with profiling
Two principal architectural features are a centralized command plane together with a hybrid scheduler. The centralized command set down is held past times Redis, a replicated key-value store. It looks similar the command set down does non convey whatever command logic inwards it, it is merely passive storage. (So TensorFlow's variables also qualify every bit command state.) The hybrid scheduler stance aims to tending amongst providing low-latency. Workers submit tasks to their local schedulers which produce upwardly one's hear to either assign the tasks to other workers on the same physical node or to “spill over” the tasks to a global scheduler. Global schedulers tin thus assign tasks to local schedulers based on global information nigh resources availability. Neither the logically centralized command set down nor the two-level hierarchy scheduling are novel innovative concepts.
The tasks creation is left totally to the application developer. Any business tin produce novel tasks without blocking on their completion, but this creates a dependency from the caller to the callee. Moreover, Ray uses the dataflow execution model inwards which tasks croak available for execution if together with exclusively if their dependencies convey finished executing. The combination of this unrestrained business creation amongst hybrid scheduling provides a lot of rope to the developer to hang himself.
Tasks are called amongst split-phase asynchronous execution model. When you lot telephone band a task, the business returns "future", which merely denotes acknowledgement, but the business volition after telephone band you lot dorsum amongst the lawsuit when its computation is completed. The caller may potentially telephone band "get" on the hereafter to block until the callee finishes execution. Ray also has a "wait" primitive to fourth dimension out from waiting on straggler tasks. Again it is the developer's responsibleness to figure out how to utilisation this correctly.
While Ray aims real-time auto learning, it doesn't convey a means for shedding load. To render charge shedding support, it is possible to adopt the SEDA architecture, thus the organisation does non grind to a stop when it is presented amongst likewise many tasks at once.
The newspaper commencement motivates the demand for real-time auto learning. For this it mentions in-situ reinforcement learning (RL) that closes the loop past times taking actions that touching on the sensed environment. (The minute paragraph mentions that such RL tin live on trained to a greater extent than feasibly past times using simulated/virtual environments: past times commencement trying multiple actions [potentially inwards parallel] to meet their touching on inwards simulation earlier interacting amongst the existent world. Again this requires real-time surgical operation every bit the simulation should live on performed faster than real-time interaction.)
Based on this application scenario, hither are their desired requirement from the ML platform.
R1: depression latency
R2: high throughput
R3: dynamic business creation (RL primitives such every bit Monte Carlo tree search may generate novel tasks during execution)
R4: heterogeneous tasks (tasks would convey widely dissimilar execution times together with resources requirements)
R5: arbitrary dataflow dependencies (BSP doesn't cutting it)
R6: transparent fault-tolerance
R7: debuggability together with profiling
The platform architecture
The newspaper does non give a scream to the platform, but the utter calls it Ray. Ray allows arbitrary functions to live on specified every bit remotely executable tasks, amongst dataflow dependencies betwixt them. Ray uses imperative programming together with does non back upwardly symbolic computation graphs, every bit far every bit I tin see. The utter mentions that programming is done inwards Python. So, at this quest Ray is to a greater extent than similar a prepare of Python libraries paired amongst Redis database for keeping command set down together with amongst Spark RDD back upwardly for maintaining object-store every bit shared memory.
Two principal architectural features are a centralized command plane together with a hybrid scheduler. The centralized command set down is held past times Redis, a replicated key-value store. It looks similar the command set down does non convey whatever command logic inwards it, it is merely passive storage. (So TensorFlow's variables also qualify every bit command state.) The hybrid scheduler stance aims to tending amongst providing low-latency. Workers submit tasks to their local schedulers which produce upwardly one's hear to either assign the tasks to other workers on the same physical node or to “spill over” the tasks to a global scheduler. Global schedulers tin thus assign tasks to local schedulers based on global information nigh resources availability. Neither the logically centralized command set down nor the two-level hierarchy scheduling are novel innovative concepts.
The tasks creation is left totally to the application developer. Any business tin produce novel tasks without blocking on their completion, but this creates a dependency from the caller to the callee. Moreover, Ray uses the dataflow execution model inwards which tasks croak available for execution if together with exclusively if their dependencies convey finished executing. The combination of this unrestrained business creation amongst hybrid scheduling provides a lot of rope to the developer to hang himself.
Tasks are called amongst split-phase asynchronous execution model. When you lot telephone band a task, the business returns "future", which merely denotes acknowledgement, but the business volition after telephone band you lot dorsum amongst the lawsuit when its computation is completed. The caller may potentially telephone band "get" on the hereafter to block until the callee finishes execution. Ray also has a "wait" primitive to fourth dimension out from waiting on straggler tasks. Again it is the developer's responsibleness to figure out how to utilisation this correctly.
Conclusions
I intend the platform is weak on "ease of use". Ray is thus minimal that it is unclear if nosotros couldn't convey gotten the same degree of back upwardly from using a systems programming linguistic communication amongst concurrency primitives together with thread safety, such every bit Rust. Rust uses the thespian model together with is really suitable for edifice a dataflow execution application, as has been demonstrated past times rewriting Naiad on Rust recently.While Ray aims real-time auto learning, it doesn't convey a means for shedding load. To render charge shedding support, it is possible to adopt the SEDA architecture, thus the organisation does non grind to a stop when it is presented amongst likewise many tasks at once.
0 Response to "Paper Summary. Real-Time Automobile Learning: The Missing Pieces"
Post a Comment