Mirage: ML kernels in the cloud (ML Workshop 2010)

Mirage: ML kernels in the cloud Anil Madhavapeddy, University of Cambridge Thomas Gazagnaire, INRIA Computer Laboratory, 15 JJ Thomson Avenue, Cambridge, UK Contributions from Tim Deegan (Citrix), Steven Hand (Cambridge), Steven Smith (Cambridge) , Jon Crowcroft (Cambridge)

Motivation: Layers Hardware Processes OS Kernel Threads Application

Motivation: Layers Hardware Processes OS Kernel Threads Application Language Runtime

Motivation: Layers Hardware Processes OS Kernel Threads Application Hypervisor Language Runtime

Motivation: Security Linux Kernel Mar 1994: 176,250 LoC May 2010: 13,320,934 LoC Most core Internet services still written in C / C++

Mirage: Approach Construct an OS designed to run on the cloud End-to-end static type safety using OCaml + DSLs More build-time analysis (“whole OS optimisation”) Simple single-threaded core, using the hypervisor to divide up cores

Mirage: OS “signature” begins module Console : sig type t val create : unit -> t val write : t -> string -> int -> int -> unit end

Mirage: A simple “hello world” Xen runs para-virtualized kernels, that cooperate with the hypervisor. Most code runs unmodified Privileged instructions (page table updates) go via Xen hypercalls $ echo ‘let _ = print_endline “hello Xen world!”’ > hello.ml $ ocamlopt –output-obj –o app.o hello.ml Linked to a “Xen MiniOS” to make a bootable kernel. Boots in 64-bit mode directly, with starting memory all mapped. Is approximately 50-100KB in size.

Mirage: 64-bit Xen Memory Layout OS Text and Data Network Buffers Reserved OCaml minor heap OCaml major heap

Mirage: Network Buffers OS Text and Data Network Buffers Reserved OCaml minor heap OCaml major heap IP Header TCP Header Transmit packet data IP Header TCP Header Receive packet data

Mirage: x86 superpages for OCaml heap OS Text and Data Network Buffers Reserved OCaml minor heap OCaml major heap Reduces TLB pressure significantly. Is_in_heap check is much simpler Improve GC/cache interaction using PAT registers?

Mirage: Typed Memory Allocators OS Text and Data Network Buffers Reserved OCaml minor heap OCaml major heap Buddy Allocator dyn_init(type) dyn_malloc(type, size) dyn_realloc(size) dyn_free(type) Heap Allocator heap_init(type, pages) heap_extend(type, pages) heap_shrink(type, pages) Page Grant Allocator grant_alloc_page(type) grant_free_page(type)

Mirage: concurrency Xen provides an low-level event interface. Optional interrupts: a perfect fit for co-operative threading! We always know our next timeout (priority queue) So adapted the excellent LWT threading library Block 5s

Mirage: extending the OS signature with timing

Mirage: concurrency using LWT Advantages: Core library is pure OCaml with no magic Excellent camlp4 extension to hide the bind monad. Function type now clearly indicates that it blocks. Open Issues: Creates a lot of runtime closures (lambda lifting, whole program opt?) Threat model: malicious code can now hang whole OS

Mirage: and parallelism? Xen divides up cores into vCPUs, LWT multiplexes on a single core Mirage “process” is a separate OS, communicating via event channels Open Question: parallelism model (JoCaml, OPIS, explicit futures) vCPU 1 vCPU 2 Mem 1 Mem 2 SHM

Mirage: I/O I/O comes in via Ethernet frames, and “zero copy” parsed via a DSL. We have Ethernet, ARP, ICMP, IPv4, DHCP, TCPv4, HTTP, DNS, SSH all implemented in pure OCaml. Performance in user-space is excellent (EuroSys 2007), now benchmarking under Xen (submission to PLDI 2011) . Zero-copy, bounds optimisation is vital to performance. Ethernet IP TCP Data

Mirage: I/O via MPL packet tcp { source_port: uint16; dest_port: uint16; sequence: uint32; ack_number: uint32; offset: bit[4] value(offset(header_end) / 4); reserved: bit[4] const(0); cwr: bit[1] default(0); ece: bit[1] default(0); urg: bit[1] default(0); ack: bit[1] default(0); psh: bit[1] default(0); rst: bit[1] default(0); syn: bit[1] default(0); fin: bit[1] default(0); window: uint16; checksum: uint16; urgent: uint16 default(0); header_end: label; options: byte[(offset * 4) - offset(header_end)] align(32); data: byte[remaining()]; } OCaml output can both construct and parse packets from this DSL. Melange: Towards a ‘Functional’ Internet EuroSys 2007, Madhavapeddy et al.

Mirage: I/O via metaprogramming Seeking a more general solution PADS MetaOCaml BER Requirements: Replace OCaml backend with LLVM parser for data plane (goal is 10GB/s type-safe network I/O). Would like to specify file-systems in this way also.

Mirage: a “multi-scale” operating system type sockaddr = | TCP of ipv4_addr * int | UDP of ipv4_addr * int module Flow : sig type t val read: t -> string -> int -> int -> int Lwt.t val write: t -> string -> int -> int -> int Lwt.t val connect: sockaddr -> (t -> unit Lwt.t) -> unit Lwt.t val listen: (sockaddr -> t -> unit Lwt.t) -> sockaddr -> unit Lwt.t end

Mirage: a “multi-scale” operating system Extremely portable signature permits it to adapt to many environments! POSIX/TUNTAP : normal OCaml runtime + Ethernet tap using ML network stack (I/O: Ethernet and higher) POSIX : with normal sockets (I/O: TCP/UDP and higher) Both use high performance kqueue/epoll backend. Javascript using ocamljs (Jake Donham) with WebSockets I/O Google AppEngine using ocamljava (Xavier Clerc) with HTTP I/O Android, iMotes using ocamlopt ARM backend.

Mirage: ML kernels in the cloud (ML Workshop 2010)

Recommended

More Related Content

What's hot (20)

Viewers also liked (20)

Similar to Mirage: ML kernels in the cloud (ML Workshop 2010) (20)

Mirage: ML kernels in the cloud (ML Workshop 2010)