"Production-quality Mesos frameworks must be able to continue managing tasks despite unreliable networks and faulty computers. Mesos provides tools to help developers do fault-tolerant task management, but putting these tools together effectively remains something of a black art. This talk will offer practical guidance to current and prospective framework developers to help them understand how Mesos deals with failures and the tools it provides to enable fault tolerant frameworks. Mesos operators will also benefit from a discussion of exactly how Mesos behaves during network partitions and other failure scenarios.  This talk will cover the following specific topics: * fault tolerance in Mesos itself: how Mesos masters and agents behave in the face of process crashes and network partitions * the tools that Mesos provides to help framework authors write reliable systems (e.g., task state reconciliation, the state abstraction, and the MasterDetector interface) * the lifecycle of a Mesos task * a collection of recommendations for how framework developers should build highly available framework schedulers and executors"

"The Apache Mesos project allows a developer to orchestrate and utilize a cluster of machines as if it were a single pool of resources. Engineers from multiple disciplines are writing “frameworks” to take advantage of the redundancy and scalability that Mesos provides. During the development of the ELK stack on Mesos (Elasticsearch, Logstash and Kibana) we found that the Mesos and Docker community is sorely missing tools to enable integration tests and projects to reduce boilerplate code. We decided to create our own tool for experimenting with Mesos and testing frameworks: minimesos, available at   https://www.minimesos.org  Minimesos is an experimentation and testing tool which can be used as a standalone cli application or from within a JUnit test. Start a containerized Mesos cluster, start the scheduler of your framework and perform assertions. At the end of the test the cluster is destroyed. In this talk I will introduce minimesos, it's features, design and implementation and what is on the horizon."

"CERN developed and is using CernVM File System (CVMFS) to distribute terabytes of software to hundreds of thousands of nodes across hundreds of datacenters. CVMFS uses a combination of lazy loading, extensive indexing, de-duplication, caching and geographic distribution. This help to significantly reduce the network load and speed-up the distribution. All these features make CVMFS a very attractive candidate for solving container-image management and distribution problem in Mesos.  In this talk we are going to present our work on integrating CVMFS client with Mesos agents as well as show how CVMFS backend can be used as a secure and scalable container image registry. We will also run a demo in which we will show how can one launch a container running a popular Linux distribution (image size is about ~300MB) by only downloading a tiny fraction of the image (~6MB)."

Containers represent a portable unit of deployment, and OpenStack has proven an ideal environment for running these workloads. However, where it becomes complex is that many times an application is often built out of multiple containers, as well as hybrid environments - diverse clouds, bare metal & even non-virtualized infrastructure. What’s more, setting up a cluster of container images can be fairly cumbersome as you need to make one container aware of another and expose intimate details that are required for them to communicate which is not trivial especially if they’re not on the same host. These scenarios have instigated the demand for some kind of orchestrator. The list of container orchestrators is growing fairly fast. This session will compare the different orchestration projects out there - from Heat to Kubernetes to Mesos & Cloudify - and help you choose the right tool.

"Waiter is an auto-scaling web proxy developed at Two Sigma investments to be open sourced soon. Waiter removes the need to maintain static allocations of web services to meet client demand. Waiter determines the ideal number of web server instances to run on a Mesos cluster for a given web service based upon its HTTP traffic. Waiter alleviates versioning pain by spinning up the correct version of a service that a client was designed to run against. Waiter provides a platform for running all types of web services and applications, removing the need for service creators to maintain their own web infrastructure.  Auto-scaling is a hard problem especially in an enterprise environment where a diverse of applications with a large variety of workload patterns compete scarce resources . We run Waiter at Two Sigma to accommodate different types of applications, from computational resource intensive apps serving programmatic requests to interactive applications like websites. These different applications impose very different QoS requirements. Waiter’s auto-scaling logic accommodates the diverse range of requirements yet keeps the simplicity and agility.  This talk will discuss how Waiter works, how Waiter scales applications based upon web traffic, the factors we considered when we design the auto scale algorithm to balance the simplicity, diversity and reliability. and lessons learned working with Mesos and Marathon."

"Docker Swarm provides a native docker experience on a cluster of Docker hosts, and supports orchestration tools such as Compose. Kubernetes offers a model based on PODs, replication controllers and services. These two orchestration and clustering solutions are currently being developed in two different communities, and adopters would pick one or the other based on their workload requirements and DevOps toolchain. Mesos provides a highly scalable platform to efficiently manage resources allocation for these two frameworks, providing a richer choice of APIs to end users. In this talk, we will share our experience with developing a container cloud service based  on Swarm and Kubernetes on Mesos. We will discuss the challenges of providing multi-tenancy for these frameworks, sharing a common software defined networking solution based on OpenStack Neutron. and integrating with a common storage service. Finally, we will share our lessons learned while developing this service, and provide a short demonstration of the container service capabilities."

"Writing your own Mesos framework is quite exciting. But, is it necessary or even a good idea? This lightning talk will take a critical look at the implications of writing your own framework, especially in regard to multitenancy and the desire to run dozens of framework instances on top of Mesos. The talk will compare this to an architecture where you launch everything with a single service scheduler such as Apache Aurora."

"When was the last time you built something that no one ended up using? When was the last time you read a customer support ticket or GitHub issue where a user couldn’t figure out how to use a particular feature that you worked on?  This happens all the time. At every company. Big and small. We can help circumvent this by conducting user research, understanding customer needs and usability testing our proposed solutions.  We’ll take a look at what it means to be user-centered, conduct user tests and really focus on putting customer experience first.  - User research methods - Testing prototypes early and often - How to setup and run DIY user tests - How Mesosphere set up a system for continuous testing - Examples of lessons learned testing Mesos, DC/OS and other developer tools"

"The ability to run both HPC and Big Data frameworks together on the same machine is a principal design goal for future Cray analytics platforms. Hadoop workloads use the YARN resource manager. Spark is a more general-purpose cluster-computing framework that can be run using YARN, Mesos or its own Standalone resource manager. The Cray Graph Engine (CGE) supports real-time analytics on the largest and most complex graph problems. CGE is a more traditional HPC application that runs under either Slurm or PBS. Traditionally, running workloads that require different resource managers would require static partitioning of the cluster leading to under-utilization of resources.  To avoid this, we leverage Apache Mesos and create interfaces to allow other resources managers (Slurm and YARN) to dynamically acquire and release resources from Mesos. In this talk, we present our experiences running mixed workloads on an experimental Cray platform with dynamic resource partitioning."

"Historically, developing new frameworks for Mesos has been a challenge for framework developers primarily due to a non-standard framework API and native library dependencies.  Support for v1 Framework API was added to Mesos, which allows developers to write frameworks/custom executors using a stable HTTP API. In this tutorial, we will demonstrate how to build a framework using the new API. The framework would use v1 Scheduler API to talk to Mesos master and a custom executor based on the v1 Executor API to talk to Mesos agent."

"One of the major initiatives in the Mesos community this year has been the addition of a CMake-based build system.  This change is necessary: As Mesos grows to support Windows (where autotools is difficult to use), it will become vital to have a robust, cross-platform build system.  But it is also very useful: CMake can generate not only makefiles, but also project files for IDEs like Eclipse. Robust support for macros make it trivial to do useful tasks that would require hours of time fiddling with autotools. Using CMake, it is also relatively easy to take advantage of (e.g.) static analysis projects like `clang-tidy`. And so on.  In this talk, we will provide an architectural overview of this new build system. We will recap existing features using a few demos that highlight the integration with developer tooling like `clang-tidy`. Finally, we will provide a roadmap of features forward."

Deploying Mesos in an enterprise-scale data center is not trivial. This talk will touch on the challenges of deploying and maintaining Mesos for production workloads at scale in enterprise cloud environments. We will discuss the puppet-dcos module, which was co-written by Mesosphere and Puppet Labs, the obstacles we faced in writing it and the challenges that still exist in administering large scale, Mesos clusters and managing those clusters with configuration management.

We will present the current state of the art for Spark on Mesos, including their origins together at UC Berkeley AMPlab, features such as dynamic allocations, major users of Mesos, such as Apple Siri, and the details of job invocation and resource allocation. We will highlight features in the Spark and Mesos integration, emphasizing new features and development in Mesos that can improve Spark deployment and scheduling, such as Quotas and GPU isolation. We’ll finish with a few stories from real-world deployments and show a demo of the Spark on Mesos integration in the context of the maturing SMACK/Infinity stack, integrating Spark, Mesos, Akka, Cassandra, and Kafka (plus other tools) for streaming applications.

The community of Mesos contributors is growing all the time, and we enjoy hearing the fresh ideas that these new faces bring. In this lightning talk, we will explain the procedure for becoming a Mesos contributor, overview the infrastructure we use for collaboration, and provide guidance on finding your first issue to work on.

"Testing software is hard enough when it’s constrained to a single machine, but testing distributed systems that consist of multiple components adds a whole new level of complexity. Usually, a new cluster of real, virtual, or emulated machines needs to be created, provisioned, and destroyed for every test run. That means release building, dependency management, cluster deployment, configuration management, and parallel provisioning all must be automated. Throw in responsiveness to continuous changes and the test harness might end up being more complex than the system it’s testing!  I’ll review several options for locally testing Mesos frameworks, Marathon applications, and DCOS services. Then I’ll go over some lessons I've learned from testing continuously, including how to containerize tests, pipelining strategies, and tactics for failure debugging."

Container technology are being evaluated by software developers and administrators with a great deal of interest. Developers want to focus on what they do best: Creating and coding new applications. That shouldn't have to change just because they need to deploy an application to a different environment. Administrators want the environment to stay reliable and stable, keeping changes at a minimum. By following a strategy that embraces good Architecture principles, use of Containers, DevOps philosophy, Apache Mesos and a Cloud based environment, architects, developers and operators can plan, create, consume and collaborate on the infrastructure configuration over the time, deploy applications and test your application infrastructure consistently regardless of the stage of the development life cycle.

"Mesos modules provides third-party developers and vendors with a way to extend Mesos functionality without modifying Mesos itself. Since its inception, Mesos module functionality has been extended to provide functionality for other subsystems such as isolation, authentication, authorization and resource estimation.  The modules started off as an expert-only mechanism making it harder for non-Mesos developers to write their own modules. In this talk, we will talk about the developments that have taken place to make module writing almost as easy as writing Mesos frameworks."

"This is a Mesos 101 session similar in nature to what was delivered at MesosCon 2014. We will talk about what Mesos does from a new users perspective. We assume you don't know anything about what Mesos is nor what it can do. We will show several ways to install Mesos as well as components in the Mesos ecosystem. By the end of this session you will be able to describe what Mesos can do as well as articulate its benefits. You will also know several ways you can deploy Mesos and get up and running quickly.   This session will be presented from a new users perspective related to the work we've done at Cisco. When we started a year ago we had no idea what mesos was, nor our options. We are now running our internal project ""PIpeline"" in production on top of Mesos."

"SMACK is about a full stack for Big Data Architecture, it is Scala/Spark, Mesos, Akka, Cassandra and Kafka. We are going to talk about the most important: the integration between the parts.  As a result the audience will be have the fundaments of: How to make Big Data architecture without using complex Greek letter architectures. How to build a cheap but effective cluster infrastructure. How to make queries, reports and graphs that business demands. How to manage and exploit unstructured and No-SQL data sources. How use tools to monitor the performance of your architecture. How to integrate all technologies and decide which replace and which reinforce.  Some large companies are using a variation of SMACK in production, particularly those looking at how to take their big data projects forward.  The fundamental premise of SMACK is build an end-to-end data processing pipeline having these components interacting in a way that integration is simple and ""get up and running"" task is quick, rather than requiring huge amounts of effort to get the tools to play nicely with each other."

"Mesos will soon reach the 1.0 milestone. In addition to new features, this would mean a more stable user-facing API and stricter support/release guarantees for operators/framework developers. The aim of this talk is to apprise the operators/framework developers/users about the new API and also discuss the support/compatibility guarantees offered by Mesos going forward.  This talk is a sequel to the last year’s MesosCon Seattle talk on “Mesos HTTP API” and continues from where it left off.  This talk will cover the following specific topics: - Discuss the newly introduced Operator APl. - Update on the recent improvements to the Framework API. - Update on client libraries for the new Framework API. - Release cadence for Mesos going forward. - Support/Compatibility guarantees for operators/framework developers e.g., backporting of patches etc. - Master->Agent renaming in the 1.0 API."

"Continuous delivery is all the rage these days but without self healing, highly available and fault tolerant infrastructure to deploy your applications to, it's really only one piece of a much larger picture.  In this session, we'll show you how can integrate the stalwart continuous integration server Jenkins with Apache Mesos and Marathon. This integration allows you to set up a continuous delivery pipeline that takes your application from code repository to DockerHub to a staging or production server with seamless automation. We'll demonstrate a working continuous delivery pipeline based on our own experiences at Mesosphere which audience members should be able to easily replicate on their own infrastructure.  Using Jenkins with Mesos allows you to spin up build agents dynamically, an approach which has allowed companies like PayPal to cut the footprint of their build farms by hundreds of nodes, saving $$$ on infrastructure by increasing utilization and reducing obstacles to providing teams with the resources they need when they need them."

"In this presentation the speakers, who work at Verizon Labs, will present a distributed service that leverages the capabilities of Mesos to implement an IoT device messaging application that directly interfaces to the wireless network infrastructure. This service must scale to support an extremely high number of end devices, be reliable, and highly available. The high-level micro services architecture will be presented, focusing on where Mesos features are leveraged. Detailing this service will show how in addition to Mesos's ability to manage compute and memory resources the team handled network connections and scaling and used virtualized storage within the Mesos cluster. The service under the load of millions of simulated devices will be presented. The simulation itself, highly leverages the Mesos platform to scale the number of devices that can be simulated, thus improving the development time. The speakers will review how this architecture, built around Mesos, compares with similar services built, and deployed, in the past. Detailing where Mesos adds value in both the development, deployment and operations of a highly resilient service."

"The goal of this session is to showcase the effort being put in to enable real-time Network telemetry on routers and switches using cluster schedulers like Apache Mesos. There are a variety of solutions today that enable provisioning of network infrastructure prior to application orchestration, such as, project calico, contiv etc. However, post-provisioning, there is no way to detect network degradation except for the myopic view that server applications may have. For this purpose, Akshat Sharma will introduce a Mesos framework called ""Qoson"" , which runs alongside traditional frameworks like Marathon, and performs real-time telemetry tasks to make intelligent decisions for app placement based on the role of a network device. With the advent of container based application-hosting on network devices running Cisco's IOS-XR , we enable a real-time distributed network telemetry database to feed into Mesos and enable intelligent actions that could range from Marathon's constraints on app-placement to SDN controller driven policy changes. This session will include a live demo of QoSon's capabilities. The current code can be found here: https://github.com/akshshar/QoSon"

When operating an Mesos Cluster, one of the challenge is efficiently using the resources. One of the benefit of mesos is scheduling application on demand and so we can take down applications and bring it up any time. One of the use case in companies which have presence across Geo Location is that the resources which are wasted as there are times where some running application won't be used. The idea is to the put the application ( like Jenkins master instances, Testing environment etc.,) which is not getting traffic for a predetermined 'N' hours/minutes and then bring the application back from hibernation when the first request of traffic comes again. This requires integration with Software LoadBalancer which needs capabilities like events based on traffic pattern and traffic hits.

"Production-quality Mesos frameworks must be able to continue managing tasks despite unreliable networks and faulty computers. Mesos provides some tools to assist developers with writing highly available frameworks (such as task reconciliation and the state abstraction), but putting these tools together effectively is something of a black art. This talk will discuss:    * how Mesos masters and agents behave in the face of process crashes and network partitions  * the tools that Mesos provides to framework authors to help them write reliable systems  * a collection of ""best practices"" for writing highly available framework schedulers and executors"

Scaling a large user base with a growing demand for a CI pipeline with an ever increasing demand on multiple platforms, we had a peculiar problem of catering a common image across the flavor and that was to “define an environment / pipeline that was specific to each request “ . our mission started with Jenkins on mesos and now we have a Dockerized pipeline that enables the multi level requirements thus creating a Dockerized CI environment.

Framework authors attempting to write frameworks which encapsulate or administer stateful services often encounter difficulties mapping the resource offer stream provided by Mesos to the framework’s provisioning, maintenance and upgrade goals. For disparate frameworks: Cassandra, Kafka, HDFS we have adopted a declarative approach to address this mapping problem. This approach has also been applied successfully to different layers of the framework lifecycle. In particular, we have applied a declarative approach to the fault-tolerant fulfillment of reservation and persistent volume requirements, and to the overall maintenance and upgrade of frameworks.

"Time Warner Cable (TWC) is one of the early adopters of OpenStack. Spurred by the automation and self-service capabilities achieved through our OpenStack journey, we were ready to take on our next ambitious project – deploying container based microservices applications on top of OpenStack. We chose Mesos / Marathon as our application and container runtime platform on OpenStack IaaS.   In this session, we will discuss our experiences with the technology, tools, and techniques that made this possible at TWC. We will make this an interactive session to learn from our collective experiences.  IaaS is only the beginning! Modernizing large corporate application ecosystems can benefit from additional building block tooling in the form of newer architectures for applications (microservices), and a generic application runtime environment.  At TWC we've chosen to deploy Mesos/Marathon as our application container runtime platform and are aggressively pursuing the deployment to facilitate our internal engineering efforts. For our initial deployment, we are deploying Mesos on OpenStack IaaS. In this session, we will discuss how we thought through our needs for the project and our considerations for deploying, monitoring, and running critical apps in production on large clusters.  Microservices-based applications deployed on containers are gaining popularity with application developers. Enterprises considering these application architectures go through a similar set of considerations and certain best practices are emerging in the process. This session is based on real-world experiences deploying a Mesos/Marathon cluster on OpenStack."

"Mesos has been supporting external volumes since the fall of 2015, but playa-mesos has not. playa-mesos is the simple Vagrant project for bringing up a quick Mesos cluster on a developer's machine using VirtualBox.  In this talk, we will look at how we've extended the capabilities of playa-mesos to support external volumes using REX-Ray. When using REX-Ray, external volumes are supported when using the Docker containerizer or the Mesos volume isolator."

"Mesos is great at allocating cluster resources for schedulers. However, there is no way to downsize your cluster, short of pulling the plug on machines! How do you defragment or roll a kernel upgrade through your cluster? Most importantly, how do you do so without violating any frameworks’ SLAs?  This talk will introduce Inverse Offers and present different use cases for them. Maintenance Primitives, a feature implemented using Inverse Offers will be used as an example to present the requirements and challenges of cooperatively deallocating resources from a running cluster.   Guidelines, strategies, and trade-offs with respect to Inverse Offers and Maintenance Primitives for framework writers as well as cluster operators will be presented."

Getting a job of your dreams might be easier than you think. Apply these five hacks and see for yourself. Study after study has showed that being satisfied and happy at a job is paramount for a healthy, productive and long life. In other words, if you’re miserable at a job, then other areas of your life will suffer as well: personal life, health, spirituality, family, friends, etc. You can use the five hacks to be presented to get the job of your dreams in pretty much any industry or field. And most of them will cost you nothing or close to nothing!

At Banno we've spent the last year moving from Chef to a stack comprised of primarily of Packer, Terraform, Mesos, and Docker. This talk is an overview of moving from a mutable automation model with Chef to a truly immutable automation model. We describe each piece of the infrastructure, why it matters, and how it composes with each other piece to create a platform wherein we can stand up a functioning mirror of production from scratch in fifteen minutes. Because we can do that, I will also talk through how we have implemented continuous delivery of our infrastructure by having Jenkins fire up a mirror of production for every change to production to ensure that the tests pass. Other topics that will be covered are: service discovery, automated rolling of environments to new server images, and disaster recovery & datacenter failover.

A scaleable and adaptable machine learning platform is essential for an organization to harness the full potential of their data. This talk outlines how Docker, Spark, Hadoop and several other building blocks can be integrated into a machine learning framework on Mesos. We will discuss how our Mesos framework leverages custom executors, framework/status messages and resource attributes to schedule tasks in a multi-tenant environment. A heterogeneous workload of Spark, Python, R & Scala tasks co-exist and run thousands of computations concurrently on an elastic Mesos cluster of hundreds of nodes. We will demonstrate how a simple user interface and a DSL allows researchers to design, test and deploy resilient machine learning pipelines.

"Our largest workloads serve tens of billions of requests per day. This requires highly scalable big data processing services. Due to the nature of the business, services can see 30x traffic spikes within minutes. Our customers, meanwhile, often demand four 9s availability requirements. This is coupled with a highly heterogeneous infrastructure.   CellOS is our solution to the challenges above. It is a “Datacenter as a Computer” approach with a couple of goals: offer reliability and efficiency built on solid grounds and focus on usability for operators and engineers.  It takes a single command and a few minutes to stand a cell up, to scale it to hundreds of nodes and back to 0, or to delete it. Once provisioned, a cell is fully equipped to run any kind of workload in a scalable and reliable fashion.  We bridged Mesos with our Big Data knowledge and designed a highly modular datacenter operating system by building on top of carefully selected open-source technology.  Independent fault / security / scalability domains allow us to run hybrid workloads making use of different configuration for VMs or hardware.  By fully separating the infrastructure layer from the clustering layer we can run this solution in multiple cloud environments and in our datacenters."

"Dealing with network failures and partitions are key challenges for large-scale distributed services. As a datacenter kernel, Mesos must both behave correctly in the face of partitions and provide application developers with the tools they need to build partition-tolerant applications.  This talk will cover the history of partition handling in Mesos, drawing on lessons learned during the course of operating large-scale Mesos clusters at Twitter. After reviewing the status quo, we’ll discuss how we plan to evolve partition handling in Mesos to give application developers more flexibility and clearer semantics in the presence of partitions."

"My company Container Solution is working extensively with Mesos for about a year now. In addition to implementation of Mesos in enterprises we created multiple Mesos frameworks: http://mesosframeworks.com/ while contributing to Cisco Mantl.io project that is based on Mesos. During this time we have realised that Mesos is a complex system that is heavily oriented towards operators and has high barriers for adoption by developers. While building our own testing environment for frameworks development that we call minimesos, we also realised, that to support really complex microservices behavior like autoscaling, cloud migration, etc. we need to create framework, which is a very complex task beyond reach of most of the businesses just trying to solve specific infrastructure problem. We think that sooner or later we will need to havre a full programming language which is flexible enough to solve complex problems, but also easy to use to drive wider adoption of Mesos. So, we have created a prototype we are calling InfraGraph which includes Groovy as a programming language and capable of creating complex cloud behavior in just few lines of code."

"At Adobe APIs are powering the next generation of Creative applications.  Mesos makes it very easy and fun to deploy and run Robust and Scalable Microservices in the Cloud. Today's technologies offer simple solutions to create RESTfull services while Mesos brings them to life faster.   As the number of microservices increase and the inter communication between them becomes more complicated, we soon realize we have new questions awaiting our answers: how do microservices authenticate ? how do we monitor who's using the APIs they expose ? How do we protect them from attacks ? How do we set throttling and rate limiting rules across a cluster of microservices ? How do we control which service allows public access and which one we want to keep private ? How about Mesos APIs and its frameworks ? Can they benefit from these features as well ?  Come and learn a scalable architecture to manage microservices in Mesos by integrating an API Management layer inside your Mesos clusters. This presentation will show you what an API Management layer is, what it's composed of and how it can help you expose microservices in a secure,managed and highly-available way, even in multi-Mesos cluster setups.  During this session you will also have the opportunity to learn how Adobe's API Platform solved this problem, where it is today and what it envisions do to with Mesos further.   If you're working with microservices already or you're creating new ones then this presentation is for you. Come and learn how Mesos together with an API management layer will make you a microservices hero in your organization."

"Mesos enables two layer scheduling by employing an offer-based resource negotiation protocol between the Mesos master and the Framework scheduler. Currently, Mesos only supports the Pessimistic Offer model. With the Pessimistic Offer model, if a resource has been offered to a framework, this same resource will not be offered to another framework until the previous offer times out. When Framework schedulers receive an offer, they will either cache unused offers, or block the offer and then trigger Mesos to resend the offer through the revive API when the Framework has a new workload request. This current method of handling offers, generally leads to tradeoffs among framework performance, resource utilization and allocation fairness.  This talk will introduce an alternative - the Optimistic Offer model - that can reduce the need for such tradeoffs leading to improved scheduling of resources. With the Optimistic Offer model, the same offer will be made available to all frameworks, which provides each framework with an equal opportunity to view for available resources in the cluster. This presentation will cover the proposed Mesos Optimistic Offer development plan, the current development status of the Mesos Optimistic Offer, the process for developing frameworks to utilize the Optimistic Offer model, as well as an evaluation of running existing Mesos frameworks with Optimistic Offer."

Now is the time to make that Mesos framework you’ve been dreaming about a reality! With the help of available libraries, building a simple framework in your favorite language is a relatively straightforward affair. This talk will focus on the well-established Mesos SchedulerDriver API and provide guidance for developers who have never written a framework before. We will review the fundamentals of Mesos frameworks, examine the scheduler API, survey the current ecosystem of libraries, and then finally show you how to build and run a small framework of your own!

Composite containers representing an application is a common requirement for modular architecture. Composition requires co-location treating a set of containers as a single unit (aka pod) for scheduling. Sidecar, ambassador, adapter patterns use container pod. docker compose in docker community is an excellent way of defining a collection of containers and can be used to represent pod. Mesos on the other hand plays the critical role of a resource and cluster manager for large clusters. The native docker integration in Mesos can only launch a single container. This presents a challenge to launch container pods in Mesos. In this presentation, we will see how a container pod defined through docker-compose spec can be launched in Mesos using popular frameworks like Marathon, thereby providing a solution missing in the current ecosystem. A developer can now write the pod spec once, run it locally in their laptop using compose, take it to docker swarm cluster or seamlessly move into Mesos without having to modify the pod spec. Audience will learn how we use custom Mesos executor to achieve this. Moving away from native container support poses an issue to clean up these pods during faults and Mesos hooks implemented as modules comes to the rescue. Audience will learn how to build and run custom mesos modules. All the code would be shared with the community and would be encouraged to contribute. We plan to end with a demo.

Craig Pottinger, in addressing some of the challenges of creating a custom Akka Mesos framework, will aim to shine light on exactly what is involved in 1) Building powerful concurrent and distributed applications and 2) Leveraging a Mesos ecosystem that is designed to abstract away the resource limit challenges that hinder building those reactive applications. As more applications are built with fault-tolerance, resiliency, and reactiveness in mind, it is important to achieve resiliency not just on the application layer, but also on the commodity machines on which they run. Firstly, there will be a run-down of the Mesos framework api and its usage. The talk will then address issues around creating a framework specifically for Akka.

"In this talk we look at what goes into cloudfoundry-mesos, a project that modifies that Cloud Foundry Diego auctioneer process to be a Mesos Framework. This includes an overview of Cloud Foundry's Diego architecture, and why it can be adapted to run on Apache Mesos.  By using a PaaS such as Cloud Foundry as an example, we can show how Mesos can serve as a common layer within the Datacenter, and does not limit the technologies on wishes to deploy. In fact, Mesos can make it easier to run multiple technologies by using pooled resources while still giving developers and users access to each technology's native tooling."

For the last few years Mesos has grown beyond just being a platform for streaming servers like spark and stateless services into a juggernaut for efficiently operating big data applications like Storm, Spark, Kafka, Cassandra and Hadoop. We are going to be talking SHMACK Stack 2.0, we have gone far beyond just SMACK. SHMACK => Streaming Hadoop Mesos Analytics Cassandra Kafka and more! We will dig down into different Streaming/Analytics systems (Storm, Heron, Spark, Kafka Streams, etc) and the down stream Kafka Log, Hadoop and Cassandra for not just where and how they work reliably in production but what else you can do on Mesos with distributed actor and remote procedure call patterns.

"Windows Server 2016 is expected to be made generally available sometime in Q3 2016, and with it, the Windows Containers API.  Supporting Windows Containers natively is a major growth opportunity for the Mesos community -- by enabling users to operate clusters containing a mix of Unix and Windows hosts, it is possible to significantly reduce administrative overhead and increase resource utilization in data centers.  In this talk we will provide a ground-up view of the work we have done over the last year to make the Windows agent a reality.  We’ll cover the changes made to Mesos core so far, discussing in detail how the Mesos agent works on Windows. We will then discuss the implications of these changes at the service level, covering how to operate clusters with a mix of Windows and Unix nodes in practice.  Finally, we will present a demo running on such a cluster, and a roadmap for future plans."

"All things Docker seem to be the defining slogan for Yammer these days. And what other way to run Docker at scale than Mesos? Yammer wrote an in-house datacenter automation platform which was fraught with problems and restrictions the Mesos landscape promised to solve, and in 2015 engineers at Yammer started exploring it as a possible replacement. This talk will focus on two core points of the implementation run at Yammer: Docker & Security Everything has to run inside Docker. From Zookeeper to the Mesos agents and even the executors. Dedication, mixed with some despair, eventually resulted in the platform for production today.  Security is the second biggest piece to honor the commitment made to Yammer customers as part of the Office365 suite. The path to provide a good solution to security aspects like compartmentalization, trust bootstrap, access control and transport encryption proved a balancing act between customer commitment and developer friendliness.  In the end Mesos helped us deliver Docker's promise of Build, Ship, Run as the foundation of environments Yammer can trust to run from Dev all the way to Production at scale and anywhere on the planet."

"Mantl.io is an integrated platform to run microservices. Most of the companies starting creating microservices and containers will quickly arrive to a conclusion that they need more than just Docker and Mesos but many more complex tools glued together in a consistent way. Mantl includes Mesos and Marathon as a key parts os the system but also provides out of the box deployment using Terraform and Ansible and pre-integrated tools suck as Docker, ELK sack, Project Calico, Vault, Consul and others to provide a comprehensive platform ready to be used by both developers and operators. Mantl is an open source project led by Cisco and already adopted in many organizations."

Traditionally, machines were statically partitioned across the different services at Uber. In an effort to increase the machine utilization, Uber has recently started transitioning most of its services, including the storage services, to run on top of Mesos. This presentation will describe the initial experience building and operating a framework for running Cassandra on top of Mesos running across multiple datacenters at Uber. This framework automates several Cassandra operations such as node repairs, addition of new nodes and backup/restore. It improves efficiency by co-locating CPU-intensive services as well as multiple Cassandra nodes on the same Mesos agent. It handles failure and restart of Mesos agents by using persistent volumes and dynamic reservations. This talk includes statistics about the number of Cassandra clusters in production, time taken to start a new cluster, add a new node, detect a node failure; and the observed Cassandra query throughput and latency.

"Since being unveiled at MesosCon Europe in 2015, external persistent volumes have quickly received a lot of attention with its potential applications. We will discuss using volume drivers as means of provisioning external storage for Mesos Frameworks. We will specifically use the Elastic Search framework (https://github.com/mesos/elasticsearch) as a reference implementation to discuss how the solution of adding external volumes was architected. A demo of the Elastic Search framework will be provided with discussions about how external storage can benefit Frameworks and the applications they manage.  Proposal Outline: - What is a Framework? * Schedulers * Executors - External Storage Considerations for Frameworks * Storage is a externally managed. * Only consider CPU and memory for Offers * Need a concept of a ""Node ID"". Describe why this Node ID is important. - ""Node ID"" * Need a mechanism in order to tie a Node to Persistent Storage * Needs to support Mesos Agent failover * How do we accomplish this? - Demo Description * Supports Docker Volume Driver Interface (DVDI) * Support mesos-module-dvdi for non-Docker Executors - Demo * Deploy ElasticSearch Framework using External Storage * Discuss the behaviors of the Framework"

"In this talk, we will take a look at how Mesos can prevent silos within the datacenter by serving as a common layer for multiple schedulers. Specifically, we will look at how Mesos can run multiple, sometimes competing, technologies in parallel, like Cloud Foundry, Mesosphere, Swarm, and Kubernetes.  Thanks to Mesos Frameworks, the schedulers from each of these different platforms has the opportunity to launch jobs on the same hardware. This reduces the amount of hardware needed, and eliminates the need to dedicate portions of your infrastructure to run a specific technology.  We will cover the current state of running Cloud Foundry, Mesosphere, Swarm, and Kubernetes on Mesos, including demonstrations of successful integrations."

Mesos and its ecosystem functions as a distributed cluster manager which enables applications and services run efficiently in an shared cluster. One of the gap we think needs a solution is how to manage the cluster which is needed for Mesos to operation and managing Mesos itself. We called this new system as (Mesos) Infrastructure Manager. The New System will enable operators and developers to spin up new mesos clusters in any cloud with optimal configuration, deploy and manage all the mesos and frameworks, monitors and collect metrics about the clusters, Auto scale up/down the cluster based on the various metrics , Remediate any failures in hardware or core services and also take care of upgrade of the cluster components with minimal or no downtime. In this talk, we will walk through the lessons learnt running a big cluster and how all the above mentioned functionalities are achieved through industry best practices.

The Oracle Data Cloud is processing billions of events and multiple terabytes of data per day in real-time. In this presentation Mike Sells and Duc Nguyen will discuss the motivation, design, and implementation of an elastic Mesos platform to handle the needs of their real-time systems. They will discuss how to think about the elasticity of applications inside of Mesos as well as Mesos itself with details on how to implement both. Why scaling down, not up, is the biggest challenge. They will explain how they built a high volume low latency web tier and a Spark streaming data pipeline on top of the same self-healing platform. How building this multi-tenant system on top of Mesos can maximize resource utilization as well as productivity while automatically adjusting to the demands of traffic and self healing on failures. This includes how they test and validate that elasticity does not impact data integrity. And will conclude by demonstrating the real-time production system, creating live failures, and showing its ability to self heal and auto correct.

Many people promise fast data as the next step after big data. The idea of creating a complete end-to-end data pipeline that combined Spark, Akka, Cassandra, Kafka, and Apache Mesos came up two years ago, sometimes called the SMACK stack. The SMACK stack is an ideal environment for handling all sorts of data-processing needs which can be nightly batch-processing tasks, real-time ingestion of sensor data or business intelligence questions. The SMACK stack includes a lot of components which has to be deployed somewhere. Lets see how we can create a distributed environment in the cloud with Terraform and how we can provision the Mesos-Cluster with Mesosphere Datacenter Operating System (DCOS) and create a powerful fast data platform.

"During our work on multiple Mesos frameworks including ElasticSearch, Logstash and Kibana we created two open source projects. First is a mesos frameworks testing tool called minimesos (minimesos.org) which is a local mesos cluster based on few docker containers that can be started in seconds and the second is Mesos-Started (https://github.com/containersolutions/mesos-starter) which is a template framework that allows to create a new Mesos framework in minutes using a simple configuration file but it also allows to extend the framework later on with java code. Recently we realised that if we combine the two and add the ability to deploy minimesos to multiple target platforms such as another Mesos or kubernetes or swarm, then we can easily create services based on Mesos-Starter and use minimesos as distributed runtime environment to run on other distributed systems. It can also add the multi tenancy capabilities and complex cloud native logic to the underlying platforms. In this talk I'll explain the concept and demo the combination of the tools."

"Ever thought of running the Mesos Master in your Top of Rack switch?  Julius Mueller and Marcel Neuhausler from AT&T Foundry take an edge-computing scenario to showcase and demo how a ToRC (Top-of-Rack Controller) and its extended resource-awareness enables a switch to orchestrate an application in a highly cost-effective and autonomous way.  ToRC is based on different open source projects (Mesos, Docker, FBOSS, ONL …) running on an OCP switch (FB Wedge)."

"Developing an end-to-end big data application right from data ingestion, data enrichment and visualisation is a very cumbersome task. In this talk, I will demonstrate how to use Apache Mesos, Marathon, Apache Spark to build a scalable, fault tolerant, responsive data platform. The result will be a real-time big data application with self-healing features — a dream for every software developer.  This talk is a collection of different recipe’s that will help the developer to understand Mesos ecosystem projects.  Choosing the right technologies and tools during the development phase has a major impact on the success of the whole project. Apache Mesos provides the best cluster management system, Marathon gives the feature for long-running applications, Akka helps to build highly responsive, resilient, elastic and message-driven system."

"Mesos modules provide a way to easily extend the inner working of Mesos. With Mesos being by nature a cluster manager, you might expect that you’d need a ton of high end hardware to do real test and development. This simply isn't true.  Steve will demonstrate how you can build a real production grade module, using open source and free technologies, running on a laptop.   This talk will cover:  - How to do repeatable builds of Mesos itself, within a container, in the version of your choice.  - How to build multiple versions of your own module, to support a range of different Mesos releases. - An easy method for automatically deploying a multi-node Mesos cluster, in the version of your choice, to support development and test.  If you have dreams of building a phenomenal new thing for Mesos, but haven’t landed that billion dollar investor yet, this session is for you.  It could also be useful if you're just some corporate drone, like the speaker, who likes to do Mesos builds at home in the middle of the night."

Mesos currently supports two types of container runtimes: one is Mesos’ own pluggable runtime which uses OS primitives directly (a.k.a. the Mesos containerizer); the other is to delegate the entire container management to Docker engine (a.k.a. the Docker containerizer). Recently, we added container image support to Mesos containerizer, which allows it to launch off-the-shelf Docker or Appc containers without depending on Docker daemon or rkt. In this talk, we are going to discuss the motivation of why we want to unify the containerizers in Mesos and how this benefits the Mesos maintainers and our users. We will then talk about the current status of this work, and show what can be built on top of this pluggable architecture. Finally, we will share our vision on the near future for the Mesos containerizer and how it’s going to enable more innovation in the Mesos ecosystem.

"There are various types of federation solutions from different open source communities, Mesos and OpenStack each has its own proposal on solving the cloud federation problem. However,these proposals are not without similarities among themselves, and therefore a unified approach which provides a combination might help users to deploy their services when they have heterogeneous infrastructures that could not rely on single solution. In this talk the speaker will present this very idea, which is an ongoing effort to combine federation solutions. The following topics will be covered: • Introduction of OpenStack Tricircle and Magnum • Introduction of Mesos Federation • Working Proposal of a united approach of Tricircle and Mesos Federation • Future Work Plan The great benefit of this approach is that we could leverage what three great open source community could offer for cloud federation solutions."

"Metrics driven infrastructure is one of the key ingredients in transforming the engineering culture of an organization. Mesos and Marathon provides rich dataset as part of its operations, that can act as a source for data for metrics, especially around application deployments, service health checks, infrastructure utilization, and capacity planning.   This session will introduce a reference architecture and design of metrics engine and dashboard, that consumes the Observability metrics data from Mesos, and Marathon events, to create actionable opportunities for developers and system operators. The session will also cover key implementation details, and showcase the ways these metrics are useful in an enterprise setting, and can further be used to improve the overall productivity of the engineering teams. We will also look at the impact of these metrics on Operational performance and team productivity. Further, we will propose improvements to help extend the current capabilities of metrics and events from Mesos and Marathon."

"Have you found a bug in Mesos that you would like to fix yourself? Have you got a feature that you would like to introduce to Mesos? Are you already a contributor, but finding it difficult to maneuver the codebase and getting your patches committed?  This talk is intended for those who would like to contribute to Mesos. Contributing to any open source project can be intimidating and trying to seek information remotely can be frustrating and demotivating. We aim to provide information that will help contributors, and will cover topics such as the project management process (JIRA, shepherds), coding standards, developer documentation, developer tools, debugging tools and techniques, running tests, and submitting patches."

"Apache Mesos is a cluster resource manager that allows you to represent many servers as a single entity. Its use is commonly associated with the rapid, repeatable deployment of containerized applications and improved datacenter efficiency. But to users unfamiliar with containers, or to those accustomed to deploying static sets of machines to host applications, deploying Mesos can seem like a daunting task.  In this workshop, you'll learn how to deploy Mesos in a development environment. Once Mesos is up and running, you'll learn to create cron jobs using Chronos, and deploy containerized applications using Marathon. By the end of the workshop, attendees should have a working Mesos cluster running on their laptop.  This workshop will cover the following: * Mesos fundamentals * Spinning up your first Mesos cluster * Creating cron jobs using Chronos * Deploying apps using Marathon"

"Writing Mesos frameworks with the original framework API came with dependencies that made implementations of new client burdensome. If you have attempted to write a framework before the HTTP API you certainly ran into issues like the lack robust documentation, dependency on platform-dependent libmesos library, network communication issues between your cluster members.   Introduced in version 0.24.0, the new HTTP API solves these problems with a simpler HTTP-based protocol for communication between Mesos cluster members. The presentation explores how the HTTP API makes it possible to create robust frameworks using the HTTP library of your favorite language. The session will do a walkthrough of the HTTP endpoint requests to arrive at a fully functioning framework while addressing the implication of using pure JSON or the Protobuf for data encoding."