Diego Components and Architecture

This topic describes the components that form and interact with the Diego system in Pivotal Web Services.

Overview

PWS uses the Diego system to manage app containers. Diego components assume app scheduling and management responsibility from the Cloud Controller.

Diego is a self-healing container management system that attempts to keep the correct number of instances running in Diego cells to avoid network failures and crashes. Diego schedules and runs Tasks and Long-Running Processes (LRP). For more about Tasks and LRPs, see How the Diego Auction Allocates Jobs.

You can submit, update, and retrieve the desired number of Tasks and LRPs using the Bulletin Board System (BBS) API.

How Diego Runs an App

The following sections describe how Diego handles a request to run an app. This is only one of the processes that happen in Diego. For example, running an app assumes the app has already been staged. For more information about the staging process, see How Applications are Staged.

The diagrams below do not include all of the components Diego. For information about each Diego component, see Diego Components.

Receives the Request to Run an App

The Cloud Controller passes requests to run apps to the Diego BBS, which stores information about the request in its database.

Several boxes represent the VMs involved in running an app with Diego. They have additional boxes within them to represent the components running on each VM. The boxes are as follows: the api VM includes the cloud_controller_ng process, the scheduler VM includes the auctioneer process, the diego-api VM includes the bbs process, the pxc-mysql VM includes the bbs db, the diego-cell VM includes the rep, loggregator-agent, garden, and route-emitter processes, the singleton-blobstore VM includes the droplets store, the doppler VM includes the doppler process, the log-api VM includes the traffic-controller process, and the gorouter VM includes the gorouter process. An arrow points from cloud_controller_ng to bbs to indicate that the cloud_controller_ng process is sending a request to run an app to the bbs process. Another arrow points from bbs to bbs db to indicate that the request gets stored in a database.

Passes Request to the Auctioneer Process

The BBS contacts the Auctioneer to create an auction based on the desired resources for the app. It references the information stored in its database.

An arrow points from a box labeled "bbs" to a cylinder labeled "bbs db" to indicate the bbs process is receiving information from its database. Another arrow points from bbs to a box labeled "auctioneer" to indicate the bbs process is contacting the auctioneer process to create an auction.

Performs Auction

Through an auction, the Auctioneer finds a Diego cell to run the app on. The Rep job on the Diego cell accepts the auction request.

An arrow points from a box labeled "auctioneer" to a box labeled "rep" to indicate that the auctioneer finds a Diego Cell VM to satisfy the request to run the app.

Creates Container and Runs App

The in-process Executor creates a Garden container in the Diego cell. Garden downloads the droplet that resulted from the staging process and runs the app in the container.

An arrow points from a box labeled "executor" to an unlabeled box inside of a box labeled "garden". This indicates that the executor sub-process of the rep process on the diego-cell VM communicates to the garden process on the diego-cell VM to create a new container. Another arrow points from garden to a cylinder labeled "droplets" to indicate that the garden process also communicates with the droplets bucket on the singleton-blobstore VM to download the droplet.

Emits Route for App

The route-emitter process emits a route registration message to Gorouter for the new app running on the the Diego cell.

An arrow points from a box labeled "route-emitter" to a box labeled "gorouter" to indicate that the route-emitter process sends a route registration message to Gorouter.

Sends Logs to Loggregator

The Loggregator agent forwards app logs, errors, and metrics to the PWS Loggregator. For more information, see the Application Logging in PWS topic.

An arrow from an unlabeled box inside a box labeled "garden" points to a box labeled "loggregator-agent". This indicates that the container running the app inside the garden process of the diego-cell VM emits logs to the loggregator-agent process of the diego-cell VM. An arrow from loggregator-agent points to a box labeled "doppler", and from doppler to a box labeled "traffic-controller" to indicate the where logs are sent.

Diego Components

The following table describes the jobs that are part of the PWS Diego BOSH release.

Component Function
Job:
auctioneer
VM:
  • Distributes work through auction to Cell Reps over SSL/TLS. See How the Diego Auction Allocates Jobs.
  • Maintains a lock in Locket to ensure only one auctioneer handles auctions at a time.
Job:
bbs
VM:
  • Maintains a real-time representation of the state of the Diego cluster, including desired LRPs, running LRPs, and in-flight Tasks.
  • Provides an RPC-style API over HTTP to Diego Core components for external clients as well as internal clients, including the SSH Proxy and Route Emitter.
  • Ensures consistency and fault tolerance for Tasks and LRPs by comparing desired state with actual state.
  • Keeps DesiredLRP and ActualLRP counts synchronized. If the DesiredLRP count exceeds the ActualLRP count, requests a start auction from the Auctioneer. If the ActualLRP count exceeds the DesiredLRP count, sends a stop message to the Rep on the Cell hosting an instance
Job:
file_server
VM:
  • Serves static assets that can include general-purpose App Lifecycle binaries
Job:
locket
VM:
  • Provides a consistent key-value store for maintenance of distributed locks and component presence
Job:
rep
VM:
  • Represents a Cell in Diego Auctions for Tasks and LRPs
  • Runs Tasks and LRPs by creating a container and then running actions in it
  • Periodically ensures its set of Tasks and ActualLRPs in the BBS is in sync with the containers actually present on the Cell
  • Manages container allocations against resource constraints on the Cell, such as memory and disk space
  • Streams stdout and stderr from container processes to the metron-agent running on the Cell, which in turn forwards to the Loggregator system
  • Periodically collects container metrics and emits them to Loggregator
  • Mediates all communication between the Cell and the BBS
  • Maintains a presence record for the Cell in Locket
Job:
route_emitter
VM:
  • Monitors DesiredLRP and ActualLRP states, emitting route registration and unregistration messages to Gorouter when it detects changes.
  • Periodically emits the entire routing table to the PWS Gorouter.
Job:
ssh_proxy
VM:
  • Brokers connections between SSH clients and SSH servers
  • Runs inside instance containers and authorizes access to app instances based on Cloud Controller roles

Additional Information

The following resources provide more information about Diego components:

Components from Other Releases

The following table describes jobs that interact closely with Diego but are not part of the Diego PWS BOSH release.

Component Function
Job:
bosh-dns-aliases
VM:
all
  • Provides service discovery through colocated DNS servers on all BOSH-deployed VMs
  • Provides client-side load-balancing by randomly selecting a healthy VM when multiple VMs are available
Job:
cc_uploader
VM:
  • Mediates uploads from the Executor to the Cloud Controller
  • Translates simple HTTP POST requests from the Executor into complex multipart-form uploads for the Cloud Controller
Job:
database
VM:
  • Provides a consistent key-value data store to Diego
Job:
loggregator-agent
VM:
all
  • Forwards app logs, errors, and app and Diego metrics to the Loggregator Doppler component
Job:
cloud_controller_clock
VM:
  • Runs a Diego sync process to ensure desired app data in Diego is in sync with the Cloud Controller.

App Lifecycle Binaries

The following three platform-specific binaries deploy apps and govern their lifecycle:

  • The Builder, which stages a CF app. The Builder runs as a Task on every staging request. It performs static analysis on the app code and does any necessary pre-processing before the app is first run.

  • The Launcher, which runs a CF app. The Launcher is set as the Action on the DesiredLRP for the app. It executes the start command with the correct system context, including working directory and environment variables.

  • The Healthcheck, which performs a status check on running CF app from inside the container. The Healthcheck is set as the Monitor action on the DesiredLRP for the app.

Current Implementations

Additional Information

The following resources provide more information about components from other releases that interact closely with Diego: