Ray Serve

Deploying a Ray Serve application through our platform follows the same process as deploying a Container application, but with an additional configuration step for Ray Serve. After configuring the basic container settings, you will upload your Ray Serve configuration YAML file, which defines the serving graph and behavior of your Ray Serve application.

info

You need to install the KubeRay add-on from Environment settings before you can deploy a Ray Serve service.

Adding a Ray Serve Application

1. Create a new Ray Serve application:

   • Click the "+" icon in the top-right corner and select "Ray Serve app".

2. Configure the application:

   • Name: Enter a name for your application (e.g., rayserve-recommender).
   • Cluster: Select the Kubernetes cluster for deployment.

3. Add a container:

   • Container registry: As with Container applications, you may deploy your Ray Serve application from either a public or private container registry.
     • Option 1: Private container registry
       • Select the container registry connection from which the container image needs to be fetched. Learn more about container registry connections here.
       • Image repository: Select the desired image repository.
       • Image tag: Select the image tag you wish to deploy.
     • Option 2: Public repository
       • Select “Public repository” from the container registry dropdown.
       • Public repository URL: Enter the public repository URL or name you would use to pull the image.
         • Examples:
           • Docker Hub:
             • nginx
             • docker.io/library/nginx
           • GitHub Container Registry (GHCR):
             • ghcr.io/kapstan-io/agent
           • Amazon Elastic Container Registry (ECR):
             • public.ecr.aws/karpenter/controller
           • Other:
             • quay.io/prometheus/node-exporter
       • Image tag: Select the image tag you wish to deploy.

4. Upload Ray Serve Config: In addition to the container configuration, Ray Serve applications require a Ray Serve configuration file. This YAML file defines your Ray Serve deployments, replicas, and routes.

   • Ray Serve Config File (YAML):
     • Copy the YAML file from your Ray Serve application into the input field.
     • Ensure that the YAML file is properly formatted and includes all necessary fields for your Ray Serve application.

   Example of a simple Ray Serve configuration:

   # Example Ray Serve config
    applications:
      - name: translator
        import_path: translator:app
        route_prefix: /

5. Finalize and Deploy:

   • Click “Confirm” to create the Ray Serve application.
   • You will be redirected to the application overview page, where you can proceed with deployment.

Adding multiple containers within a Ray Serve application

You can run multiple containers as part of your Ray Serve application, just as you would with a standard Container application.

1. Initiate the process of creating a new Ray Serve application by following the steps outlined in Adding a Ray Serve application.
2. On the application configuration tab, click the + Container button situated at the bottom right. This allows you to configure additional containers (e.g., an init container or sidecar) alongside your primary Ray Serve container.
3. Modify configurations of individual containers using the container dropdown menus.

Configure application

Go to application configurations to learn more about application configurations like environment variables, networking, hardware and more.

Deploy

Go to deployment and event history to learn more about deploying the application and viewing the event history.