RayServe

Deploying a RayServe application through our platform follows the same process as deploying a Container application, but with an additional configuration step for RayServe. After configuring the basic container settings, you will upload your RayServe configuration YAML file, which defines the serving graph and behavior of your RayServe application.

Adding a RayServe Application

1. Create a new RayServe application:

   • Click the "+" icon in the top-right corner and select "RayServe".

2. Configure the application:

   • Name: Enter a name for your application (e.g., rayserve-recommender).
   • Cluster: Select the Kubernetes cluster for deployment.

3. Add a container:

   • Container registry: As with Container applications, you may deploy your RayServe application from either a public or private container registry.
     • Option 1: Private container registry
       • Select the container registry connection from which the container image needs to be fetched. Learn more about container registry connections here.
       • Image repository: Select the desired image repository.
       • Image tag: Select the image tag you wish to deploy.
     • Option 2: Public repository
       • Select “Public repository” from the container registry dropdown.
       • Public repository URL: Enter the public repository URL or name you would use to pull the image.
         • Examples:
           • Docker Hub:
             • nginx
             • docker.io/library/nginx
           • GitHub Container Registry (GHCR):
             • ghcr.io/kapstan-io/agent
           • Amazon Elastic Container Registry (ECR):
             • public.ecr.aws/karpenter/controller
           • Other:
             • quay.io/prometheus/node-exporter
       • Image tag: Select the image tag you wish to deploy.

4. Upload RayServe Config: In addition to the container configuration, RayServe applications require a RayServe configuration file. This YAML file defines your RayServe deployments, replicas, and routes.

   • RayServe Config File (YAML):
     • Copy the YAML file from your RayServe application into the input field.
     • Ensure that the YAML file is properly formatted and includes all necessary fields for your RayServe application.

   Example of a simple RayServe configuration:

   # Example RayServe config
    applications:
      - name: translator
        import_path: translator:app
        route_prefix: /

5. Finalize and Deploy:

   • Click “Confirm” to create the RayServe application.
   • You will be redirected to the application overview page, where you can proceed with deployment.

Adding multiple containers within a RayServe application

You can run multiple containers as part of your RayServe application, just as you would with a standard Container application.

1. Initiate the process of creating a new RayServe application by following the steps outlined in Adding a RayServe application.
2. On the application configuration tab, click the + Container button situated at the bottom right. This allows you to configure additional containers (e.g., an init container or sidecar) alongside your primary RayServe container.
3. Modify configurations of individual containers using the container dropdown menus.

Configure application

Go to application configurations to learn more about application configurations like environment variables, networking, hardware and more.

Deploy

Go to deployment and event history to learn more about deploying the application and viewing the event history.